Between 1637 and 1642 Gašpar Ilešházi — chief count of Trenčín, Liptov and Orava — built the manor house that still stands in the middle of Dubnica, on the site of an older Gothic residence. It is a four-winged early Baroque building around an arcaded courtyard, one of the largest of its kind in Slovakia. It was made for comfort and display rather than war: the corner bastions, the loopholes along the upper floor and the moat with its drawbridge were insurance, not preparation for a siege. The family left Trenčín Castle, which no longer suited them, and brought their furniture, their pictures and probably the miraculous statue of the Virgin from the castle chapel down into the valley.V rokoch 1637 – 1642 dal Gašpar Ilešházi, hlavný župan trenčiansky, liptovský a oravský, postaviť uprostred Dubnice kaštieľ, ktorý tu stojí dodnes – na mieste staršej gotickej kúrie. Je to štvorkrídlová ranobaroková stavba okolo arkádového dvora, jedna z najväčších svojho druhu na Slovensku. Vznikla pre pohodlie a reprezentáciu, nie pre vojnu: nárožné bašty, strieľne po obvode horného podlažia aj priekopa s padacím mostom boli poistkou, nie prípravou na obliehanie. Rodina odišla z Trenčianskeho hradu, ktorý jej už nevyhovoval, a zniesla do doliny nábytok, obrazy a pravdepodobne aj zázračnú sochu Panny Márie z hradnej kaplnky.
