At the heart of the estate lies Elvaston Castle - a Gothic Revival masterpiece designed by James Wyatt in the early 1800s, based on the original house dating back to 1633. Wyatt's design, built for the 3rd Earl of Harrington, became home to the Stanhope family until the estate was sold by the then Earl of Harrington to Derbyshire County Council in 1968.