The first Ferne House was the manor house of the de Ferne family: Philip de Ferne is recorded as living there in 1225. From the Ferne family, in 1450 it passed to the Brockway family, and in 1561 to William Grove of Shaftesbury. By 1809 the house had become so dilapidated that it was demolished. The 18th-century gatepiers to the park remain; they are Grade II listed structures.
Second house
The second Ferne House was built by Thomas Grove, "on an enlarged scale in the year 1811 on the site of the old structure … in an elevated situation, commanding a pleasing view of the surrounding country". An 1850 photograph of this house is reproduced in The Grove Diaries. This house was remodelled some time after 1850 and assumed a square ground-plan. In 1902 the house passed out of the ownership of the Grove family, when it was sold to A. H. Charlesworth, who further enlarged it the following year. The house was bought in 1914 by Alfred Douglas-Hamilton, 13th Duke of Hamilton, who also bought nearby Ashcombe House around the same time. During World War II the house was used as an animal sanctuary by his wife Nina, co-founder in 1906 of the Animal Defence and Anti-Vivisection Society with Lizzy Lind af Hageby. She used the sanctuary to enable well-off London families to evacuate their pets to safety. The house remained in the Hamilton family's possession until the estate was bequeathed by the Duchess to the Animal Defence and Anti-Vivisection Society, for the purpose of maintaining the sanctuary. Nikolaus Pevsner briefly described the house in his 1963 edition of Wiltshire in The Buildings of England series. A clause in the Duchess's will stated that it should remain as an animal sanctuary in perpetuity, but the restrictions she laid down were so stringent, according to The Observer, that the house was unsaleable, and as a result it was demolished in 1965. The animal sanctuary moved to Chard, Somerset, where it still operates; in 1985 the Animal Defence Trust still owned the property, including the still-standing stable block and lodges. In 1991, the Ferne Estate was sold at auction for £1,040,000. The buyer was Francis Dineley, whose father had made money from arms manufacturing.
Third house
Sometime after 1991 the estate passed into the ownership of the 4th Viscount Rothermere and his wife. According to Private Eye, despite residing here, the 4th Viscount Rothermere claims to domicile in France for tax avoidance. In 2001 the third and present Ferne House was built to the design of the architect Quinlan Terry, in Palladian style and at a reported cost of £40m. The house won the award for Best Modern Classical House from the Georgian Group in 2003, and in 2006 permission was sought to build two additional wings. The house was featured in the November 2006 edition of Vanity Fair. A summerhouse in the grounds, called the New Pavilion and also designed by Terry, won the 2008 Georgian Group award for a New Building in the Classical Tradition. Terry is also known for the large plot of buildings in the 18th-century style by the Thames at Richmond.
