The Domesday Book records the site as belonging to "Gunni The Dane", however the structure that occupies the site was built sometime between 1615 and 1620 by John Poulett, 1st Baron Poulett. The castle was designed as a hunting lodge for Lord Poulett, a Somerset MP. The English Civil War saw the decline of Poulett's fortunes, and by 1791 the castle was derelict and being used as a dairy by a local farmer. In the early 19th century, the manor of Walton in Gordano, including Walton Castle was bought by Philip John Miles who also held properties and extensive estates elsewhere including Kings Weston House, Leigh Court, Cardigan Priory and Underdown in Ledbury, Herefordshire, whence his family had originally come. The castle passed through the Miles family by descent and in 1975, the castle's owner, Sir William Miles, the 6th Baronet, offered to give the ruin to the public if some money could be found to preserve it. The estimate for this rose from £6,000 to £46,000 and the plan fell through, as did a plan to set up a trust.
Restoration
In 1979, Sir William Miles's daughter and his son-in-law, Martin Sessions-Hodge, spent £100,000 and over a year restoring the ruin into a five-bedroom, three-bathroom house. In 1984 they sold it to Rai and Margarita Hamilton, a financier and record studio owner respectively. The couple made further alterations to the castle, though some of these had to be reinstated due to problems with unauthorised removal of crenellations on a listed building. Margarita regularly hosted charity events at the castle into the late 1990s, but in 1996 the couple separated. After the council formally warned Margarita over the number of noise abatement complaints, in 2013 she sold her share of the castle for £1.5M to Walton Properties Ltd., a Guernsey company in which her estranged husband has a minority interest. She moved out of the property supposedly on the condition that she could still arrange events at the castle. However no mention of any such arrangement is contained in the Purchase & Sale Contract and it appears that any such arrangement would have been a private agreement between Margarita Hamilton and her ex-husband and has no binding effect on the new owners. Margarita Hamilton commenced proceedings in the Bristol High Court in an attempt to obtain reparation for her perceived injustice. In 2009 comedian Bill Bailey was guest of honour at an event designed to raise funds for the Teenage Cancer Trust.
