The Manor House at Cholesbury was most probably built towards the end of the 16th century. It is a two storey, Grade II listed building. Originally constructed of wood, it has retained its timber framework but acquired a brick casing in the 18th century. It is suggested by English Heritage that the original building was much larger than it is today, consisting probably only the cross-wing of a once much larger house. The house was built on an area previously occupied by a section or rampart which formed part of the ringwork of Cholesbury Camp, an Iron Agehillfort. Also, close by the house and within the boundary of the hillfort is St Lawrences Church. Today the house still retains much of the original roof structures and timber interior structure, an older thin brick chimney stack at the west end of the building and a much more modern chimney at the east end. Major refurbishment work in 2011 has retained the extant original features and removed or renovated the more modern additions.
The manor of Cholesbury is not recorded in the Domesday Book and was first recorded in the 13th century as Chelwardisbyry. In 1086 it was, most likely, included within the manor of Draitone, which according to the Victoria County History for Buckinghamshire was under the control of Magno le Breton and was assessed at 6 hides, and 3 virgates. Prior to the Conquest the manor which contained the lands known then as Chelwardisbyry had been overseen by Aluric a thane or servant of King Edward I. The le Breton family stewarded the lands for King William I and those that succeeded him to the Crown of England for almost two hundred years. The manor of Cholesbury was first recorded in a conveyances of 1248 and another in 1251 to Hughle le Breton who was living in Wolverton. Thomas Perot was keeper of the manor in 1330. His name lives on to this day as Parrott's Farm. The next mention was in 1362 certifying it was in the hands of Mary the Dowager Countess of Norfolk. Thomas was the first of several in the Cheyne family who resided nearby at Chenies and Chesham Bois and become Lord of the Manor in 1384. In 1541 the manor at Cholesbury was sold to Chief Justice Baldwin by Robert Cheyne. and was subsequently sold in 1571 by John Baldwin to Robert Maldred Thomas Stile, who was an attorney of the Court of King's Bench. The ownership of Cholesbury passed via several families including the Hobys and Sayers through marriage between 1666 and 1689 when Mary Sayer married her second husband Loftus Brightwell. John Seare was Lord of the Manor in 1650 and was the first to be at the same time, Lords of both the manor of Cholesbury and the neighbouring manor of Hawridge. It was passed to Richard Seare in 1712 and until his death in 1714 when his son John took over, selling it in 1748 to Robert Darell. Edward Darell bequeathed the manorship in 1813 to his nephew the Rev. John Jeffreys, Rector of Barnes, Surrey. His son the Rev. H. A. Jeffreys the Rector of Hawkhurst, Kent inherited it and on his death in 1899 the lordship was bought by Mr. Henry J. Turner, J.P., the first Lord of the Manor to reside within the area for 200 years at Braziers End House. The Turner family lived in the village until 1935 when the manor was sold to Malcolm Stewart. Henry Moore took it on after the Second World War in 1948 and it was purchased from him by John Randall in 1953. After his death in 1979 Elma Randall became the first female Lord of the Manor before it was bought by Dennis Smith in 1982. From 1987 the Lordship was held jointly by Michael Smith and Christine Stott, and from 1996 solely by Christine Stott.
