The 15th century buildings are the remains of the Palace of the Bishop of Lincoln. Although it is often stated as being built in the 15th century, the first Palace was actually built in the late 12th century, when records show it as being used to house the Bishops of Lincoln. Subsequently, the wooden structure was replaced by more substantial buildings and a tall brick tower was added in 1475, protected by walls and a moat, and surrounded by an outer bailey. Parts of the complex were demolished in 1632 on the orders of the Ecclesiastical Commissioners. The Victorian house now present on the site was built in 1872. The Claretian Missionaries were given the site by the Roman Catholic Bishop of Northampton in 1956. Initially it was used a Junior Seminary carrying out the preliminary training of 11- to 18-year-olds aspiring to become Claretian Missionary priests or brothers. The Junior Seminary closed in July 1965. Since then the Towers complex has been developed by the Claretians as a Retreat and Conference Centre. Also located on the site is the Catholic Parish Church of St Hugh of Lincoln, which stands on the site of the great chamber of the medieval palace and was originally built as the chapel for the Junior Seminary. The grounds, but not the buildings, of the Towers are open to visitors at all reasonable times but dogs are not permitted. It is a scheduled monument and a Grade I listed building. Little now remains of the bishops' moated palace except the great tower, the inner gatehouse, part of the battlemented wall, which used to surround the inner court within the moat, and the outer gate and wall. On 16 July 1551, Henry, Duke of Suffolk and his brother Lord Charles died here from the sweating sickness. They had come to Buckden to avoid the sickness at Cambridge. In the 1630s Bishop Williams held state at Buckden, entertaining his neighbours with lavish displays of hospitality. The antiquary Edward John Rudge published a history, Illustrated and Historical Account of Buckden Palace, in 1839.
Henry VIII & Catherine Howard in 1541. They stayed at Buckden Palace in 1541, during a summer tour that the King and Queen took of England before her coronation. It was during this tour that Catherine would be accused of committing adultery with Thomas Culpeper which led to her beheading for treason in 1542.