There are records of a manor here since 1040. In June 1781 the previous house was advertised for rent in the Norfolk Chronicle and described thus:- "To be Lett, and entered upon at Old Michaelmas next, all that modern built Capital Messuage situate in Crimplesham, in the County of Norfolk; consisting of a Hall, two Parlours, four very good Lodging Rooms, and compleat Garrets, Kitchen, Dairy, Pantry, Cellars, and other convenient Offices, Coach-house and Stabling for eight Horses, with a Granary over the same, Dove-cote well stocked with pigeons, Yards and Garden, well planted with Fruit Trees, with an Orchard and Paddock adjoining, containing together, by Estimation, five Acres, and with or without eighteen Acres of excellent Pasture Ground, now in the Occupation of Mr James Drew. The said Premisses exceedingly well adapted for the Residence of a Gentleman fond of Country Diversions, and are pleasantly situate within half a Mile of the Turnpike Road from Lynn to London, three of Downham-market, nine of Swaffham, and ten of Lynn, all considerable Market Towns".
In 1881 John Grant Morris paid for a new hall to be built and gave it to his daughter as a wedding present. She married a local landowner Sir Alfred Bagge RN, second son of Sir W Bagge MP for West Norfolk, between 1837–1859 and 1865 - 1881. The new hall, a 10-bedroom mansion of white brick structure set in a wooded park, was designed in 1880 by Alfred Waterhouse and it was erected in 1881. The first payment was on 30 July 1880, and the last payment on 17 November 1881. The payments totalled £6,994 and 15 Shillings. £5,994 and 15 Shillings being the construction costs and £1000 for the finishings. The finishings were from Maples in London and were of very high quality, particularly the fine doors and other woodwork of matched Canadian Pine. The house design was very modern for the time, with warm air ducted central heating to all the rooms, its own electricity generator in the stable block, and its own sewerage system and water supply. In the garden are an artificial lake and a fine Victorian folly in the form of a Gothic Chapel, the materials of which are thought to have come from the ruins of West Dereham Abbey or a building reclaim yard. It may predate the new house. Waterhouse had previously built for Morris Allerton Priory, near Liverpool; he later built also a villa near Cannes.
Notable residents
In 1845, the former building was occupied by Mrs Elizabeth Doyle, an eminent Quaker and close friend of Elizabeth Fry. Bible Society meetings were held in the house from time to time.
