Following the murder of Lord Frederick there were a number of memorials proposed to him; the proposal for a pillar or obelisk at Bolton Abbey was first made in September 1882 by electors of the Northern West Riding of Yorkshireparliamentary constituency. The proposal was led by William Foster, owner of the Black Dyke Mills, but he died before the proposal came to fruition. A committee, chaired by Angus Holden was formed to take the idea forward and £2,000 was raised in donations. Bids for a memorial were sought and the chosen bid was one designed by Manchester architects Thomas Worthington and John Garrard Elgood. The design was for a tall crenellated tower to be sited on South Nab, a promontory overlooking the River Wharfe. In 1884, the Duke of Devonshire notified the committee that objections to the planned tower had been received and that it "would be detrimental to the landscape of the district and mar the beauty of the outline". On the basis of this and also the Duke's own opinion about the unsuitability of the design and location, the committee resolved to find another solution. Worthington and Elgood came up with a different design, this time for a fountain. The revised design was accepted by the committee and it was resolved to erect the fountain just north of the priory at a location known as Hartington Seat.
Worthington and Elgood's design for the fountain was constructed by Stephenson and Co of Manchester with the carving work conducted by Earp and Hobbs. The fountain stands approximately high and is constructed of local stone in the form of ashlar blocks. Standing on a plinth the fountain is surrounded by six buttresses in a hexagonal plan. A frieze connects the buttresses and is inscribed "This fountain was erected by electors of the West Riding as a tribute to his memory, Frederick Charles Cavendish Born November XXX. MDCCCXXXVI. Died May VI. MDCCCLXXXII". Each buttress has a gargoyle at the level of the frieze and in the parapet above each section of the frieze there is a coat of arms. Two of these are those of the Cavendish family and the other four are the arms of the Lyttleton, Boyle, Clifford and Howard families, all of whom are families in the Cavendish lineage. Above the frieze the memorial is topped with an open lantern. The Leeds Mercury described the architectural style as Gothic.
