Cavendish lived at Billing Hall, Northamptonshire which he commissioned John Carr to substantially remodel in the fashionable Palladian style around 1776. The house passed to the Elwes family in 1790. He died unmarried in December 1796, aged 64.
Legacy
Cavendish's friend Edmund Burke penned several eulogies of him after his death: "The world never produced a more upright and honourable mind; with very considerable talents, and a still more considerable improvement of them".
There is lost to the world, in every thing but the example of his life the fairest mind that perhaps ever infor'd a human body. A mind totally free from every Vice, and fill'd with Virtues of all kinds, and in each kind of no common rank or form; benevolent, friendly, generous, disinterested, unambitious almost to a fault; Tho' cold in his exterior, he was inwardly quick and full of feeling, and tho' reserv'd from modesty, from dignity, from family temperament and not from design, he was an entire stranger to every thing false and counterfeit: so great an Enemy to all dissimulation active or passive, and indeed even to a fair and just ostentation, that some of his Virtues, obscur'd by his other Virtues, wanted something of that burnish and lustre which those who know how to assay the solidity and fineness of the metal wish'd them to have. It were to be wish'd that he had had more of that Vanity of which we who acted on the same stage had enough and to spare. I have known very few men of better natural Parts, and none more perfected by every species of elegant and usefull erudition. He served the publick often out of Office, sometimes in it, with Fidelity, and diligence, and when the occasion call'd for it, with a manly resolution. At length when he was overborne by the Torrent, he retir'd from a world that certainly was not worthy of him. He was of a character that seems as if it were peculiar to this Country. He was exactly what we conceive an English Nobleman of the old Stamp, and one born in better times.
Lord John Cavendish was listened to, whenever he rose, with...deference or predilection. His near alliance to the Duke of Devonshire; his very name, connected with the Revolution of 1688, which secured the liberties of Great Britain; his unblemished reputation, and his talents, though very moderate;—all these qualities combined to impress with esteem, even those who differed most from him in political opinion. Nature had in the most legible characters stamped honesty on his countenance.