His active criticism helped to effect a change in the method of electing Scottish representative peers to the House of Lords. He was offered the post of Secretary at the British Embassy in Madrid in 1766-67 but declined it due to the illness of his father who died in 1767. He was also the Grand Master of Scottish Freemasons from 1782 to 1784. He was friends with Benjamin Franklin, originating probably from Franklin's visit to Scotland in 1759 and met him several times in London in 1764. A supporter of the American cause, he corresponded with George Washington and sent him a box made from the oak that hid William Wallace after the Battle of Falkirk. The box was designed by the Goldsmith Company of Edinburgh, and the earl requested that Washington, upon his death, pass it on to the man in his country who he believed should merit it best, and upon the same conditions that induced the earl to send it to Washington. In his will Washington thanked the earl for presenting the box to him along with sentiments which accompanied it; feeling incapable of making the choice asked of him he bequeathed the box back to the earl. In 1780 he founded the Society of Antiquaries of Scotland. As he outlined in a letter of November 1780 he wished to create a body to promote antiquarian researches in that part of Great Britain. It drew on traditions that involved Sir John Scot of Scotstarvit, Sir Robert Sibbald, and Sir John Clerk of Penicuik. The Society of Antiquaries of Scotland was formally constituted on 18 December 1780. He later disagreed with some policies of the Society and withdrew in 1790. However, after withdrawing from the management of the Society of Antiquaries he remained active in the area, contributing papers and became involved with the Literary and Antiquarian Society of Perth in 1785. Erskine was elected a member of the American Antiquarian Society in 1816. He was also elected to honorary membership of the Royal Danish Academy of Sciences and Letters in 1785 and of its Icelandic counterpart in 1791. Spending much of his later time at Dryburgh he embellished the grounds with monuments including commemorations of his ancestors and of Robert Burns and William Wallace. He also commissioned a cable-stayed bridge over the River Tweed at Dryburgh. He opened this bridge on 1 August 1817 but it collapsed within months. A replacement was built after a redesign, but this too collapsed in 1838. A more permanent bridge did not arrive until 1872, when the suspension system was used instead. In later years he became increasingly eccentric a trait which tended to obscure his talents, as was noted by Sir Walter Scott. His correspondents included Horace Walpole, and he produced an Essay on the Lives of Fletcher of Saltoun and the Poet Thomson and several other writings. He died at his residence at Dryburgh in April 1829, leaving no legitimate children, and the earldom passed to his nephew Henry though all his non-entailed properties including Dryburgh were passed to his illegitimate son, Sir David Erskine There is an interesting story concerning the Earl in which the writer George Dyer brought him to meet Charles Lamb at his flat in Mitre Court Buildings. Charles Lamb was not home and his sister Mary Lamb was deeply flustered at having to greet an Earl unannounced in her household.