He married at Bath, on 11 December 1813, Elizabeth Brodie, who was twenty-four years his junior. Brodie was the daughter of Alexander Brodie of Arnhall in Kincardineshire. Elizabeth Grant described her thus: However, at the time of his marriage and, in fact, until he inherited the Dukedom, George found himself in almost constant financial difficulties. He was referred to as "Lord Huntly now in the decline of his rackety life, overwhelmed with debts, sated with pleasure, tired of fashion, the last heir male of the Gordon line". While his marriage remedied some of these problems, it did not supply the much sought-after heir. Like his father, George acquired many of the positions which the Gordon family could expect almost as of right. These included the posts of Lord Lieutenant of Aberdeenshire, Chancellor of Marischal College, Aberdeen and Lord High Constable of Scotland. He held the latter post of Lord High Constable for the coronation of King George IV in 1820. By the time of his succession to the dukedom, he had established a reputation as an extreme reactionary. He steadfastly opposed the Great Reform Bill and when the majority of Tory Peers opted to abstain, he remained one of the twenty-two "Stalwarts" who voted against the Third Reading of the Bill in the House of Lords on 4 June 1832. Throughout much of this period, his wife served Queen Adelaide at court. Indeed, she was given the Queen's coronation robe, which is now to be found with many other Gordon memorabilia at Brodie Castle. Nathaniel Parker Willis, the American journalist, has left us with an interesting account of life at Gordon Castle in the twilight years of the 5th Duke's life. He described the "canonically fat porter" at the lodges who admitted him to a "rich private world peopled by ladies cantering sidesaddle on palfreys, ladies driving nowhere in particular in phaetons, gentlemen with guns, keepers with hounds and terrier at heel, and everywhere a profusion of fallow deer, hares and pheasants. At the castle a dozen lounging and powdered menials." Willis continued: "I never realised so forcibly the splendid results of wealth and primogeniture." Just before dinner the Duke called at his room, "an affable white-haired gentleman of noble physiognomy, but singularly cordial address, wearing a broad red ribbon across his breast, and led him through files of servants to a dining room ablaze with gold plate."
Legacy
The Duke died at Belgrave Square, London, on 28 May 1836, aged 66. The Dukedom of Gordon became extinct, but the Marquessate of Huntly passed to his distant cousin the Earl of Aboyne while the Gordon estates passed to his nephew, Charles Gordon-Lennox, 5th Duke of Richmond. The Gordon moveable property was left by the Duchess to the Brodies of Brodie. Elizabeth Brodie, the last Duchess of Gordon, retired to Huntly Castle Lodge, where she became more fervently religious than she had previously been until her death on 31 January 1864, when the last trace of the original Dukedom of Gordon was also extinguished. The Duke and Duchess of Gordon established the Gordon Chapel in Fochabers that contains a memorial tablet to the 5th and last Duke. The Duke had three illegitimate children: Charles Gordon, Susan Sordet, and Georgiana McCrae.