For some time he was captain in a company of Lord Loudoun's regiment of foot, afterwards the 54th. Murray sat as Member of Parliament for Perthshire from 1761 to 1764. On 8 January 1764, his uncle and father-in-law, the 2nd Duke of Atholl, died. Murray should have been heir to the dukedom, which was only able to descend through the male line; but he was ineligible since his father had fought in the Jacobite Rising of 1745 and had consequently been attainted in the blood. However, on 7 February 1764, the House of Lords deemed Murray the rightful heir to his uncle's title and he succeeded him as 3rd Duke of Atholl. He was elected a Scottish Representative Peer in 1766. His wife, on the death of her father, the second duke, succeeded to the sovereignty of the Isle of Man, and to the ancient English barony of Strange, of Knockyn, Wotton, Mohun, Burnel, Basset, and Lacy. For some time negotiations had been in progress with the English government for the union of the sovereignty with the English crown; and in 1765 an act of parliament was passed to give effect to a contract between the lords of the treasury and the Duke and Duchess of Atholl for the purchase of the sovereignty of Man and its dependencies for £70,000, the duke and duchess retaining their manorial rights, the patronage of the bishopric and other ecclesiastical benefices, the fisheries, minerals, &c. The arrangement rendered them very unpopular in Man, and the 42nd Regiment of Foot, or Black Watch, under Lord John Murray, had to be stationed in the island to maintain order. The money received by the duke and duchess was directed to be laid out and invested in the purchase of lands of inheritance in Scotland, to be inalienably entailed on a certain series of heirs. The duke and duchess had also a grant of an annuity of £2,000 for their lives. Atholl was chosen a representative peer in succession to the Earl of Sutherland, who died 21 August 1764, and he was reelected in 1768. In 1767, he was invested with the Order of the Thistle. He was Grand Master of the Ancient Grand Lodge of England from 1771 until 1774, and Grand Master of the Grand Lodge of Scotland from 1773 to 1774. He died at Dunkeld on 5 November 1774.
Atholl died in November 1774, aged 45, after drowning himself in the River Tay in a fit of delirium and was buried at Dunkeld. His eldest son John succeeded him in the dukedom.