He was the only son of Honorable George Anson Byron and Charlotte Henrietta Dallas, and grandson of the admiral and explorer The Hon. John Byron, who circumnavigated the world with George Anson in 1740–44. He married Elizabeth Mary Chandos Pole on 18 March 1816. She was the daughter of Sacheverell Pole Esq., of Radbourne Hall, b. 16 June 1769. During this man's lifetime, he became of representative of Sir John Chandos, K.G., and by sign manual, or deed poll assumed the additional surname of Chandos. Elizabeth was descended from a well documented long line of the Pole family, including Cardinal Pole, who at the time of Henry VIII, was the last Catholic Archbishop of Canterbury and the son of the last Yorkist heiress, Margaret, countess of Salisbury. Elizabeth's mother, Mary, was the daughter of the Rev. Henry Ware, D.D., Rector of Balrothey. The couple had seven children:
Byron joined the Royal Navy as a volunteer in December 1800, serving in the Napoleonic Wars, and attaining the rank of captain in 1814. In 1824 Byron was chosen to accompany homewards the bodies of Hawaiian monarchs Liholiho and Queen Kamāmalu, who had died of measles during a state visit to England. He sailed on in September 1824, accompanied by several naturalists and, amongst his lieutenants, Edward Belcher. He toured the islands and recorded his observations. With the consent of Christian missionaries to the islands, he also removed wooden carvings and other artifacts of the chiefs of ancient Hawaii from the temple ruins of Puuhonua O Hōnaunau. On his return journey in 1825, Lord Byron discovered and charted Malden Island, which he named after his surveying officer, Mauke and Starbuck Island. Starbuck was named in honour of Capt. Valentine Starbuck, an American whaler who had sighted the island while carrying the Hawaiian royal couple to England in 1823–1824, but had probably been previously sighted by his cousin and fellow-whaler Capt. Obed Starbuck in 1823. Malden may have been the island sighted by another whaling captain William Clark in 1823, aboard the Winslow. Byron became a rear-admiral in 1849 and a vice-admiral in 1857. Byron retired as admiral in 1862.