Holder was born in Happy Valley, South Australia, the son of James Morecott Holder and his wife, Martha Breakspear Roby. He was educated at Pulteney Grammar School and St Peter's College, Adelaide before first becoming a teacher, schoolmaster, and Methodist preacher, and later the editor and proprietor of the BurraRecord; he also wrote for the Adelaide Register. Holder married Julia Maria Stephens in 1877. His wife proved to be a great boon to his career, providing political advice and serving as South Australian President of the influential Women's Christian Temperance Union. Speculating that it contributed to his poor health, Holder had failed to seek suitable medical attention following an accident involving a mule in 1899.
With considerable experience as a Councillor and Town Clerk, and just five months after his election as mayor of the Corporate Town of Burra, Holder was elected to the South Australian House of Assembly in 1887 as the member for Burra, and soon gained a sound reputation in parliament. As a result, he served as Treasurer of South Australia from 1889–90 in the J. A. Cockburn ministry, and Leader of the Opposition from 1890–92. He again served as Opposition Leader in 1899. He sat on many royal commissions during his parliamentary career in South Australia, and his reasonableness and sincerity made him a very valuable committee man. In June 1891 he carried a vote of want of confidence in the Playford ministry, and took office as Premier and Treasurer. He had only a small majority and it was a time of great financial difficulties due to a severe drought and Holder was forced out as Premier after just four months. Holder then served as Commissioner of Public Works in Charles Kingston's government from 1893–94, followed by a third stint as Treasurer from 1894 until his re-election as Premier and Treasurer in late 1899. As Premier, his most notable innovation was to introduce one standard time zone throughout South Australia, while he also played a prominent role in the movement towards a federal union, and, as such, was a member of the convention that framed the Commonwealth constitution in 1897–98. . Holder took over the liberal leadership from Charles Kingston and was again Premier, this time from 1899 to 1901. He was succeeded in both roles by John Jenkins. The Liberal and Democratic Union would not be formed until the 1906 election.
Federal politics
As Premier, Holder considered himself to be the logical choice for a ministerial position in the new federal cabinet, and was offered a cabinet position by William Lyne after Lyne was invited by the Governor-General, Lord Hopetoun to form a government and become the inaugural Prime Minister. Holder initially accepted, and was in Melbourne en route to Sydney to officially accept his ministry when he was convinced by Alfred Deakin to refuse Lyne and instead support Edmund Barton's claim to the premiership. Believing that Barton would invite him to join the ministry, Holder was embarrassed and angry when Barton instead chose Kingston. Nonetheless, Holder resigned as Premier to successfully contest the 1901 federal election for the Free Trade Party and entered the new federal parliament in the single statewide Division of South Australia. Elected Speaker of the House of Representatives, Holder was re-elected to parliament in the 1903 and 1906 elections in the Division of Wakefield, contesting as an independent candidate.
Death
Knighted in 1902, Holder served as Speaker until his death on 23 July 1909. A 14-hour parliamentary session had started the previous afternoon. At 5 am the House was in committee, but Holder was present, having been called to the chamber to receive the committee's report, and was seated on the front bench, next to the Minister for Home Affairs, George Fuller. During a rowdy exchange, he exclaimed "Dreadful, dreadful!", then slumped sideways in his seat. He was taken to his room, where a cerebral hemorrhage was diagnosed by three members with medical qualifications and a doctor from outside the house. He died at 4:18 pm that same day without having gained consciousness. He was given a state funeral in Adelaide.
Legacy
The Canberra suburb of Holder was named in his honour when gazetted in 1970.
Family
On 29 March 1877, he married Julia Maria Stephens. She was president of the Women's Christian Temperance Union in South Australia, and a vice-president of the National Council of Women.