He was born in Oystermouth, Glamorganshire, Wales in 1864. His family migrated to New Zealand when he was eight years old and grew up in the Auckland suburb of Newton. Upon completing his education, Rosser became a builder by trade. Whilst working as a carpenter he married Sarah Louisa Craig on 30 November 1886.
After he was blacklisted by conservative building contractors due to his links with the Liberal Party, Rosser took up a new career as a union organiser, the first in Auckland. Within twelve years he was involved in the formation of nine new trade unions and was himself the secretary of many of them, demonstrating a skill for arbitration. Over time arbitration was overtaken by collective bargaining as most new unionists favored method. As a result, Rosser's more moderate views were at increasing odds with the more militant forces in the labour movement. This came to a head in 1910, when he was replaced as the president of the Auckland Trades and Labour Council by the more radical Michael Joseph Savage.
Political career
Through his role as a union secretary Rosser openly participated in politics. In the early 1900s he played a key role in establishing the Independent Political Labour League in Auckland, the first New Zealand Labour Party and the United Labour Party. Throughout his career, Rosser was an advocate of the moderate wing of the labour movement, arguing that organised labour should improve rather than replace capitalism. In the, he stood in the three-member electorate and came fourth. At the next election in, he came seventh. He was one of three candidates in the in the electorate and stood for the Independent Political Labour League when he came last. Rosser was elected to the Auckland City Council in 1901 but lost his seat in 1903 and later failed to secure re-election in 1907, 1913 and 1931. He regained a seat in 1933, though in 1938 he lost his seat on the council, after standing as an independent. He was unexpectedly denied re-nomination by the Labour Party alongside sitting councillors Ted Phelan and George Gordon Grant. The next year he was expelled from a Labour Party as he had stood against Labour's official municipal candidates.
