Born in Wellington, New Zealand, and one of six children, Scott had a difficult childhood. His father had served with the New Zealand Expeditionary Force in Gallipoli during the First World War where he was chronically wounded. Although his father was employed money was scarce and Scott frequently went hungry. His father was employed by the Public Works Department, and the family moved to Kapuni, then Tangarakau and later Ohura. When Scott was nine in 1930 his parents split up and he moved into a Salvation Army children's home, Cecilia Whatman Home in Masterton. Although after two years the parents reconciled, it was short-lived, and they separated permanently within a year. Scott stayed with his father while his five siblings were sent again to a children's home. Scott worked part-time and went to school until his father died of cancer in 1934. After his father's death he moved back with his mother, who had established a home with his siblings.
After working in a warehouse from the age of 13 he enlisted in the New Zealand Army when the Second World War started. He was posted to the Motor Transport Pool in Auckland. They had a team which entered Auckland's senior rugby union competition, and in 1941 won the Gallaher Shield. In 1942 he was posted to Egypt with the Army Service Corps in the New Zealand Division as a truck driver. He served in North Africa and Italy, and described driving trucks of ammunition to the front lines as the most lonely experience of his life. Company rugby teams within the New Zealand Division competed in a tournament called the Freyberg Cup – named after Lieutenant-General Bernard Freyberg who commanded the division. Scott was in the Ammunition Company team that made the Freyberg Cup final in 1944. Although they lost the final to the 22nd Division, Scott, who played at fullback was selected to trial for the Army's Kiwis team. During World War II, he served in Italy and was one of the players in the New Zealand Army rugby team along with Bob Stuart as fullback.
All Blacks
As an All Black he made his debut in 1946 against Australia. He toured South Africa in 1949 where he played in all four test matches. He also played against the 1950 British and Irish Lions. He did not play in the 1951 tour of Australia and announced his retirement from first-class rugby that year. He was bribed to play one game for the Auckland union in 1952. He then played for the Petone club in Wellington during 1954. In a publication he was described as: "for me there will never be anyone as great as Scott. The man amazed me even in his final moments. His positional play was of course something out of the box. He was a genius and what other fullbacks had to do, Scott didn't, because he had the greatest balance, the greatest poise, that I've ever seen in any man." In his autobiography, Hennie Muller stated: "Scott always appeared to have plenty of time, even under pressure. He loved coming into the line and his speed and elusiveness were such that he was always a danger. Altogether, the greatest player I've ever played in any position." Scott was part of the Petone club committee between 1966 and 1970.