Harper made his Scottish Football League debut for Hibernian against Airdrie on 1 September 1920. He spent five years at Easter Road, making over 200 appearances. He also played in two consecutive Scottish Cup Finals, in 1923 and 1924, but finished on the losing side on both occasions. While with Hibs he also won nine of his 11 Scotland caps and represented the Scottish League XI on two occasions.
Arsenal
In November 1925 Harper became one of Arsenal manager Herbert Chapman's first signings, joining the club for £4,000, then a record fee for a goalkeeper. He made his Arsenal debut against Bury on 14 November 1925, which Arsenal won 6–1. Harper became Arsenal's first-choice keeper straight away, displacing Jock Robson and young Welshman Dan Lewis. Harper finished 1925–26 and started 1926–27 as Arsenal's first choice keeper, but Chapman blamed Harper for a 4–2 defeat in a North London derby against Tottenham Hotspur on 18 December 1926 and ruthlessly dropped him in favour of Dan Lewis. After three years playing in the American Soccer League, Harper rejoined Arsenal. Dan Lewis had by now left the club, and Harper became first choice over Charlie Preedy and Gerard Keizer. In his first season back at Arsenal, Harper won an English First Division winners' medal, as the club won their first ever league title in 1931. However, during the 1931–32 season Harper was dropped once again. He made his final senior league appearance for Arsenal away to Blackburn Rovers on 31 August 1931. During his two spells with Arsenal, Harper made 63 league appearances and 10 further appearances in the FA Cup.
American Soccer League
In between his two spells with Arsenal, Harper played in the American Soccer League. Between 1927 and 1929 he made 79 league appearances for Fall River Marksmen. He also played a further 3 games in the National Challenge Cup and 6 in the Lewis Cup and helped the Marksmen win the league title during the 1928–29 season. Harper’s time in the ASL coincided with both the Great Depression and the Soccer Wars, a power struggle between the ASL and the US Football Association. During this era many clubs relocated, merged or folded. Towards the end of the 1928–29 season Sam Mark, owner of the Marksmen, took over the struggling Boston Soccer Club and Harper then switched from the Marksmen to the Boston team. By the time the Fall 1929 season started they had become the Boston Bears. When the Soccer Wars were resolved, new league rules forbid the ownership of two or more clubs and Mark’s decided to sell Boston. However unable to find a buyer, he was forced to dissolve the team after just 4 games of the Spring 1930 season. Harper then joined New Bedford Whalers for the remainder of the season, making 31 league appearances. He also made a further 4 cup appearances, two in the National Challenge Cup and two in the Lewis Cup. During the summer of 1930 Harper rejoined Fall River Marksmen and played in friendlies against Rangers and Kilmarnock and was a member of the Marksmen squad that toured Central Europe.
Plymouth Argyle
In December 1931 Harper signed for Plymouth Argyle and went on to make 82 appearances in all competitions between then and 1939. During the Second World War he worked in the dockyards in Rosyth, Scotland. After the war he returned to Argyle where he served as a trainer, groundsman and even laundryman. Such was his contribution to the club that he was awarded a testimonial match against Arsenal in October 1972. Plymouth's training ground, Harper's Park, is also named after him.
Scottish international
Between 1923 and 1926 Harper made 11 appearances for Scotland. He won his first nine caps while playing for Hibernian and his last two while with Arsenal. Harper made his international debut on 3 March 1923 in a 1–0 win against Ireland at Windsor Park. He subsequently played in all three games of the 1923, 1924 and 1925 British Home Championships and played in two of the three games in the 1926 competition. He made four appearances against Ireland, four against England and three against Wales. His record with Scotland included eight wins, two draws and just one defeat. With Harper in goal, Scotland also won the British Home Championship in 1923, 1925 and 1926 and finished as runners up in 1924. He made his last international appearance on 17 April 1926 in 1–0 win against England at Old Trafford.