White rose through the youth ranks at hometown club, Colchester United, having joined the U's at the age of eleven. He was eventually handed his first team debut by Phil Parkinson in a League Cup tie at home to Cheltenham in August 2004, before retaining his place for his Football League debut four days later in a 4–1 win over Doncaster. He went on to make 24 appearances for the U's during the remainder of the 2004–05 season and displayed his versatility by covering a number of positions, proving to be particularly effective in a man-marking role that had been assigned to him by Parkinson. White then played an important role the following season as the U's won promotion to the Championship. Having established himself as a first-team regular, he made 41 appearances in all competitions as Parkinson's side finished second in League One, as well as enjoying a run to the fifth round of the FA Cup where they were beaten 3–1 by Chelsea at Stamford Bridge, a game in which White played the full 90 minutes. In June 2007, White was offered an improved contract to remain at Layer Road, but after making only a handful of appearances in the opening months of the season, he moved to Stevenage in a month-long loan deal. The spell revitalised White and, with George Elokobifalling out of favour at the U's after handing in a transfer request, White forced his way back into the side with his fitness improved. In August 2008, with the club now back in League One following relegation from the Championship, White signed another new contract to extend his stay at the club. Under the reins of new manager Paul Lambert in summer 2009, White was told that he could leave the club. White was part of a group of players who were being frozen out under Lambert, and he moved on a one-month loan deal to Essex rivals Southend for the start of the 2009–10 campaign along with Matt Heath. White started every game of his spell at Roots Hall and admitted to having "no idea" where his future would lie, but during his spell with Southend, Lambert left the U's to join Norwich City, allowing White to force his way back into the Colchester team under new manager Aidy Boothroyd, going on to make 39 league appearances that season. He expressed his gratitude to Boothroyd for resurrecting his career, explaining how Lambert had left him to be forgotten: "I felt I never got the chance to impress him. It wasn't that I was given the chance and messed it up, he just decided he didn't fancy me and that was that, I wasn't involved. The new manager has been like a breath of fresh air. It's nice for the forgotten players that were left disappointed by the old manager to feel wanted around the place again." With Boothroyd departing for Coventry City, John Ward took over as U's manager and White once again found himself back on the fringes of the first-team after Brian Wilson had been signed at full-back. Ward insisted that White remained part of his plans, and an injury to Wilson in December 2010 ruled him out of the remainder of the campaign, allowing White an extended run in the first-team. White signed a further two-year contract extension in May 2011 and went on to feature in the unfamiliar left-sided role ahead of both Michael Rose and Ben Coker during the 2011–12 campaign. During the season, White also enjoyed his 200th league game for Colchester aged 25, admitting that it was "special that Colchester is my home town club, and that I've been able to stay here this long." The 2012–13 season saw Colchester narrowly avoid relegation, with White playing 22 league games for the club under new manager Joe Dunne. His contract was due to expire at the end of the season as Dunne expressed an interest at keeping the defender at the club. White declined the offer of the contract in June 2013, bringing to an end a nine-year stay with the U's. He had been offered a reduced salary to stay on with the club, opting instead to look for another club. He made over 250 appearances during his time with Colchester.
Southend United
On 27 June 2013, White signed a two-year deal with Southend United, the club where he had spent time on loan in 2009, becoming manager Phil Brown's first summer signing. On 6 June 2017 White extended his stay with Southend signing a one-year deal. On 9 May 2018 White signed another new one-year deal following 37 appearances in the 2017/18 season. In July 2019 he signed a new contract with the club. John White played his 500th league appearance on 1/2/20 as part of the Southend team in their 2–1 victory over Lincoln. He was substituted in the second half due to injury.
