Born in Cramlington, Northumberland, Brown joined Hartlepool's youth set-up from Cramlington Juniors football Club. He had joined too late to go through the Youth Scholarship system and was subsequently offered a temporary deal. He was part of the successful Hartlepool under 19s side that won the Dallas Cup XXV in 2004. James Brown was included on the substitutes bench for Hartlepool's match against Bournemouthon the last day of the 2004–05 season. He was expected to play a part in the Hartlepool's 2005–06 campaign and participated regularly during pre-season. However, during pre-season, Brown ruptured his cruciate ligament against Hibernian and missed the majority of the subsequent season. On 6 May 2006 Brown made his comeback and debut for the club, coming off the bench to score in the 1–1 draw with Port Vale at Victoria Park. The game was bittersweet for Brown as Hartlepool's failure to win saw them relegated to League 2. James's finishing earned him his first professional contract with the club in 2006. During the 2006–07 campaign, he played nearly all of Hartlepool's games on the right wing. In August 2008, Brown signed a new long-term contract with Hartlepool. Brown scored against Colchester on 17 April 2010 which saw Hartlepool United winning 3–1, this once again led to Jeff Stelling revealing his James Brown singing doll. Brown was released by Hartlepool on 10 May 2012.
Brown signed for Conference Premier side Gateshead on 6 June 2012. His debut was delayed due to requiring surgery on a double hernia. He eventually made his Gateshead debut on 6 October 2012 as a second-half substitute in a 1–1 draw against Hyde. He made his first start for Gateshead three days later in a 2–2 draw with Macclesfield Town. Brown scored his first goal for Gateshead on 22 February 2013 in the return game against Macclesfield Town. He made his 50th appearance for Gateshead on 11 January 2014, scoring in a 2–1 win over Nuneaton Town. He retired from professional football in November 2014 due to a persistent knee injury. Following this, he joined Northern Premier League Premier Division side Whitby Town.
of Sky Sports, a Hartlepool fan, has been known to exclaim "I feel good!" when Brown scores for Hartlepool, a reference to the more famous James Brown's "I Got You ". Brown's goals on the opening day of the 2008/09 season against Colchester United led to Jeff Stelling revealing a James Brown singing and dancing doll toy with bobbing head singing "I feel good". In Stelling's book "Jelleyman's Thrown a Wobbly", Stelling reveals that a fan of the show had waited for 3 hours outside hospital which he was visiting for charity to present with the doll to be used whenever James Brown scored.
