Born in Streatham in south London, Brown grew up as a fan of Barnet as his father was a keen fan of the club, regularly attending games at Underhill Stadium via the London Underground. At the age of 18, Brown joined his boyhood club as his father had local connections to the club and he was offered a contract by manager George Wheeler. On his arrival, club captain Alf D'Arcy described Brown as "A very gifted centre forward, lean and so quick but unusually with a sharp brain to go with it! He would take a row with a centre half in his stride and yet out-think his opponent, and he was incredibly fit too". In his first season at the club, he scored 41 times in all competitions to help the club win the Athenian League in the 1958–59 season and reach the final of the FA Amateur Cup, losing 3–2 to Crook Town with Brown scoring both goals for the Bees. However, his appearances were limited during the following season due to injury and Brown finished the season with 17 goals. During his time at Barnet, Brown also worked as a fishmonger in Billingsgate Fish Market.
Football League
Brown remained with Barnet until September 1960, when he joined Fulham, remaining an amateur, working part-time at a cricket bat manufacturer owned by Stuart Surridge. He then moved to Watford, becoming a full-time professional for the first time in his career, and made 28 League appearances for them before transferring to Northampton Town. Brown remained with Northampton for three seasons.
Cardiff City
He moved to Cardiff City in October 1966 for a fee of £15,000, winning the Welsh Cup in his first season at Ninian Park, scoring in the second leg of the final during a 2–1 victory over Wrexham. As a result of winning the Welsh Cup, Cardiff qualified for the European Cup Winners' Cup the following season and Brown made his European debut on 4 October 1967, scoring a penalty during a 2–0 win over Irish side Shamrock Rovers in the second leg of the first round. He featured in both legs of the second round against Dutch side NAC Breda, scoring an early goal in the second leg as Cardiff advanced after a 4–1 win. In the remainder of the campaign, Cardiff would go on to reach the semi-final, the furthest a Welsh club has ever progressed in European competition but, on 26 December 1967, Brown injured his knee during a 3–0 win over Aston Villa. It was later discovered that he had damaged his cruciate ligament and spent six months in plaster, stating that "It was obvious I would never play professionally again". He returned to pre-season at Cardiff and also received an offer from Yeovil Town but later took the decision to retire after being told that he could be confined to a wheelchair if he re-injured the knee.
International career
England amateur
Brown was handed his debut for the England national amateur football team on 20 May 1959, alongside Barnet teammates Roy Sleap and Alf D'Arcy, scoring twice during a 3–1 victory over the Netherlands at Zuiderpark Stadion. He played in two further matches in the following week, a 3–1 defeat to Luxembourg, during which he scored again, and a 2–0 defeat to West Germany. Over the following two seasons, Brown was an integral part of the amateur side, rarely missing a match and scoring consistently for the side, including braces in matches against Scotland and France in 1961. He went on to appear in 14 matches for the side, scoring 12 times. Following his decision to turn professional with Watford at the start of the 1961–62 season, he became ineligible to represent the amateur side.
1960 Olympics
Having represented England at amateur level, his performances for Barnet attracted the attention of the manager of the Great Britain Olympic football team, Norman Creek, who had been alerted to Brown following his brace in the 1959 FA Amateur Cup final. While still an amateur, Brown was chosen in the Great Britain squad for the 1960 Summer Olympics, along with his Barnet teammate Roy Sleap. Brown scored in all three of teams group matches, including a brace in their opening game during a 3–2 defeat to Brazil, as Britain finished the group stage in third-place resulting in their elimination. After seeing Brown score against his Italy side during the tournament, Giuseppe Viani offered Brown a contract at his club side A.C. Milan, however Brown kept the offer on hold as he was due to tour Australasia with an FA XI, which included England internationals such as Tom Finney and Bobby Moore. During the tour, he played in a number of the games against the New Zealand teams, and scored hat-tricks against both Otago and the Minor Provinces. When he returned to England, Brown discovered that Viani had suffered a heart attack and had subsequently stepped down as Milan manager.
Later life
Brown later worked as a youth coach at Cardiff City for a decade before working a shipping company.