Brown was born in 1935 in Lewes, Sussex, where he attended the localsecondary modern school. He played football for Lewes St Mary's and represented East Sussex at schoolboy level; it was while playing schools football that he was spotted by Brighton & Hove Albion manager Don Welsh. After a trial, he was taken onto the club's ground staff in 1951 and signed professional forms on his 17th birthday in September 1952. National Service obligations and inability to dislodge captain Ken Whitfield from the centre-half position meant Brown did not make his senior debut until 9 November 1957, in a Third Division South match against Northampton Town. Albion won 4–2, and "the critics accorded him a fair share of the credit for ... their first victory at Northampton for 11 years", but he returned to the reserves for the next match. Whitfield was reported to have asked for a transfer the following month, but this came to nothing, and Brown appeared just twice more for Albion's first team before moving on after seven years with the club. On 17 September 1958, Don Welsh signed Brown for a second time, for Third Division club Bournemouth & Boscombe Athletic. The following day, Brighton signed his brother Alan, also a centre half. Brown impressed immediately for Bournemouth: on his debut, the People described him as "a shining light with the coolness of a veteran ... not only kept Higham in check but maintained complete command of the middle". He went on to play 65 Third Division matches for Bournemouth before joining Poole Town of the Southern League in 1962. He remained at Poole for several years, and was for a long time associated with nearby Hamworthy United, as player, manager and physiotherapist. After his death, the Hamworthy club set up a trust fund to raise money to build a stand in his memory, and since then have held an annual Irvin Brown Day to raise funds for the club. Brown was one of six brothers who all played football; two younger brothers, Alan and Stan, played at Football League level. He married Sheila Walter in 1956. By the time he signed for Bournemouth, the couple had a daughter. After giving up full-time football, he remained in the Poole, Dorset, area where he worked as a carpenter and where he died in 2005 at the age of 70.
