The club was formed by White in 1963 as team for former students of Primet High School. They initially played in the Nelson and Colne League and were based on a pitch at Holt House. After promotion through the local leagues the team moved from the old pitch to a spot on the current Holt House site. It was originally used for cricket, but after the council granted planning permission the club developed a stadium on the site. They joined the Lancashire Combination in 1975 and saw a sustained period of success, with seven consecutive top-seven finishes, including being runners-up three years in a row between 1980 and 1982. The club were founder members of the North West Counties League in 1982 and were placed in the Third Division. They won the division at the first attempt, and were promoted to the Division Two. By the mid-1980s White had become a millionaire through his property and timber business interests, and he began to invest money into the club. Following 12 clubs leaving the North West Counties League to form the Northern Premier League's new First Division, Colne were elevated to the First Division for the start of the 1987–88 despite only finishing 8th in the Second Division in the previous season. In the following season the club won the First Division championship on goal difference from Rossendale United, only losing three games and conceding 14 goals all season. They also won the FA Vase, beating Emley 1–0 in the Wembley final. As North West Counties League champions, the club were promoted to the First Division of the Northern Premier League. With more investment in the playing squad, including the signing of former Liverpool player Alan Kennedy, 1988–89 saw the club win the First Division with only a single defeat, and subsequent promotion to the Premier Division. In the summer of 1989 the club turned full-time, with some players earning more than those at Football League clubs. They won the Premier Division by a margin of 26 points with crowds averaging over 1,300, and reached the semi-finals of the FA Trophy where they lost to Conference side Barrow. Despite winning the NPL, the club was refused promotion to the Conference as their Holt House ground did not meet the required grading. White attempted to arrange a groundshare with Burnley, reportedly offering them £500,000 to play at Turf Moor, but was turned down. Following the rejection, plans were unveiled for a new stadium on the edge of neighbouring Nelson. However, following a pre-season friendly against Newcastle Blue Star in the summer of 1990, White told the players that the club was to fold. Following the Colne fiasco he quit football amidst allegations of death threats and that the money had run out. North West Counties League football returned to the town when Colne F.C. were founded and elected straight into the league in 1996, with the new club taking over the Holt House ground.
