Forced to stop his playing career at the age of 20 due to injuries, he turned to coaching and had his first assignment as coach of Terengganu FA under-20 youth team in 1972. In his four years with the team, the team won Burnley Cup in 1974. In 1976, Abdul Rahman was appointed the assistant coach for Malaysia national football team. His first stint with the national team to 1982 also saw his overseeing the Malaysia 'B' team that competes in Thailand King's Cup tournament.
Terengganu
In 1983, he returned to Terengganu to become the new head coach. He inherited a team that was a finalist in the previous years' Malaysia Cup.
Second stint with the national team
He returned to Malaysia national team setup in 1984, again as assistant coach. In 1987, he had his first spell as head coach of Malaysia as he led the team to silver medal of 1987 Southeast Asian Gamesfootball competition. He then returned to Terengganu in 1988 to begin an 8-year spell as general manager of Terengganu FA football team.
Perlis
appointed Abdul Rahman in 1996 as head coach, marking his return to the touchline. He guided Perlis to the quarter-final of 1996 Malaysia Cup, then goes one better in 1997 as Perlis reaches semi-final of Malaysia Cup that year.
Terengganu again
He returned to Terengganu in 1998 to become head coach for the second time. He led them to 1998 Malaysia Cup finals, their first appearance in the finals since 1982, where they were runners-up again.
Third stint with the national team
Abdul Rahman was appointed in October 1998 to succeed Hatem Souissi as the head coach of Malaysia, after the dismal performance in the 1998 Tiger Cup. With Malaysia, he guided them to third place in 2000 Tiger Cup, although Malaysia also were eliminated at the group stage of 1999 Southeast Asian Games football competition and failed to qualify for the 2000 AFC Asian Cup in Lebanon. He leaves his post after the 2000 Tiger Cup, succeeded by Allan Harris.
Selangor, Kelantan TNB and second stint with Perlis
He were signed by Selangor FA in 2001 as head coach, guiding them to Malaysia FA Cup title in 2001. After his contract were ended in June 2002, he were back in Terengganu as director of football in 2003. Abdul Rahamn returned to coaching in 2004, taking the helm of club side Kelantan TNB FC. He returned to Perlis in 2005. He was successful during his second spell as Perlis coach, winning 2005 Super League Malaysia, 2006 Malaysia Cup, 2007 Malaysia Charity Shield, as well as 2005 Malaysia Cup runners-up, and two-times FA Cup runners-up in 2006 and 2007.
He returned to Malaysia and signed for PKNS FC in 2009. He guided the team to the 2011 Malaysia Premier League championship, and promotion to the 2012 Super League Malaysia. Abdul Rahman successfully maintained the club in the Super League for two seasons, and also guided the club to the Malaysia Cup quarter-finals for two seasons in a row. At the end of 2013 season, Abdul Rahman accepted Terengganu's offer to become their head coach once more, starting from 2014 Malaysia Super League season. He relinquished his post on May 2015, after a poor run in the league and issues with the team management. He has yet to return to football coaching after leaving Terengganu.
