Kleff joined Borussia Mönchengladbach from amateur outfit VfL Schwerte, where he wasn't a regular starter, in 1968 to become the leading goalkeeper of the Bundesliga side until injuries forced him to lay-off periods in the late 1970s. From 1968 to 1976 he did not miss any domestic game of Borussia Mönchengladbach and was able to lift five Bundesliga one DFB-Pokal and one UEFA Cup trophy with Die Fohlen. He was also in the Borussia Mönchengladbach team when they lost to Liverpool F.C. in the UEFA Cup final in 1973 and the European Cup in 1977. He was also part of the Mönchengladbach team which won the 1979 UEFA Cup. After a season at Hertha BSC, he returned to Mönchengladbach, Kleff played two more years in the Bundesliga for his old club, joining local rival Fortuna Düsseldorf for two years in 1982. After a fall-out with Düsseldorf he joined Rot-Weiß Oberhausen of the 2. Bundesliga and was on the verge of becoming 40 years of age when he enjoyed one more year in the best German division with VfL Bochum. At the age of forty Kleff played the majority of games for promoted FSV Salmrohr in the 2. Bundesliga of 1986–87. The season ended in swift relegation for Salmrohr and in Kleff's decision to hang up his gloves. However, for some time he continued to be a goalkeeper for amateurs such as SV Straelen. At the end of his illustrious career, Kleff had taken part in 433 matches in the Bundesliga and 56 games in the 2. Bundesliga.
Kleff's life in goal of West Germany was limited due to his competitor Sepp Maier occupying the role of West Germany's leading goalkeeper in that decade, Kleff won just six caps for his country in between 1971 and 1974. His first of those six matches took place in Oslo on 22 June 1971, and was a 7–1 thrashing of Norway in a post-season friendly. Despite not making many appearances for his country, Kleff won the 1972 UEFA European Championship trophy in Brussels and the 1974 FIFA World Cup.
His resemblance to German actor and comedian Otto Waalkes earned him several minor roles in successful cine films such as Otto – Der Film in 1985 and Werner – Beinhart! in 1990.
