Born in Espinho, Pereira only played amateur football, and retired at the age of 28. He started managing on junior level, and his first head coach experience arrived midway through the 2004–05 season, when he was appointed at A.D. Sanjoanense in the third division. Subsequently, Pereira worked with S.C. Espinho in the same level, being fired with ten games left in his second campaign and returning to FC Porto's juniors for a further season. In 2008 he returned to head coaching again, with C.D. Santa Clara in division two, leading the Azores side to the third position in his first year and the fourth in the second, on both occasions narrowly missing out on Primeira Liga promotion.
Porto
In the summer of 2010, Pereira left Santa Clara to become assistant manager to André Villas-Boas at Porto. On 21 June 2011, following the head coach's departure to Chelsea, he was promoted to first-team manager, winning his first official match – and title – against Vitória de Guimarães, for the season's Portuguese Supercup. Despite a less than stellar performance both in Europe – with Porto being knocked out of the UEFA Champions League in the group stage and in the UEFA Europa Leagueround-of-32 – and in the Portuguese Cup, Pereira led the club to the league title in his first season in charge. In March 2013, following the team's elimination from Champions League contention, he came under heavy criticism, notably due to his decision of benching James Rodríguez during the first half on both games. The Colombian stated his disappointment in the coach's decision, but said that he respected him and trusted his reasons, adding that the situation was possibly created because of his questionable fitness, even though he claimed to be 100% fit. In early May, as Porto ranked second in the league, Pereira deemed the Portuguese league as a "dirty competition". Only a few days later, after his team defeated S.L. Benfica at home to surpass its opponents – eventually winning the league title, conceding six draws in 30 games– he considered it to be a "highly competitive and prestigious league".
Abroad
In late May 2013, Pereira was interviewed for the vacant job at Premier League club Everton, but eventually signed a two-year deal with Al-Ahli SC of the Saudi Professional League. On 7 January 2015 he moved clubs and countries again, replacing fired Míchel at the helm of Olympiacos F.C. and eventually winning the double. On 10 June 2015, Olympiacos announced a mutual contract termination with Pereira. The following day, he was appointed at Fenerbahçe SK for two years. The Turkish club cut ties unilaterally with Pereira on 15 August 2016, with the case being subsequently taken to the Court of Arbitration for Sport. On 18 December he was named head coach of TSV 1860 Munich in the German 2. Bundesliga, with the one-and-a-half-year deal being made effective the following 1 January. The team finished the season third from bottom, and eventually got relegated on 30 May 2017 after a 1–3 loss on aggregate to SSV Jahn Regensburg in the play-offs. Pereira became the manager of Shanghai SIPG F.C. on 12 December 2017, replacing Villas-Boas at the Chinese Super League club. In his first season in charge, he guided them to their first-ever title in the competition. In December 2019, Pereira turned down a new approach from Everton.