After graduating from Union College, Carmody served as head coach of Fulton-Montgomery Community College in Johnstown, New York, and led the team to a 17–10 record and conference title in his only season there. He returned to Union the following year as an assistant coach under head coach Bill Scanlon. In 1980, Carmody became a part-time assistant at Providence College, where he worked for 2 seasons under head coach Gary Walters.
From 1982 through 1996, he was an assistant basketball coach at Princeton University under the Tigers' legendary coach, Pete Carril. After fourteen years, he became the head coach in 1996 when Carril retired. Despite not being able to offer athletic scholarships due to Ivy League rules, Carmody's 1997–1998 team reached a ranking as high as 7th nationally, and was ranked 8th nationally going into the NCAA Tournament. This led to a number-five seed in the NCAA Tournament. That team lost in the second round of the tournament to #4 seed Michigan State, and was ranked 16th nationally at the conclusion of the tournament. He is considered one of the leading practitioners of the Princeton offense. While coaching Princeton, he established the Ivy League career winning percentage record of 78.6%, going 92–25.
In 2000, he succeeded Kevin O'Neill as the head coach of the Northwestern Wildcats Men's Basketball Team. One of his top assistants from 2000 to 2006 was Craig Robinson, the brother of former First Lady Michelle Obama. From 2008–2014, Robinson was the head coach at Oregon State University. In 2003–04, Carmody led the Wildcats to an 8–8 record in Big Ten play, their first non-losing record in conference play since 1967–68. On January 18, Northwestern defeated the then-number-seventeen Minnesota Golden Gophers. On January 21, 2009, Carmody's Kevin Coble-led Wildcats defeated number-seven Michigan State University at the Breslin Center in East Lansing, Michigan earning their second consecutive win over an opponent ranked in the AP top 25, marking the first time in school history for such a feat. The 2008–09 unit became the first in school history to win 20 games and briefly flirted with the first NCAA Tournament appearance in school history. On December 28, 2009, Northwestern was ranked number 25 in the Associated Press Basketball Poll, marking the first time Northwestern had been ranked in the AP Poll since 1969. The 2009–10 team also notched the school's second-ever 20-win season. Despite Carmody's efforts to upgrade the Wildcat program, his teams never finished higher than fifth in the Big Ten, and his 2003–04 team was the only one that finished with even a.500 record in conference play. After the Wildcats suffered their first losing season in six years, Carmody was fired on March 16, 2013. He left as the second-winningest coach in school history, behind only Dutch Lonborg.
Holy Cross
After spending the 2014–15 season as a special assistant and advisor to Fairfield coach Sydney Johnson, Carmody was hired as the head coach of the Holy Cross Crusaders in March 2015. In Carmody's first year with the Crusaders, his team won the Patriot League Tournament Championship. After going 0–9 on the road in league play, he completed a magical conference tournament run of 4–0 on the road to claim the crown. Carmody would coach three more seasons at Holy Cross before ultimately retiring after the 2018-2019 season.
