McCarthy first came to fame for his 10-year stint at Mississippi State, where his teams won 169 games, lost 85, and won four Southeastern Conference titles. While coaching at MSU he was named SEC Coach of the year 3 times. When he left Mississippi State he was the school's all-time leader in wins but has since been passed by Richard Williams and Rick Stansbury. McCarthy may best be remembered for his team crossing the color line in the segregated south of the 1960s. Even before it was certain that Mississippi State would face Loyola and their four black starters, racist elements in the Mississippi media got into the act. On Thursday, March 7, 1963 the Jackson Daily News printed a picture of Loyola's starters to show that four of them were African Americans. As a caption to the picture, Daily News editor Jimmy Ward wrote that "readers may desire to clip the photo of the Loyola team and mail it today to the board of trustees of the institution of higher learning" to prevent the game from taking place. At the time, a longstanding state policy barred college teams at state schools from playing games against racially integrated teams. The Bulldogs had been forced to turn down three previous NCAA Tournament bids for this reason, including when they won their first two outright SEC titles in school history. The editorials were in response to the decision by Mississippi State President Dean W. Colvard's March 2, 1963 to accept the automatic bid to the NCAA Tournament as outright SEC champions. The College Board of Mississippi met on March 9, 1963 and upheld Colvard's decision. But on March 13, just a day before the team was scheduled to travel to East Lansing, state senator Billy Mitts and former state senator B.W. Lawson sought and obtained a temporary injunction against the team leaving the state. While sheriffs were on their way to Starkville, Mississippi to serve the injunction, the team was participating in a pep rallythe night before their departure, where effigies of state senators Mitts and Lawson were hung. The team's original plan was to leave Starkville at 8:30 a.m. on Thursday morning. But learning that sheriffs would be expected to arrive in town at 11:30 p.m. Wednesday night, MSU put their sophisticated contingency plan into effect. McCarthy, the athletic director, and the assistant athletic director drove to Memphis, and then flew to Nashville. The team itself sent the freshman squad to the airport as scheduled-posing as the varsity team. The real varsity team hid in a dorm on campus. The next morning, they boarded a private plane at the airport and flew to Nashville to meet the coach and team officials. From Nashville, the whole group took a commercial flight to the game at East Lansing, Michigan. These events were chronicled in the DVD produced by Starkville-based McCarthy Gymnasium on the campus of MSU was named for him in 1975. McCarthy later coached the George Washington University's men's basketball team, going 9–18 with the Colonials in 1966–1967.
McCarthy was known as "Ol' Magnolia Mouth" for his cement-thick Mississippi accent and short, funny phrases known as were short funny phrases that earned McCarthy his nickname of. A few of the more famous Babe-isms were: "Boy, I gotta tell you, you gotta come out at 'em like a bitin' sow," "My old pappy used to tell me the sun don't shine on the same dog's butt every day," "Why panic at five in the mornin' because it's still dark out?" and "Now, let's cloud up and rain all over 'em."
