Frazier began his administrative career as a graduate assistant at Arizona State University in 1977. He would later become a full-time administrator and remained with the University for a total of 23 years, eventually becoming the Senior Associate Athletics Director. He would earn his first athletics director job at the University of Alabama at Birmingham in 2000. In 2002, he left UAB to accept the athletic director position at the University of Hawaii where he would remain until 2008. He has served on the U.S. Olympic Committee in numerous capacities. In 1996, he was elected as one of three vice-presidents for the Committee and accompanied the team to the 2000 games in Sydney, Australia. Most recently, he served as the chef de mission for the 2004 games in Athens, Greece. He has also served on the U.S. Olympic Athletes Advisory Council and the U.S. Olympic Overview Commission. As a member of the Fiesta BowlBoard of Directors, he was vice-president in 1996 and chairman in 1998 and 1999.
Honors
Frazier has received numerous honors, including the NCAA Silver Anniversary Award, which recognizes "distinguished former student-athletes on their 25th anniversary as college graduates". Frazier was one of 461 athletes to receive a Congressional Gold Medal due to the US boycott of the 1980 Olympics. In 2003 he was cited as one of the 101 Most Influential Minorities in Sports by Sports Illustrated. He was named as one of The 50 Most Powerful African Americans in Sports in the March 2005 issue of Black Enterprise magazine. In 2002, the Herman R. Frazier Political Science Scholarship was established by his friends and colleagues at Arizona State University to "celebrate and honor Mr. Frazier’s lifetime achievements" and award a "deserving political science student."
Criticism
Frazier was criticized by some for leaving UAB with a $7.5 million deficit. His supporters noted that he had inherited a deficit and did not receive institutional support, while his detractors claimed that he was ineffective in increasing revenue. On January 8, 2008, Herman Frazier was fired from his position as athletic director at the University of Hawaii following heavy criticism from the fans and media of the State of Hawaii. On March 3, 2007, he was loudly booed by the crowd following the final home game of the University of Hawaii's head basketball coachRiley Wallace. Frazier is widely viewed as having forced Wallace to resign by including a "no-extension" clause in his final contract. However, Wallace has accepted responsibility by saying, "The bottom line is Riley Wallace signed the contract." Frazier was also criticized by both the media and public for ongoing delays and a perceived lack of integrity in finalizing the full 2007 Hawaii Warriors football team schedule. Frazier admitted that he may have miscalculated in regards to the schedule but stands by his record at Hawaii citing, in part, his balancing of the athletics budget. He inherited a $2.5 million deficit from his predecessor. Finally, Frazier also received the brunt of the blame for then UH football coach, June Jones, leaving the University for Southern Methodist University. On April 6, 2009, Jim Bolla—who Frazier hired in 2004—was fired as the women's basketball coach for Hawaii after former players complained of a "pattern of verbal abuse" from Bolla. Former UH player Pamela Tambini told the Honolulu Advertiser that players complained for years about their treatment by Bolla to Frazier but Frazier apparently took no actions. The paper quoted Tambini stating: "Everybody knew about it and did nothing about it... We were going through hell. No student should have had to go through that. It's not fair."