Gersh was born in New York City to poor Russian Jewish immigrant parents, Ida and Louis Gershowitz. His parents owned a deli on 25th and 9th Avenue. When Gersh was 8 or 9 years old, his family moved to Washington Heights and opened a restaurant there. As a child Gersh hoped to become a baseball player, and used to sell peanuts outside baseball stadiums. Gersh moved to Los Angeles in the early-1930s to study at UCLA, graduating in 1934. He began working on a swing gang for Paramount Pictures, where his brother-in-law Sam Jaffe was an executive. When Jaffe left Paramount in 1936 to set up his own talent agency, Gersh moved with him to work as an office boy earning $15 a week. His first client as an agent was director Mark Robson, with whom he had been in a fraternity at UCLA some years before, and who was looking to move into directing from a job as an editor at RKO Pictures. Gersh subsequently signed other directors from the studio including Robert Wise, Richard Fleischer and Joseph Losey, thereby establishing a reputation as an effective spotter of directing talent. During World War II, Gersh saw active service, joining an anti-tank unit of the 3rd Infantry Division in North Africa and Italy for four months, before being transferred to Special Services, where he met Beatrice Aberle, his future wife. In 1949, on his return to Hollywood, he bought Sam Jaffe's talent agency, which he renamed as the Phil Gersh Agency in the 1960s. His film-industry clients included Eddie Albert, June Allyson, Mary Astor, Richard Benjamin, Lloyd Bridges, Humphrey Bogart, Richard Burton, Lee J. Cobb, Harrison Ford, Gloria Grahame, Arthur Hiller, William Holden, Karl Malden, Fredric March, James Mason, Dorothy McGuire, Zero Mostel, David Niven, Don Siegel, Raoul Walsh and Robert Wise. He also represented writers including Julius J. Epstein, Ernest Lehman, Abraham Polonsky, and Budd Schulberg. Gersh helped his clients Mostel and Polonsky when they were blacklisted during the anti-Communist witch-hunts of the 1950s. Gersh was known for his brash and assertive approach on behalf of his clients, as well as his strong New York accent and signature gravelly voice. He was credited with masterminding Humphrey Bogart's move, during the 1950s, away from typecasting in gangster films to more diverse roles. He also fell out with close friend David Niven when Niven was persuaded to fire Gersh by an agent for the William Morris Agency, who died a few days later. Niven later revealed the story and publicly apologised to Gersh in his memoirs. In 1965 Gersh won the role of director in The Sound of Music for his client Robert Wise, and the substantial returns to his agency from this deal allowed for the purchase of plush new headquarters in Beverly Hills. Renamed The Gersh Agency in 1990, today the company is run by Gersh's sons Bob and David Gersh, and by the time of Phil Gersh's death in 2004 had 60 agents working from offices in Beverly Hills, California and New York City. Phil Gersh continued to play a role in the company's management until 10 weeks before his death.