Rivers was born in Los Angeles on March 19, 1892 to Andrew Ybarra. He was a fourth generation Californian. He was of Spanish-Indian descent. On February 22, 1911, he defeated Jimmy Reagan, a former Bantamweight World Champion by a technical knockout in the thirteenth round of a scheduled twenty. Reagan took a "terrific beating" and was knocked down four times prior to the thirteenth round when he was knocked down twice more by Rivers before the fight was called by Referee Eyeton. The bout took place in the Arena in Vernon, California, considered in the city of Los Angeles. The referee was Charles Eyton. On January 1, 1912 he knocked out former World Bantameight Champion Frankie Conley. At the time, Rivers was described 'as fast as chain lightning, and a stinging puncher to boot', and 'is a Mexican. His appearance tells that, but his accent does not differ from that of any other American stripling. He is a product of the Los Angeles fight game, and is probably the best youngster they have turned out down that way.' Annecdotally, he was 'christened Ybbarranda, or some other Mexican patronym with the customary deckload of Y's and R's' when asked his name by a southern US boxing announcer prior to a bout, he was then asked where he lived. Replying 'Down the river', he was announced as Joe Rivers. Johnny Kilbane took on Rivers in 1911 and outboxed him in the twenty-round bout. A rematch also went to Kilbane. Rivers was described as 'a regular bear-cat scrapper, a hit-and-miss, hammer-and-tongs battler, who took a lot of stopping'. On July 4, 1912, he fought Ad Wolgast for the lightweight title. At the beginning of the thirteenth round, he and Wolgast both landed punches that sent the other down. Wolgast fell on top of Rivers, and referee Jack Welch helped Wolgast to his feet and began the count on Rivers. Welch declared Wolgast the winner by a technical knockout, in one of the most controversial decisions in the history of boxing. His father, Andrew Ybarra, died at aged 58 years on January 23, 1913 of tuberculosis. On March 1, 1913 he married Pauline Slert of Santa Monica, California. Rivers was cited incorrectly to have died in 1918 when, after enlisting in World War I, the SS Tuscania was torpedoed. By 1955 he was living alone, in a windowless room on West Second Street in Los Angeles. His only possession of value was his father's 200-year-old violin, which he played daily. He died on June 26, 1957 in Inglewood, California. He was buried in Calvary Cemetery in East Los Angeles.
