On December 21, 1896, he won the World Colored Heavyweight Championship vacated by long-time colored champ Peter Jackson by knocking out Charley Strong in the 19th round in a fight held in New York City. On New Year's Day 1897 he fought Joe Butler in Philadelphia and was knocked out in the second round. They met again on the sixth of March in a title bout, and Armstrong prevailed, winning by a technical knock-out in the 6th round. The following day, he fought Frank Childs and lost to him on points in a non-title bout. His next fight was a title bout with Sam Pruitt in San Francisco on April 23, 1897, which he won via a 1st round knock-out. His next official fights resulted in a loss on points to Joe Sheehy in a four-rounder in June and a win against Jack Douglass via a TKO in the 2nd round on September 13. Some sources also report that he beat Jack McCormick on September 25. In 1897, he also fought two exhibitions against Tom Sharkey in May and fought two exhibitions against two different opponents, Jimmy Barry and Childs, on May 29. He fought an exhibition against Joe Choynski in August and finished out the year with an exhibition against John Holtman in November. In the new year of 1898, Armstrong put his championship on the line in a rematch with Childs. On January 29, 1898, and Childs took the title by knocking Armstrong out in the second round.
Post-Championship
On August 5, 1898, Armstrong fought top white heavyweight contender James J. Jeffries, who was less than a year away from winning the world's heavyweight championship, in a 10-round bout at the Lenox Athletic Club in New York City. Armstrong lost on points. He fought colored heavyweight champDenver Ed Martin for the colored heavyweight title in the Crystal Palace in London on July 25, 1902. According to the Associated Press coverage of the fight:
"Martin proved the cleverer and never gave his opponent a chance, being declared an easy winner on points at the close of the fifteenth round; Martin, it is announced, will challenge the winner of the Jeffries-Fitzsimmons fight in San Francsico."
Martin was not given a title shot and met Armstrong for a rematch in Philadelphia on December 10 According to the Daily Gazette and Bulletin:
"The bout was fast from the start and both men narrowly escaped a knockout. The bell saved Armstrong in the third and fifth and Martin was floored six times in the fourth round."
The six-round bout ended in a . In his next fight, Martin lost his title to Jack Johnson, who would one day break the color bar and become the first African American heavyweight champion of the world. Armstrong retired after being KO-ed by Walter Johnson in Haverhill, Massachusetts on December 26, 1904. He ended his career with an official record of 17 wins against 11 losses and five draws. He also had a newspaper decision record of one win and one draw. On January 29, 1932, he worked the corner for King Levinsky in his fight against Max Baer. He died on January 5, 1933 in Chicago, Illinois.