Born in Baltimore, Maryland, Davis grew up in East Baltimore. His father was a schoolteacher, his mother was a food services worker and his stepfather was a steel worker, according to an October 12, 2000 article in the Baltimore Sun. Davis attended Phillips Academy for high school, and earned a Bachelor of Arts degree in American history from the University of Pennsylvania in 1971. Although he had planned to become a college professor, Davis chose to pursue a career in the law after taking an undergraduate course in constitutional law, according to the October 12, 2000 article in the Baltimore Sun. Davis earned a Juris Doctor from the University of Maryland School of Law in 1978. At the University of Maryland, Davis won the Myerowitz Moot Court Competition in 1977 and was thus elected to the law school's three-member National Moot Court Team.
On May 4, 1995, President Bill Clinton nominated Davis to be a United States District Judge of the United States District Court for the District of Maryland. The United States Senate unanimously approved Davis to the seat in a voice vote on August 11, 1995. He received his commission on August 14, 1995. His service terminated on November 12, 2009, due to elevation to the Fourth Circuit. In discussing his judicial philosophy, Davis told the Baltimore Sun in an article that was published on October 12, 2000 that "I want the loser—and I know there's always going to be a loser, that's the nature of the beast—but I want the loser to be able to say, 'I lost, but I was heard, and I believe that judge gave me every consideration in hearing my side.
On October 12, 2000, President Clinton nominated Davis to be a judge for the United States Court of Appeals for the Fourth Circuit, to replace Murnaghan, who had died. The nomination was a part of Clinton's effort to integrate the Fourth Circuit, which up to that point had never had an African-American judge; however, since Davis was nominated after July 1, 2000, the unofficial start date of the Thurmond Rule during a presidential election year, no hearings were scheduled on his nomination, and the nomination was returned to Clinton at the end of his term. President George W. Bush chose not to renominate Davis to the Fourth Circuit. On April 2, 2009, President Barack Obama nominated Davis to the United States Court of Appeals for the Fourth Circuit. On June 4, 2009, the Senate Judiciary Committee endorsed the nomination by a vote of 17-3. On November 9, 2009, the full Senate voted 72–16 to elevate Davis to the Court of Appeals. He received his commission on November 10, 2009. Davis assumed senior status on February 28, 2014. On May 3, 2017, it was announced that Davis will become City Solicitor for the City of Baltimore, effective September 1, 2017. He retired from active federal judicial service on August 31, 2017.