By late 1660 Davenport and three other actresses were living in Lincoln's Inn Fields in London as protégées of Lady and Sir William Davenant joining the Duke's Company of actors under his management and with whom in 1661 she appeared as Lady Ample in his play The Wits; for him she also played Gertrude in Hamlet, Evandra in Love and Honour and Clerora in The Bondman. The first well-known English actress described as "a charming, graceful creature and one that acted to perfection", June 1661 saw her in what was to become her most famous role which was to be associated with her for years to come: Roxalana, in a revival of Davenant's The Siege of Rhodes, originally written in 1656 but rewritten to take advantage of the talents of the young actresses now in his Company. The diarist Samuel Pepys was pleased to see beautiful and talented women like Davenport now playing the female roles previously given to young men.
The Earl of Oxford became infatuated with her, pursuing her ardently for 9 months, but she rejected his advances and returned his gifts. The diarist John Evelyn saw her on 9 January 1661-62, she being soon after taken to be "My Lord Oxford's Misse". On 18 February 1662 Pepys recorded in his Diary that Roxalana was no longer performing on the stage, the supposition being that she was by then living with Aubrey de Vere, 20th Earl of Oxford. Davenport had refused to become the Earl's mistress, and it was her belief that the two had legally married, a claim later refuted by de Vere in a court case of 1686. Davenport claimed that she had married de Vere some time in 1662 or 1663 in the dining room of Elizabeth Farlow, who had a chandler's shop in London in a ceremony performed by a man 'in a Minister's Habit', who was probably de Vere's groom or one of his trumpeters in disguise. Whatever the truth of the matter, from that time onwards the Earl and Davenport lived together as man and wife in Drury Lane until the couple moved to Covent Garden when she fell pregnant. Their son Audrey de Vere was born on 17 April 1664 and baptized at St Paul's church in Covent Garden; de Vere openly acknowledged that he was the father of the child and granted a pension to Davenport and took responsibility for her debts. On 4 January 1665 Samuel Pepys paid a visit to the chaotic home of the couple and recorded in his Diary that 'his Lordshipp was in bed at past 10 a-clock: and Lord help us, so rude a dirty family I never saw in my life'. However, Aubrey de Vere later stated in his lawsuit of 1686 that Davenport had married a man named Barber, alias another man named Radcliffe, while de Vere himself married Diana Kirke shortly before 12 April 1673 in a legally recognised ceremony. While the court case confirmed that Hester Davenport and Oxford had indeed gone through some sort of ceremony it ruled that she had failed to establish that it had been performed by a genuine clergyman; she was therefore unable to prove that she was anything other than a discarded mistress. Although Davenport had lost the case against her erstwhile "husband" she did not accept the result and continued to style herself as 'Countess of Oxford'. At the same time she insisted that their son was legitimate and attempted to establish him as the heir to the earldom. Popular opinion at the time was that she was an innocent victim who had been duped into a sham marriage. It was claimed that Davenport appealed to Charles II for support, and that she received a pension of 1000 crowns a year, but there is no evidence for such claims.
Later years
Following her liaison with Aubrey de Vere Davenport remained single as befitted her conviction that she was married and on his death in March 1703 she styled herself as the ‘dowager Countess of Oxford’, while on 25 July 1703 she married the Flemish merchant Peter Hoet of Gray's Inn. Her son Audrey de Vere died and was buried at the Church of St Andrew in Holborn on 4 June 1708 as 'Earl of Oxford, from Grays Inn'. Her husband Peter Hoet died in 1717. She signed her own will on 16 November 1717 as 'Hester Oxford’ in it cutting her widowed sister Anne Walker out of a share of her estate, leaving her just one shilling while the remainder of her estate was bequeathed to her friends and executors John Hardy, baker, and Dorcas Magenis. Hester Davenport died the same day just hours after signing her will on her deathbed and was buried on 20 November at St Anne's church in Soho in London.