In 1850, Pottinger purchased a commission in the Grenadier Guards and served in England until 1854. Active in social life, he lost much of his adoring mother's wealth on the race-course. In 1856, he succeeded his father as second baronet and soon dissipated his inheritance. Forced by debt to leave England, he migrated to Sydney. After failing on the goldfields he joined the New South Walespolice force as a mounted trooper. A superb horseman, he spent the next few years on the gold escort between Gundagai and Goulburn. Probably because of conditions imposed by his family who still supported him with funds, Pottinger kept his title secret but in 1860 it was discovered by the inspector-general of police, John McLerie, and promotion came rapidly. In November he became clerk of petty sessions at Dubbo and on 1 October 1861 assistant superintendent of the Southern Mounted Patrol. Although determined to succeed in his career he was involved in a drunken brawl at Young on 20–21 December 1861. Sued, he received a public rebuke from Charles Cowper for his 'highly discreditable' behaviour. Posted to the Lachlan, he proved himself an indefatigable but unlucky hunter of bushrangers.
Inspector of police
Under the 1862 Police Regulation Act, Pottinger was appointed an inspector of police for the Western District. The Act was bitterly criticized and Pottinger seen as a symbol of its defects. In April 1862, he arrested Ben Hall at Forbes on a charge of highway robbery, but he was acquitted. Soon afterwards Hall joined Frank Gardiner's gang which robbed the Lachlan escort of some £14,000 on 15 June 1862. Quick in pursuit, Pottinger remained on the trail for a month, and arrested two of the bushrangers. They escaped several days later in a gun battle but Pottinger recovered the stolen gold taken by the prisoners. Criticized for his failure to send an adequate guard with the escort and his return without prisoners, Pottinger was praised by others for his determination and endurance. On the night of 9 and 10 August Pottinger and a party of police surrounded the house of Gardiner's mistress, Kate Brown, but the bushranger escaped when Pottinger's pistol misfired. They arrested a young boy on suspicion of being an accomplice and allowed him to remain in the lock-up without comforts; his death in March 1863 from gaol fever further diminished Pottinger's reputation. On 27 September 1862 Pottinger had appeared before a Bathurst court on a charge of assault. In February 1863, Pottinger attended the Sydney trials of the escort robbers; jostled by larrikins in the street he again became the subject of public notice. He also threatened politician Joseph Harpur with his whip for charges made against him in the New South Wales Legislative Assembly. Meanwhile, the bushrangers in his district became more active. He later captured Patrick Daley, but on 17 August 1864 failed to arrest James Alpin McPherson. In May 1863, the inspector-general had directed the police to act on their own initiative. Early in January 1865 hoping to lure Hall and his associate, fellow bushranger John Dunn, into the open, Pottinger rode in the Wowingragong races in breach of police regulations. Despite his justifiable claim that his action 'fully warranted the discretionary departure in point from the letter ' of the regulation he was dismissed from the police force on 16 February 1865. Protest meetings against his dismissal were held on the diggings and in the towns, with petitions for his reappointment.