In round 23 of the 2004 NRL season Smith made his NRL debut for the Melbourne Storm against the Canberra Raiders. He scored a try on debut. Smith didn't play first-grade in 2005 but became a regular first-grader in 2006. The Storm reached the 2006 NRL Grand Final and Smith was selected to play from the interchange bench in the loss to the Broncos. He played in the 2007 NRL Grand Final win over the Sea Eagles, again selected off the interchange. In 2010, the premiership that Smith won with Melbourne was stripped by the NRL due to the club's major and deliberate breaches of the salary cap. After winning the 2007 NRL Grand Final with the Melbourne Storm, Smith was selected for the 2007 All Golds tour. Not to be confused with compatriot and namesake Jeremy James Smith, both Jeremy Smiths were named in the rugby league Centenary clash of 14 October 2007, Jeremy James Smith in the number 7 jersey and Jeremy Jon Oscar Smith in the number 13 jersey. In August 2008, Smith was named in the New Zealand training squad for the 2008 Rugby League World Cup. Later in the season Smith played in the 2008 NRL Grand Final defeat by Manly, starting in the. Smith signed a 3-year contract with the St. George Illawarra Dragons starting in 2009 to be a part of the new Wayne Bennett era at the Dragons. In October 2008, he was named in the final 24-man New Zealand Kiwis squad. Smith was a member of the World Cup-winning New Zealand team beating Australia in the 2008 World Cup final at Suncorp Stadium in Brisbane, Queensland. In 2010, the premiership that Smith won with Melbourne in 2007 was stripped by the NRL for major and deliberate breaches of the salary cap.
St. George Illawarra Dragons
In January 2009, Smith was named the 2008 New Zealand Rugby League Player of the Year. In his first year at the Dragons, the team won the 2009 Minor Premiership after finishing first in the regular season and again won the Minor Premiership in 2010. In August 2010, Smith signed a 3-year contract with the Cronulla-Sutherland Sharks starting in 2011. Smith played at in the Dragons' team that won the 2010 NRL Grand Final, defeating the Sydney Roosters. This was Smith's second premiership after the Storm's 2007 title. After that win, he had played in more grand finals than any other New Zealander. After winning the 2010 NRL Grand Final with the St. George Illawarra Dragons, Smith was a part of New Zealand's successful 2010 Four Nations campaign.
Cronulla-Sutherland Sharks
Despite having a disappointing year with the Cronulla-Sutherland Sharks in 2011, Smith was again chosen to represent New Zealand in the 2011 ANZAC Test and 2011 Four Nations. After a successful start to year with the Sharks, Smith was selected in the 2012 ANZAC Test. On 26 September 2012, Smith signed a three-year contract with the Newcastle Knights starting in 2013 to rejoin his former Dragons coach, Wayne Bennett. He was released from the final year of his Sharks contract due to personal reasons. He was named the New Zealand Rugby League's 2012 Player of the Year.
Newcastle Knights
Smith commenced playing with the Newcastle Knights in the 2013 NRL season. On 22 August 2014, Smith became one of the current NRL players and former Sharks players to accept reduced bans from the Australian Sports Anti-Doping Authority for his role in the club's 2011 supplements program. On 13 February 2015, Smith played for the NRL All Stars against the Indigenous All Stars in the annual All Stars match. On 6 June 2015, he re-signed with the Knights on a 1-year contract. On 26 February 2016, Smith was announced as one of three co-captains for the Knights alongside Trent Hodkinson and Tariq Sims. On 13 May 2016, he announced his retirement from rugby league at the end of the 2016 season.