In the 1950s he was a music director for several churches in Baltimore, Philadelphia, and in 1957 moved to New York City to become music director at the Church of the Incarnation. In 1959, he succeeded Arthur Mendel and Alfred Mann as director of New York's Cantata Singers. He expanded the group's repertoire to include 19th and 20th-century works, but gained notice for presentations rarities from its traditional repertoire, especially Handel's Belshazzar and Rameau's Les Indes galantes. Harold Schonberg assessed a 1959 concert of works by Purcell and Britten with these words: Also in 1959, he launched the ten-year career of the Festival Orchestra of New York. With that group in 1964 he led the New York premiere of Haydn's C Major Cello Concerto, a work discovered in 1961, with Janos Starker. He gained a wider reputation with a series of concerts of works by Bach and Handel at Carnegie Hall in 1961-2. In 1963, he presented Handel's Messiah four times in four different editions, further enhancing his reputation as a conductor of Baroque music. In 1963 Time magazine described him as "The hero of the baroqueniks." Dunn appeared as an imposter on the January 31, 1966 episode of the CBSgame showTo Tell the Truth. He revealed his true identity at the end of the segment. This led to his appointment in 1967 as Music Director of the Handel and Haydn Society of Boston, a post he held until 1986. During his tenure there, he transformed the organization from a traditional amateur oratorio society to a fully professional ensemble with a repertoire that ranged from early music to modern works. Michael Steinberg described his work: "His performances are clean, transparent, rhythmic and, in a broad repertory from Schütz to Dallapiccola and Stravinsky, he is particularly effective in works with chorus." The New York Times said that his work "helped animate the early music revival that took place in the mid-20th century and afterward". His ensembles used modern instruments but comprised fewer players; the ensembles' "clean, spare sound harked back to 17th- and 18th-century models".
Published works and later life
Many of his editions of choral music were published by E.C. Schirmer in Boston, where he was Editor-in-Chief in the 1970s. He held several faculty appointments at music schools in the U.S., including Boston University, Stanford University, and finally from 1990 to 1999 at the Indiana University School of Music. After his retirement, he devoted himself to writing and mentoring younger conductors. He composed works suitable for small church choirs, described by their publisher, Cantate Music Press, as "adaptable to the exigencies and emergencies that are apt to befall Sunday mornings." He died on October 26, 2008, in Bloomington, Indiana, and was buried in Green Mount Cemetery.