Millard was born in Boston. At the age of about eight he made such progress in the singing school he attended that after one winter he could sing the alto part of any church song. He then joined the choir of one of Boston's leading churches. At age ten he became a member of the Handel and Haydn Society, and was the leading alto in the chorus for several years. He became a singer in the Trinity Church of Boston which at the time was known for having the best music in town. His salary was $150 per year, a sum never heard of before for someone as young as he was. At the age of 15 Millard had to fill-in for a famous tenor, Mr. Jones, who had suddenly taken ill and could not perform in the oratorio Samson, in which he had been the lead.
In 1851, at the age of 21, Millard went to Europe to study for three years in Italy and other places, with the continent's best instructors. He stayed in London for two years after leaving Italy. There he sang in Louis Antoine Jullien's concerts at Exeter Hall, Surrey Gardens. He also sang in Boosey & Co.'s Verdi festival. He sand with Clara Novello, Miss Charlotte Helen Dolby, and Mr. Sims Reeves. Later he went with Catherine Hayes to perform in Ireland and Scotland. He worked as a correspondent for Dwight's Journal and other US papers. While in Europe he also composed music.
In 1854 he returned to Boston where he remained until 1858, teaching music and Italian as well as singing in concerts. He then moved to New York permanently. Within the year of his move he wrote his first hit song, "Viv L'America." Only two years later he marched to this tune after he enlisted to fight in the Civil War.
He served as a first lieutenant of the New York Nineteenth Regiment in the Union forces in the American Civil War. After serving for four years he was severely wounded at the Battle of Chickamauga and was forced to resign his commission and returned to New York.
offered Millard a job in the Custom House as a recognition of the importance to the country the song "Viva L'America" took on during the Civil War. He wrote another patriotic song, "Flag of the Free," which was also popular.
Compositions
Among other works by Millard was a four-act opera entitled Deborah. Millard wrote about 300 songs and published about 400 adaptations from German, Italian, French and Spanish. He also wrote a good number of sacred compositions, including four services for the Episcopal Church, four Te Deums, a grand mass, and a vesper service for the Catholic Church. Millard was also a poet, but set relatively few of his poems to music.
Millard was an active Freemason who took the thirty-second degree. He also wrote some Masonic music. They say he was a good conversationalist, and especially popular with "the ladies."