Small was born in 1828 in Edinburgh to Margaret and John Small. He was educated at the Edinburgh Academy and the University of Edinburgh where he graduated with an MA in 1847. In the same year he succeeded his father, who was acting librarian of the university library until his death. In 1854 he obtained the full status of librarian, with an official residence. He held the office, also in succession to his father, of acting librarian to the College of Physicians, for which he prepared a catalogue in 1863. He also served for many years as assistant clerk to the Senatus Academicus and editor of the University Calendar. He was president of the Library Association in 1880, and on 21 April 1886 the University of Edinburgh awarded him a LLD. He was for some time treasurer of the university musical society. Small devoted his leisure time to literary work. His first larger publication was a volume, English Metrical Homilies … Edited, with an Introduction and Notes, Edinburgh, 1862. He was the chief associate of Cosmo Innes in editing the Journal of Andrew Halyburton, published in 1867. Thereafter his chief labour was expended on editing, with careful glossaries and indices, the works of early Scottish poets, viz. The Poetical Works of Gavin Douglas, 4 vols. Edinburgh, 1874; Sir David Lyndesay's Monarchie for the Early English Text Society, and The Poems of William Dunbar for the Scottish Text Society. In 1885 he re-edited David Laing's Remains of Early Scottish Poetry, prefixing a bibliographical notice of his predecessor. To the British and Foreign Evangelical Review he sent an elaborate article on the authorship of the Ode to the Cuckoo, and he contributed numerous papers to the Transactions of the Royal Society of Edinburgh and the Society of Antiquaries. He also gave assistance to Sir Alexander Grant in writing the History of Edinburgh University. Small's brother-in-law was William Purdie Dickson,, a Scottish Professor of Divinity at the University of Glasgow from 1873 to 1895. The William Dickson Prize is named in his honor. Small's nephew was Andrew Munro,, a Scottish fellow, lecturer in mathematics and bursar at Queens' College, Cambridge from 1893 to 1935. The Munro scholarships and studentships at Queens' College, Cambridge are named in his honor. After a long illness John Small died unmarried in Edinburgh on 20 August 1886, and was buried in the Grange cemetery in Edinburgh. In 1924, the estate of John Small's sister, Jemima, left £5,000 to establish a fund in the name of both her father and her brother for the purchase of books and to subsidize general purpose expenses within the library.
Works
Besides the works mentioned above, Small wrote:
Some Account of the Original Protest of the Bohemian Nobles, 4to, Edinburgh, 1861.