The manor of Cardinham is not mentioned in the Domesday Book and may thus have acquired its name from its later holders the de Dynham family which took its name from Dinan in Brittany. The de Cardinham family may thus have been a branch of the de Dynham family of Hartland in Devon to whom the barony later passed from Isolda de Cardinham, for reason unknown. This was the opinion of Leland who writing of Robert de Cardinan as founder of Tywardreath Priory, called him quidam ex Dinamiis from the Dinam Moreover, Lysons pointed out a further factor giving this opinion validity in that the arms of the two families were similar. Cardinan was later quartered by their descendants the Prideaux family of Prideaux Place, Cornwall, as Sable, three lozenges in fesse ermine, whilst the arms of Dynham were Gules, four lozenges in fesse ermine. It was suggested to Lysons by Mr Austen, of Place-House, in Fowey, that the Castle was named by its builder, possibly a member of the Dynham family who had inherited the property by marriage to the FitzWilliam heiress, Car - Dinham.
Robert FitzWilliam, recorded in 1166 Cartae Baronum to be holder of lands formerly held by Richard FitzTurold. In 1169 he and his son Robert made a gift to Tywardreath Priory in Cornwall.
Robert "son of Robert FitzWilliam", who following his father's death paid in Devon a large feudal relief for his inheritance. In 1194 he was excused scutage on 71 knight's fees, approximately 71 separate manors, which were known to have been held from the successor of the Count of Mortain. A drawing of his seal, appendant to an undated grant of his mill of Cardinam to the Priory of Tywardreath, is reproduced in Lysons, Magna Britannia, Volume 3: Cornwall. It shows an eagle displayed circumscribed by the words: SIGILL ROBERTI DE CARDINAN.
Robert de Cardinan, who in 1194 is recorded as heir of Robert FitzWilliam and exempt from payment of scutage. He made gifts to Tywardreath Priory in Cornwall.
Andrew de Cardinan, son and heir, who left a daughter Isolda de Cardinan as his sole heiress.
Thomas de Tracy, first husband of Isolda de Cardinan, heiress of the Barony of Cardinan. At some time before the death of Thomas she had granted away her manors. Secondly she married William de Ferrers, probably the son of Reginald de Ferrers of Newton Ferrers, Devon and Shillingham, Cornwall.
Oliver de Dinham, 1st Baron Dynham of Hartland and Nutwell in Devon, to whom Isolda de Cardinham granted at some time before 1270 the barony of Cardinham and also Bodardle, stated in some records to be a separate barony. In 1270 his claim was disputed unsuccessfully by Hugh de Treverbin, son of Odo de Treverbin by his wife Emma de Cardinan, daughter of Andrew de Cardinan. His peerage title was not apparently hereditary.
John Dynham, 1st Baron Dynham, summoned by writs of Kings Edward IV and Henry VII to attend parliaments from 28 February 1467 to 16 January 1497, the writs being addressed to Johanni Dynham de Care Dynham, by which he is held to have become Baron Dynham. He died without male progeny when his heirs became the descendants of his five sisters.