Rhind was born in New York City, New York, the son of Charles Rhind, a prominent shipowner who also served as Minister to Turkey from 1827. His mother, Susan Fell, was a descendant of Cadwallader Colden, the Governor of the colonial Province of New York from 1769 to 1771. Rhind was appointed midshipman on September 3, 1838, and between 1839 and 1841 he served on the Mediterranean Station aboard the frigate and the sloop. He then served aboard the sloop in the West Indies in 1842-43, then on the frigate off the coast of Africa in 1843-44, before attending the Philadelphia Naval School in 1844-45. Promoted to passed midshipman on July 2, 1845, Rhind served on the brig on the Coast Survey in 1845-46, which was then attached the Home Squadron on the coast of Mexico during the Mexican–American War, being present at Alvarado and Tabasco. Rhind served on the steamer in 1848, before rejoining the Coast Survey aboard the schooner Ewing on a voyage to the coast of California in 1849-50. He then served aboard the sloop in the East Indies in 1850-51, before returning to the Coast Survey, where he remained until 1854, receiving promotion to the rank of master on April 30, 1853. Rhind was commissioned as a lieutenant on February 17, 1854, and served on the sloop on the Pacific Station, but in May 1855 was court-martialed after a disagreement with his commander, and left the Navy in September 1855. Eventually reinstated, Rhind served on the sloop, the flagship of the Africa Squadron from 1859-61.
On December 14, 1861, during the American Civil War, Rhind was ordered to take command of the screw steamer ; and, while commanding her, earned the Thanks of the Navy Department in a letter dated September 7, 1864, for the capture and destruction of Confederate works commanding the South Edisto, Dawho, and Pon-Pon Rivers, in April 1862, and received promotion to lieutenant commander on July 16, 1862. Promoted to commander on January 2, 1863, in that year he participated in the attacks on Charleston's defenses as commanding officer of the ironcladram. During the attack on April 7, 1863, Keokuk was struck over 90 times in 30 minutes, suffering 19 holes at or near her waterline. Retiring, she was kept afloat until the following morning, before finally sinking, by which time the crew had been taken off. Later, after commanding the gunboat and the screw frigate, on October 23, 1863, he assumed command of the side-wheel gunboat on the North Atlantic Blockading Squadron, and earned praise from Rear AdmiralSamuel Phillips Lee for the "gallantry and endurance displayed" by himself and his crew during an engagement with three batteries at Deep Bottom on August 13, 1864. In December 1864 he was detailed to command the powder boat, which was loaded with 215 tons of gunpowder, then towed by to a point 250 yards off Fort Fisher. There Commander Rhind and his crew set the fuzes and started a fire before escaping to Wilderness. The blast from the explosion, although loud, did little damage and two days later Rhind returned to close proximity to the fort to plant a marker buoy as near to the fort as possible to allow the fleet to bombard Fort Fisher at close range. Admiral Porter, in his official report to the Navy Department, wrote; In 1866 he became a Companion of the First Class of the Military Order of the Loyal Legion of the United States - a military society composed of the officers of Union armed forces and their descendants. He was assigned MOLLUS insignia number 208.
