Wild, a physician and ardent abolitionist, lost his left arm at the Battle of South Mountain in 1862. After recovering, he raised a unit of former slaves called Wild's African Brigade. During the winter of 1863–64, Wild led these soldiers in an expedition on the coast of North Carolina, terrifying a local white population accustomed to African slavery since the early 18th century. Wild's brigade landed in Virginia in May 1864 and began building the fort at Wilson's Wharf, one of a series of protective outposts guarding supply lines for Union Maj. Gen. Benjamin Butler's Bermuda Hundred Campaign. The wharf was at a strategic bend in the James River, overlooked by high bluffs, from Sherwood Forest, the home of former U.S. President John Tyler. By this time, Wild's unit had a frightening reputation among Southerners. Wild's subsequent actions alarmed them all the more. His soldiers freed and recruited slaves and in one case whipped a plantation owner who had a reputation for harshness to his slaves. The Richmond newspapers denounced these activities and put intense pressure on the government of Jefferson Davis to put a stop to Wild's depredations. Succumbing to the political pressure, Davis's military adviser, Gen. Braxton Bragg, ordered Fitzhugh Lee's cavalry division to "break up this nest and stop their uncivilized proceedings." While his uncle, Robert E. Lee, was battling Ulysses S. Grant at the North Anna River, Fitz Lee took elements of three cavalry brigades plus the 5th South Carolina Cavalry Regiment on a march from Atlee's Station to reach Wilson's Wharf. The Confederate general expected to fight a rabble, but instead found the defenders of Fort Pocahontas alert and ready for action. Wild commanded 1,100 men and two cannons. The Union force consisted of the 1st USCT and four companies of the 10th USCT. Battery M, 3rd New York Artillery was the only all-white unit in the defenses. The gunboat USS Dawn lay in the James River to deliver fire support to the fort's defenders. The fort was crescent-shaped facing north, about long, straddling the road to the wharf. It was anchored on both ends—to the west on a bluff and on the east by a branch of Kennon Creek—so it could not be flanked. It was fronted by a deep, broad ditch and abatis.
Battle
Around noon on May 24, Lee's men charged and drove in the Union pickets who were posted near the Charles City Road, about a north of the fort. By 1:30 p.m. the fort was invested and Lee sent two officers under a flag of truce with a message demanding the surrender of the garrison. He promised that the black soldiers would be taken to Richmond and treated as prisoners of war, but if they did not surrender, he would not be "answerable for the consequences." Wild and his men interpreted this to mean that some of the men would be returned to their former masters and others would be tried by state authorities for inciting insurrection. Wild sent back a written reply that said "We will try it" and verbally told the two officers "Take the fort if you can." Lee planned a two-pronged attack. Brig. Gen. Williams C. Wickham's brigade moved east of the fort, concealed in ravines of Kennon Creek. To distract the Federals from Wickham's attack, Col. John Dunovant of the 5th South Carolina demonstrated on the western end of the fort. Dunovant's men advanced as far as the ditch and abatis, but were driven back by heavy fire. Wickham's men rushed forward across an open field and were met by interlocking fields of musket fire, canister rounds from two 10-pounder Parrott rifles, and naval gunfire from the Dawn. As Lee looked for a weak point in the fort's defenses, Union reinforcements arrived at about 4 p.m. on the steamer George Washington, carrying four companies of the 10th USCT. Lee ordered his men to withdraw to Charles City Court House and the next morning they rode back to Atlee's Station.
Aftermath
About 200 Confederate were killed or wounded in the abortive attack. Federal losses were six killed and 40 wounded. A few African-American soldiers were captured, and of these some were shot and one was sent to his master in Richmond. Materially, this action had little effect on the outcome of the war, but the North scored a propaganda victory. It was the first significant combat encounter between the Army of Northern Virginia and black soldiers, who had fought well in a defensive battle against a larger attacking force. Southerners, unwilling to acknowledge their defeat against a predominantly African-American force, claimed that six gunboats and substantial numbers of white Union soldiers were involved in the action. In his report, Fitz Lee minimized both his strength and his losses.
