Share This Article

Everybody knows about Marse Robert—but Smith lee was the major player when the war began. What happened?

On March 24, 1847, as Sydney Smith Lee commanded a 24-pounder long-range naval cannon brought onshore during the U.S. siege of Veracruz, his younger brother Robert joined him at his post. “I stood by his gun whenever I was not wanted elsewhere,” Robert later wrote. “I am at a loss what I should have done had he been cut down before me. I thank God that he was saved. He preserved his usual cheerfulness, and I could see his white teeth through all the smoke and din of the fire.” The two men shared a distinguished family history (see sidebar, p. 60) and a devotion to the military, but their careers would follow dramatically different trajectories. Robert’s culminated in an indomitable allegiance to the Confederacy and crushing defeat; Smith’s began with distinction and ended in unremarkable obscurity.

In the 13 years between the end of hostilities in Mexico and the outbreak of the Civil War, Robert had his share of success. He supervised construction of Fort Carroll in Baltimore, was superintendent of the U.S. Military Academy for three years and commanded cavalry in Texas in Albert Sidney Johnston’s regi­ment. But he stayed out of the national spotlight until October 1859, when he was sent to Harpers Ferry to deal with John Brown and his raiders.

Smith Lee, on the other hand, was very much in the public eye during this period, making a reputation for himself on both the national and world scenes. He began his rise by serving as the commander of the Philadelphia Naval Yard for three years, and followed that success by serving three years as commandant of the U.S. Naval Academy in Annapolis, Md. There Smith influenced many of the officers who would later fight both for and against the Confederacy.

In 1852, the year Robert became West Point’s superintendent, Smith received a plum assignment: commanding Commodore Matthew C. Perry’s flagship USS Mississippi in Perry’s expedition to Japan, helping to bring Western civilization to the hermit kingdom. On July 8, 1853, Smith anchored in Tokyo Bay and presented gifts to the startled Japanese. Among the symbols of Western civilization were 100 gallons of Kentucky bourbon whiskey, fine wines, two telegraph transmitters with several hundred feet of wire, a quarter-scale railroad with 370 feet of track, John James Audubon’s Birds of America and a selection of firearms.

The mission convinced the Japanese to open their ports to world trade, and the men of the expedition immediately became heroes to the American public.

Following his success in the Orient, Smith Lee was appointed, along with future Union Navy leaders David G. Farragut and David Dixon Porter, to a naval board to meet and entertain the Japanese ambassador. He was then named chief of the Bureau of Coast Survey in Washington, D.C. He was at the peak of his career and at the political center of things when the Civil War broke out in April 1861.

Neither Smith nor Robert wanted to see Virginia join the Confederacy. They agreed, nevertheless, to make their decision jointly if Virginia chose to leave the Union. On April 18, 1861, Smith and Robert met with their cousin Samuel Phillips Lee to discuss what to do if Virginia seceded. Phillips Lee, a naval officer, made it clear he would stay with the Union, and Smith promised to blow him out of the water by placing a battery on the Virginia shore. Phillips was the son-in-law of Francis Preston Blair Jr., one of the most influential figures in the United States, with a father and brother then serving in Lincoln’s Cabinet. He later attempted to obtain the U.S. Army commanding general’s position for Robert and an equally important position for Smith, but it was in vain, as both brothers refused to desert their native state.

Diarist Mary Boykin Chesnut preferred Smith (whom she knew well) to Robert. Her opinion initially was echoed by the military establishment, which generally ignored Robert while placing Smith in a position of importance.

By November 1861, the fledgling Confederate Navy had 29 vessels afloat, manned by nine captains, 25 commanders, 24 lieutenants and a full complement of sailors. It was a magnificent performance, but it was not enough to challenge the might of the United States, which was amassing a navy of 264 vessels, carrying 2,557 guns and manned by 22,000 seamen, by December 1861. As befitted his rank and experience, Smith Lee was sent to command Norfolk, the key naval command in the Eastern Theater.

The importance of Norfolk to the Confederacy could not be underestimated. At the beginning of the war, Virginia had only 50,000 pounds of gunpowder. The capture of Norfolk in April 1861 provided an additional 300,000 pounds, though admittedly most of it was coarse cannon powder.

As far as fieldpieces were concerned, Virginia had only 30, including a dozen new Parrott guns, but with the addition of Norfolk, several new fieldpieces were added to the Confederate armory. USS Merrimack had been scuttled by the Union Navy, but its hull was retrievable.

Along with this advantage, the dock used for the cleaning and care of ships, foundries and forges along with mounds of tools and equipment were virtually untouched.

Navy Secretary Stephen Mallory audaciously proposed construction of an ironclad that would tip the balance of power to the Confederacy. Mallory had intuitively seen the Achilles’ heel of the mighty Union Navy—the wooden vessels of its blockade. If he could destroy that wooden wall, the South would be spared a strangling death and could perhaps even levy tribute on the great urban centers of the North.

There was nothing revolutionary about an ironclad ship in 1861. The French had learned the value of ironclads in the Crimean War, and as early as 1859 had Gloire, an iron-plated vessel protected by armor 4.5 inches thick. By 1861 France had another 20 vessels under construction. Their British rivals began work in 1859 on Warrior, also iron-plated, to meet the threat. They had 10 vessels under construction, with armor plating 6.5 inches thick. Surprisingly, the U.S. naval establishment could find no use for such innovations, leaving the field free for spectacular Confederate success.

Unfortunately, Lee belonged to the establishment. He was a good officer, and he could follow orders, but he doubted the value of an ironclad ship.

Nevertheless, Merrimack was brought to the surface and recommissioned as Virginia. The first question Smith Lee asked its proud inventor was, “Mr. Porter, do you really think it will float?”

Despite Lee’s misgivings, Virginia did float, and despite the unreliability of its creaking engines, it was a weapon that could break the blockade. On March 8, 1862, the cumbersome vessel departed from its mooring at Norfolk to challenge the enemy fleet. Moving at no more than 6 knots, it rammed Cumberland and left Congress a burning hulk.

It looked as though the Confederacy had scored a major success. Union Secretary of War Edwin Stanton panicked and advised Lincoln to leave Washington. The mayor of New York, Fernando Wood, hinted at an agreement with the Confederacy to use his city as a free port. All seemed lost for the Union.

The next day a strange craft steamed up to meet the Virginia at Hampton Roads. It was Monitor. The U.S. Navy had produced an ironclad of its own. Arriving in the nick of time, Monitor checked Virginia and ended the Southern ironclad monopoly.

In its duel with Virginia, the Union vessel had given as good as it got, forcing extensive repairs on the Southern ship. Entrusted to Lee’s care, Virginia would undergo several repairs before being pronounced ready for sea duty. Never really in favor of the ironclad, Lee nevertheless gave his best effort in attempting to make the project work. He dutifully oversaw the repairs, but some observers felt that he was “wanting force, energy and vim,” and that “he allows everyone to browbeat him.”

If he was not a great commander, he was still a capable subordinate. On March 26, Mallory ordered him to “begin at once, without attracting special notice to the subject, to carefully pack and get ready for transportation all the fine machinery and tools not required for your workshops.” Lee realized Monitor had provided an effective check on the further activities of Virginia, and that with Union Maj. Gen. George B. McClellan’s army on the march it was necessary to salvage as much of the equipment at Norfolk as possible. It was a credit to his military ability and organizational genius that most of the materiel at Norfolk eventually was saved. It was largely due to Smith Lee that the Confederacy could still field an army and navy in early 1862.

But he disagreed with Robert, who was acting in the unofficial capacity as military adviser to President Jefferson Davis, on what to do with Virginia. Robert wanted the vessel removed from Norfolk to stop McClellan’s advance up the Virginia Peninsula. Smith demanded Virginia remain under his command.

Its new commander, Josiah Tattnall, thoroughly confused by conflicting orders, steamed back and forth between the two commands, accomplishing nothing.

On May 10, 5,000 Union troops landed at Willoughby’s Point, seven miles north of Norfolk. The Confederate commander, Maj. Gen. Benjamin Huger, sent a message to Smith Lee warning him of the Union advance. But Lee neglected to pass it on to Tattnall, resulting in the forced destruction of Virginia. The ship attempted to retreat up the James River, but even after attempts to lighten Virginia, the river proved too shallow.

For only 45 days Tattnall had commanded the famous ironclad. The vessel had been back in the water for 13 days when its commander reluctantly ordered it destroyed. He had never even fired a round at the enemy. Mallory blamed Tattnall, but the newspapers savaged the unfortunate Huger for the disaster. On December 5, 1863, with Smith Lee sitting on the court-martial, Tattnall was exonerated. The feud with Huger would last longer.

The Battle of Drewry’s Bluff on May 15, 1862, offered Smith Lee perhaps a final chance for immortality in battle. Again he failed. Sent to relieve the commander of Fort Darling, an important James River stronghold seven miles below Richmond, Smith arrived with the battle underway.

He was unwilling, however, to assume control of the fort from Commander Ebenezer Farrand. Serving as second in command, Smith coordinated with Captain Augustus Drewry’s Southside Heavy Artillery in directing the fort’s batteries against the Union’s five-vessel flotilla steaming up the James toward the Confederate capital.

After a few hours of fighting, the engagement ended in a Confederate victory, as the battered Union vessels retreated back down the James toward Hampton Roads. Smith assumed command of the fort after the battle, but was not even mentioned in dispatches about the event.

He was soon sent to Charlotte, N.C., against his wishes. From this point on, Smith Lee’s career declined and the career of Robert E. Lee began its meteoric rise.

He served the Confederacy in minor capacities for the rest of the war, as examiner in the Confederate Naval Academy, chief of the Bureau of Orders and Detail and as a member of several courts-martial. But never again was he to play a key role in any military action.

On April 2, 1865, Smith Lee performed his last service to the Confederacy. With Richmond about to be abandoned, General Samuel Cooper ordered Lee to transform his sailors into soldiers. Four hundred men in two skeleton regiments filed out to the trenches at Danville.

Smith Lee was not among them—he was too old, or perhaps unwilling, to fight. Entrusted with the care of Robert’s wife, he waited out the last seven days of the war in the Eastern Theater.

Smith would not meet Robert again until May 4, 1869. He had retired to Richland in Stafford County, where he vainly attempted to scratch a living from the depleted soil of the old plantation. Robert was busy in his new duties as president of Washington College in Lexington.

On May 6, Ascension Day, they knelt together at Christ’s Church for the last time. Robert had been attending a church council meeting in Fredericksburg and stopped to see Smith. For two days, he stayed at Smith’s home.

This was to be their last meeting. On July 24, suffering from a liver ailment and worn out by hard manual labor, Smith Lee died. Robert arrived the next day, too late for a funeral.

He called Smith’s death “a sad gap in our family…a grievous affliction to me which I must bear as well as I can.”

Little more than a year later, on October 12, 1870, Robert died. Smith was the closest friend he ever had.

Adapted from The Handsomest Man in the Confederacy, originally published in the July 1994 issue of America’s Civil War.