The joint committee on the conduct of the war found its Ball’s Bluff scapegoat in Brig. Gen. Charles Stone.
The day was clear, bright and bitterly cold at West Point. Packed snow crunched monotonously under the feet of the horses drawing the black draped caisson on which rested a rosewood casket covered with the Stars and Stripes. Paced by the sound of muffled drums, the caisson, flanked by six cavalry corporals, and the fur-bundled mourners advanced solemnly toward the gravesite on the southern side of the cemetery.
The cortege had come up by train from New York City earlier that morning, following a high requiem Mass at St. Leo’s Catholic Church. It was met by the Military Academy’s band, a company of engineer troops and the post’s officers, respectfully drawn up in line to salute the casket as it was transferred from the train to the caisson.
Among the mourners were numerous high-ranking Army officers and veterans of the Civil War, including the superintendent of West Point, General Wesley Merritt, who personally directed the burial arrangements. The list of pallbearers included Generals William Tecumseh Sherman, Charles Devens, Fitz John Porter, John Schofield, Z.B. Tower and William W. Burns, among others. It was mid-afternoon, January 27, 1887, and Brig. Gen. Charles Pomeroy Stone was about to be laid to rest will full military honors.
Twenty-six years earlier, Stone had been a rising star in the Union Army. He was a West Pointer (class of 1845) with solid Mexican War credentials and was well respected both as an engineer and an ordnance officer. Stone had served until late in 1856, when he left the army in California to better provide for his wife, Maria Louisa, and their infant daughter Hettie.
When war threatened again he found himself in Washington, where he had a fateful dinner with his old commanding officer from Mexico, General Winfield Scott, on the last day of 1860. Over cigars, Scott convinced Stone to reenter the Army and organize the defense of the capital.
Stone’s efforts over the next few months only reinforced Scott’s high opinion of him as he accomplished several essential tasks. He first eliminated the pro-secession elements from within the Washington, D.C., militia, especially from the “National Rifles.” As inspector-general, Colonel Stone partially disarmed that company and maneuvered its commander, Captain Frank Schaeffer, into a corner so that he either had to proclaim his loyalty to the Union or be dismissed. Schaeffer and his supporters within the unit were soon gone.
In March, Stone directed the security arrangements for the inaugural ceremonies, even including himself in the cavalry detachment that ringed the president-elect’s carriage, thus shielding Abraham Lincoln from possible assassins. Within days of Fort Sumter, Stone had crafted a plan for the defense of Washington while continuing to organize additional militia companies. Then, on the morning of May 24, mere hours after Virginia formally seceded, Stone led one of the first columns of troops to enter Virginia when the Federals moved to secure the southern approaches to the capital.
In early June, he commanded the Rockville Expedition, a month-long mission designed to clear the western approaches through Maryland. It familiarized him with the region directly across the Potomac from Loudoun County, Va. On August 12, Stone—now a brigadier general—took command of the Corps of Observation, a small brigade that eventually became a division of over 10,000 men assigned to watch that same area. No real fighting occurred during the next two months, but it was an anxious time as everyone awaited the rematch between the two armies that had clashed at Manassas in July.
Public attention came to focus on Stone’s sector on October 3-4, 1861, when he was reinforced by the large California Brigade under Colonel Edward D. Baker. Baker was a senator from Oregon and President Lincoln’s closest friend. In fact, he had been riding with Lincoln in the carriage that Stone and his cavalrymen protected on inauguration day.
On October 9, Brig. Gen. George McCall crossed the Potomac with his 12,000-man division and established a camp in Langley, Va., 25 miles east of Leesburg. Colonel Nathan “Shanks” Evans, the commander of Confederate forces in the Leesburg area, certainly noticed that within a matter of days enemy forces in striking distance of his 2,800-man brigade had been significantly strengthened. As there were no Confederate troops within easy supporting distance, Evans began to worry about a Union attempt to envelop and cut him off.
Following some skirmishing upriver near Harpers Ferry, Evans abandoned Leesburg late in the evening of October 16, apparently believing that the feared envelopment was close. His movement south was observed from across the river during the daylight hours of October 17 and duly reported to Maj. Gen. George B. McClellan, who responded by ordering McCall to advance from Langley to Dranesville, about halfway to Leesburg. From there McCall was to probe toward Leesburg to see whether Evans’ movement was genuine or merely an attempt to bait a trap.
Interestingly, Evans had withdrawn on his own authority. On hearing this, General P.G.T. Beauregard expressed his displeasure, and Evans returned to Leesburg in short order. He was back by the evening of October 19, and McCall was scouting a few miles away. It is to Evans’ somewhat precipitate withdrawal from Leesburg that the accidental Battle of Ball’s Bluff, which would have such enormous consequences for General Stone, can ultimately be traced.
A full tactical analysis of the battle is beyond the scope of this article. It should be understood, however, that contrary to the traditional interpretation, Ball’s Bluff was not an attack on Leesburg. While Union movements on October 21 at first appeared to be the envelopment that Evans had feared, none of the Federal forces in the area that morning had orders to attack the town. They were supposed to conduct limited missions and then withdraw. Indeed, McCall would have withdrawn from Dranesville on October 20 had he not opted to remain an extra day in order to complete the mapping of some roads. As the fighting commenced, he was unaware of what was happening and led his men away from the action.
After granting McCall’s request to remain the extra day, McClellan notified Stone of McCall’s presence in the area and suggested that Stone conduct a “slight demonstration” to see what effect it might have. He did not inform Stone that he had ordered McCall to withdraw. All the next day, Stone believed McCall was within supporting distance.
Stone conducted his demonstration on the afternoon of October 20, very visibly moving some troops to Edwards Ferry and positioning others at various points upriver in case they should be needed. Evans knew a feint when he saw one, however, and did not respond. By nightfall, it was over. Nothing that happened after that was part of the “slight demonstration,” though historians often have confused it with the events of October 21.
To learn whether the demonstration had any effect, Stone ordered a reconnaissance patrol to cross at Ball’s Bluff and scout toward Leesburg. The patrol returned that night and mistakenly reported that it had seen a small, vulnerable enemy camp (which was actually a row of trees mistaken for tents). Sensing a target of opportunity, Stone ordered Colonel Charles Devens to take 300 men of his 15th Massachusetts, raid the camp and “return to your present position.” As a diversion in Devens’ favor, he also ordered Major John Mix to cross three miles downriver at Edwards Ferry with 30 troopers from the 3rd New York Cavalry, attract the enemy’s attention to his force, then withdraw to Maryland.
Even though McCall’s division was about to leave the area, Devens’ raiding party was to conduct its raid and return, and Mix’s small cavalry force was a mere distraction, the combination of appearances and the expectations of the participants has resulted in a faulty historical interpretation of these movements as a three-pronged envelopment of Leesburg. What eventually occurred, however, merely developed as a result of Devens’ decision—on discovering that his raiding party had nothing to raid—not to return immediately but to send for further instructions.
Colonel Devens had the discretionary authority to do this, but his decision was a pivotal moment for Stone. Had Devens recrossed the river when he discovered the patrol’s mistake, there would have been no battle and thus no foundation on which Stone’s political enemies could later base their attacks.
About midmorning Stone spoke with Colonel Baker, who had come to find out what all the activity was about. Knowing of the patrol’s mistake but unaware that fighting had already begun, Stone explained to Baker what he knew and ordered him to evaluate the situation at Ball’s Bluff, then either cross more troops or withdraw those already there. As neither knew that combat was underway, both were thinking only of an expanded reconnaissance.
On his way upriver, Baker learned of the fighting and immediately began ordering all the units in the area to prepare to cross the river, despite the fact that there were few available boats. A transportation bottleneck immediately developed, and Baker did not personally go to Ball’s Bluff for some four hours.
The actual engagement consisted of three skirmishes separated by lengthy lulls, followed by continual fighting from about 3:30 p.m. until after dark. It was a slugfest that could have gone either way but eventually resulted in a rout of the Federal forces. Baker became the only U.S. senator ever to die in combat, and his death became a political club to be wielded against Stone.
McClellan exonerated Stone almost immediately, announcing on October 24 that “the disaster was caused by errors committed by the immediate commander, not General Stone.” There, for the moment, the matter rested, though a nascent problem lay in the fact that “some few colored men” had crossed the river with the Union troops during the retreat.
There were wildly inaccurate press accounts and considerable grumbling among Senator Baker’s friends, but Stone’s troubles did not really begin until a month later, when the runaway slave question came up. Both Maryland and Federal law required that runaways be returned to their owners. Administration policy winked at those requirements as they applied to rebellious states, but, in order to keep slave-owning Maryland loyal, it insisted that runaways from there be returned. The Radical Republicans in Congress strongly opposed this conciliatory attitude.
On November 24, two black men appeared in the camp of the 20th Massachusetts and were identified as runaway Maryland slaves. Per the standing orders, they were returned under guard to their owner. A few days later, an unidentified officer, most likely either Lieutenant John W. LeBarnes, whom another officer described as a “long haired abolitionist and spy of Gov. Andrew’s,” or Lieutenant Norwood P. Hallowell, who later commanded the black 55th Massachusetts, wrote to Massachusetts Governor John Andrew complaining about the incident. The governor responded with letters to Lt. Col. Francis Palfrey, commanding in place of Colonel William R. Lee (who had been captured at Ball’s Bluff), and to Secretary of War Simon Cameron.
Around the same time, Stone was informed that two slaves who had been brought back from Ball’s Bluff wanted to return to Virginia. Jack and Bob McCoy were brothers who belonged to John Smart, owner of Smart’s Mill. The two had gotten caught up in the action but, because their families were still in Virginia, asked to go back. After a delay of several weeks, Stone arranged for their return. That they returned voluntarily was entirely lost on Stone’s critics.
Governor Andrew demanded that Secretary Cameron exercise his “official authority” to prevent any repetition of these kinds of acts and also enlisted the support of staunch abolitionist Senator Charles Sumner.
Ignoring the fact that Stone, as a soldier, was obliged to enforce the laws and policies of his civilian superiors, Sumner savaged him in a December 18 speech on the floor of the Senate: “Brigadier General Stone, the well-known commander at Ball’s Bluff, is now adding to his achievements there by engaging ably and actively in the work of surrendering fugitive slaves. He does this, sir, most successfully. He is victorious when the simple question is whether a fugitive slave shall be surrendered to a rebel.”
Stone had earlier labeled Governor Andrew’s correspondence with Palfrey “unwarrantable and dangerous interference” in military affairs and undiplomatically compared Andrew to the governor of South Carolina. A tense exchange of letters between McClellan and Andrew hardened everyone’s positions and contributed to the anti-Stone and anti-McClellan attitude of the Joint Committee on the Conduct of the War.
On December 23, Stone surely made the blunder of his life when he responded directly to Sumner, accusing him of uttering “a slander and a falsehood” and calling him a “well known coward.” From that point on, Sumner did everything in his considerable power to ruin Stone’s career.
The Joint Committee on the Conduct of the War first met on December 10, 1861. Sumner was not a member, but his friend and political ally, Senator Benjamin Wade of Ohio, was the chairman, and most of the other members were of like mind.
The committee’s ostensible purpose was to determine the causes of the defeats at Manassas, Wilson’s Creek and Ball’s Bluff, but that quickly expanded into a self-defined mandate to investigate whatever it deemed important. Unfortunately for Stone, the committee’s investigations and his personal dispute with Sumner soon coalesced, and Stone became both target and pawn in a highly partisan effort to force upon the Lincoln administration and the Army the committee’s view of what the war should be about. That involved immediately abolishing slavery and planning harsh punitive actions to be implemented against a defeated South.
In enforcing existing law, Stone earned both Sumner’s and, through him, the committee’s lasting enmity. Stone’s fall from grace had much less to do with losing at Ball’s Bluff than it did with what Brig. Gen. Horatio Gibson later called “the blatant vaporings of a prejudiced Senator.”
Between December 27, 1861, and February 27, 1862, the committee examined 39 witnesses, few of whom had been present at Ball’s Bluff. Two in particular had personal grudges against Stone based on his having reprimanded them for various offenses against military discipline.
The committee members encouraged witnesses to second-guess Stone’s decisions and to question both his competence and his loyalty. Witnesses portrayed normal flag-of-truce communications as treasonable acts. Hearsay testimony that his men did not trust him or that prominent secessionists liked him was accepted as legitimate evidence. Stone never really had a chance.
On January 27, Senator Zachariah Chandler and Representatives George Julian and Daniel Gooch met with new Secretary of War Edwin Stanton to present the committee’s “findings” on General Stone. The next day Stanton ordered Stone’s arrest.
McClellan, to his credit, withheld the arrest order and convinced Stanton to give Stone a chance to rebut the accusations against him. Stone testified again on January 31, though it was a mere formality. Wade told him of evidence “which may be said to impeach you,” but when Stone asked to see this evidence, the senator replied, “I propose merely to state the heads; I do not desire to discuss them.” Stone was incredulous, but Wade knew of the arrest order, while he did not.
On February 5, Stone attended a reception at the White House, where he spent the evening “mingling with his friends” and receiving “as much consideration from all about him as any man then present.” Two evenings later he attended a similar reception at the home of General McClellan, on whose desk the arrest order sat like a ticking bomb.
As Stone came home from a meeting near midnight on February 8, 1862, the bomb went off. Approaching his house, he saw his old friend, Brig. Gen. George Sykes, and a squad of soldiers. Sykes told Stone that he had “the most disagreeable duty to perform that I ever had,” then placed him under arrest. Ironically, he was arresting his own commanding officer. Stone’s Regular Army commission was as colonel of the 14th U.S. Infantry; Sykes was that regiment’s major.
Sykes allowed Stone to see his wife before taking him to provost marshal headquarters. The next morning, Stone was sent to Fort Lafayette in New York Harbor, escorted by Lieutenant Daingerfield Parker of the 3rd U.S. Infantry. Due to a mix-up in the travel arrangements, Stone had to lend money to Parker to purchase their tickets. General Stone paid his own way to jail.
The story of his six-month incarceration is one of numerous failed attempts by him and his friends to get a hearing. He wrote letters up the chain of command all the way to President Lincoln. Over 700 prominent citizens of Massachusetts signed petitions to the president in his favor, all to no avail. He was never charged with any crime, but remained in prison as the controversy swirled around him.
Eventually, Senator James McDougall of California introduced a bill requiring that any officer placed under arrest must be shown the charges against him within eight days or be released. By then, the war had moved on and Stone had little symbolic political value. On August 16, Stanton ordered his release. From prison, he went to Washington, where he continued to seek a hearing and a new assignment.
Though it was not exactly a formal hearing, Stone did testify before the Joint Committee for a third time on February 27, 1863, and was able to forcefully state the case that he had never before been allowed to make publicly. The committee members were polite but noncommittal. Three months later to the day, however, he finally got an assignment, when he was ordered to report to Maj. Gen. Nathaniel P. Banks in New Orleans.
Banks gave him a variety of tasks, from inspecting the defenses of New Orleans to arranging the surrender of Port Hudson. On July 25, Banks appointed Stone his chief of staff.
Though he served creditably under Banks for nearly a year, their relationship quickly became strained. The cause seems to have been a combination of Banks’ uncertainty about his own standing in Washington and some indirect concerns about Stone’s Catholicism in heavily Catholic Louisiana.
With regard to Stone’s religion, the problem involved his leniency in granting travel passes in and out of New Orleans. At least one Catholic priest who received a pass turned out to be a spy. Father Gilbert Raymond of Opelousas, “by availing himself of the means of acquiring information which his position and calling afforded him,” managed to forewarn Confederate authorities about the Federal expedition up Bayou Teche in October 1863. Banks feared that Stone was being used.
Stone’s wedding created another problem. Maria Louisa Clary Stone had died in February 1863. Perhaps seeking a mother for Hettie, Stone married Annie Jeannie Stone (coincidentally, Stone was her maiden name) on November 5, 1863, in New Orleans. Jeannie had a brother in the Confederate Army. The wedding got linked to the failure of the Red River campaign the following April. One Washington diarist described the defeat at Mansfield, La., as “another Ball’s Bluff at the hands of Genl Stone—who married a secesh wife six months after losing his first one.”
General Banks relieved Stone as chief of staff on April 16, 1864, using the Red River failure as his reason. Curiously, he cited an order from Lt. Gen. Ulysses S. Grant, though that order from the newly appointed general-in-chief had been written on March 28, well before the collapse of the campaign. Grant, who knew and respected Stone, apparently only intended to bring him back east for service. Instead, Stone not only was relieved under an undeserved cloud for the failed campaign but was also stripped of his volunteer brigadier’s commission and reverted to his Regular Army rank of colonel. He also cooled his heels for another four months before receiving further orders. Neither the reduction in rank nor the delay was part of Grant’s March 28 order.
The likely culprit was Senator Sumner. In a June 18 letter to Ralph Waldo Emerson, Sumner wrote, “The Presdt. told me, that Genl. Stone was dropped at the order of Genl. Grant just before he commenced his march on Richmond. This is an answer to the suggestion of your correspondent (author’s emphasis). Sumner thus took credit for Stone’s predicament.
Grant did give Stone a brigade in the V Corps on August 21, but—weary of the ordeal and sick with typhoid—Stone had finally had enough; he resigned from the Army, effective September 13. Two days before, he wrote an extraordinary letter to Maj. Gen. George G. Meade, actually asking “to be shot by your order tomorrow.” Referring to recent negative comments about him in the press, Stone said that any officer who was as bad as they claimed “has no right to live and I earnestly request that my execution may be ordered.” Presumably, he was not serious, but considering all that he had been through, the letter certainly sheds some light on his frame of mind.
Stone had a remarkable postwar career. In 1870, he became chief of staff to Khedive Ismail of Egypt, a position he held until the British took control of that country 13 years later. Returning to New York, he became chief engineer on the Statue of Liberty project. He constructed the base and pedestal, then oversaw the actual assembly of the statue, symbolically driving the first and last rivets himself and serving as grand marshal of the dedication on October 28, 1886. His great work accomplished, Stone collapsed in his office on January 19, 1887, and succumbed to pneumonia on January 24.
The vindictiveness of Charles Sumner and the rabid partisanship of the Joint Committee destroyed General Stone’s Army career and damaged his reputation with historians. The men who knew him best, however, understood that he was neither inept nor a traitor. No less a soldier than Phil Kearny once called him “the ablest man in the army.” The old soldiers standing at attention in the snow at West Point as the fired salute echoed across the plain in 1887 no doubt agreed.
The United States Military Academy today remembers General Stone through its annual Brigadier General Charles P. Stone Memorial Award for Excellence in Arabic Studies. Given to a graduating cadet each year since 1980, the award recognizes Stone’s long career in Egypt and his contributions to U.S.–Egyptian relations during the 19th century.
James A. Morgan III, who writes from Lovettsville, Va., is the author of A Little Short of Boats: The Fights at Ball’s Bluff and Edwards Ferry, October 21-22, 1861.
Originally published in the June 2006 issue of Civil War Times. To subscribe, click here.
