Modern American intelligence gathering, not yet a science, can still leave military forces lost in the fog of war.
Out of a blinding snowstorm burst an unexpected host of panzer divisions, attacking a weakly held front in the last major German offensive of World War II. The Battle of the Bulge had begun, and the Allies were not prepared. Yet they might have been, because Allied intelligence was aware of most of the German units involved and even knew where they were gathered. The Allies simply hadn’t predicted what would happen next.
The British Ultra code breakers issued eleven thousand decrypts during the buildup and the Bulge offensive itself, from October 1944 until January 1, 1945. They made substantial data available on the relief and rebuilding of the German panzer troops, and more concerning railway movements. Just the unloading points for the formations mentioned in these messages should have been a tip-off. An intercept on December 7 even revealed the combat readiness of several of the German infantry divisions that would participate in the Ardennes offensive.
Luftwaffe messages signaled an increase in air strength, special security measures, the preparation of airfields in the offensive sector and of aircraft for ground attack missions, and a command reorganization. All these were indications of preparations for a contingency operation. Yet this information made no difference. Catching the Americans surprised and unprepared, German troops overran one U.S. First Army position after another.
The failure of American intelligence at the Bulge highlights the importance of good information in the field. Before battles, much takes place behind the scenes. Acquiring the intelligence needed to make sound strategic decisions is among the most important of these activities. Lack of warning means surprise and increases the risk of defeat, or it brings about battles—and wars—that ought never to have been fought. Good intelligence enables commanders to deploy their forces in the most efficient and deadly way possible, contributing to victory or at least avoiding defeat.
Since 1941, the United States has engaged in conflicts ranging from a world war to the invasion of Grenada, and each required gathering intelligence before com- mitting forces. Just enumerating this list of alarums and excursions suggests the difficulty of maneuvering through the intelligence terrain: World War II; the Korean War; Vietnam; interventions in the Dominican Republic, Grenada, and Panama; assorted cold war contingencies; the Gulf War of 1990–91; Afghanistan (first as a covert operation, now overtly fighting insurgency); and the current conflict in Iraq. Most of these conflicts have provided multiple instances when the ability to penetrate the fog of war was central to the outcome of combat engagements, and sometimes to the intervention’s outcome.
World War II proved a formative passage in intelligence practice. Of course, American participation in the war itself began with a signal intelligence failure in December 1941, when American military and naval intelligence did not detect the Japanese intention to strike at Pearl Harbor, the major American naval base in the Pacific. American intelligence gathering was actually pretty good at the time, with naval attachés reporting extensively on what was visible to them, and radio operators intercepting Japanese messages and even deciphering some codes. But prevailing assumptions about likely Japanese moves in the event of war, the way observable movements conformed to those notions, and the glut of information that backlogged the system left Washington in the dark. The failure to anticipate Pearl Harbor left a lasting stain on American intelligence and later served to buttress those who argued for the creation of a civilian agency to inform American presidents and supplement the data reported by military sources.
As the United States participated in the war, its intelligence resources mushroomed, and American mastery of information-gathering methods also grew. Aerial photography and photographic interpretation evolved, huge resources were devoted to radio interception, and code breaking became a major enterprise. By 1943, American fleet intelligence in the Pacific alone was staffed at a higher level than the entire prewar American intelligence establishment. In both the European and Pacific theaters, legions of spies—local resistance networks plus a small number of active agents recruited from among the enemy’s ranks—provided a great deal of what has become known as “human intelligence.”
The American alliance with Great Britain opened the door to close intelligence cooperation with an ally that had broken the German machine codes, resulting in the famous Ultra intelligence. The British also assisted the United States in developing counterespionage and deception operations. In short, the nation gained intelligence expertise during World War II that has served as the foundation for America’s understanding of its adversaries ever since.
Nevertheless, as widely as this revolution in intelligence ranged, it did not succeed in dissipating the fog of war. For example, take the Normandy invasion. The Anglo-American landing on the Continent, designed to eject Hitler’s armies from the Atlantic Wall, was among the most extensively prepared endeavors in military history, in both intelligence and operations. For months the Allies followed every enemy development, photographing German fortifications, logging spy reports on troop movements, and meticulously constructing terrain models of Atlantic Wall fortifications. They listened to German radio transmissions, collected the reports of the Japanese naval attaché on the strength of German positions, and even sent commandos to surreptitiously collect samples of beach sand.
Despite all that preparation, when the landing took place on June 6, 1944, the Americans wading ashore at Omaha Beach did not know that a fresh, powerful German formation, the 352nd Infantry Division, had reinforced the defenses in that sector. Lore has it that the French Resistance had indeed discovered the enemy move and sent a report to London, but the carrier pigeon bearing the message fell victim to a German soldier’s shotgun. According to the British intelligence history, the Allies had had some indications of the movement of the 352nd from its reserve position near St.-Lô as early as April, but by D-Day the intelligence mavens had yet to obtain definitive evidence. As a result, the Omaha Beach invaders faced much stronger defenses than anticipated and the outcome in that sector was a near thing.
Similarly, intelligence failed in the winter of 1944, as the Allies pushed toward the German frontier. American armies were the principal targets of the full-scale German counteroffensive that became the Battle of the Bulge, and it fell to their intelligence to detect the enemy.
Instead, even the warnings of last-minute deserters were discounted, and the German offensive burst over the Allies like a lightning bolt. A review of the periodic intelligence summaries of the relevant Allied commands shows that some had detected that the Germans were marshalling their forces, particularly the powerful Sixth Panzer Army, but German cover and deception activities succeeded in misleading the Allies.
The Germans accomplished this in part by giving Gen. Dwight D. Eisenhower and his lieutenants something logical to believe: that Hitler intended to intervene against one of the Allied offensives then underway. The Supreme Headquarters Allied Expeditionary Force (SHAEF), Eisenhower’s headquarters, bought the deception, even arguing that the Germans were raiding the actual attack sector in the Ardennes for troops to reinforce other threatened areas. The intelligence officer for the responsible army group, Omar Bradley’s Twelfth, prognosticated that the Germans were nearly beaten: “the crust of defenses is thinner, more brittle, and more vulnerable than it appears.”
Both commands interpreted the assembly of German panzer forces as the gathering of a fire brigade of reserves—just as their adversary intended. Their idea was that the Germans might counterattack as American offensives thrust across the Roer River or captured the city of Aachen, then the scene of pitched battle. The force that actually held the Ardennes front, Courtney Hodges’s First Army, issued one estimate that foresaw a German counterattack, but their other estimates played down that possibility.
The aerial reconnaissance group supporting First Army flew 361 photographic missions during the period. Though weather was poor, there was only one day when photo flights were impossible. The truth was that the Americans were diverted by their own operational focus—flights were overwhelmingly concentrated over the Roer and Saar regions and German reserve areas. Between December 10 and 15 (the day before the offensive began) there were just three sorties over the attack sector. Neither British (Secret Service MI-6 or Special Operations Executive) nor American (Office of Strategic Services, or OSS) intelligence furnished significant information; the OSS had yet to establish itself in Germany. The Dutch underground provided some data on German paratroop units and headquarters departing from the front, but this fitted easily into the fire brigade hypothesis.
One lesson of the Battle of the Bulge is that predisposition can affect intelligence work. The Allies had won a great victory in Normandy and were on their way to Berlin, or so they thought. Intelligence staffs were as susceptible to victory disease as the operations gurus. The game became predicting the relative utility of the assorted offensive schemes proposed, with intelligence serving to validate strategic choices. Evidence of German offensive preparations could be discounted because of the impression of Germany’s overall weakness, and because Allied commanders refused to believe their own forward momentum could be blunted. The net result would be that despite the great advances made in intelligence throughout World War II, even in its final stages the fog of war could still settle over the battlefield.
The first great cold war encounter was the Korean War, which erupted on June 25, 1950, and continued for three years. Incremental advances had taken place in the means of collecting intelligence—most significantly in the capability of cameras for overhead reconnaissance. The United States had lost ground in code breaking: the Soviet Union and its allies used codes that were less vulnerable to decryption, so there was no Korean War equivalent to Ultra intelligence. On the other hand, radio traffic analysis and direction finding, an important tool in World War II, now became a mainstay of signals intelligence.
The single greatest development, however, was that the United States established a civilian intelligence unit, the Central Intelligence Agency (CIA). Preventing another Pearl Harbor was a major aim of proponents of the full-time civilian intelligence agency; Korea would be that agency’s first test, and its intelligence efforts stumbled. In this new encounter, the North Koreans initially invaded South Korea. They were stopped by South Korean and American forces at the Pusan perimeter. A following United Nations counteroffensive, coupled with an amphibious invasion at Inchon on September 15, 1950, substantially cleared North Korean troops from the south.
The issue then became whether to invade and occupy North Korea. Beyond the question of inadequate warning of the initial North Korean invasion, the biggest intelligence failure of the war would concern the People’s Republic of China’s intervention in Korea in October and November 1950.
As in the pre–World War II period, intelligence resources were thin at the start of the Korean War. There was a substantial organizational effort in the field, but much of it was directed at the Soviet Union, the primary cold war adversary of the United States. To take the code breakers as an example, the Armed Forces Security Agency (the predecessor to today’s National Security Agency) had the equivalent of just two analysts working on Korean issues before the war, with two part-time cryptanalysts and a single Korean linguist. Since April 1950, a single radio set had been devoted to intercepting Korean traffic.
There were eighty-three people watching China, out of an organization that, including contributing components embedded within the military services, numbered in the thousands. Similarly, there was no CIA station in South Korea and military intelligence there depended on a small staff section within the U.S. advisory group. The U.S. Air Force in the Far East included only two dozen reconnaissance aircraft, among them just four long-range (modified) Lockheed RF-80 or Boeing RB-29 photo planes.
All that changed quickly with the coming war. The CIA quickly ramped up spy operations, although most were ineffective. The code breakers shifted to round-the-clock activity, got an advance unit of radio monitors to Korea by mid-September, and set up the 1st Communications Reconnaissance Company in October, based on a unit drafted in from Fort Lewis, Washington. The air force doubled the staff of its scout unit and quickly augmented its planes. Aerial reconnaissance became the primary pillar of intelligence, though the United States started way behind, having first to expend so much effort simply to map North Korea. The cryptographers had to depend on traffic analysis and on any unencrypted messages sent by the Chinese and the North Koreans.
In spite of these limitations, a picture began to emerge of the Chinese redeploying units of their People’s Liberation Army from central China to Manchuria, across the Korean border. The code breakers gleaned signs of this from their interception of open communications. Surveying the successive issues of a weekly summary of Soviet bloc military movements that the CIA began to publish with the onset of the war, Washington seems to have had a fair handle on the Manchurian buildup.
Gen. Douglas MacArthur, head of the Far East theater command (FECOM), had an intelligence staff that compiled similar summaries displaying the same knowledge. Headed by Maj. Gen. Charles A. Willoughby, FECOM intelligence projected Chinese strength in Manchuria at about one hundred thirty thousand in July 1950, building to four hundred fifty thousand by late September. Beijing’s intentions became a paramount question.
The CIA issued its assessment among a series of reports circulated on October 6. The agency’s analysis held that China was capable of full-scale (but not necessarily decisive) intervention. After laying out a series of pros and cons on an attack, however, the CIA judged that “a consideration of all known factors leads to the conclusion that barring a Soviet decision for global war, such action is not probable in 1950.” The civilian intelligence agency was wrong—but so was military intelligence. FECOM’s Willoughby repeatedly ruled out any Chinese intervention, even as he noted Beijing’s growing military strength on the Korean border. MacArthur explicitly told President Harry Truman at an October 15 meeting on Wake Island that there was little chance the Chinese would intervene.
In truth, Beijing was making its decision at that very moment. The Chinese had even threatened to act if American troops went into North Korea. General Willoughby rejected the warning in a FECOM intelligence bulletin as “probably in the category of diplomatic blackmail.”
Chinese general Peng Dehuai, who supported intervention and became the People’s Volunteer Army commander in Korea, crossed the Yalu River into Korea on October 18, three days after MacArthur told Truman it would never happen. On October 21, Chinese troops had their initial engagement with South Koreans, followed a few days later by one with Americans. The Chinese then pulled out of contact for a month, but followed up by launching a major offensive in late November.
On November 6, ahead of the big attack, the CIA published a formal national intelligence estimate on the Chinese intervention. This time, after a process of reasoning through a set of pros and cons from Beijing’s perspective, the agency concluded by presenting a list of the advantages the Chinese would see accruing from their intervention. The CIA had changed its mind.
FECOM did not. General Willoughby was still arguing the merits of Chinese action when the offensive broke that drove the United Nations forces almost back to the Pusan perimeter.
The Korean case shows the dangers of wishful thinking. General MacArthur set himself upon a certain course of action, capturing North Korea, and his intelligence officer produced interpretations to please. While the United States was wrong across the board at first—the CIA’s initial interpretation being as mistaken as Willoughby’s—the agency changed its tune as Beijing demonstrated seriousness of purpose. MacArthur chose not to believe. More than two years of bitter war ensued, with thirty three thousand six hundred Americans dead and one hundred forty two thousand total casualties among overall United Nations losses (including South Korean) of three hundred ninety-eight thousand.
Then came the Vietnam War. By the 1960s, the pillars of intelligence had been strengthened considerably. The American intelligence community had grown larger than ever, with a new Pentagon-wide military unit, the Defense Intelligence Agency, to supplement the armed services branches and compete with the CIA. The National Security Agency (NSA) had brought the code breakers into their own as a fully coequal intelligence unit. The NSA’s cryptologists by now also had a lot more experience with Soviet and bloc systems and devices, assisted by the first sophisticated computers.
Other means of intelligence gathering had also improved. Better cameras made high-altitude spy planes possible, and, after 1960, even spy satellites. And the United States now had substantial global capability, not just focused on Russia. In short, the fog of war ought to have dissipated. Yet Vietnam offers prominent examples of its persistence. Examples include the lack of intelligence in the air war, continuing uncertainty about the size of the North Vietnamese forces and the National Liberation Front, and the Americans’ surprise at the Tet offensive. But a lesser known and equally important failure that occurred at the end of the war, the Mayaguez incident, is worth attention.
By the end of the Vietnam War, Southeast Asia had been wired for sound. American troops had placed sensors over wide swaths. They had deployed a new generation of sophisticated reconnaissance satellites and regular photographic reconnaissance missions, and employed radio monitors at many stations 24-7. All these supported their dedicated analytical staffs. A substantial body of experience among intelligence watchers developed over the lengthy period of active operations that had continued through 1972. Yet the system did not perform well in the context of this specific tactical situation.
The Mayaguez incident occurred suddenly in the chaotic days after the fall of Saigon, a sort of spasmodic shudder of a war that refused to end. On May 12, 1975, Cambodian Communists of the Khmer Rouge boarded an American merchant ship belonging to the Sea-Land Corporation, impounding it and taking the crew prisoner. President Gerald R. Ford determined to free the ship and crew. In a crisis that unfolded over ninety-six hours, the U.S. Marines staged a costly helicopter-borne assault on Koh Tang Island, some fifty miles off Cambodia’s coast, while American planes bombed the Cambodian mainland, even though the ship and crew had already been released.
The incident began as Mayaguez managed to radio word of the Cambodian seizure. Company officials in Jakarta, Indonesia, relayed the messages they received to the American embassy, the first arriving in Washington at about five o’clock on the morning of May 12. A delay resulted from the initial message identifying the incident as a “reported” seizure, and because the alert had not come from an actual intelligence source. Subsequent dispatches revealed the true situation.
Before six, the Defense Intelligence Agency asked the Pacific Command for a photo mission over the incident area. At the CIA, Director William E. Colby learned of Mayaguez on the phone, five minutes after a critical message confirming other reports arrived. Staff members wrote a briefing note that CIA officer David A. Peterson used to inform President Ford at 7:40 a.m., ten minutes after the White House Situation Room told Deputy National Security Adviser Brent Scowcroft. Secretary of State (and national security adviser) Henry A. Kissinger heard of the incident at his morning staff meeting shortly after eight. Photo planes were en route to the area from Thailand by 9:57 a.m. Washington time. Radio monitors were focused on Cambodian traffic.
Ford met with Kissinger and Scowcroft even before spy planes had taken to the air, and held his first National Security Council (NSC) meeting about Mayaguez at noon. Colby related the information already compiled by the CIA, including a reminder that Cambodians had instigated two recent, similar incidents. They had fired on a South Korean tanker but not boarded it, and had taken a Panamanian freighter but quickly released it.
The consensus favored a strong reaction. Initial talk focused on mining Cambodian waters. Ford ruminated that he might want to “scramble” a force to take an island near where the ship had been seized.
That night the White House learned Mayaguez was underway toward the port of Kompong Som (formerly Sihanoukville). A midmorning meeting on May 13 framed President Ford’s decision. Colby told the group that Mayaguez had stopped at Koh Tang, and the crew had been taken ashore there. The CIA could not say how many Cambodians were on the island (“presumably few,” Colby related). Both Ford and his vice president, Nelson Rockefeller, complained about the imprecise information. Rockefeller felt the misleading data had foreclosed military options. Ford’s NSC mulled over whether to use U.S. Marines to capture Kompong Som, or to take Koh Tang and the ship.
A navy destroyer would reach the area the next night. A company of marines from the Philippines already stood ready at U Tapao, Thailand. More marines moved from Okinawa to prepare for the contingency. The Koh Tang option was selected to prevent the Khmer Rouge from moving the crew off the island or the ship to Kompong Som, where it would be much better protected against efforts at recapture.
No American intelligence mechanism reported that the Mayaguez crew were, in fact, being moved to Kompong Som (they were, but not for long), and orders to prevent ship movements led to air attacks on the very Americans they were trying to rescue. Navy Lockheed P-3 ocean surveillance planes flew overhead, maintaining watch over Koh Tang, but they too apparently missed the crew movement. That afternoon the CIA reported that the crew had likely been moved—but to the interior of Koh Tang, not the mainland. At a late-night NSC session, Ford turned to Scowcroft, not Colby, for the update.
By then leaders were clearly assuming the crew had been taken aboard a fishing boat that American aircraft were attempting to prevent from reaching the mainland. The secretary of defense wanted more tear gas dropped to turn the boat aside. Ford now complained about military trans- Mayaguez mission of orders as opposed to intelligence. Colby’s main contribution was to estimate that there were only one hundred Khmer Rouge on Koh Tang. He pushed for an assault on the island and the ship. By morning the CIA was reporting that at least some crewmen were at Kompong Som and could even be farther inland. In truth, no one in the United States had any idea where the crew might be.
The crisis reached its apogee on May 14. Colby’s briefing that day differed in detail but covered the same ground as the day before. American destroyers were on the scene, aircraft flew overhead, air strikes were hitting Khmer Rouge bases on the mainland. About the time these forces went into action, the Cambodians released the merchant seamen. They were neither on Koh Tang Island nor at Kompong Som—though they had been held there briefly—but on another island, Rong Sam Lem.
Cambodia broadcast a statement, picked up and reported by the CIA’s Foreign Broadcast Information Service, that the ship was being released. News reached Kissinger while the American planes were in the air. Lacking information on the crew, Ford and Kissinger let the attacks go ahead as planned. Americans recaptured and boarded the ship, but the crew was gone.
The Mayaguez crew reached one of the American destroyers while marines were fighting on Koh Tang, and a second wave of assault troops actually landed after both crew and ship had been recovered. Fighting went on for hours. It turned out the spotty intelligence on the Khmer Rouge played against the United States. Five of eight assault helicopters were destroyed or damaged. Eighteen marines died on the island and twenty-three more died aboard a chopper that crashed and burned in Thailand. Fifty Americans were wounded. There were more rescuers dead than there were Mayaguez crewmen.
Three main charges have been leveled at American intelligence in the Mayaguez crisis. The first concerned CIA’s alleged failure to tell Ford of other ship seizures by the Cambodians in the days before Mayaguez, and to issue warnings to mariners. The transcripts of Ford’s NSC meetings clearly show that Colby, the CIA director, brought up the other incidents at the very first meeting. Warning merchant ships was not a CIA responsibility.
The second charge castigates the agency for not appreciating the strength of the Khmer Rouge on Koh Tang. There is some truth to this, since the CIA failed to deliver a comprehensive estimate of Cambodian strength—not just on Koh Tang—until May 14, when American military measures were already in motion. Even then the data was generic and not specific to the opposing forces. This fault was compounded when Colby went beyond his role as CIA director to become an advocate of the attack.
The last charge, that agency communications failed, is overstated. The initial reports moved quickly. At the end, notice of the Cambodian announcement was received, translated, and reached the White House in less than an hour. Both the CIA and the American military experienced critical transmission failures, but even if they had performed perfectly it would have made no difference: the NSC transcripts show the Ford administration was determined to display a show of force.
Nevertheless, it is instructive that main features in the recommendations of the CIA postmortem on Mayaguez concerned ways to increase the velocity of communications from the field to Washington and keep analysts informed of operational data. The CIA also bemoaned the growing role of the telephone. By its nature, with one person speaking to another over an audio circuit, telephones restricted knowledge to direct participants and whomever they informed. This effectively cut knowledge of intelligence and orders, which, for the CIA, left colleagues in the dark and failed to generate the written record that could have informed others. The practice conforms exactly to what officials today call “stovepiping.”
The advent of the mobile cell phone has magnified this problem enormously. Neither failing has ever been resolved, and both played roles in such recent episodes as 9/11, the American invasion of Afghanistan, and the Iraq war.
The Mayaguez incident is notable because, among other things, it took place at the dawn of a new era in communications. time American leaders presumed they could com- mand a military operation in real or near-real time. Gone was the SHAEF of General Eisenhower, where Mayaguez was really the first the leader could do no better than listen to the weather prediction for the Normandy invasion, say, “Go!” and then settle back to observe the outcome. Leadership today works by remote control.
In April 1980, President Jimmy Carter tried something similar to Mayaguez in Iran; it ended in tragedy at Desert One when a Special Forces mission to rescue hostages failed miserably. Ronald Reagan led the 1983 invasion of Grenada by remote control. George Herbert Walker Bush exerted White House command over the Panama invasion at the end of 1989. The Gulf War of 1990–91 was something of a throwback because it involved a lengthy period of buildup and aerial offensive, but it too had episodes when remote control came from the White House.
Bill Clinton suffered, one could argue, for not attempting remote control in Somalia. And the current President Bush has swung both ways. At Tora Bora in Afghanistan at the end of 2001, remote controls from theater leaders and Washington negated the plans of field commanders to trap Osama bin Laden in a mountain fastness. The failed blitz bombing of Saddam Hussein’s supposed headquarters on the first night of the 2003 invasion of Iraq, arranged by the Pentagon at the behest of the White House in a matter of hours, was not merely unsuccessful but preempted a carefully planned “shock and awe” air offensive. The long war in Iraq led George W. Bush to back away from efforts at remote control, at least from what the public can see, though no doubt the eventual declassification of secret records will show at least episodic exercise of real-time command.
All this has implications for dispelling the fog of war. The evolution of technology has not only solidified the pillars of intelligence but has also increased the velocity of command decisions in a way that challenges intelligence to produce accurate appreciations on an urgent basis. As the scene of battle shifts rapidly from one front to another, requirements for up-to-date data in multiple arenas become immensely complicated. The old Pearl Harbor question—how to overcome the “signal to noise” ratio and find the real gems among the glut of information collected—has also become harder to resolve, as the array of data multiplies from a host of sophisticated technical systems.
Failure to predict the 1998 testing of three Indian nuclear weapons shows that intelligence shortcomings remain. Moreover, serendipitous events can still keep key information from reaching intelligence staffs, just as the shooting of a carrier pigeon over the Normandy coast thwarted the Allies in 1944.
Problematic assumptions of military and political leaders, and their predisposition to believe in a certain interpretation, have bedeviled military and civilian analysts from the Battle of the Bulge to the search for Iraqi weapons of mass destruction. Wishful thinking magnifies those effects, as with MacArthur in Korea.
One lesson from the Iraq war is that there is danger in obscuring one intelligence question by obsessing over another, for example, when the fixation on the enemy as being “former regime elements” hindered realization that a popular resistance had formed, and again when insisting that the enemy were all al-Qaeda terrorists transplanted to Iraq prevented the occupying force from seeing the rise of sectarian violence.
The commission that examined the September 11, 2001, tragedy specifically cited the problem, revealed in the Mayaguez crisis, of stovepiping intelligence—reporting up a single chain of command and thus not sharing information—as a reason for the intelligence failure. In short, the huge advances in technology and technique, for both collecting and interpreting information, have not banished the fog of war. Nor has the advent of civilian intelligence agencies that supplement military intelligence. Perhaps one day, but not today.
Originally published in the Autumn 2008 issue of Military History Quarterly. To subscribe, click here.
