Shakespeare, Cryptography and the "Investigation of Numerous Betrayals"
In the play Troilus and Cressida, the problems of history, literature, and cryptography are tied into such an interesting and intricate knot that solving even one of the mysteries leads to unraveling all the others.
The initial concept of this investigation (see the beginning here [i1] and here [i2]) involved delving into the ancient mysteries of the Rosicrucians and Masons, gradually leading through them to the secrets of modern history. However, the materials and documents discovered during these excavations naturally form a picture that clearly has its own structure and logic.
Since the main guiding principle here, in the affairs of scientific-mystical investigations in real-time [i3], is always new leads and clues in the studied documents, rather than pre-formulated concepts, the initial plan is set aside. Because the set of facts and evidence being revealed distinctly directs the investigation toward studying a somewhat different topic.
And it turns out unexpectedly convenient to start this branch of research with Russian classics…
Anton P. Chekhov, as is well known, was not only an outstanding playwright but also a talented humorist and satirist. One of his famous quotes beautifully conveys the peculiarities of Chekhov's humor [o1]:
If your wife has betrayed you, rejoice: that she betrayed you, and not the fatherland.
American novelist Joyce Carol Oates, though most renowned as a master of fiction, also made a notable mark in literary criticism. The sharpness of her unique perspective as a literary critic is well conveyed by this quote from an Oates article dedicated to the most mysterious of Shakespeare's works:
Troilus and Cressida, that most vexing and ambiguous of Shakespeare's plays, strikes the modern reader as a contemporary document – its investigation of numerous infidelities, its criticism of tragic pretensions, above all, its implicit debate between what is essential in human life and what is only existential are themes of the twentieth century.
Readers even slightly attentive will immediately notice what is common between the two quotes from such different authors as humorist Chekhov and critic Oates. The common subject here is Betrayal.
Or, to generalize a bit, the theme of various, big and small acts of treachery, since both marital infidelity and all other numerous kinds of betrayals – to friends or the motherland, to high moral principles or scientific truth – are all an inescapable element of human social life. Where there is deceit, there are betrayals. And a social human somehow doesn't know how to live without lies...
Joyce Carol Oates' essay The Tragedy of Existence: Shakespeare’s Troilus and Cressida was published in 1967 [o2]. By an interesting coincidence, two more books of note to us were also published in that same year of 1967, also dedicated to "investigations of numerous betrayals," but not in theater and literature, but in cryptographic and military-spy affairs.
One of this pair of books, David Kahn's The Codebreakers: The Story of secret writing [o3], quickly became famous and today rightfully holds the title of "The Bible for Cryptography Historians." The specific features of this extensive work and its not accidental flaws, directly related to the problems of Shakespeare studies, were detailed earlier in the material "Bacon, Shakespeare, and the NSA's 'Cut the Ends!' Reflex" [i4].
The second book, Ladislas Farago's The Broken Seal: The Story of ‘‘Operation Magic’’ and the Pearl Harbor Disaster [o4], though far less famous, has also taken its notable and quite special place in the history of cryptography. The peculiarity of the book lies in the fact that its author, L. Farago, managed to introduce into science such a sophisticated lie about betrayal, which all cryptography historians have long seen and noted... but still hesitate to fully debunk to this day.
Scholars hesitate to fully restore the truth here because it turns out to be extremely uncomfortable. It radically changes the long-established, yet thoroughly false image of "the great cryptologist" William F. Friedman. At the same time, this truth brings science back to the controversial figure of George Fabyan and the strange affairs of his Riverbank Laboratories. Namely, to experiments with levitation, Baconian ciphers and Baconian magic, to Francis Bacon as the son of Queen Elizabeth and the author of Shakespeare's works.
Thus, with a careful reconstruction of events, one can clearly see how the consistent and complete restoration of truth in this convoluted story leads science to the epic picture of an outright, almost indecent farce. A farce of numerous betrayals at the deep foundations of humanity's existential tragedy. Which vividly reminds us of scenes from the irritatingly strange Baconian play Troilus and Cressida...
The Two Faces of Herbert Yardley
It so happened that the unintended main character of the current episode became the distinguished American cryptologist Herbert Osborn Yardley. Probably the most infamous Enfant terrible in the history of 20th century cryptology.
On one hand, an undeniable natural talent, who managed as a young man – without experience and essentially without education – not only to create but also successfully lead a new intelligence service of a great power. On the other hand, an incorrigible show-off and egotist, a drinker and womanizer, a gambler and adventurer...
And since it turns out that it's convenient to present the heroes of our investigation with support from the new scientific-historical book by Betsy Smoot "From the Ground Up: American Cryptology during World War I" [o5], we'll use this same source for a brief introduction to Yardley's biography.
HERBERT OSBORN YARDLEY
Herbert Yardley was a prominent and controversial figure in American cryptology during World War I and the decades that followed. Born April 13, 1889, in Worthington, Indiana, he was the son of a railroad station agent/telegrapher and learned to operate the telegraph during his teens.
In 1907, after graduating from a high school, Yardley worked as a railroad telegrapher. In 1912, he took a civil service examination for a telegrapher job, came to Washington, DC, and embarked on a career in the State Department code room. Yardley was fascinated by codes and ciphers and concerned about the insecurity of the department’s cryptography. He studied the small amount of subject material available in English.
Shortly after the United States entered World War I, he focused on being assigned to the newly revived military intelligence effort in the War Department. He began active duty on July 5, 1917, and was soon put in charge of MI-8.
Just over a year later, as MI-8 was beginning to make its mark, Yardley was sent to Europe to assist the G2A6 and to learn more about the British and French cryptologic efforts. After the Armistice was signed on November 11, 1918, he found a place for himself providing cryptologic support to the 1919 Paris Peace Conference.
When Yardley returned to Washington in April 1919, MI-8 efforts were winding down, but a decision was made to establish a new organization, based on the 1917 discussion about creating a central cipher bureau for the government. The so-called “Black Chamber” was born in July 1919 in New York City with Yardley in charge.
The rest of Yardley’s story is well known and well documented. His book, The American Black Chamber, published in 1931 (and excerpted that same year in the widely read Saturday Evening Post), caused great consternation in the small American cryptologic community. William Friedman spent many years gathering material to refute the inaccuracies in Yardley’s work.
After providing cryptologic services in China and Canada (very briefly) and doing some work for the US Army cryptologic system during World War II, Yardley ran a variety of businesses. He died on August 7, 1958, and is buried in Arlington National Cemetery.
The brief excerpt of the quote mentioning W.F. Friedman, highlighted here in bold italics, is for the following reason. The years-long, intensely hostile, and even bitter fixation of Friedman on Yardley's book and on the personality of this vividly contradictory individual was far from a random episode in the relations between two outstanding cryptographers.
The documents and evidence presented next vividly show how closely intertwined were the destinies of Yardley and Friedman literally from the first days of their involvement in the field of US military cryptography. From those early days, in other words, when such a field, in essence, did not exist.
Because interesting American laws at the beginning of the 20th century simply prohibited the government from having intelligence and decryption services in peacetime. That is, intelligence was to be established in case of war, and disbanded again at its conclusion...
And in the biographies of our pair of talented cryptographers, Yardley and Friedman, it turned out that the former, Yardley, was immediately favored by fate, quickly making him the head of the new special service. But for the latter, Friedman, his ascent was thwarted by his authoritative and jealous boss, Colonel Fabyan at Riverbank... At least, that's how the reasons for such an unsuccessful start appeared to Friedman later on.
But here, however, we are not talking about career highs and lows, but about betrayals and treacheries. Including both actual betrayals and those fabricated intentionally to defame people.
Two Compromising Documents in Farago's Book
Hungarian-American writer and journalist Ladislas (Laszlo) Farago published more than a dozen very lively books dedicated to little-known aspects of military-spy history. Professional historians, as well as heroes of Farago's books, who knew the subject from the inside, rated his work low, to put it mildly, due to his fondness for excessive sensationalism, careless treatment of real people and dates, frequent distortion of facts, and fabrications about important historical events.
Regardless, Farago insisted that he worked in a strictly documentary genre, and he supported his books with numerous references to real, often rare and hard-to-access historical materials. For this reason, Farago's particularly interesting book for us, Broken Seal [o4], dedicated to the confrontation between American and Japanese cryptographic services during 1921-1941, continues to surface in the far from concluded debates among historians.
The main subject of the ongoing debates surrounding this book is a very colorful episode describing the betrayal by Herbert Osborn Yardley, the head of the highly secretive "Black Chamber" in the United States government, which was engaged in intercepting and decrypting the diplomatic correspondence of various states, from potential adversaries to closest allies.
The story of betrayal begins with the new U.S. Secretary of State, Henry Stimson, who had lofty notions about the service of diplomats and assumed office in early 1929. After some time, he received a compilation of the most interesting fragments from foreign diplomatic telegrams decrypted by Yardley's "Black Chamber" for review.
“Gentlemen do not read each other's mail,” Stimson exclaimed, and gave orders that “all State Department funds be withdrawn from [Yardley's] support” at once.
By then the “black chamber” existed solely for the benefit of the State Department, which was footing its entire bill. Yardley had no advance warning and was stunned when the blow fell. “There was nothing to do,” he wrote ruefully, “but close the Black Chamber…. Its chapter in American history was ended.”
Secretary Stimson's seemingly naïve decision to dismiss Yardley was, unbeknown to him, one of the most fortuitous acts of his distinguished public service. Despite the secrecy in which the “black chamber” was shrouded within the American government, the Japanese knew all about Yardley's activities because none other than Herbert O. Yardley had sold its secrets to them. And he was not even the first one to betray to the Japanese the cryptological efforts of the United States.
It's the scourge of a secret service that it's sometimes less successful in safeguarding its own secrets than in procuring those of others. The very atmosphere of the institution — its basic melodrama, its lack of moral scruples, its dependence on people with aberrant personalities — renders it highly vulnerable.
The American cryptological establishment was no exception. For all its many loyal workers, it also had its quota of traitors. They were lured into treason by their unbridled ambition and an exaggerated notion of their own importance.
On March 7, 1925, an American employee of the War Department's cryptographic bureau contacted the Japanese embassy in Washington and warned Counselor Isaburo Yoshida, the acting chief of the mission in the absence of Ambassador Masanao Hanihara, that “the War Department was maintaining a supersecret code-breaking bureau.” According to Yoshida's informant, “there was no foreign code that could not be decrypted” by this mysterious agency. The man went even further. He volunteered the advice that “the only possible means of protecting the [Japanese] codes [would be] to change them as frequently as possible.”
The sensational information was immediately forwarded to Tokyo, in document “Secret — No. 48,” dated March 10, 1925. But nothing in the Japanese archives indicates that they undertook anything to tighten up their crypto-security as a result of this warning, or that Yoshida's informant followed up his “advice” with additional information or other useful services.
In the summer of 1928 Yardley attended a party at a friend's house in New York because he hoped to pick up some intelligence from one of the other guests, Koshiro Takada, who represented a Tokyo newspaper in the United States.
By then Yardley was thoroughly disillusioned and disgruntled. Even as a telegraphist in the State Department he had had the dream of running a vast central cryptological agency in the United States, with hundreds of men and women working under him. His pioneering work in M.I.8 and his great coup in 1921–22 turned the dream into an obsession. But instead of gaining the authority, influence and scope he craved, he was gradually cut back while a “bunch of amateurs,” as he called his colleagues in similar agencies, were established in positions of importance.
In his isolation, Yardley had persuaded himself that his government was ungrateful and disloyal to him. And he decided that this ingratitude had automatically released him from any moral bonds on his part. Moreover, he was troubled by increasing financial difficulties. He needed more money to finance his private life, his gambling for high stakes and his huge consumption of liquor during those Prohibition days. Now he was ready to have his revenge and also to get the money he urgently needed.
A few weeks after their meeting in New York, Yardley called on Takada and told him that he had some valuable information that would interest the Japanese embassy. Could Takada introduce him to Ambassador Tsuneo Matsudaira in Washington? Takada made the arrangements and suggested that Yardley, on one of his next trips to Washington, call at a house on Crescent Place where he would find someone to whom he could give the information.
At 1661 Crescent Place, an elegant little graystone house off Connecticut Avenue, he was received by Setsuzo Sawada, counselor of the Japanese embassy. Yardley went to the heart of the matter at once. He introduced himself as the United States government's senior cryptologist, briefly sketched his background, then told Sawada that he was prepared to sell his country's most closely guarded secret—for $10,000 in cash.
The offer was so staggering that it aroused Sawada's suspicions. According to his first report to Tokyo, in which he described this strange encounter, he had said to Yardley: “But you're making a lot of money in your job! Why are you willing to sell your country?” “Simple, sir,” Yardley replied, according to Sawada. “It just so happens that I need more money.”
A deal was made, but not without some haggling. Contrary to the popular belief that the Japanese had unlimited funds for such transactions, the Foreign Ministry operated on a very tight budget from secret funds. Sawada countered Yardley's demand by offering him $3000 at first, and then $5000. For a while Yardley refused to lower his price, but an agreement was finally reached. Yardley received $7000, with the understanding that he would be paid more if he decided to continue to work for the Japanese.
It was an amazing bargain at any price. In return for their money, the Japanese obtained all the secrets of the “black chamber” — Yardley's methodology in breaking their codes, copies of his work sheets, and his solutions of other codes as well, including those of the British Foreign Office, which they were especially anxious to get. Moreover, Yardley agreed to cut back his work on Japanese messages.
For a while Counselor Sawada remained in Washington to maintain liaison with Yardley. In 1929, however, he was recalled to Tokyo to take over the denshin-ka, the Cable Section of the Foreign Ministry.
At the end of his book, in the Notes section, Farago provided the following references to source documents for this sensational story of betrayal:
The account of Yardley's arrangement with the Japanese, presented here for the first time, is entirely from the original Japanese docs, microfilm copies of which are in AMFA, Reels UD29–UD30. The War Department employee's contact with the acting Japanese ambassador in Washington is described in “Secret Document No. 48” in this series, dated Mar. 10, 1925.
The Yardley transaction was recapitulated in great detail on June 10, 1931, in a memorandum from the chief of the Cable Section, Shin Sakuma, to For. Min. Shidehara, Doc. No. 0157, a microfilm copy of which is in the author's possession. Another recapitulation of what Sakuma called “Yardley's treachery” was contained in a memo dated Apr. 6, 1933. Foreign Correspondent Takada's participation is described in documents dated Nov. 26, 1939, and Mar. 5, 1940.
Presenting all these details from Farago's book in full citation is very useful and instructive for the following reason. Although the book's author refers, seemingly, to a multitude of different sources, actual informative and authentic documents, as will be shown by subsequent historical checks, are essentially two: the secret telegram No. 48 from 1925 and the memorandum from June 1931.
Indeed, a truly astonishing fact of modern historical science is that initially, and for many decades, ALL scholars and researchers who studied this episode in detail and from various angles, for some reason, always implicitly assumed that both Japanese documents were talking about the same cryptographer traitor.
Only nearly fifty years later, in 2011, a publication finally appeared from the prominent historian of cryptography, John Dooley [o7], in which he first extracted from the archives, translated, and published the full text of that initial Japanese telegram about the events of March 1925.
And here unexpectedly it was discovered that the U.S. War Department cryptographer who had proactively approached the Japanese with important information was actually named in this dispatch. And this informant was not Herbert Yardley at all, but William Friedman...
And the most interesting part here is what the historical science does (or doesn't do) with this so-called "new" sensational information. And what, accordingly, could be quite possible to do – if approaching the scientific search for truth without omissions and deceit.
Additional Reading
[i1] History Science as an Art of Cutting Out
[i2] NSA Hints for the Mysteries of Shakespearean Studies
[i3] Cryptographic Baconiana and Mythology of Shakespeareana (rus.)
[i4] Bacon, Shakespeare, and the NSA's 'Cut the Ends!' Reflex
Main Sources
[o1] Anton P. Chekhov. Life is Beautiful! (First published in 1885) Complete works and letters in 30 volumes. M.: Nauka, 1975. – Vol. 3
[o2] Joyce Carol Oates. The Tragedy of Existence: Shakespeare’s Troilus and Cressida. Philological Quarterly, Spring 1967
[o3] David Kahn. The Codebreakers: The Story of secret writing. New York: Macmillan, 1967
[o4] Ladislas Farago. The Broken Seal: The Story of ‘‘Operation Magic’’ and the Pearl Harbor Disaster. New York: Random House, 1967
[o5] Betsy Rohaly Smoot. From the Ground Up: American Cryptology during World War I. National Security Agency, Center for Cryptologic History, 2023
[o6] Herbert O. Yardley. The American Black Chamber. New York: 1931
[o7] John Dooley, “Was Herbert O. Yardley a Traitor?”, Cryptologia 35, no. 1 (January 2011): 1–15