Skeletons in the Closet
According to the Merriam-Webster Dictionary, the idiom "skeletons in the closet" means "something bad or embarassing that happened in someone's past and that is kept secret..." In the history of cryptography, there are not just many such skeletons, but a great many. However, it is beneficial to clean out these closets (both wardrobes and toilets) from time to time, simply for the sake of health.
The subject for this mini-investigation arose quite spontaneously during the analysis of materials for a much larger series about the Baconian-Shakespearean mysteries. The current story doesn't really concern Bacon-Shakespeare matters at all, but it vividly and instructively demonstrates the side effects of internet censorship [i1]. Or, in other words, the "Master and Margarita effect" [i2], which clearly manifests exactly what the authorities would very much like to hide deeper.
Stories about the role and place of cryptography in real life are inevitably intertwined with the secrets of espionage in international relations. One way or another, everyone here tries to spy on each other, but some do it much better than others. Advantages can be achieved by very different means, sometimes clean [i3], sometimes dirty or even very dirty [i4]. And, naturally, it is common and customary to hide unclean deeds, thus giving rise to numerous "mysteries of history."
When you start to look more closely at what exactly they are trying to hide from the people, an amazing picture unfolds. It's a picture of how, from different sides, several researchers manage to independently approach the answers to historical puzzles... And how they all immediately bounce away from the already clearly identified approaches while simultaneously trying to cover up the traces of their searches.
This study will not attempt to understand WHY scientists do this. But it will demonstrate through specific examples HOW it is done, by whom it is done, and what ultimately science (does not) gain from this, making this mini-investigation necessary and useful.
The problem of obtaining additional information arose unexpectedly and, as they say, out of nowhere. It all started [i5] while delving into primary sources in the remarkable for its findings article by Professor John Dooley "Was Herbert O. Yardley a Traitor?" [o1].
In one of the key sections of this article, the outcomes of analyses by two different researchers who studied the same source documents are compared. Namely, a fragment from Robin Denniston's work "Yardley’s Diplomatic Secrets," from 1994 [o2] is first quoted, where the author describes "Yardley's betrayal" as a fact firmly established, immediately followed by counterarguments from a response letter by another very knowledgeable historian of cryptography, Louis Kruh (1923-2010), well acquainted with the documents at the foundation of this entire mysterious story:
Unfortunately, Denniston’s [the main new] source is wrong. Louis Kruh, in a Letter to the Editor in the journal Cryptologia in 1995 [o3] demolishes the [main source, anonymous] Surveillant article and hence Denniston’s claim...
When researchers studying the same documents come to diametrically opposite conclusions, it's natural to wonder about the specifics of the facts and evidence these historians relied on. In other words, the task arose to obtain two Cryptologia journal publications for more thorough analysis: Denniston's article [o2] and Kruh's letter [o3].
And here, one after another, surprises began to emerge in succession. The so-called "Surprises of Absence"...
Surprise #1: From the Cryptologia Journal Website
Obtaining Denniston's article presented absolutely no problems. However, acquiring Louis Kruh's letter turned out to be very interesting.
The CRYPTOLOGIA, the most respected journal on the history of cryptography, has a quite substantive website with a digital archive of all issues and publications. Or, more precisely, almost all. Because when you visit the page for Issue 4 of 1995, where the letter from Louis Kruh we need was published on pages 377-379, you'll find that these specific pages, for some reason, have not been digitized. That is, they are simply not present neither in the issue's table of contents nor on the internet…
The CRYPTOLOGIA journal, however, has been published in print since 1977 up to the present day. Therefore, a kind people, who know how to do so, managed to find the required issue of the journal and scan the hidden pages [o11], for which many thanks.
For the sake of brevity, there won't be a recount of the arguments and quotes from Louis Kruh with which he proves the fabrication of the accusations against Yardley, since these things are now well known (thanks to extensive researches in the 2000s). The discussion here will focus on something else. On the document and circumstances that even now usually don't figure in discussions about Yardley (key points can be conveniently highlighted in bold font).
Denniston attacked Yardley with phrases such as: "[a] reprihensible betrayer of secrets," "his treacherous agreement with Japanese," "a mercenary and a traitor" and much more. His extravagant tirade includes more than a dozen slanderous comments.
But where is the evidence that jusifies Denniston's diatribe?
... In [his book] "The American Black Chamber," Yardley mentions that his office was broken into after an incompetent attempt [of a foreign state] to seduce him.
Some support for this view comes in a June 6, 1931, letter from Yardley to Frederick Sullens, editor, Jackson [Mississippi] Daily News. In his letter, Yardley tells how the American Black Chamber was forced to move to another location "when our files were rifled by the secret agent of a foreign government" (National Archives, SRH-038, p. 154).
In the same letter, Yardley's penultimate paragraph could rebut Farago's [book "Broken Seal"] original charge:
"When the American Black Chamber was closed, should I have desired to continue in my profession my only employer could have been a foreign government.
One of the great powers, learning through their secret agents of the abandonment of cryptography in the United States, approached me with a view to my creating such a bureau and training their subjects in the science of cryptography.
The United States Government paid me $7,500 per annum. This foreign power offered twice this amount and expenses for myself and family.
Although I have felt no hesitancy in revealing the secrets of the American Black Chamber, I did not feel that I could accept such a position for my knowledge would have been turned against my native country in the reading of her diplomatic secrets."
The excerpt from Yardley's correspondence presented here, or rather, his entire letter from June 6, 1931, turns out to have fundamentally important significance for the secret history of the intelligence services. It cannot be said to be proof, but at the very least, a clear demonstration of the special significance of this document are all the subsequent surprises that accompanied this investigation.
Surprise #2: From Cryptography Historian D. Kahn
In 2004, David Kahn, the most famous among authoritative specialists in the history of cryptography, published the first and undoubtedly very substantial biography of Herbert Osborne Yardley: "The Reader of Gentlemen's Mail" [o4]. Some of his thrilled colleagues, already confident in the exhaustive completeness and quality of the work, immediately called this book the "first and last" of Yardley's biographies. [o5]
Fans' excitement is never objective. But in any case, when suddenly little-known information about an attempt to entice the United States' chief cryptanalyst by some other "great power" is discovered, it's natural to wonder what the particularly knowledgeable David Kahn dug up on this matter. Simply flipping through the pages of his "exhaustive biography of Yardley," however, provides no answers to this question. Because such a spicy episode is simply not present in the main text of the book...
After a slightly more careful study, a mention of this episode is indeed found - among the notes to the main text. And this note looks quite strange, to put it mildly:
Main text, p. 105:
[Having lost his job as a cryptanalyst and finding nothing worthy in exchange, ] Yardley, broke and desperate, turned to his main marketable asset: his secret knowledge.
Note at the end of the book to p. 105:
...his secret knowledge: Yardley claimed that "One of the great powers learning through their secret agents of the abandonment of cryptography in the United States, approached me with a view to my creating such a bureau and training their subjects in the science of cryptography."
He said he was offered twice his American salary of $7,500 plus expenses for his family and himself. "Though I have felt no hesitancy in revealing the secrets of the American Black Chamber, I did not feel that I could accept such a position for my knowledge would have been turned against my native country" (B-M, 6 June 1931).
I don't believe this story. Somebody might have approached him in a general way, but, in view of the fact that Yardley was broke and that several years later he did work as a cryptanalyst for two other countries, I think that if such an offer had been made and he felt that it would not harm the United States, he would have taken it.
The bolded fragments show that there's something significantly amiss with the logic and sources of the famous historian.
Firstly, in his note, the author of the book immediately accuses his hero of lying, without having any proof besides his own conjectures.
Secondly, Yardley's letter clearly speaks about his feelings that his cryptanalytic knowledge in the position offered to him by "one of the great powers" would have been "turned against my native country". Yet, David Kahn starts to speculate on what Yardley would've done if his feelings had been different.
Thirdly, and this is especially important here, David Kahn not only fails to provide information about the document from which Yardley's words are quoted (the letter to Frederick Sullens, editor of the Jackson Daily News in Mississippi, in the southern United States) but also gives a very vague reference to the actual source of the document (B-M, 6 June 1931).
This reference – "B-M" – in D. Kahn's book stands for "Indiana Historical Society, Bobbs-Merrill file on Yardley" meaning Yardley's archive from Bobbs-Merrill publishers in the collections of the Indiana Historical Society, a state in the Northwest USA. This archive contains about a dozen folders of documents related to Herbert Yardley's correspondence with Bobbs-Merrill, which published his book The American Black Chamber. But it's quite odd to look for Yardley's letter, sent to a newspaper at the other end of the large country, there.
Especially since in previous publications by historians about Yardley, the exact location of this document was indicated quite differently: National Archives. College Park, Md. Record Group 457. National Security Agency. SRH-038. A Selection of Papers Pertaining to Herbert O. Yardley.
David Kahn in his book about Yardley makes extensive use of this archival source SRH-038 (Special Research History, file 38). Twelve times before page 105 and another four times after. But precisely where he should have referred to the document SRH-038, p. 154 (as Kruh did), Kahn for some reason gives a vague reference completely off the mark...
Surprise #3: From Professor John Dooley
In 2023, the cryptography historian we already know, John Dooley, who prompted all these meticulous investigations to begin, released his own detailed monograph on Yardley, his colleague-rival W. Friedman, and the "birth of modern American cryptology" [o6].
This book, naturally, also makes intensive use of materials from NSA archives. In particular, documents from the Yardley file SRH-038 are cited almost 30 times. However, interestingly, the especially important document for us (SRH-038, p. 154) is also not mentioned EVEN ONCE by Dooley.
Moreover, unlike David Kahn, Professor Dooley in his book chose not to touch on this sensitive topic at all, neither in the text nor in the notes. The important letter from Yardley to Sullens simply doesn't exist here for the history.
Since John Dooley also completely excised any mentions of William Friedman's betrayal (which he himself had revealed a few years earlier [o1][i5]), it leads to the conclusion that this is not a coincidence. It is yet another attempt to construct such a carefully whittled down and polished version of the birth of modern American cryptology, which looks like "a lady pleasant in all respects"...
Surprise #4: From NSA Historians
Finally, since the missing information should be sought in all sources, not just the easily accessible ones, it's prudent to seek additional information from declassified NSA documents. Although readers from Russia are now blocked from accessing the NSA website, there are workarounds.
In short, after some searching on the official site http://www.nsa.gov, it was indeed possible to find a relevant document – from the CCH or the Center for Cryptologic History at the NSA. For some time, historians from this institution have been publishing (or had published?) a news feed called History Today in the NSA's internal network.
And so, as part of this information initiative, a small page was published on September 28, 2018, with a title along the lines of: "Herbert Yardley – Has the Story Changed?" [o7]. And although this document is quite small, it raises big and serious questions.
The most curious moments, as before, are highlighted in bold.
The CCH, with assistance from the NSA Archives, recently came across some comments Yardley made about the book [The American Black Chamber] as it was coming out.
In a June 6, 1931 message to Lincoln Foster of The Living Age magazine *, Yardley called it “inconceivable” that US officials would ever deny that his organization had existed (the government, not surprisingly, claimed that there never was a Black Chamber).
...
The next part of his letter to Foster is somewhat of a head scratcher. In writing his book, he “felt no hesitancy” in revealing the secrets of his organization to the outside world. Yet, he also revealed that he was offered twice as much money as the government paid him to work for a foreign power to run its own black chamber and train its personnel.***
* This is probably Littell’s Living Age, a weekly magazine.
*** CCH had never heard that Yardley was offered a job by another power at this time, and can only speculate about the truth of the claim.
So, what do we see here?
We see how NSA's own historians (albeit anonymously) literally create on the fly not just a trimmed and polished version of history, but essentially an entirely different, fundamentally fabricated story.
The real recipient of the letter, Frederick Sullens, here becomes Lincoln Foster, and the Jackson Daily News, which Sullens headed, becomes the weekly Living Age…
How Should We Understand All This?
Reflecting on the discovered array of surprises, it's probably not too hard to deduce that the trick here is staged not only for NSA employees interested in history but also for anyone else who wished to find additional information on this matter. Following such an initial presentation of material by contemporary historians, all seekers are likely to face not just difficult, but most likely fruitless searches.
Because not only the agency's own scholars, but also the main sources of information about the life and affairs of Yardley from so-called "independent" researchers, Kahn's and Dooley's books, will absolutely not help in locating the necessary document in the NSA archives. And the material that provides a genuinely informative link in the CRYPTOLOGIA journal is completely absent from the journal's website, not only in the texts but also in the table of contents.
But why exactly is this document [o8] so diligently and unanimously hidden by historians now?
A comprehensive answer to this question, as promised in advance, will not be provided here. Simply because there are no reliable documents on this matter yet.
However, there are plenty of documents that detailedly and convincingly show how "one great power," known as Great Britain, became terribly displeased with Herbert Osborne Yardley following the publication of his book "The American Black Chamber."
Two sources turn out to be particularly informative on this matter, one English and one American.
The English source is the book by the already mentioned cryptography historian Robin Denniston "Thirty Secret Years: AG Denniston's Work in Signals Intelligence, 1914-1944" [o9]. The coincidence of the author's and the main character's last names in this biographical book is far from accidental, of course. Because Robin Denniston is the son of the legendary Alastair Denniston, who led Britain's cryptographic intelligence service for nearly three decades.
R. Denniston's book about his father is largely based on personal diaries and family memoirs of the intelligence officer. It contains nothing about the British attempt to recruit Herbert Yardley before his book was published in 1931. However, it does provide sufficient information on how British cryptographers read the encrypted correspondence of "friendly" USA during the interwar period. It also details how Alastair Denniston personally flew to Canada in 1941 to ensure Canadian authorities immediately expelled Yardley, who had just begun creating a national cryptologic service and training staff there.
The American source is a very informative and richly documented 1989 article by another well-known to us historian, Louis Kruh, "Tales of Yardley" [o10] (adding particular value to this source is the fact that it's not mentioned even once in John Dooley's latest book on Yardley [o6]). In the condensed format of a journal article, L. Kruh managed to include a great deal of detail about this truly extraordinary person, cryptographer, and organizer, as well as about the plethora of problems Yardley created for the intelligence services of the USA, Great Britain, and Canada.
Specifically, about how the American cryptologic community was thrown into confusion when, after the Black Chamber was shut down, Yardley candidly described in his book how they successfully read the encrypted correspondence of "friendly" Great Britain. About how quickly and successfully Yardley organized the cryptologic agency for Canada, how Canadian authorities squirmed and lied when expelling Yardley at the British's demand, and how Canadian cryptographers rebelled against this, having appreciated the professionalism of their boss. About how long and hard the FBI's counterintelligence searched for compromising material on Yardley but found nothing...
In short, the rich archives of cryptography history contain a wealth of information that significantly contradicts the false, but "ideologically correct," picture that contemporary historians are now trying to construct. Herein, likely, lies the persistent desire of authorities to hide not just one of Yardley's important letters but the entire NSA archival file SRH-038 with Yardley's dossier, which was declassified in 1980 but only now can be found in our collection, as well as on the Internet Archive.
In conclusion to this mini-investigation, it only remains to clarify why all this was done. The essence of the venture lies in the signal. An important signal that, in reality, in almost any modern scientific field today, it is precisely such an — essentially false, but "ideologically correct" — picture of the state that dominates.
If one were to scratch just a little deeper and more attentively at each of these pictures — as in the case of the history of cryptography analyzed here — literally everywhere, "skeletons in the closet" are uncovered. A lot of skeletons. In archaeology and anthropology [i6], in economics and history [i7], in biology and medicine [i8], in psychology and philosophy [i9], in literary and theater studies [i10], in fundamental physics and mathematics [i11].
Everywhere, in short.
This means that for new generations of young researchers, there is definitely a gigantic field of not only unknown but also long-explored, yet still unutilized knowledge and things. And these things are very interesting. It just takes removing the skeletons...
Additional Reading
[i1] Scientific Censorship (rus.)
[i2] The "Master and Margarita" Effect, or Files Don’t Burn (rus.)
[i3] The OSINT Model (rus.)
[i4] Total Hagelin, or Finita la Commedia (rus.)
[i5] Aristotle at the Trojan War and "Traitorous founders" in the NSA Hall of Honor
[i6] Pseudo-Archaeology (rus.); A New History of Humanity (rus.); The Guardian of Antiquities and The Great Tomography of Egyptology (rus.)
[i7] OSINT as a Metaphor (rus.); As if a New Law of Nature (rus.)
[i8] Deciphering Codes and Meanings of Bioelectricity (rus.); The Secret Life and Intelligence of Plants (rus.); Life and Death in the Paths of Science (rus.)
[i9] Deception as the Basis of the State, Self-Deception as the Basis of Worldview (rus.); Mushrooms, Entangled World, and Distributed Intelligence (rus.); CIA, the Mystery of the 25th Page, and the Universe as a Hologram (rus.)
[i10] Undermining the authorities' credibility and Shakespeare as "Our Everything"
[i11] The Mythology of the Big Bang and Alfvén's Cosmology (rus.); The Nature of Self-Deception in the Exact Science (rus.); Ostrogradsky – Voevodsky, or What Is Not Discussed (rus.); The Wonders of Floquet Engineering, or What Is Not Seen, Not Heard, and Not Discussed (rus.)
Main Sources
[o1] John Dooley, “Was Herbert O. Yardley a Traitor?”, Cryptologia 35, no. 1 (January 2011): 1–15. DOI: 10.1080/01611194.2010.533254
[o2] Robin Denniston (1994): Yardley’s Diplomatic Secrets, Cryptologia, 18:2, 81-127 DOI: 10.1080/0161-119491882784
[o3] Louis Kruh (1995): Letter to the Editor. Cryptologia, 19(4):377–379.
[o4] David Kahn. The Reader of Gentlemen’s Mail: Herbert O. Yardley and the Birth of American Codebreaking. New Haven: Yale University Press, 2004
[o5] "Until now, Yardley has never had a biographer. Thanks to David Kahn, he will never need another." Quotation from: Robert Angevine. (Book Reviews) The Reader of Gentlemen’s Mail: Herbert O. Yardley and the Birth of American Codebreaking by David Kahn. The Journal of Military History, January 2005, pp.262-263. DOI: 10.2307/3397085
[o6] John F. Dooley. “The Gambler and the Scholars: Herbert Yardley, William & Elizebeth Friedman, and the Birth of Modern American Cryptology”. Springer, 2023
[o7] History Today: Herbert Yardley — Has The Story Changed? NSA Center for Cryptologic History, 2018 September 28
[o8] SRH-038. A Selection of Papers Pertaining to Herbert O. Yardley (Dates: 1918-1950; Total Pages: 192), pp 154-155. National Archives. College Park, Md. Record Group 457. National Security Agency.
[o9] Robin Denniston. Thirty Secret Years: A. G. Denniston’s work in signals intelligence, 1914 — 1944. Clifton-upon-Teme: Polperro Heritage Press, 2007
[o10] Louis Kruh (1989) Tales Of Yardley: Some Sidelights To His Career. Cryptologia, 13:4, 327-358, DOI: 10.1080/0161-118991863998
[o11] Letters to the Editor. Cryptologia 19:4, 375-379