Wikileaks today announced the launch of the Public Library of US Diplomacy (PlusD), a searchable database with the release of Special Project K: the Kissinger cables -- ostensibly, PlusD will include other records in the future. WikiLeaks has published more than 1.7 million U.S. diplomatic records -- including cables from previously released Cablegate cables, intelligence reports, and congressional correspondence -- from January 1, 1973 to December 31, 1976, the period during which Henry Kissinger was secretary of state and national security advisor. The documents were formerly confidential, classified, or labeled "NODIS" ("no distribution") or "Eyes Only". The database can be accessed at http://search.wikileaks.org/plusd/.

According to Wikileaks:

...Most of the records were reviewed by the United States Department of State's systematic 25-year declassification process. At review, the records were assessed and either declassified or kept classified with some or all of the metadata records declassified. Both sets of records were then subject to an additional review by the National Archives and Records Administration (NARA). Once believed to be releasable, they were placed as individual PDFs at the National Archives as part of their Central Foreign Policy Files collection. Despite the review process supposedly assessing documents after 25 years there are no diplomatic records later than 1976. The formal declassification and review process of these extremely valuable historical documents is therefore currently running 12 years late.

According to the Guardian:

The data, which has not been leaked, comprises diplomatic records from the beginning of 1973 to the end of 1976, covering a variety of diplomatic traffic including cables, intelligence reports and congressional correspondence.

Julian Assange said WikiLeaks had been working for the past year to analyse and assess a vast amount of data held at the US national archives before releasing it in a searchable form.

WikiLeaks has called the collection the Public Library of US Diplomacy (PlusD), describing it as the world's largest searchable collection of US confidential, or formerly confidential, diplomatic communications.

Assange told Press Association the information showed the vast range and scope of US diplomatic and intelligence activity around the world.

Henry Kissinger was US secretary of state and national security adviser during the period covered by the collection, and many of the reports were written by him or were sent to him. Thousands of the documents are marked NODIS (no distribution) or Eyes Only, as well as cables originally classed as secret or confidential.

Assange said WikiLeaks had undertaken a detailed analysis of the communications, adding that the information eclipsed Cablegate, a set of more than 250,000 US diplomatic cables published by WikiLeaks from November 2010 and over the following year. He said WikiLeaks had developed sophisticated technical systems to deal with complex and voluminous data.

Top secret documents were not available, while some others were lost or irreversibly corrupted for periods including December 1975 and March and June 1976, said Assange.

There's a new digital archive in town, from the Wilson Center's Cold War International History Project and its new Digital Archive of declassified official documents called www.digitalarchive.org.

Digital collections include: the Berlin Wall, Chinese nuclear history, Cuban foreign relations, Geneva Conference of 1954, Mitrokhin archive, and much more.

From the Wilson Center Web Site:

The Wilson Center [recently] launched a new Digital Archive of declassified official documents from nearly 100 different archives in dozens of different countries that provide fresh, unprecedented insights into the history of international relations and diplomacy.
[Clip]
The new website – www.digitalarchive.org – features uniquely powerful new search tools, an intuitive user-interface, and new educational resources such as timelines, analysis from leading experts, and biographies of significant historical figures. The Digital Archive will continually expand with new documents, translations, and analysis as they become available.
[Clip]
The new Digital Archive has been designed from the ground-up to make these historical document collections available to the broadest possible audience, from high school students through world-renowned scholars. Thousands of official documents from dozens of governments are now accessible through intuitive searching with filters such as location, date, subject, or language. Users can also browse topics by exploring themes or collections like the Database on Inter-Korea Relations and popular subjects such as the Warsaw Pact or the Cuban Missile Crisis.

[HT to InfoDocket!]

This is the first in a series of guest posts from the National Security Archive, a non-governmental research organization, archive and publisher of declassified documents. Thanks to James Jacobs for the invitation to write about the Archive. Over the next 3-4 posts, I’ll describe the organization and some of our projects, starting today with a bit of history about our founding and mission in life. Meanwhile, we invite any and all to visit us at www.nsarchive.org or in person at Gelman Library on the campus of George Washington University.

* * * * *

For anyone who loves anniversaries, this month is a biggie. Fifty years ago the world survived one of the seminal events of the nuclear age -- the Cuban missile crisis. I mention it because almost from the very start, the National Security Archive’s been an active promoter of studying the crisis (we’ll have a series of postings of the latest findings on our site in the coming weeks), and it makes for a good case study of what our organization’s mission is and how we go about our work.

(Today I’ll touch on our substantive projects; later posts will deal with other Archive activities.)

In 1985, the Archive officially opened its doors to the public as an innovative non-profit research institute and library facility focused on making available the underlying government documentation all of us as citizens need to understand what our elected officials (and permanent bureaucracies!) are doing in our name. That basic mission reflected the mix of individuals who over time in the early 1980s coalesced around the idea of forming such an organization -- journalists (Scott Armstrong, Washington Post; Raymond Bonner, New York Times; Strobe Talbott, Time; and others), scholars (John Lewis Gaddis, Catherine Kelleher, Ernest May, Anne Cahn, John Prados, among others), public interest group leaders (John Shattuck, Mort Halperin, Margaret Carroll, and others), former officials (Anthony Lake, Walt Slocombe, Joseph Onek, etc.), and even current members of Congress like Jim Moody (D-WI). Each in their own profession had an interest in following how the government worked, and each understood the power of the historical record in educating the public.

Our main tool for breaking loose documentation was the Freedom of Information Act (FOIA) -- passed in 1966 and later amended -- which in principle grants anyone access to the inner workings of the Executive Branch. (States have their own version of the law.) Building on donations by Armstrong, Bonner and many others, the Archive began to develop a large repository of declassified records, which has always been open to researchers at our facility (originally at the Brookings Institution, but since 1995 at GWU). Right now, our holdings total somewhere around 8 million pages and run the gamut of foreign policy topics covering the Cold War and its aftermath.

How do we organize our work? Here’s where the Cuban missile crisis comes in. Generally, our projects center around the efforts of a staff analyst whose job it is to become an expert not only in the history of a given subject but, critically, in the way that subject was handled by the U.S. government. Since accumulating the key paperwork underlying U.S. policy is our goal, you have to know how it flowed within the halls of government, especially when it would be utterly impractical (not to say impossible) to expect to get your hands on more than a tiny fraction of the millions of pages federal agencies produce on these kinds of topics in the course of their activities.

In the Cuba case, we had a couple of analysts who filed FOIAs for critical material, such as Kennedy’s correspondence with Khrushchev, CIA studies, and so on. As you might expect, they initially got stonewalled from time to time on much of the more sensitive items, and had to file appeals, as provided for under the Act. Part of the problem was that agencies claimed they simply couldn’t locate the files. That seemed hard to imagine, given the high level of involvement (the president and his men) during the crisis. It was only after an active round of additional research and interviews with a range of outside experts including former officials that it began to become clear that much of the most important material had been removed from its normal archival locations on orders of Lyndon Johnson, who decided that if another similar crisis erupted one day it would be a good idea to have the record available so that future presidents and officials could study the Cuban crisis as an example. From there it took a bit more effort to determine that the materials had been stored away in a special office inside the State Department, but once my colleagues were able to provide that information (down to the level of box titles) to the Department, it removed a major obstacle to getting the documents out into the public domain.

But a second hurdle remained -- one that anyone who’s ever researched recent foreign policy or intelligence records knows all too well: how to overcome restrictions on access to classified information? This is something our analysts deal with every day. Protecting reasonably classified information, especially if it concerns American foreign relations, is one of several legal justifications under FOIA for withholding records from the public. Sometimes the official reasoning is entirely understandable -- for instance if it relates to the specifics of manufacturing a nuclear bomb, or to planned troop deployments. But bureaucrats and politicians being who they are, it won’t come as a surprise to anyone that often the rationales employed to keep our history hidden are either flimsy or entirely bogus. Scholars and journalists confront these problems all the time, and it frequently takes persistent effort to argue one’s case through the appeals process to a point where an agency will reverse its earlier denial. This was another reason for creating the Archive -- to have an institution in place that could afford to wait the months, years and -- yes -- even decades it sometimes takes for requests to be fulfilled. (We’ve had numerous requests take 10, 15, 20 or even more years to be completed.)

When it came to arguing our case on the missile crisis, we eventually were able to take advantage of some of the monumental political changes that had taken place around the collapse of the Soviet Union and the (putative) end of the Cold War. As former Soviet archives began to open a crack, and as Boris Yeltsin learned the power of documents as a weapon to expose the misdeeds of his former Communist Party cronies, some astonishing materials from “the other side” of the Cold War started to emerge. Among these were records of leadership meetings, correspondence and transcripts of conversations with foreign heads of state. These included some dealing with what the Soviets used to call “the Caribbean crisis.” Since (a) our main global enemy no longer existed, and (b) its successor regime was releasing some of the same kinds of materials we and other researchers were seeking from the U.S., it was possible argue with U.S. agencies that there was no longer a need to protect once-sensitive documentation about the inner debates of the ExComm, photos from U-2 flights over Cuba, and reams of intelligence about events that, after all, had occurred 30 years earlier.

Through sheer persistence and a little creative thinking, the Archive was able to accumulate an extraordinary record of the crisis that has since become part of the larger public record amassed by scholars and journalists around the world.

Yet, one of the lessons one learns fairly soon in this line of work is that government records -- surprise -- do not tell the whole story! A document is only as accurate and reliable as its author, who may or may not have had access to good information, and may have been influenced by motives we can sometimes only guess at. How then to get past this basic historiographical challenge? By asking the people who wrote or received those documents!

But here we were able to go about that task with a twist. It was our tremendously good fortune in the course of our work on the missile crisis to come into contact, and quickly partner with, an unusual husband-and-wife team -- James G. Blight and janet [sic] M. Lang, then of Harvard’s Kennedy School (now at the Balsillie School of International Affairs in Ontario, Canada). Neither of these two were originally trained historians (he is a psychologist, she is an epidemiologist!), but together they came up with a methodology for studying recent historical events that has produced some extraordinarily important results. The approach is called Critical Oral History and it’s complex enough to have had books written about it, so I’ll only give a thumbnail sketch here. It involves bringing to the table (literally) a group of individuals (often former antagonists) who participated in the events under scrutiny, adding to the mix a small contingent of scholars who know the literature, and underpinning the exercise with declassified documents from the time. The “veterans” are there to discuss among themselves how they viewed the events at the time, what they sought to achieve, what they believed their adversaries were up to, and so on. The scholars and documents are there to refresh memories and keep the discussion anchored in the facts insofar as that’s possible.

This was the methodology Jim and janet -- usually with the Archive’s help on the document front (and at times in other ways) -- used to explore the missile crisis at a depth and level of detail not otherwise attainable before then. Starting in 1987 and running initially through 1992, they organized a series of conference in Cambridge, Mass.; the Caribbean; Moscow; and Havana itself, at which the participants included Robert McNamara, Arthur Schlesinger Jr., Ted Sorenson and additional U.S. luminaries from the period; Foreign Minister Andrei Gromyko, his deputy Georgi Kornienko, long-time Ambassador to Washington Anatoly Dobrynin, and various other Soviet diplomats, generals and intelligence experts; and, from Cuba, Fidel Castro plus several of his senior advisers. As a principal part of the Archive’s contribution, we were able to gather a pretty rich array of declassified records not just from the United States but from literally all over the world -- Russia, the Soviet bloc, Cuba, China, Brazil, and elsewhere. As a result of these accumulated resources (human and documentary), the sessions produced immediate headlines, and in the longer term fundamentally changed the way the missile crisis is understood -- literally rewriting the text books.

Here are a few of the revelations:

- The U.S. did not know at the time that tactical nuclear weapons were on the island that might well have been used in the event of an American invasion, almost certainly touching off a nuclear war
- Washington had no idea that 43,000 Soviet troops and thousands more civilians were in Cuba by late October 1962, numbers that significantly raised the likelihood of a major retaliation in case of a U.S. invasion
- The Americans were unaware of the importance of Castro’s role and the pressure he put on Soviet leaders to sharpen their responses to U.S. actions, including emotionally advising Khrushchev at the height of the crisis that if the U.S. invaded the island Moscow should deploy its missiles before the Americans had the chance to use theirs
- Soviet subs around the quarantine line carried nuclear-tipped torpedoes that one captain tried to launch but failed when he couldn’t persuade other key holders on board to go along
- A major cause of the Soviet captain’s motivation was the fear of being under attack after the U.S. Navy, unaware of the nuclear-tipped torpedoes, began harassing the subs by tossing the equivalent of grenades onto them
- Ultimately, the crisis was not resolved by dint of nuclear superiority and boldly staring down the adversary -- which was the original accepted wisdom; it involved a willingness to make a deal, to compromise, as Kennedy did in secretly offering a trade of the Cuban missiles for the Jupiter missiles in Turkey

To make revelations like these broadly available, the Archive does a number of things. First, we make our materials accessible to researchers at no charge (except photocopying) at our GWU facility. Another avenue is through a subscription product of highly curated selections -- the Digital National Security Archive (DNSA) -- via the publisher ProQuest. We also publish shorter compilations of materials as books. And of course we post selected items on our Web site (www.nsarchive.org). We currently have 391 “Electronic Briefing Books” of documents on newsworthy topics on our site.

(I should clarify -- and emphasize -- that all these publications contain no editorializing from us on U.S. government policy. We’re a non-partisan 501(c)(3) group and the only issues we take a stand on are freedom of information and the principle of open government access.)

The Cuban missile crisis project in many ways became a model for our other historical documentation projects at the National Security Archive, including studies of U.S. policy toward the Vietnam War, the Soviet Union and the superpower rivalry, a series of crises in Eastern Europe during the Cold War, and more. Most of the basic stories of these projects are available on our site and, as mentioned, the underlying documentation we and our partners and colleagues around the world have collected is also available here in Washington, D.C.

Next week I’ll talk (maybe at a bit shorter length!) about our freedom of information activities. If you have any questions as we go along, don’t hesitate to write to me at mbyrne@gwu.edu.

Malcolm Byrne
Deputy Director, Research Director
The National Security Archive
at George Washington University

Kel McClanahan of National Security Archive submitted to the Central Intelligence Agency a Freedom of Information Act request for "the CIA's copy of its new regulation 32 C.F.R. 1908." This is a public document: a regulation in the Code of Federal Regulations. It is available, for example, here: 32 CFR 1908.

The CIA even has a copy of 32 CFR on its own website (though evidently not the current version!):

http://www.foia.cia.gov/32CFR.asp
http://www.foia.cia.gov/txt/32CFR.pdf

But the CIA responded to this request this way:

We did not locate any records responsive to your request... our searches were thorough and diligent, and it is highly unlikely that repeating those searches would change the result...

Read the story here:

• The CIA Cannot Find Their Own Regulations about Declassification, by Nate Jones, Unredacted, The National Security Archive (June 8, 2012).

Pentagon Papers to be Officially Released, by Steven Aftergood, Secrecy News (May 11, 2011).

The National Archives announced this week that it “has identified, inventoried, and prepared for public access the Vietnam Task Force study, United States-Vietnam Relations 1945-1967, informally known as ‘the Pentagon Papers’.” As a result, 3.7 cubic feet of previously restricted textual materials will be made officially available at the Nixon Library on June 13, the Archives said in a May 10 Federal Register notice.

Nixon Presidential Historical Materials: Opening of Materials Federal Register Volume 76, Issue 90 (76 FR 27092) (May 10, 2011).

Previously restricted textual materials. Volume: 3.7 cubic feet. A number of textual materials previously withheld from public access have been reviewed for release and/or declassified under the systematic declassification review provisions and under the mandatory review provisions of Executive Order 13526, or in accordance with 36 CFR 1275.56 (Public Access regulations). The materials are from National Security Council (NSC Files), Presidential Acquisition Files, Pentagon Papers.

For all you (de)classification geeks out there, here's an interesting new .gov blog to add to your blogrolls. NARA's Public Interest Declassification Board now has a blog called Transforming Classification. They'll be posting a bunch of white papers on various topics over the coming months. I'm looking forward to the conversation.

The Public Interest Declassification Board is an advisory board established by Congress to promote the fullest possible public access to a thorough, accurate, and reliable documentary record of significant U.S. national security decisions and activities. The Board’s mandate includes advising the President and other government officials on policies deriving from the issuance by the President of Executive orders regarding the classification and declassification of national security information...

...President Obama has charged the Board with designing a more fundamental transformation of the security classification system. In response to his request, we are proposing new solutions that address the shortcomings of the current system and tackle the challenges of digital records. By reducing inefficiencies and increasing public access, our proposals aim to improve the classification/declassification’s system capacity to protect and serve the American people.

Every other Wednesday over the next eight weeks, we will post either two or three “white paper” synopses to the blog describing an element of our proposed transformation.

[HT to Meredith Stewart]

So the CIA just got around to declassifying 6 of the U.S.'s oldest classified documents from WWI (1917 + 1918). They've posted them in their CIA FOIA reading room and the CIA Records Search Tool (CREST) at the National Archives (but to use CREST, a researcher must physically be present at the National Archives, College Park, Maryland :-|). That also means that the documents will also soon be available at the archive-it FOIA collection (I'm harvesting them as we speak ;-)).

These documents, which describe secret writing techniques and are housed at the National Archives, are believed to be the only remaining classified documents from the World War I era. Documents describing secret writing fall under the CIA's purview to declassify.

"These documents remained classified for nearly a century until recent advancements in technology made it possible to release them," CIA Director Leon E. Panetta said. "When historical information is no longer sensitive, we take seriously our responsibility to share it with the American people."

One document outlines the chemicals and techniques necessary for developing certain types of secret writing ink and a method for opening sealed letters without detection. Another memorandum dated June 14, 1918 - written in French - reveals the formula used for German secret ink.

"The CIA recognizes the importance of opening these historical documents to the public," said Joseph Lambert, the Agency's Director of Information Management Services. "In fiscal year 2010 alone, the Agency declassified and released over 1.1 million pages of documents."These documents, which describe secret writing techniques and are housed at the National Archives, are believed to be the only remaining classified documents from the World War I era. Documents describing secret writing fall under the CIA's purview to declassify.

"These documents remained classified for nearly a century until recent advancements in technology made it possible to release them," CIA Director Leon E. Panetta said. "When historical information is no longer sensitive, we take seriously our responsibility to share it with the American people."

One document outlines the chemicals and techniques necessary for developing certain types of secret writing ink and a method for opening sealed letters without detection. Another memorandum dated June 14, 1918 - written in French - reveals the formula used for German secret ink.

"The CIA recognizes the importance of opening these historical documents to the public," said Joseph Lambert, the Agency's Director of Information Management Services. "In fiscal year 2010 alone, the Agency declassified and released over 1.1 million pages of documents."

Declassified CIA documents (all pdf):

• Document 1
• Document 2
• Document 3
• Document 4 -- This is the one with the steps for checking for invisible ink.
• Document 5
• Document 6

This was such cool news that Rachel Maddow went gaga over the news!

[HT to Gary Price for posting CIA Declassifies Oldest Documents in U.S. Government Collection (1917 + 1918), View Them Online on InfoDocket. Thanks Gary!]

Steven Aftergood points to an article by an historian who says that the Pentagon Papers are still classified as Top Secret:

You might be dismayed to learn that the Pentagon Papers are still classified as TOP SECRET!

This is despite the fact that The Pentagon Papers have long been in the public domain. Indeed, US government historians use them in official accounts of the Vietnam War and they are referenced and republished in official US government records, such as Foreign Relations of the United States. Senator Mike Gravel even entered them into the Congressional Record!

Aftergood notes that "This means that every public and private library in the country that has a copy of the Papers is technically in possession of currently classified material."

• Twelve Million Pages Opened by Declass Center in 2010, by Steven Aftergood, Secrecy News (January 26th, 2011).
• Can Government Employees Read the Pentagon Papers?, by John Prados, Unredacted (December 14, 2010).

Prados goes on to say:

The classification of the Pentagon Papers takes on an even stranger significance when one considers the federal government's recent pronouncement that "unauthorized disclosures of classified documents (whether in print, on a blog, or on websites) do not alter the documents' classified status or automatically result in declassification of the documents."

This is the reason –in the case of Wikileaks– why the Government has been demanding that US government employees refrain from looking at any of these documents, even if doing so hampers their ability to fulfill their mandates. If this standard holds true, government employees should not be allowed to read (or reference, or cite) the Pentagon papers either.

Yesterday, President Obama issued an executive order on classified national security information that declared that “No information may remain classified indefinitely.” The order is “part of a sweeping overhaul of the executive branch’s system for protecting classified national security information,” which includes overturning Executive Order 13292 of March 25, 2003. That order, put in place by President George W. Bush, allowed the leader of the intelligence community to veto decisions by an interagency panel to declassify information. This order also establishes a new National Declassification Center at the National Archives (sec3.7) which, according to the AP is expected to speed the declassification of “more than 400 million pages of Cold War-era documents” that are currently backlogged.

For more background on the process for putting together this executive order, check out the National Security Archive's Unredacted Blog (also love their Document Friday!)

[Thanks Think Progress!]

Yesterday, the National Security Archive (NSA) released documents that recounted the abuses committed by the Colombian Army. According to the NSA article, these documents describe how the Colombian military officers collaborated with the drug-trafficking paramilitary groups and used death squad tactics to tackle guerrillas.