The Sunlight Foundation announced today a new bill introduced by Congressman Steve Israel (NY-2) called the Public online Information Act (POIA) (read the bill (PDF)). POIA will require that all "public" executive branch documents be permanently available on the Internet at no cost. POIA also creates a:

"special federal advisory committee to coordinate the development of Internet disclosure policies. These policies promote information best practices, including data interoperability standards, and will keep the government up-to-date with new technology. The advisory committee’s 19 members – six appointed by each branch of government, plus one by GSA – are drawn from the public and private sectors and serve as watchdogs, synthesizing the needs of agencies and the public and making recommendations on updating federal law."

While I wholeheartedly support the spirit of POIA -- free permanent internet access to executive branch documents! -- and will definitely be contacting my representative to support its passage, I have 2 concerns that I hope will be discussed by the Sunlight community, the soon-to-be federal advisory committee, libraries and the public:

1) preservation: There was an article in today's NY Times -- "Fending Off Digital Decay, Bit by Bit" -- that highlights the many issues surrounding digital preservation. Just putting something on the Web does not mean that it will be preserved. The GPO has been working on their Federal Digital System (FDsys) since 2004 (and really since 1994 when they started GPOaccess) to deal with the inherent digital issues. Many researchers, librarians, academics, computer programmers etc have been working on these issues pretty much since the 1960s. And the issues are still here today.

So I'd like to see as part of this bill an acknowledgement that online information is expensive to preserve AND that there will be continued funding for research and sustainability of digital archives through the National Digital Information Infrastructure & Preservation Program (NDIIPP). Readers are encouraged to explore the issues here and here.

2) privatization of govt information: The following from the Sunlight announcement caught my eye and concerned me:

Freeing government information from its paper silos provides the private sector with raw material to develop new products and services and gives the public what they need to participate in government as active and informed citizens.

Federal government information is in the public domain. That's a good thing. However, there's a fundamental issue at stake here. One can't have "permanent free public access" to government information where the private sector is involved. The private sector has been involved in giving access to government information for a long time (see LexisNexis, Thomson West, Readex etc). They do it well but they certainly don't do it for free. Libraries and other organizations have paid many millions of dollars to license access to govt information for the communities they serve. Here's more background and context on privatization. For all intents and purposes, these private sector companies take public domain information and privatize it. Any digital govt information accessible on the internet should already be findable, usable and accessible in bulk at minimum.

But there needs to be more. What I'd like to see in this bill and in the discussion after it passes (devil's in the details right?!) is not only a requirement that all govt information is online permanently and for free, but that there be the inclusion of a viral GNU General Public License-like piece of the public domain whereby anything IN the public domain (i.e., govt information) has to STAY IN the public domain. There are plenty of folks (I'm looking at you Sunlight, Govtrack.us, OpenCongress, OpenCRS etc) excited about making govt information more available, more usable and more shareable and this would support their public service.

Please help Sunlight get the word out about POIA and contact your representative and let them know that they should co-sponsor POIA and assure its passage.

C-SPAN has posted their archives online. That's 23 years worth, 160,000 hours - online (almost all of their content). This is extremely cool. Get ready to waste a chunk of time today going through their archive. It should be noted that while all their programming is available, popular programs like Book TV are not embeddable (although you CAN send the link to facebook, twitter etc). Go ahead and browse the committee list for a little vicarious legislating :-)

The C-SPAN Archives records, indexes, and archives all C-SPAN programming for historical, educational, research, and archival uses. Every C-SPAN program aired since 1987, now totaling over 157,000 hours, is contained in the C-SPAN Archives and immediately accessible through the database and electronic archival systems developed and maintained by the C-SPAN Archives.

[HT to Paul Blumenthal (@PaulBlu) at Sunlight Foundation!]

Calling all 21st century librarians: the fine folks at Citability and the League of Technical Voters Project are organizing a weekend code-a-thon in Washington DC April 9th - 11th. The goal is to create open source tools aimed at improving government accessibility and accountability. But you don't have to be a coder to participate. They're also looking for librarians! Now's your chance to put your govt information skills toward an amazing project.

If you live in Washington DC area, please Sign up for the DC Code-a-thon today Join with lots of smart people working hard and having fun for the great cause of govt transparency!

[UPDATE: Scroll down for list of library happenings for Sunshine Week]

Spring has sprung with a vengeance here in SF. And that could only mean one thing: Sunshine Week!! Yes it's time once again to feel the warm FOIA on your cheek, to discuss and raise awareness of the importance of free and open government information, transparency and the Freedom of Information Act. Be on the lookout for editorials in your local newspaper (like this one in the Cleveland Plain Dealer), discuss FOIA with your friends and family (you'll be glad you did :-)) and highlight it in your libraries -- perhaps by having a public showing of the OpenTheGovernment Webcast!

OpenTheGovernment.org is having a Sunshine Week Webcast 12-2PM EST on Friday March 19 entitled "Building Transparency." The Webcast will include a host of great speakers including Norm Eisen, Special Counsel to the President for Ethics and Government Reform, Jim Harper, Director of Information Policy Studies at the Cato Institute, John Wonderlich, Policy Director at the Sunlight Foundation, Kevin Goldberg, American Society of News Editors (ASNE) counsel, Miriam Nisbet, Director of the new Office of Government Information Services (OGIS), Melanie Sloan, Executive Director, Citizens for Responsibility and Ethics in Washington (CREW), Melanie Pustay, Director of the Department of Justice (DOJ) Office of Information Policy (OIP), Eric Gundersen, President and co-founder of Development Seed and Sean Moulton, Director of Federal Information Policy at OMB Watch. It should be a great discussion so hope you can tune in.

What libraries and others are doing for Sunshine Week:

• Northern CA Association of Law Libraries (NOCALL), in association with the Special Library Association Sierra Nevada Chapter, is sponsoring 2 Sunshine Week events; one in Sacramento and one in San Francisco. Both have interesting lists of speakers and require registration for a small fee ($20 for Sacramento event and $15 for SF event). In addition, the SF event immediately precedes the NOCALL Spring Institute on information piracy, "Piracy on the Barbary Coast" which NOCALL and SLA members can attend at the NOCALL member rate, and later in the evening, a celebration of NOCALL's 30th anniversary.
• Freedom of Information Day at the New York Public Library. Tuesday, March 16, 2010, 10:30 - noon. Conference Room 18 on the lower level of New York Public Library (188 Madison Ave. @ 34th St.).

  This year's guest speaker is Heather Joseph, Executive Director, the Scholarly Publishing and Academic Resources Coalition, (SPARC), an international alliance of academic and research libraries working to create a more open system of scholarly communications. FOIA day has been held at NYPL annually since 1993.

• California State University San Bernardino Pfau Library has partnered with the San Bernardino League of Women Voters to be a site for the OpenTheGovernment.org webinar on government transparency. This is the second year that Pfau Library has participated. You can see video of last year.
• The web site www.TalkStandards.com will focus on open government during its monthly online forum. The forum will take place on Thursday March 18th from 8-12 Pacific / 11-3 EST / 4-8pm GMT.

  TalkStandards is an active online community where ICT developers, researchers, policymakers and other interested parties can share ideas and collaborate on the global standards system. Each month, a timely topic is chosen (last month, it was eHealth, for example).

The folks over at radicalcartography.net have just made available the Statistical Atlas of the 9th US Census (1870) as a bulk download. It's great that this amazing government publication is finding interest by the public -- and that the radical cartographers are doing lots of cool projects like Census Demographics.

However, it should be noted that it's been available online for a while from both the Library of Congress and the Federal Reserve Archival System for Economic Research (FRASER). And of course it's also available in paper from Federal Depository Libraries across the US. I'd recommend that all you radical cartographers, cartographer wanna be's, history buffs, data geeks etc get thee to your local Federal Depository Library to see what the Federal govt has published over the last 200+ years and also check out what your libraries are digitizing and putting online. You'll be glad you did.

Presented here are all of the maps and charts from the first statistical atlas of the US Census, widely praised in its time and still a wonderful example of sophisticated graphics, the out-of-date racial/psychological nomenclature notwithstanding. The atlas is available page-by-page from the Library of Congress, but you can download it in bulk here.

[Thanks BoingBoing!]

Our pals over at the Sunlight Foundation have just posted a great infographic showing the cycle of transparency. There's just one thing missing and it fits in all parts of the cycle: policy, technology, reporting, engagement. That piece is libraries. But who's quibbling, it's a great graphic of the entire government ecosystem. Thanks sunlight!

With data being made easily accessible, journalists and bloggers can begin to dig into it, mix it up, identify relevant information and give the data context. As that critical context is provided, citizens absorb it and spread the information to others – both online and face-to-face – and make the data actionable.

Ultimately, informed citizen action creates greater public awareness; citizens become more effective, responsible advocates; holding government accountable becomes informed by data rather than inside-the-Beltway pundits, and better decisions can be made for our democracy.

As each element of the Cycle of Transparency moves forward concurrently, bringing about the changes we need to create a more transparent government, we also identify new needs.

At the end of the day, the process that the Cycle of Transparency describes is about creating a government more deserving of our trust, and ultimately, a government that allows its citizens to fully participate and hold government accountable as our Founders intended.

We mentioned GovPulse a few months ago as it was one of 3 finalists in the Sunlight Foundation's apps for America 2 Contest. But here's a reminder to check it out.

GovPulse is an easy-to-use, open-source Federal Register browser. It lets you find any kind of notice, notification and solicitation that a federal agency puts out. GovPulse parses that data flow and gives you a way to browse the tens-of-thousands-of-pages-log register by agency, category or date. It also includes tools for visualizations and analysis of the register. For instance, check out the agency page to see sparklines of the notices from each agency, or the map of places mentioned by an agency. or search the Federal Register for proposed activities by location.

GovPulse is a great addition to the documents/policy junky digital toolbox that includes govtrack.us, OpenCongress, OpenCRS (or, to toot my own horn, the CRS digital archive!) OpenSecrets, Legal Information Institute (LII), Justia. What are others that should be in this toolbox? Please leave us a comment with other suggestions.

Yes it's census season again. And to mark the coming of the 2010 census, The US Census Bureau has digitalized all the decennial censuses in pdf from 1790 through 2000. Check out how your city/town/state/district has changed over the 210 years of the census. Census geeks might also want to check out this handy guide to the census called Measuring America: The Decennial Censuses From 1790 to 2000 where one can read the actual questionnaires for each census and get background history on each census. Oh and don't forget American Factfinder, the Census's database for the 1990 and 2000 census, American Community Survey, Economic Census, and annual economic surveys. Factfinder includes quick facts, mapping tools and more.

Happy data hunting!

According to Aliya Sternstein at NextGov there's a proposal in the draft federal broadband plan to create a .gov video archive called video.gov. It'll be similar to the government's data.gov initiative. Wonder when they'll create a documents.gov? Oh yeah, they already have that! It's called the FDLP and it's been around for almost 200 years!!

A proposal in the draft of the government's imminent broadband plan would create a YouTube-like online archive called Video.gov to preserve agencies' Web content and possibly information provided by the media, an official with the Federal Communications Commission said on Monday.

The planned national digital archives for the 21st century would expand upon the government's Data.gov Web site, a warehouse of downloadable federal statistics, and be maintained by the National Archives and Records Administration, the Library of Congress and other agencies, said Eugene Huang, FCC's director of government performance and civic engagement for the national broadband plan.

[Thanks for the tweet Michael Riedyk at DotGov]

Back on September 18, 2007, the House Judiciary Committee chaired by John Conyers (D-Michigan) held a hearing entitled "Warrantless Surveillance and the Foreign Intelligence Surveillance Act". In that hearing, Conyers posed some questions to the Justice Department to get at the Department’s views on the legal framework governing electronic surveillance under the amended Foreign Intelligence Surveillance Act (FISA) -- we've been tracking FISA for some time on FGI. The Committee hearing volume (pdf) was published in June 2008 without the Justice Department’s answers to these questions, because they were provided to Congress too late to be included in the published record.

As you might remember, back in December, 2005 the NY Times broke a story about the Bush administration secretly authorized the National Security Agency (NSA) to eavesdrop on Americans and others inside the United States to search for evidence of terrorist activity without the court-approved warrants ordinarily required for domestic spying, according to government officials. FAS as well as the Electronic Frontier Foundation (EFF) and other civil liberties organizations have been tracking the NSA warrantless surveillance controversy.

Many thanks to Steven Aftergood and the Federation of American Scientists (FAS) for submitting a FOIA request to make public Assistant Attorney General Kenneth Wainstein's written responses to those questions posed about this important program and bringing to light the legal perspective that held sway within the Bush administration's Justice Department.

“If the so-called Terrorist Surveillance Program (TSP) was perfectly legal as has been claimed, why would companies who cooperated in it need immunity?” the Committee asked. (To protect classified information, among other reasons, the Department responded.) “Is the President free to disregard any provisions of FISA with which he disagrees?” (No, not exactly.) “If an individual in the United States is suspected of working in collusion with persons outside the United States–such that an investigation of one is in effect the investigation of the other–under what circumstances, generally, would you use criminal or other FISA wiretaps?” (Targeting of persons in the United States can only be done under FISA procedures.)

Open source intelligence -- not to be confused with Open-source software -- is "a form of intelligence collection management that involves finding, selecting, and acquiring information from publicly available sources (my emphasis) and analyzing it to produce actionable intelligence." Libraries in the Federal Depository Library Program have since the early 1940s received output from this process in the form of Foreign Broadcast Information Service (FBIS) materials *for free*. FBIS materials offered translation of foreign news sources, and via the Joint Publications Research Service (JPRS) foreign language books, newspapers, journals, unclassified foreign documents and research reports. FBIS became the World News Connection in 1996, but it is a severely limited version (about half) of what's available for internal government use.

The Federal of American Scientists has more on FBIS. Check out FBIS and JPRS materials in library collections near you!

All that background as context to a very troublesome turn of events as described by a recent post on the govdoc-l list (see the email below stripped of personal information). This important piece of the govt information universe is now only available via a very expensive commercial database (World News Connection), depriving the academic and larger research communities of full access to all that is done by FBIS at taxpayer expense. Please help us by contacting the Open Source Center (OSCinfo@rccb.osis.gov 202-338-6735, or 1-800-205-8615) and Robert Tapella (PublicPrinter@gpo.gov) at the Government Printing Office and request that the Open Source Center offer free access of opensource.gov to depository libraries. Thanks!

>>>>>>>>>>>>>>>>>>>>>>>>

Date: Wed, 24 Feb 2010 10:25:58 -0600
Subject: OpenSource.gov access

Has any library successfully gained access to OpenSource.gov?

For those who are unfamiliar with this resource, here is the what their web page says about them:

"OpenSource.gov provides timely and tailored translations, reporting and analysis on foreign policy and national security issues from the OpenSourceCenter and its partners. Featured are reports and translations from thousands of publications, television and radio stations, and Internet sources around the world. Also among the site's holdings are a foreign video archive and fee-based commercial databases for which OSC has negotiated licenses. OSC's reach extends from hard-to-find local publications and video to some of the most renowned thinkers on national security issues inside and outside the US Government. Accounts are available to US Government employees and contractors. Register today to see what OpenSource.gov has to offer."

When we tried to register, they informed that we would have to justify why we needed access to the information and that we could get the information through World News Connection (via Dialog) OR, and I quote:

"In addition to the World News Connection, individuals may be able to access OSC products through university libraries, or the Federal Depository Library Program. Many Depository Libraries received CDs from the US Government Printing Office that contain select Open Source Center products." [The CDs that they are referring to are the FBIS materials (PREX 7.10/3:)]

In our response, we informed them that WNC was an expensive database they we could not afford and that their information regarding OSC being distributed through the FDLP was sorely out of date since the CDs have NOT been distributed for over 5 years.

In their response, they say they are considering adding additional agencies such as the Federal Depository Library (FDL) as part of the approved list of agencies in OpenSource.gov., but such a review would take a considerable amount of time to do. (I took this to mean, when 'ell freezes over.) Now here is the strange part--they think the FDLP is under the Dept of Interior and we could sign up that way--but our email address would need to have .gov or .mil in it. I am not sure, but I think they are actually referring to the Natural Resource Library in the U.S. Dept of Interior, which is a federal depository library, with which we are not associated, so this is NOT an option.

At this point I am stymied as to how we can have access to information that was formerly available FOR FREE through depository but is now only available through commercial ($$$) means. I know that GPO is aware that the CDs are no longer being distributed because of the creation of the OpenSource database. The only message I could find about this situation via the GOVDOC-L archives was from 2007 when they said "FDLP is still working with the agency OSC to get an agreement with how we are going to access their database." It is now 3 years later and we still do not have access to this information.

In the meantime, we have a professor on campus doing research in Middle East affairs and would like to have access to more recent information than what we have in our library via microfiche and CDs. We can not afford WNC, so I don't know what else we can do--except get access to OpenSource.gov. If anyone has been successful, I would be happy to hear how you did it.

The site Cryptome has been shut down over a Digital Millennium Copyright Act (DMCA) notice from Microsoft alleging copyright infringement after Cryptome published a 22-page Microsoft document outlining how the company stores private user data in its web-connected servers. The document also explains how government agencies can access that personal data. John Young has put up an alternative website while the original domain is locked by Network Solutions. Wired news blog "Threat Level" and ReadWriteWeb have more context.

Feel free to download the document entitled "Microsoft® Online Services Global Criminal Compliance Handbook" (.pdf).

Good thing libraries have collected Cryptome archives on CDROM and have harvested the site as well!

[Thanks BoingBoing!]

Please check out the new book published by the MetaArchive Cooperative called A Guide to Distributed Digital Preservation. It's both timely and handy.

[Full disclosure: the book is primarily about LOCKSS and mentions specifically the project that I'm working on LOCKSS-USDOCS, FGI and I receive no compensation from the sales of the book.]

Announcement: publication of A Guide to Distributed Digital Preservation

Authored by members of the MetaArchive Cooperative, A Guide to Distributed Digital Preservation is the first of a series of volumes from the Educopia Institute describing successful collaborative strategies and articulating specific new models that may help cultural memory organizations work together for their mutual benefit.

This volume is devoted to the broad topic of distributed digital preservation, a still-emerging field of practice for the cultural memory arena. Replication and distribution hold out the promise of indefinite preservation of materials without degradation, but establishing effective organizational and technical processes to enable this form of digital preservation is daunting. Institutions need practical examples of how this task can be accomplished in manageable, low-cost ways.

This guide is written with a broad audience in mind that includes librarians, archivists, scholars, curators, technologists, lawyers, and administrators. Readers may use this guide to gain both a philosophical and practical understanding of the emerging field of distributed digital preservation, including how to establish or join a network.

Readers may access A Guide to Distributed Digital Preservation as a freely downloadable pdf and/or as a print publication for purchase. Please visit http://www.metaarchive.org/GDDP to download or order the book.

******

The MetaArchive Cooperative provides low-cost, high-impact preservation services to help ensure the long-term accessibility of the digital assets of universities, libraries, museums, and other cultural memory organizations. In addition to preserving members' digital content in a distributed digital preservation network, the Cooperative also offers consulting and education services to institutions that seek training in digital preservation planning, policy creation, and implementation, including setting up and running Private LOCKSS Networks (http://www.lockss.org).

For more information, please contact Program Manager Katherine Skinner (katherine.skinner@metaarchive.org).

I found this NPR story this morning very interesting. The U.S. Supreme Court hears arguments today in a case that pits an individual's right of free speech and association against USAPA. The case is being brought by the nonprofit Humanitarian Law Project. Too bad the briefs for this case aren't publicly available yet (at least not on FindLaw :-( ). This would be a slam dunk for the Humanitarian Law Project if their name was followed by "LLC."

Carl Malamud announced yesterday the inaugural meeting of the International Amateur Scanning League (IASL) (I'm already imagining cool swag!). Malamud is taking FedFlix program to the streets! Fedflix, a joint venture between the National Technical Information Service and Public.Resource.Org, digitizes NTIS video and makes them available on YouTube, the Internet Archive, and the public.resource.org Stock Footage Library.

Well now a gang of volunteers including members of DC CopyNight and Smithsonian employees working on their own time are going to the National Archives and Records Administration (NARA) and copying over 1,500 DVDs to be uploaded to the net.

Malamud said:

What makes this grassroots digitization effort so remarkable is that it has the full support of the government. Indeed, David Ferriero, the U.S. Archivist, joined me in the initial meeting where we taught volunteers how to rip DVDs!

Kudos to Malamud and the IASL!

And this makes me think that more libraries and librarians should be doing the same thing for govt documents. Why not set up your own scanning operations in your depository library (Book Liberator or DIY Book Scanner can show you how to digitize on the cheap!) and then deposit those scans into the Internet Archive's US Documents Collection (don't forget to follow FDLP digitization standards!). Scans could also be ingested into FDSys (when they've got that capability working ;-)). So get to it; what are you waiting for?!