The Sunlight Foundation announced today a new bill introduced by Congressman Steve Israel (NY-2) called the Public online Information Act (POIA) (read the bill (PDF)). POIA will require that all "public" executive branch documents be permanently available on the Internet at no cost. POIA also creates a:

"special federal advisory committee to coordinate the development of Internet disclosure policies. These policies promote information best practices, including data interoperability standards, and will keep the government up-to-date with new technology. The advisory committee’s 19 members – six appointed by each branch of government, plus one by GSA – are drawn from the public and private sectors and serve as watchdogs, synthesizing the needs of agencies and the public and making recommendations on updating federal law."

While I wholeheartedly support the spirit of POIA -- free permanent internet access to executive branch documents! -- and will definitely be contacting my representative to support its passage, I have 2 concerns that I hope will be discussed by the Sunlight community, the soon-to-be federal advisory committee, libraries and the public:

1) preservation: There was an article in today's NY Times -- "Fending Off Digital Decay, Bit by Bit" -- that highlights the many issues surrounding digital preservation. Just putting something on the Web does not mean that it will be preserved. The GPO has been working on their Federal Digital System (FDsys) since 2004 (and really since 1994 when they started GPOaccess) to deal with the inherent digital issues. Many researchers, librarians, academics, computer programmers etc have been working on these issues pretty much since the 1960s. And the issues are still here today.

So I'd like to see as part of this bill an acknowledgement that online information is expensive to preserve AND that there will be continued funding for research and sustainability of digital archives through the National Digital Information Infrastructure & Preservation Program (NDIIPP). Readers are encouraged to explore the issues here and here.

2) privatization of govt information: The following from the Sunlight announcement caught my eye and concerned me:

Freeing government information from its paper silos provides the private sector with raw material to develop new products and services and gives the public what they need to participate in government as active and informed citizens.

Federal government information is in the public domain. That's a good thing. However, there's a fundamental issue at stake here. One can't have "permanent free public access" to government information where the private sector is involved. The private sector has been involved in giving access to government information for a long time (see LexisNexis, Thomson West, Readex etc). They do it well but they certainly don't do it for free. Libraries and other organizations have paid many millions of dollars to license access to govt information for the communities they serve. Here's more background and context on privatization. For all intents and purposes, these private sector companies take public domain information and privatize it. Any digital govt information accessible on the internet should already be findable, usable and accessible in bulk at minimum.

But there needs to be more. What I'd like to see in this bill and in the discussion after it passes (devil's in the details right?!) is not only a requirement that all govt information is online permanently and for free, but that there be the inclusion of a viral GNU General Public License-like piece of the public domain whereby anything IN the public domain (i.e., govt information) has to STAY IN the public domain. There are plenty of folks (I'm looking at you Sunlight, Govtrack.us, OpenCongress, OpenCRS etc) excited about making govt information more available, more usable and more shareable and this would support their public service.

Please help Sunlight get the word out about POIA and contact your representative and let them know that they should co-sponsor POIA and assure its passage.

Our pals over at the Sunlight Foundation have just posted a great infographic showing the cycle of transparency. There's just one thing missing and it fits in all parts of the cycle: policy, technology, reporting, engagement. That piece is libraries. But who's quibbling, it's a great graphic of the entire government ecosystem. Thanks sunlight!

With data being made easily accessible, journalists and bloggers can begin to dig into it, mix it up, identify relevant information and give the data context. As that critical context is provided, citizens absorb it and spread the information to others – both online and face-to-face – and make the data actionable.

Ultimately, informed citizen action creates greater public awareness; citizens become more effective, responsible advocates; holding government accountable becomes informed by data rather than inside-the-Beltway pundits, and better decisions can be made for our democracy.

As each element of the Cycle of Transparency moves forward concurrently, bringing about the changes we need to create a more transparent government, we also identify new needs.

At the end of the day, the process that the Cycle of Transparency describes is about creating a government more deserving of our trust, and ultimately, a government that allows its citizens to fully participate and hold government accountable as our Founders intended.

[UPDATE: I added the slides for Tom Bruce's talk]

Shinjoung and I submitted a panel on the future of govt information for iConference 2010 in Champaign, IL. We had a good far-reaching discussion with Tom Bruce (Cornell Legal Information Institute), Daniel Schuman (Sunlight Foundation) and Cindy Etkin (GPO). Below are my slides and notes. I've also attached the notes and abstract as PDFs. As Tom tweeted, "World's problems: solved."

If the other panelists agree, I'll post their notes/slides as well. This is of course an ongoing conversation so please feel free to leave comments, questions, rants etc.

--that is all!

3:45 - 5:15 pm Thursday, February 4, 2010
Roundtable 4 : : Technology Room
"Gone today, Here tomorrow: assuring access to government information in the digital age." ShinJoung Yeo, University of Illinois; and James R. Jacobs, Stanford University

Panelists:

• Shinjoung Yeo, Moderator
• James Jacobs, Stanford University Library
• Thomas Bruce (Legal Information Institute, Cornell University)
• Daniel Schuman (Sunlight Foundation policy director)
• Cindy Etkin (Govt Printing Office)

[SLIDE 1: govt documents]

Right up front, I'm a librarian and a collaborator in the LOCKSS distributed digital preservation project (Lots of Copies Keep Stuff Safe). I've been in academia/education my whole life as a student, teacher, librarian and technologist. I've been a government information/FDLP librarian since 2002 and currently am serving a 3 year term on the Depository Library Council, the body which informs and advises the Govt Printing Office regarding issues of the Federal Depository Library Program (which Cindy talked about). So my mindset/perspective/bias is from one who assists in the scholarly communication process, one who believes that libraries have a place in the digital information landscape, and one who believes strongly in the idea that access to govt information is a fundamental right. As Ralph Nader has said, “There can be no daily democracy without daily citizenship.” And there can be no citizenship without access to government information.

[SLIDE 2: mmm documents]

With that in mind, I'd like to talk about the underlying historical ideals of the FDLP, discuss how those ideals have been under fire from both within and without the library community and argue that those ideals applied to today's information landscape give us the best chance at access to and long-term preservation and assurance of govt information.

[SLIDE 3: FDLP logo]

The federal depository library program (FDLP) has been around since 1813 in one form or another. The basis underlying the need for an FDLP is to give the public free access to government information. Depository libraries have long safeguarded the public's right to know by cooperating with and receiving for free the govt publications published by the Govt Printing Office (GPO), organizing, maintaining, and preserving those publications, assisting users in accessing said information in a geographically dispersed system and most importantly, assured that govt information is freely available and tamper-proof -- think Napster for govt information. Taken together, the collections of the 1238 depository libraries make up the historic corpus of govt information available for free to every citizen. Jessamyn West of librarian.net, recently called the FDLP the longest running open source project. I would add that it's the longest government-run public-centric open-source project to support the democratic ideal.

[SLIDE CHUCK QUOTE]

Over the last 20-30 years, developments in publishing and Internet technologies have affected the way government information is produced, disseminated, controlled, and preserved. These changes have affected the policies and procedures of the GPO and, in turn, have affected the depository library program. Despite the often-heard promises that Web technologies will bring more information to more people more quickly and easily, the actual effects have been decidedly mixed. The highly visible, short-term successes of rapid dissemination of single titles directly to citizens (e.g., the large number of downloads of the 9/11 report) mask the loss of a secure infrastructure (GPO's Federal Digital System (FDsys) notwithstanding) for long-term preservation of and access to government information as more and more agencies publish content on their own Web sites rather than using the GPO conduit (which librarians call "fugitive documents") and very few agencies publish to any standards or have policies in place that deal with archiving and preservation. As Chuck Humphrey, a data librarian friend of mine, once said, “there seems to be an inverse relationship between convenience of dissemination and preservation standards.”

In addition to this lack of a secure infrastructure, the growing din of the call for digitization of historic govt publications (most recently the Ithaka/ARL report "Documents for a Digital Democracy: A Model for the Federal Depository Library Program in the 21st Century"), while no doubt a boon for access today, is somewhat of a red herring that makes library administrators believe that they will soon be able to dispose of their physical collections and use that space for today or tomorrow's buzz word. This call for digitization may instead have the deleterious affect of damaging the long-term preservation of govt publications.

Lastly, the growing trend toward privatization of govt information has actually caused a decrease in public access despite it's digital nature. This is not a new trend. Herbert Schiller noted this in 1986 in his book "Information and the Crisis Economy." Speaking of machine readable formats, he wrote that, "Library information capability is greatly enhanced. Yet this benefit is accompanied by the abandonment of libraries' historical free access policy. User charges are introduced. The public character of the library is weakening as its commercial connection deepens. No less important, the composition and character of its holdings change as the clientele shifts from general public to the ability-to-pay user."

[SLIDE: GAO contract]

We've seen over the last 30 years a disturbing rise in Federal Agencies entering into contracts with private companies whereby public domain govt documents are digitized and then taken out of the commons via licensing agreements. See for example, the Government Accountability Office (GAO)'s deal with Thomson-West whereby Thomson-West digitized the GAO's 20,597 legislative histories of most public laws from 1915-1995 and in return received exclusive license to sell access to the content. GAO received nothing in return but an account on Thomson's service while the public received nothing at all.

Rapid technological change and the misplaced assumption that "it's all in google" have caused some in the FDLP community to question the need for the FDLP and some others to drop out of the program altogether. I believe that the inherent nature of digital information actually increases the need for a distributed network of dedicated, legislatively authorized libraries. It would be prudent to draw upon the existing infrastructure of FDLP libraries and the almost 200 years of cumulative experience of these institutions in assuring preservation of and access to government information. We must reinforce FDLP’s traditional mission of selection, collection, free access, and preservation of government information in the digital era in order to assure free access to this information into the foreseeable future. Some in the depository community, like my library, are doing just that by participating in the LOCKSS-USDOCS network, harvesting digital govt information -- for example, harvesting openCRS that Daniel mentioned along with other sites that post CRS reports -- and yes digitizing parts of their collections. But we need more libraries not less.

[SLIDE: FDLP ecosystem]

Nobody knows for sure how to preserve digital content for the long-term. This means to me that a loosely coupled, independently administered, distributed ecosystem is the best way to assure long-term preservation -- many organizations with many funding models and a distributed technical infrastructure(s) have a better shot at preservation than 1 or 2 organizations -- especially if one of those organizations has a tenuous budget, or is a private corporation etc.

Imagine if you will 2 future govt information systems: on the one hand, the system where there are one or two digital collections (say for example GPO's Federal Digital System (fdsys) and Portico, the dark archive currently housing digital journals); and on the other hand, one with many digital collections in fdlp libraries. How would each of these deal with or react to different stress situations or threat models (e.g., reduced budgets, increased demand for privatization, increased demand for censorship or control or removal of information, media/hardware/software/network failure, natural disaster, organizational failure etc.)? It's easy to see that a highly replicated, distributed FDLP model of preservation would deal with these situations much better than a centralized model. A web is much stronger than a silo.

[SLIDE: Federal Register XML]

law.gov, Carl Malamud’s proposal for a registry and repository of all legal information -- from what I've seen and heard and read, is a compelling proposal for a significant piece of the federal (and state) legal information ecosystem. What we ought to be doing is a) figuring out how to make law.gov a reality; b) figuring out how to expand it beyond legal materials to include ALL federal information -- information from all 3 branches of government, federal agencies as well as the regional and local offices of those agencies, data and statistics, the entire Congressional/legislative process including the funding that goes into that process to grease the skids so to speak, and making sure public information stays in public control; and c) MOST IMPORTANTLY from my perspective as a librarian, figure out how to preserve that ecosystem for the long term so that the public can inform itself not just today or tomorrow but 100 years from now. Now the 4 of us on this panel are just 4 players with dogs in this fight. But if we agree on the goals, then we ought to work together to proceed toward them and mobilize our communities and the public to support this endeavor.

It's going to take the government (and not just GPO) being serious about transparency and funding the necessary changes in its own federal information distribution system to include open format standards with no DRM, bulk data channels, indexing, description, collection and authentication of information resources, multiple digital preservation strategies to not only assure preservation but also to insure against tampering and deletion of vital information (which, as I've stated earlier, the FDLP historically has done very well!). It's also going to take libraries being serious about and applying the ideals of the FDLP to build a distributed digital infrastructure that takes into account access to as well as preservation of digital govt information.

I agree with Tom and am absolutely convinced that the changes in the information ecosystem that are needed should not be left to the market because the information market leans heavily toward monopoly, proprietary standards, licensing restrictions, lack of access, "rights management" and the like.

If an evolving ecosystem that is free, open, standards-based, authenticated, and privacy-protecting is built and sustained correctly then citizens, libraries, non-profit watchdogs, hackers, activists, AND government will thrive.

[SLIDE 7: THANKS! lockss, archive-it]

digital changes a lot of things about information, but it doesn't change the need to fund it, collect it, share it, preserve it, and give access to it. As my friend and colleague Jim Jacobs recently stated, "lots of collections keep stuff safe!"

Los Angeles Time business columnist Michael Hiltzik discusses key players in the Free Government Information movement in the article These crusaders bring transparency to government (28 September 2009).

The Sunlight Foundation announced the winners of their "Apps for America 2: The Data.gov Challenge" at the O'Reilly/techweb Gov 2.0 Expo Showcase in DC yesterday. The web applications were to be built with data available from the U.S. Data.gov site. And the prizes go to...

First prize: DataMasher.org. Combine and compare government data at the state level.

Second prize: GovPulse. Making Federal Register access easier.

Third prize: ThisWeKnow.org. Find government info by zip code.

Prize for best data visualization: QuakeSpotter.org. Global view of earthquakes with links to quake-related tweets.

Get complete information from Sunlight's press release.

The one-day Gov 2.0 Expo is over, and the two-day Gov 2.0 Summit (with bold face names) is in progress. They are tweeting up a storm over at the Hyatt, using the hashtag #gov2s.
[Update: They are using the #g2s tag, too.]

I will blog more later on how the Gov 2.0 Expo went.

The Sunlight Foundation has announced three finalists for their Apps for America 2 Contest. Sign up for a free account and Vote for the winner!

The finalists include:

GovPulse.usis a "Federal Register browser" allowing you to browse the by agency, category or date. The visualizations are pretty cool too.

ThisWeKnow.org allows you to explore government data about your community. I did a search for my city, and it gave me a summary, but I also can look at how many people in my county were diagnosed with Cancer. Dang!

DataMasher lets you mashup two different data sources and you can share/comment on others. Sunlight Foundation really likes this mashup: High School Graduation vs Guns in Household.

Thank goodness for data.gov and the opening up of more government data so we can make cool applications like these!

Just a quick note to let folks know that I'm currently at Transparency Camp this weekend. It's a great meeting of activists and technologists concerned with all kinds of transparency and government. To follow what's happening, you can track on the twitter hashtag #tcamp09. Since it's an unconference, talks can be proposed on the fly. I gave a talk on the FDLP which got a lot of interest. Contact me if you've got other ideas for presentations. There are still slots available for sunday.

Data is definitely getting sexy. Jonathan Gray of the Open Knowledge Foundation asks "What features should be included in a catalogue of open government data?" and points to a few other data repositories being built on the state and country level (like my own city of SF's CivicDB!). He also mentions the Sunlight Foundation's plan to build on and expand data.gov with a national data catalog that I had meant to write about a couple of weeks ago (go Sunlight!). So I'm putting this question to you, FGI's faithful readers. I'm sure you'll have a thing or three to add to the list of requirements for open data catalogs.

Here are a few suggestions for those building catalogues for (open) government data based on our experience developing CKAN:

• Make the catalogue itself open!
• Let others download the catalogue data in bulk (not just via an API)
• Include information on how to get the data, and how it can be used
• Make it versioned!

[originally tweeted by @EllnMllr. Go ahead and follow her!]

The Sunlight Foundation, Open The Government and the Center for Democracy and Technology have done it again. They've created yet another cool initiative called Show Us the Data for you to "identify the 10 Most Wanted Government Documents, Reports or Data Sets that should be available on the Web"!

Is the federal government putting the information you need online? Are there categories of unclassified documents or data that you know exist–on paper or in government computers and databases–that would be of value to the public if posted and regularly updated on an agency's Web site? If so, then help Open The Government and the Center for Democracy and Technology identify the 10 Most Wanted Government Documents, Reports or Data Sets that should be available on the Web. Use this site to tell us what data you want and who has it, (“Request a Document”) or add your vote (up to three times) to the suggestions others have made.
The deadline to submit documents and vote is March 9, 2009

Thanks to Amy West's tweet for the heads-up about this video, "Tech Agenda 2009: Open Government." John Wonderlich turns a nice phrase (@ 3:10) in response to the question about defining what is open government. John says, "the Federal Depository Library Program is like the internet of the '40s and '50s." Check this one out while nibbling on your cheese (yes that's for you Rebecca :-) ).

This is the second in a series of town halls, "Tech Agenda 2009: Creating New Opportunities for Open & Participatory Government" focused on how technology can help government become more accountable, transparent and participatory. Panelists for Panel II, "Changing Policy to Promote Open Government," include Chris Barkley, Meredith Fuchs, Karina Newton, and John Wonderlich. The discussion was moderated by Andrew McLaughlin of Google. This event took place on December 12, 2008 at Google's offices in Washington, D.C.

Greetings all, I'm honored to be guest blogging this month here at FGI.

I'm a web developer at Sunlight Labs, which involves the development of sites and projects that aim to enable citizens and journalists to more easily access government data.

Those of us that spend our days wrestling with government data often spend a lot of time talking about the data that should be available but isn't. An issue of equal if not greater importance is how to make the already available data useful to a general audience. Anyone that has dealt with raw data from any government agency knows that simply passing government data along is typically not sufficient.

One example of a project done here at Sunlight that emphasizes making some of the complex outputs of the federal government meaningful to the average citizen is Capitol Words - a site that provides a daily and monthly view of what the most commonly used word in the Congressional Record was.

The Congressional Record is the official journal of the daily proceedings of Congress. It is mandated by Article I, Section 5 of the Constitution, which emphasizes how essential it is for the people to know what their legislators are up to. It is even published today in a digital format. Unfortunately, today it is far too large to be of any real benefit to the general public.

Capitol Words was born out of a suggestion that it would be interesting to see simply the "word of the day" as a way of getting a sense of what was on Congress' mind. By giving the average citizen a window into what Congress is doing, it is possible that they will become more engaged then they otherwise might have been. Some citizens may even be inspired to dig deeper and look at the Congressional Record.

Simple presentations of government information such as the popular tag cloud, or even just a simple word, can provide access to data that may be freely available, but is still not accessible to the general public. As great as it is to see more and more government data being made available, hopefully people will also develop new and interesting ways to present government information in a manner useful to all citizens.

The Sunlight Foundation has launched a new web site, Party Time!, which aims to document the political party circuit -- not "political parties" as in "GOP" and "Democratic," but parties as in champagne, food, golf... and money: "the social whirl surrounding politicians in their quests to raise cash to run their campaigns."

There is a searchable database that lets you track parties thrown at the 2008 Democratic and Republican National Conventions as well as fundraising activities by all lawmakers running for Congress that happen all year round going back to 2006.

So you think you're a policy wonk? Well, why don't you take the Sunlight Foundation's PoliQuiz, an interactive, multiple choice political trivia game. Or better yet, play the game below as part of the Freegovinfo team? Everyone who plays PoliQuiz on freegovinfo.info will have their scores associated with the Freegovinfo team. Help us climb up the team leaderboard!

[cross-posted from Legalresearchplus.com]

As I wrote about earlier, I was having a fun, albeit geeky Saturday night following links hither and yon, and I ran across the fascinating PublicMarkup.org site.

PublicMarkup is a project of the Sunlight Foundation , and just like sunlight is the best of disinfectants, a bit of public scrutiny and participation should make legislation stronger and more viable. The folks at PublicMarkup drafted the Transparency in Government Act of 2008, and hope that the wonders of online collaboration will enhance and improve the legislation and will inspire lawmakers to carry it forward.

From the PublicMarkup site:
“The Sunlight Foundation has put together what we think can become model transparency legislation—the Transparency In Government Act of 2008—and we now need your help to further shape and refine it.”

Interesting? Inspired?
Well, you better move fast if you want contribute — You only have until May 19th before they close the site for comments and then try to re-craft/re-draft the legislation.
-Erika

Last month we posted about OMB Watch's survey to choose 5 (and only 5) favorite questions on government openness and transparency for congressional and presidential candidates. Last week, OMBWatch released their report (PDF) of the survey. Given the way things have been going over the last 30 years -- but especially over the last 8 years of the imperial presidency! -- it's no wonder that those surveyed feel that presidents should be more accountable, the public should have increased access to administrative information from both the legislative and executive branches (especially health, safety, and environmental information), and that those within the government should have increased legal protections when they report on government wrong-doing.

I think PublicMarkup's first effort at drafting openness legislation -- the Transparency in Government Act 2008 -- jibes perfectly with this effort.

Five questions to ask the presidential candidates to gauge where the candidates fall on the openness-secrecy spectrum:

1. Manipulation of Facts: "Do you support disclosure of all communications between the White House (including the Office of Management and Budget and other executive offices) and agencies regarding administrative decision-making and information disclosure?"
2. Executive Privilege: "What do you believe are the appropriate limits of executive privilege in the disclosure of information to Congress and the public?"
3. Whistleblowers: "In order to strengthen accountability against corporate crimes, would you support pending legislation that expands whistleblower protection rights to private sector workers who report violations of any federal public health and safety laws?"
4. Presidential Records: "Do you commit to reversing Executive Order 13233 to restore public access to presidential records after twelve years?"
5. Health, Safety & Environment: "Given the importance of health and safety information, how would you ensure that the public has easy access to understandable information about the air they breathe, the water they drink, and the products they use?"