Readers of Slashdot asked Carl Malamud about his experiences and hopes in his project to prod the U.S. government into scanning archived documents. They asked questions about metadata, digitizing rare books, what he thinks about corporate partnerships in the process to get public data released, other projects like Ancestry.com and PACER, and even "Which government agency is the worst to get information from?"

Malamud's answers are posted at the link below "with a mix of heartening and disheartening information about how the vast project is progressing."

• Carl Malamud Answers: Goading the Government To Make Public Data Public, Slashdot, Your Rights Online section, Posted by timothy on Monday January 23, 2012.

Earlier this month, we posted about the "Open letter and petition to President Obama to create a federal scanning commission and digitize all .gov publications". The petition closed on 1/20 and now David Ferriero, the Archivist of the US at the National Archives, has given the official NARA response. I'd say this is a positive first step, but much discussion is still needed. Please join the conversation over at the NARA Blog. I think documents librarians will be invaluable to this effort going forward!

Digitizing Federal Public Records
By David Ferriero

Thank you for signing a petition asking the Obama Administration to digitize all public records.

The Obama Administration believes increasing access to our collections by digitizing our records is a great idea. Our most recent efforts to do this ourselves as part of our OpenGov initiative, include the Citizen Archivist project, a Wikipedian in Residence, Tag it Tuesdays, and Scanathons. We are also moving forward on implementing the President’s recent Memorandum on Managing Government Records, which focuses on the need to update policies and practices for the digital age.

But all those things aren’t enough. Your petition, and the Yes We Scan effort broadly, calls for a national strategy, and even a Federal Scanning Commission, to figure out what it would take to digitize the holdings of many federal entities, from the Library of Congress to the Government Printing Office to the Smithsonian Institution.

These ideas bring up a host of questions that still need to be answered: What should the National Archives’ priorities be? Do we focus on preserving deteriorating paper records, still bound with red ribbons from two centuries ago? Do we make digital copies of Vietnam Era film footage? Should we focus on preserving those older paper records while citizens volunteer to digitize more recent, and better preserved, records?

The National Archives – which houses the Nation’s permanent records – is looking for your input to help answer these important questions on how we move forward. What are your thoughts on how the National Archives and other agencies should proceed? What questions should we be asking ourselves?

You can add your thoughts over on the National Archives blog, and I’m looking forward to having a longer discussion with the creators and signers of this petition on this important issue in the coming weeks– more details on that will follow.

Thank you again for your interest in this important issue. I’m looking forward to your ideas on how we can proceed with digitizing federal public records.

David Ferriero is the Archivist of the United States

Time once again for a selection of news and new resources that we hope will be an interest to the FGI community. The following posts are from INFOdocket.com (@infofodocket) where we compile and post new items daily.

1. New iOS App From SEC: U.S. Securities and Exchange Commission News

2. Just Released: Library of Congress Offers Congressional Record As iPad App

3. National Library of Medicine Announces Latest Release of Its “History of Medicine Finding Aids Consortium”

4. Asked and Answered Questions: U.S. Department of Education Releases Mobile Version of Answers.Ed.gov

5. Smithsonian Gets Dedicated Funds for Digitization and New Media, Will Spend $8.7 Million In FY 2012

6. The Library of Congress Names Gayle Osterberg Director of Communications

7. EPA Releases Comprehensive Greenhouse Gas Emissions, Online Database and Dataset Available

8. White Paper: “Authentication of Primary Legal Materials and Pricing Options”

9. Federal Elections Commission (FEC) Launches Mobile Web Site

10. Full Text Reference Resource: Trade & Development: UNCTAD Handbook of Statistics 2011
From the United Nations Conference on Trade and Development.

11. NARA Launches “Citizen Archivist Dashboard”

12. U.S. Bankruptcy Courts Begin Offering Online Chat Services

13. USDA Launches New Online Nutrition SuperTracker

14. New Database: Time-Series Plots of Phrases in U.S. Supreme Court Opinions (Legal Language Explorer)

15. Statistics: NLM Updates MEDLINE Indicators With FY2011 Numbers

Many of us in the government documents world woke up to 2012 with the following message posted on the Web site of the National Biological Information Infrastructure (NBII) and distributed around to various library listservs:

In the 2012 President's Budget Request, the National Biological Information Infrastructure (NBII) is terminated. As a result, all resources, databases, tools, and applications within this web site will be removed on January 15, 2012.

NBII has been a critical program since 1994 (See Bill Clinton's Executive Order 12906 which created the "National Spatial Data Infrastructure" ("NSDI")). NBII was set up to coordinate a broad array of information at the federal level about biodiversity and ecosystems.

Todd Carpenter, director of National Information Standards Organization NISO, put it nicely and succinctly when he tweeted:

What is particularly sad about NBII shutting down is it's precisely the thing we need MORE of not less=>trusted data repositories #opendata

Well have no fear, the Library of Congress, Internet Archive and Stanford Libraries have all harvested (separately) the NBII Website -- Stanford harvested twice between January 5 and January 13, 2012for its Fugitive US Agencies collection.

Daniel Schuman describes a House commitment to openness that resulted in action!

• House Launches Transparency Portal, by Daniel Schuman, Sunlight Foundation (Jan. 13, 2012).

  Making good on part of the House of Representative's commitment to increase congressional transparency, today the House Clerk's office launched http://docs.house.gov/, a one stop website where the public can access all House bills, amendments, resolutions for floor consideration, and conference reports in XML, as well as information on floor proceedings and more. Information will ultimately be published online in real time and archived for perpetuity.

  ...the ongoing process of releasing documents online, in real-time, and in machine-readable manner is a tremendous sea change from the slow and ponderous paper publications that are often late, fairly difficult to use, and unfriendly to computers.

Daniel rightly emphasizes the availability of XML, but the site does make PDFs available as well. It also has as RSS feed:

• docs.house.gov
• RSS feed.

The HathiTrust announced that it will be adding a new field to its inventory files that will indicate volumes that have been identified as U.S. federal government documents.

• Update on December 2011 Activities, HathiTrust (January 13, 2012).

  Changes to Tab-delimited Files

  On February 1, HathiTrust will be adding three additional columns to the tab-delimited inventory files (“hathifiles”) available at http://www.hathitrust.org/hathifiles. The files are frequently used by partners and non-partners as a means to obtain full bibliographic records for HathiTrust items to load into local catalogs (see HathiTrust Data Availability and APIs). The additional columns will identify the publication date and publication location of volumes in HathiTrust, as well as volumes that have been identified as U.S. federal government documents.

The Federal Reserve has released transcripts of meetings of the Federal Open Market Committee from 2006, after a standard five-year delay.

• FOMC: Transcripts and Other Historical Materials, 2006.

As the New York Times notes, the transcripts...

... clearly show some of the nation's pre-eminent economic minds did not fully understand the basic mechanics of the economy that they were charged with shepherding. The problem was not a lack of information; it was a lack of comprehension, born in part of their deep confidence in economic forecasting models that turned out to be broken.

• Inside the Fed in 2006: A Coming Crisis, and Banter, By Binyamin Appelbaum, New York Times (January 12, 2012).

  For a famously private institution known for its cryptic, formulaic statements, the meeting transcripts offer a rare glimpse of senior officials in relatively unguarded conversation, somewhat akin to the tapes that some presidents have made in the Oval Office. The Fed officials exchange jokes, gossip about people who are not present, and speak much more frankly about the economy and policy than they did in the public remarks that they made contemporaneously.

  The results are unlikely to burnish any of their reputations, inasmuch as they could not see the widening cracks beneath their feet.

The Federal Open Market Committee (FOMC) is responsible for "open market operations" -- purchases and sales of U.S. Treasury and federal agency securities. These are the Federal Reserve's principal tools for implementing monetary policy. "Monetary policy" refers to actions undertaken by the Federal Reserve to influence the availability and cost of money and credit to help promote national economic goals. The FOMC consists of twelve members: seven members of the Board of Governors of the Federal Reserve System; the president of the Federal Reserve Bank of New York; and four of the remaining eleven Reserve Bank presidents, who serve one-year terms on a rotating basis.

The most detailed record of FOMC meeting proceedings is the transcript. Beginning with the 1994 meetings, the FOMC Secretariat has produced the transcripts shortly after each meeting from an audio recording of the proceedings, lightly editing the speakers' original words, where necessary, to facilitate the reader's understanding. Meeting participants are given an opportunity within the subsequent several weeks to review the transcript for accuracy.

Transcripts of FOMC meetings are made available to the public with about a five-year lag.

EconomyWatch.com has a beta version of "Econ Stats," an economic statistics database service. They say that coverage is worldwide, by country, economic region and geographical region from 1980 to 2016 forecasts. It currently includes over 50 indicators. Its sources are IMF, World Bank, UN, OECD, CIA World Factbook, Internet World Statistics, The Heritage Foundation and Transparency International.

• Econ Stats: The Economic Statistics and Indicators Database

  "You use this data by finding country, economic indicator or year that you are interested in, and then browse through the data from there. We have raw numerical data as well as some basic textual analysis and supporting notes."

Hat tip to beSpacific!

The UK National Archives is preserving government information published on the web by archiving UK Central Government Websites.

• UK Government Web Archive.
• Information on web archiving.

hat tip to BeSpacific!

The Minnesota Historical Society has several papers on authenticating digital legal information. Here you will find white papers that address authentication issues as well as information on the Uniform Electronic Legal Materials Act. Links to additional resources are also provided.

• Preserving state government digital information

  Project partners have identified authentication of digital material -- the process by which information is assured to be what it appears or claims to be -- as a common interest. The trustworthiness of online state statutes and session laws is of particular interest.

The newest paper discusses five methods of authentication and their associated costs pertaining to authenticating primary legal materials in electronic format:

• Authentication of Primary Legal Materials and Pricing Options (pdf)

Hat tip to INFOdocket!

This is from last year, in case you missed it. (I did.):

• Documerica Returns!, EPA blog (May 2nd, 2011).

  Almost 40 years ago, EPA’s Documerica project captured thousands of images of environmental problems and everyday life. Now it’s your turn!

  On Earth Day 2011, EPA put out a global call for current photos of life and our environment, PLUS a challenge to photograph the ‘now’ of places in Documerica. Your photo could be exhibited around the U.S. in 2012!

  Join In!
  Sign up and submit photos through Flickr!

See also:

EPA wants your environment pictures, issues public photo challenge, by Michael Cooney, Network WorldBy (01/06/12).

Welcome to the first State Agency Databases activity report of 2012! We have a lot of activity to report from the State Agency Databases Across the Fifty States Project.

STATISTICS FOR 2011 - PARTIAL YEAR

We started making note of project statistics in May 2011. This enables us to report partial year statistics (May - Dec 31) for our project pages. Here are a few highlights for 2011:

Total visits to state database pages - 73,606
Total visits to subject based database listings - 10,325

Top five states by number of visits:

Missouri 5592
Florida 3694
Alaska 3438
California 2786
Ohio 2437

Bottom five states by number of visits:

New York 478
Vermont 470
West Virginia 397
Maine 383
Minnesota 307

Most popular subject page - Prisoner Locaters - 6,239. This is also the most popular page in the entire project.

The full set of project statistics for our site can be found at http://tinyurl.com/StateAgencyDBstats2012 as a Google Spreadsheet.

NEW ORPHAN STATES FOR 2012

As a result of the yearly page review procedure put in place last year, the following states have become available for adoption:

Maryland
Minnesota
New York
Rhode Island
Utah
West Virginia

If adopting any of the above states interests you, please review our Volunteer Guide. If you think you can fulfill the basic responsibilities listed AND can start modifying your page within two weeks of receiving a wiki login, please e-mail me at danielcornwall@gmail.com.

SUBJECT PAGE ACTIVITY

Prisoner Locater Tools - An offender lookup tool for Utah was added to the page.

OTHER WIKI ACTIVITY

For a blow by blow list of changes to our project over the past 14 days, please see http://tinyurl.com/statedbs14d. Here are some highlights:

DATABASES ADDED

NEW MEXICO (Adrienne Walker)

The Judicial Branch of New Mexico - This online case lookup is maintained by The New Mexico Courts. This database gives access to the lower court cases of the New Mexico District Court, Magistrate Court and Municipal Court data. Municipal court data is limited to criminal Domestic Violence and DWI historic convictions from September 1, 1991 onwards. Searches can be conducted by name, case number or DWI.

NEW YORK (Orphan - Added while reformatting page)

New York State Kosher Food Registry Search - Registry is searchable by product (type or name, packaged or unpackaged), by food establishment name, by store (name or location), by name of certifier. Records for products include brand, name, size and certifying agency.

NORTH DAKOTA (Kathryn Thomas)

State Document Depository - State Documents can be searched in the library catalog. Lists of recent arrivals can be viewed by agency name and then by document title.

UTAH (Orphan - Added while reformatting page)

Utah Burials Search - Database of people buried in Utah. May be searched by deceased name, dates of birth and death, and burial place (any, county, city/town, cemetary). Records contain name, birth date, death date, burial date, place of birth (when available), place of death, grave location, source (when available), comments, when available.

Watching Them Watching: Issa Touts Video Archive of Oversight Hearings, by Nick Judd, TechPresident (January 6 2012).

As of today, the House Committee on Government Oversight under Rep. Darrell Issa has released 1,139 videos of hearings going back to the 103rd Congress of 1993-1994, committee staff announced today.

These videos, dusted off from the House committee's archives, join hundreds more going all the way back to 1987 on House.Resource.org, a repository for archived video and hearing transcripts gleaned from C-SPAN, the House and the Internet Archive as part of a collaboration between Carl Malamud's Public.Resource.org and House Speaker John Boehner. At the start of this Congress, Boehner asked Issa's Oversight committee — which had been recording its own video of hearings, doubling up on video already recorded by the House Broadcasting Studio, since the 2010-2011 session of Congress — to take on archiving and publicising video of committee hearings as a pilot project. The House this year also launched its own streaming of floor proceedings.

Last month Rep. Darrell E. Issa introduced legislation, the Research Works Act that would effectively prohibit the federal government from requiring open, free access to federally funded research.

The Association of American Publishers issued a statement in support of this legislation.

• Publishers Applaud “Research Works Act,” Bipartisan Legislation To End Government Mandates on Private-Sector Scholarly Publishing, Friday, 23 December 2011.

Peter Suber has been tracking this and Gary Price has links to Peter's posts and other information about this issue here.

Gary also has an interesting post about some members of the AAP that you would think might oppose the Research Work Act, but who have not yet spoken out against it or the AAP announcement.

See also:

• Why Is Open-Internet Champion Darrell Issa Supporting an Attack on Open Science?, by Rebecca J. Rosen, The Atlantic (Jan 5 2012).

  [The] bill would prohibit all federal agencies from putting any privately published articles into an online database, even -- and this is the kicker -- those articles based on research funded by the public if they have received "any value-added contribution, including peer review or editing" from a private publisher. This is a direct attack on the National Institutes of Health's PubMed Central, the massive free online repository of articles resulting from research funded with NIH dollars. Similar bills have been introduced twice before, in 2008 and 2009, and have failed both times.

As the federal government attempts to consolidate its web presence and reduce the number of dot-gov web sites, it faces a huge task. When the British government did something similar, it reduced the government's 2,000 websites by more than 75 percent and shifted its online organizing structure from being based on the interests of agencies creating content to focusing on the interests of the citizens consuming that content. That effort took five years. The U.S. Government has 16,000 or more web sites. Currently it is hard for citizens to find the information they need because the sites are so badly done that typical web-wide searches often list government data well below less authoritative, outdated or recycled sources and the agencies themselves have clunky internal search engines.

An article in NextGov about the current state of dot-gov web sites has a number of interesting tidbits of information worth thinking about.

• Feds aim to serve citizens better by revamping Dot-Gov, by Joseph Marks, NextGov (Jan. 3, 2012).
  • While the government is publishing more information than ever through about 18,000 websites, it's become increasingly difficult for agency information to reach the public.
  • Much of the dot-gov reform effort has so far focused on eliminating excess government sites that sprouted up during the Web-crazed 1990s and now do little but diminish the dot-gov domain's gravitas.
  • The federal Web presence is also pockmarked with stand-alone sites such as MLKday.gov, which are officially top-level domains but don't have much content and aren't regularly updated.
  • an informal survey in October with a custom-built Web crawling tool showed at least 200 ... had likely been unofficially retired.