The Securities and Exchange Commission announced a new service that allows you to "instantly compare what 500 of of the largest American companies are paying their top executives."

• Chairman Cox Unveils New Internet Tool With Instant Comparisons of Executive Pay (press release) Washington, D.C., Dec. 21, 2007

The announcement says that The Executive Compensation Reader is "available today on the SEC's Web site at http://www.sec.gov/xbrl."

When you go to that web site, you'll find "Interactive Financial Reports", "Executive Compensation", and (coming soon) "Financial Explorer" and "Investment Management Risk Return Summary".

But...A big, boldface disclaimer is on the bottom of the page:

These applications are on servers not controlled by the SEC. The SEC does not endorse these sites, their sponsors, or any policy, activity, product, data, or service they offer. These sites may be removed from the Internet with little or no warning. The purpose of the data reported is to test XBRL technology. The data is largely unaudited and unreviewed. It is not an official SEC filing. Use these applications at your own risk. Do not rely on the reported data, or documents rendered from the data, to make investment decisions.

The Interactive Financial Reports is a link to "http://216.241.101.197/viewer" which is run by viawest.net which describes itself as a "provider of colocation, managed hosting and business continuity solutions ViaWest has become a production partner to thousands of companies. We can design, implement and support your critical data and infrastructure – allowing you to focus on your core business."

The Executive Compensation link points to "http://216.12.130.224/" which is run by servervault.com which is also a hosting company. It's home page says it provides "secure, robust IT infrastructure management and hosting services, specifically engineered to help federal agencies and commercial entities achieve the most stringent security posture as well as comply with a wide range of IT security regulations, including FISMA, DITSCAP, Sarbanes-Oxley and HIPAA."

Both sites sport an SEC logo and a numeric url that obscures the fact that you are looking at a .net or .com site, not a .gov site.

Is this a temporary situation and will these sites become .gov? is the SEC simply outsourcing the hosting? Or will these sites become commercial after a trial period with the companies or the SEC charging for services?

At minimum, this kind of outsourcing obscures the authenticity of and provenance of the information and makes the job of GPO, libraries, web-archivists, and citizens more difficult.

Recently the Office of Management and Budget (OMB) announced the launching USASpending.gov, which is a relaunch of www.FederalSpending.gov, to provide citizens with easy access to government contract, grant, and other award data (See New OMB Federal Spending Database). But now, Secrecy News says that several defense intelligence agencies will withhold unclassified information about their contracts from USASpending.gov.

• Intel Agencies to Withhold Contract Info from Public Database by Steven Aftergood, Secrecy News, December 18, 2007.

Several defense intelligence agencies will withhold unclassified information about their contracts from a new public database of government spending.

The new database at USAspending.gov is intended to provide increased transparency regarding most government contracts.

But when it comes to intelligence spending, there will actually be a net loss of public information because categories of intelligence contracting data that were previously disclosed will now be withheld.

Secrecy News has made available a PDF of a A U.S. Army Field Manual that explains the rudiments of map reading. Distribution of the manual is restricted, and it has not been approved for public release. Maps and Legends Secrecy News, December 28, 2007.

• MAP READING AND LAND NAVIGATION. FM 3-25.26 January 2005 HEADQUARTERS DEPARTMENT OF THE ARMY Washington, DC, 30 August 2006

A new study by the Pew Internet and American Life Project and the University of Illinois at Urbana-Champaign says that "members of Gen Y are the leading users of libraries for help solving problems and in more general patronage.... Furthermore, it is young adults who are the most likely to say they will use libraries in the future when they encounter problems: 40% of Gen Y said they would do that, compared with 20% of those above age 30 who say they would go to a library." 

• Information searches that solve problems: How people use the internet, libraries, and government agencies when they need help (December 30, 2007) Leigh Estabrook, Evans Witt, and Lee Rainie. (full report. PDF)
• Information Searches That Solve Problems (Press Release)
• Young Adults Heavy Library Users by Anick Jesdanun, Associated Press December 30, 2007.

Thanks to Docuticker for pointing out this new Congressional Research Report on Federal data mining efforts:

Data Mining and Homeland Security: An Overview (PDF; 231 KB)
RL31798
Source: Congressional Research Service (via Federation of American Scientists)

Aside from cataloging currently known datamining efforts by the federal government, the report identifies four areas of concern:

As with other aspects of data mining, while technological capabilities are important, there are other implementation and oversight issues that can influence the success of a project’s outcome. One issue is data quality, which refers to the accuracy and completeness of the data being analyzed. A second issue is the interoperability of the data mining software and databases being used by different agencies. A third issue is mission creep, or the use of data for purposes other than for which the data were originally collected. A fourth issue is privacy. Questions that may be considered include the degree to which government agencies should use and mix commercial data with government data, whether data sources are being used for purposes other than those for which they were originally designed, and possible application of the Privacy Act to these initiatives.

I've heard people say that data mining by the government is no big deal since advertisers and other corporate interests do it all the time in efforts to focus marketing and improve profits. If we don't have privacy from corporate types, why should the government worry us? Because ratty data used by a marketer might result in a bald man getting shampoo ads, but when the government relies on ratty data for law enforcement, innocent people can get jailed or harrassed.

Hopefully Congress will be more vigilant on this issue.

In a govdoc-l posting on December 21, 2007, Jan Goldsmith informed the depository community that the Federal Depository LIbrary at UCLA celebrated it's 75th anniversary. Her post states how they celebrated:

The UCLA Library celebrates its 75th anniversary as a member of the Federal Depository Library Program this year.  We were designated a depository in 1932 by Senator Samuel Shortridge. Over the course of these past years, we have provided free access to federal government information for the University community and beyond.  In honor of our anniversary, Kris Kasianovitz and Jan Goldsmith, with the professional assistance of Ellen Watanabe and Dawn Setzer, mounted an exhibit in the Young Research Library lobby.  Gary Strong signed letters notifying all legislators whose district we serve, including our Senators and Congress representatives.  For more information regarding 75 years of collection and services of the Federal Government Documents at the UCLA Library visit our Anniversary web page, at: http://www.library.ucla.edu/libraries/yrl/depository_exhibit.html

Let's hear it for UCLA for providing three quarters of a century of public service to the people of Los Angeles!

Back in November, Citizens for Responsibility and Ethics in Washington, along with Project on Government Oversight, Public Citizen, Electronic Frontier Foundation and the Sunlight Foundation launched a new website,  GovernmentDocs.org, which will house governmetn documents obtained through Freedom of Information Act (FOIA) requests. From the press release:

The database will house Freedom of Information Act (FOIA) responses, and other government documents, from a number of organizations, that can be browsed, searched and reviewed. It is the only one of its kind.

Traditionally, government watchdog groups have either posted FOIA documents on their websites as unsearchable PDFs, or statically highlighted several pages within a document to bolster their findings. This has historically limited the public's access to FOIA documents, and minimizes the opportunities for use by researchers, journalists and citizen reviewers for further research and disclosures. Governmentdocs.org changes that:

• Each and every document goes through an optical character recognition (OCR) process, so that the text of each document is entirely searchable.
• A powerful search engine provides full-text searches and hit highlighting.
• Citizen reviewers can add information to each document page and highlight important findings, allowing for more robust and targeted searches.
• Every page of every document has its own unique URL so that documents can be linked, shared, or posted onto websites.
• The database is a coalition effort, so all of the organizations' documents will be housed on governmentdocs.org and searches will work across collections.

Still No Directory of Federal Websites, E-Gov Act Ignored.   By Coby Logen, .gov Watch. November 05. 2007

How many government websites are there? How many HHS or DOJ sites are there? You and I have no way to know. American taxpayers cannot even know how many public websites their government is funding. By law, we should—but the system is broken.

The E-Government Act of 2002 set a deadline of two years to develop a "public domain directory of public Federal Government websites" (Section 207(f)(3)). But this directory still does not exist 5 years later.

As we've noted here before (Is your search engine finding the government information you need?), the problem of relying on commercial search engines to find government information is that a lot of government information on the web goes un-indexed by those search engines.

• Most fed data is un-Googleable By Jason Miller FCW (December 17, 2007). "After five years, a major E-Gov Act provision goes unmet because of search problems."

Sen. Joseph Lieberman (I-Conn.), chairman of the Homeland Security and Governmental Affairs Committee, says "There are more than 2,000 federal government Web sites not included in commercial search engine results. Is it accidental, or is there a policy, or it is just laziness? I would like to know why" and "Agencies do not let commercial search engines index their sites."

I wonder if that is true? I wonder if there is any document librarian who can answer that question or point to which sites are not indexed?

It is probably closer to the truth to say, along with John Needham, Google's manager for public-sector content partnerships, that government  "databases" are being missed by web crawlers and that  "Agencies are concerned more about how information is presented than if users are finding it."  In other words, agencies would probably like to have their information indexed, but haven't figured out how to do so, or don't have the budgets to do what is necessary.  It probably isn't "laziness" but lack of funds and other resources; it probably is sometimes "accidental" in that some may not know what to do.  It is probably sometimes even "policy" -- but probably less often.

But, one big problem is that we don't really know the scope of the problem or the cause.  FDLP librarians should be pushing GPO, researchers, and library schools to research these issues so we have answers.

Carl Malamud says his motivation is to make the workings of the government more accessible at no cost and that "This is society's operating system."

• Documents of Library in Boston to Go on Web. By John Markoff, New York Times (December 27, 2007).

A digital library partnership, including two nonprofit organizations and the Boston Public Library, is preparing to begin making digital copies of the library's paper-based government documents collection, which will then be made available on the Internet.

The project, which will take two years and require the hand scanning of millions of pages of government hearings and related publications, will cost an estimated $6 million, according to the project's sponsors.

Boston Public Library librarians said they planned to begin by digitizing the House Committee on Un-American Activities hearings from the 1950s, which is regularly sought after by its patrons.

The project is being undertaken by Public.Resource.Org, a nonprofit group seeking to open public access to government records, and the Internet Archive, a San Francisco-based digital library.

The project is the brainchild of founders of the two organizations, Carl Malamud and Brewster Kahle...

Where are those rules? Be careful where you search to be sure you have found the correct and complete source.  While there are some kinds of information (e.g., phone books) that are just easier to use online than in print (and the semi-annual regulatory agenda may be one of them), just putting them online may create new access problems (for those who don't have online access) and preservation problems (will these be saved and unaltered and by whom at what cost?).  They may also present new problems of discovery when it is not easy to determine which of many versions is the complete one or the current one or which search produces the correct result.

And don't forget that one of the main motivations behind this switch is that the government wants to save money -- not expend more on preservation.  GovExec.com  says the shift to online publication of the Unified Agenda will "save" the government $800,000 (Agencies put regulatory plans online by Elizabeth Newell, December 11, 2007).

• 1,700 Pages of Rules, Fewer Dead Trees. By Cindy Skrzycki, Washington Post (December 18, 2007).

Until now, the Federal Register, the government's daily accounting of rules, printed the entire agenda in several bound volumes. It has been available online since 1995. Its search system, connected to the U.S. Government Printing Office, was challenging for some users....

The migration to http://www.reginfo.gov saves money and makes rulemaking more available to the public, administration officials said....

The agenda is available on two other government sites. The Federal Register is still required to print two categories of rules, offering the partial agenda and plan in print and at http://www.gpoaccess.gov/ua/index.html.

The other site is http://www.regulations.gov, the electronic docket for commenting on rules.

OMBWatch notes that "Since the Unified Agenda has gone paperless, you will also be unable to find entries in the PDF version of the Federal Register on the Government Printing Office website" (Agencies' Regulatory Plans Available Online,  December 12, 2007)

Challenging Cheney. Newsweek Web Exclusive (Updated: 12:57 PM ET Dec 24, 2007) By Michael Isikoff.

J. William Leonard learned the hard way the perils of questioning Vice President Dick Cheney. The veteran National Archives official challenged claims by the Office of Vice President (OVP) to be exempt from federal rules governing classified information. His efforts touched off a firestorm--and a counter-strike by Cheney's chief of staff, David Addington, who tried to wipe out Leonard's job....

Now, Leonard is quitting as director of the Archives' Information Security Oversight Office (ISOO)--the unit that monitors the handling of government secrets. He tells NEWSWEEK that his fight with Cheney's office was a "contributing" factor in his decision to retire after 34 years of government service.

..."The global struggle that we're engaged in today is more than anything else is an ideological struggle. And in my mind....that calls for greater transparency, not less transparency."

Public Access Mandate Made Law, The Alliance for Taxpayer Access. December 26, 2007.

President Bush has signed into law the Consolidated Appropriations Act of 2007 (H.R. 2764), which includes a provision directing the National Institutes of Health (NIH) to provide the public with open online access to findings from its funded research. This is the first time the U.S. government has mandated public access to research funded by a major agency.

CONGRESS DIRECTS EPA TO RE-OPEN ITS LIBRARIES. Public Employees for Environmental Responsibility (PEER). December 21, 2007

Buried within the omnibus appropriations bill Congress sent this week to President Bush is a Christmas present for the beleaguered library network of the U.S. Environmental Protection Agency. Congress ordered EPA to restore library services across the country and earmarked $3 million for that purpose...

The rationale for the library closures was never clearly spelled out by the agency, which maintained that it wanted to digitize all of its holdings. Its original claim of cost savings did not bear up under scrutiny and clashed with the enormous expense of digitizing hundreds of thousands of documents. In addition, the agency did not anticipated copyright restrictions, which barred many of its holdings from being digitized.

See more coverage of EPA libraries here.

According to today's NY TImes, J. Edgar Hoover, the head of the Federal Bureau of Investigation (FBI) from 1924 - 1972, sent a plan on July 7, 1950 to President Truman to suspend habeas corpus and imprison some 12,000 Americans he suspected of disloyalty, saying that mass arrests were necessary to “protect the country against treason, espionage and sabotage.”

Hoover’s plan was declassified Friday as part of a collection of cold-war documents concerning intelligence issues from 1950 to 1955. The collection makes up a new volume of “The Foreign Relations of the United States,” a series that by law has been published continuously by the State Department since the Civil War.   

Greetings all and happy winter solstice or your favorite year-end holiday! It's been an exciting and busy year for us here at FGI. As you can see by the recent blog posts, things have been a little slow for us in the past week or so, but that's to be expected with all the travel/planning/family/cooking etc that goes on. We'll no doubt delve back in to the news in full-force in the new year. If any of our readers are interested in being our blogger of the month in January, please drop us a line at admin AT freegovinfo DOT info.

Peace and free and open government in 2008!

The Government Printing Office released a new technical document on the Federal Digital System (FDSys) called FDsys System Release and Capabilities v5.0, December 2007.

In paging through this document, I was pleased to learn that when the latest release (1C) is deployed, FDSys will support delivery of documents in formats including PDF by FTP and RSS. The sections making note of these new features are brief:

4.10.5 Delivery by RSS
FDsys will allow users to sign up to receive DIPs via RSS.

4.10.8 Delivery by FTP
Release 1C.4 builds upon 1C.2 by allowing users to request delivery based on user defined criteria.

Depending on how these features are implemented, it could be exciting for federal documents stakeholders including depository libraries.

As previously announced, I believe, people will also be able to get notifications of new content without having to get full file downloads:

4.10.22 User Notifications
Release 1C.4 will build upon Release 1C.2 by providing additional email and RSS notifications. Users will be able to sign up to receive notifications for system events, business events, and job processing events. This includes receiving a notification when now content is added to a collection or when there is a match to a user defined string.

According to the release schedule, some of these features will appear by November 2008 and others won't be ready till 2009. But it looks like GPO is on the right track and bringing out features long sought by many in the documents community. We at FGI are cautiously optimistic.

Information Magic Eight-Ball, OMB Watch, 12/18/2007. "Over the past year, there has been a great deal of activity on issues related to government transparency and secrecy, but it can remain difficult to figure out exactly what all the discussions, reports and hearings actually mean. To try to get to the bottom of this murky issue, we are breaking out our Magic Eight-Ball of Information Policy to ask a few key questions about the past year — the progress and setbacks, laid out in simple terms. We wish there was a better approach, but unfortunately, 2007 was that kind of year for government transparency, with vague and unclear answers for most questions."

The 50-State Agency Databases Registry, which I coordinate, has launched a new set of subject-focused database collections under the heading of history:

* Biographical Databases - Databases that provide biographical sketches of authors, state officials, famous state residents, etc.
http://wikis.ala.org/godort/index.php/Biographical_Databases

* Historical Media Databases - Databases that provide online access to photographs, video, or audio.
http://wikis.ala.org/godort/index.php/Historical_Media_Databases

* Historical Newspaper and Magazine Indexes - Databases that index articles in older newspapers, journals and magazine that contain historical information. These databases will usually lead one to microfilmed items that may be obtainable through Interlibrary Loan.
http://wikis.ala.org/godort/index.php/Historical_Newspaper_and_Magazine

* Museum Collection Databases - Catalogs of state museum holdings which often have historical notes. Museums listed here are either run by a state or by one of the state's political subdivisions
http://wikis.ala.org/godort/index.php/Museum_Collection_Databases

* Official Records Databases - vital records, (birth, death, etc), war pensions, etc.
http://wikis.ala.org/godort/index.php/Official_Records_Databases

These pages just launched, so they are a little light on content. The Registry volunteers will be adding to these pages in the next few weeks.

If you are registered with the ALA GODORT wiki and would like to help the effort along, please browse the state pages or search for words from the historical categories and copy and paste databases from the state pages to the appropriate subject page.

Why these topics? Because after making solicitations here, on govdoc-l and on Facebook, more people wanted subject pages for historical materials than anything else. It's my personal hope that when these annotated historical materials pages are filled out, they will be of special value to historians and to schoolchildren stuck with "I need to do a report on the State of _____." It will help them go beyond state symbols and the like.

Hope you enjoy them!

And speaking of usaspending.gov (thanks Jim for the original post!),  Radar Magazine has analyzed federal spending on paper shredding contracts since 2000 (see chart at left) and shown that there has been a 600% increase on these contracts since George W. Bush took office. In 2000, the federal government spent $452,807 on shredding services (no doubt some appropriate and some not); by 2006, the "Cheney Effect" had bumped that number up to $2.9 million. And by halfway through 2007, the feds almost matched that number, with $2.7 million and counting. Here's the full breakdown and data from usaspending.gov.

The Office of Management and Budget is launching USASpending.gov a relaunch of www.FederalSpending.gov to providecitizens with easy access to government contract, grant, and other award data.

News stories:

• OMB Offers an Easy Way to Follow the Money, by Elizabeth Williamson, Washington Post, (December 13, 2007; Page A33)
• OMB beats deadline to unveil federal spending database, by Jason Miller, FCW (December 12, 2007)

"The database’s initial functionality will include the ability to search contracts and grants but not other types of spending data such as loans, said Robert Shea, OMB’s associate director of administration and government performance." (FCW)

The database will have an API (application programmming interface) so that programmers will be able to users to write software to access the data directly, and a wiki where users will be able to suggest changes and add information.

Yesterday, ALA submitted a statement to the U.S. Senate Committee on Homeland Security and Governmental Affairs hearing, "E-Government 2.0: Improving Innovation, Collaboration, and Access", addressing the critical yet unacknowledged role public libraries play in delivering E-government services to the American people. Witnesses included:

• Karen S. Evans, Administrator, Office of Electronic Government and Information Technology , Office of Management and Budget (PDF)
• John Lewis Needham, Manager, Public Sector Content Partnerships , Google, Inc. (PDF)
• Ari Schwartz, Deputy Director , Center for Democracy and Technology (PDF)
• Jimmy Wales, Founder , Wikipedia (PDF)

All of the witnesses had interesting things to say, and Schwartz's testimony included mention of a just-released report from the CDT and OMB Watch called, "Hiding in Plain Sight: Why Important Government Information Cannot Be Found Through Commercial Search Engines."

I really hope that the committee members hear both the great advantages that e-government can have for citizens AND the great need for much more financial support to public libraries and other organizations on the front line of e-government services. The great fallacy of new Web technologies and social media is that these tools will allow governments, libraries and other public service organizations to save money and cut physical service points. YES, e-government helps get services into citizens' hands, BUT these new technologies also necessitate MORE spending on public service points. What agencies (and libraries) seem to think is that if they have a Web service, then they don't need a physical service. But, like IM in libraries, you can't just cut hours of your service point. Instead, you need MORE staff to be able to handle the Web service and the inevitable increase in service requests (both in-person and online).

ALA statement on e-government (PDF)

"Libraries strongly support the E-government Act, since it has enhanced access to government information. However, since its enactment, public libraries are often the only organizations that can help individuals interact with government agencies and access E-government services...

Libraries have a critical role in E-Government not only as portals to access, but also organizing and categorizing information and providing the necessary tools and expertise to provide community service. Librarians provide the front line reference service that informs the public how to access and evaluate government information through both physical and virtual collections and how to train people in the use of electronic resources. Libraries help the public become information literate...

Public libraries are open to taking on the challenge of E-government initiatives, yet the library community has seen little collaboration or support from federal agencies for the significant increase in services public libraries provide on their behalf...

-- Lynne Bradley, Director of ALA's Office of Government Relations

A new ACLU report says that government "fusion centers" have not been planned in a public, open manner and that some are engaging in improper intelligence activities and endanger the privacy of citizens.

• What’s Wrong With Fusion Centers?, by Michael German and Jay Stanley, American Civil Liberties Union, December 2007.

This report is intended to serve as a primer that explains what fusion centers are, and how and why they were created. It details potential problems fusion centers present to the privacy and civil liberties of ordinary Americans, including: Ambiguous Lines of Authority... Private Sector Participation... Military Participation... Data Mining... and Excessive Secrecy.

See also: Groups seek definition of terrorism at government and private sector information-sharing centers.

Congress Wants To Allow New Web Tools by Aliya Sternstein, National Journal's Technology Daily, December 12, 2007 PM edition [subscription required]

Congress is slowly moving ahead with rule changes that would allow House members and senators to enhance their Web sites with links and content from commercial sites, like social networks and video-sharing services.

Options under consideration include permitting members to use video-streaming sites if the sites are ad-free and protect the uploaded material from tampering.

S. Res. 401--110th Congress (2007): A resolution to provide Internet access to certain Congressional Research Service publications

Bill at GovTrack

Bill at Thomas

Story at Secrecy News

Summary:

Directs the Sergeant at Arms of the Senate, in consultation with the the Director of the Congressional Research Service (CRS), to make publicly available through a centralized electronic system the following CRS-produced information: (1) Issue Briefs; (2) Reports that are available to Members of Congress through the CRS website; and (3) Authorization of Appropriations and Appropriations Products.

Exempts from such requirements: (1) any information determined to be confidential by the CRS Director or the head of a department or agency that provided the information to CRS; and (2) documents, other than those described in clauses (1) through (3), that are the product of a congressional research request.

Allows the Sergeant at Arms, in consultation with the CRS Director, to revise the information made publicly available by removing information concerning CRS employees, removing material that may infringe copyright protection, and making necessary changes to ensure accuracy and currency.

Requires access to the information made available pursuant to this resolution through Senate committee and Member websites. Makes the Sergeant at Arms responsible for maintaining and updating the information and for establishing a database within six months after adoption of this resolution.