Earlier this year, I wrote about the budding momentum behind governments making their data more readily available to the public for reuse. Open data initiatives have generated dozens of commercial and non-commercial websites that add value to the government data. Some make the data more understandable by using interactive maps to provide visuals about where activities are taking place (e.g. government stimulus spending). Others make the data more accessible by offering services to customize or deliver government information (e.g. postal codes to allow public interest groups to launch advocacy campaigns).
The crucial aspect behind these initiatives is that the government makes the data available in open formats free from restrictive licences so companies and civil society groups can create innovative websites, tools, and online services.
My weekly technology law column (Toronto Star version, homepage version) notes that last week the global open data movement received a big boost in three countries that is sure to leave Canadians wondering why their government has been so slow to move on this issue.
The U.S. issued its much-anticipated Open Government Directive, instructing every federal department and agency to take specific actions to open its operations to the public. Rather than simply identifying principles, the directive issued strict timelines for action.
For example, it requires agencies to publish "information online in an open format that can be retrieved, downloaded, indexed, and searched by commonly used web search applications. An open format is one that is platform independent, machine readable, and made available to the public without restrictions that would impede the re-use of that information." Each agency is required to publish at least three datasets within 45 days and to establish an open government section on its website.
Not to be outdone, the Australian Government 2.0 Task Force issued its draft report on how to make government information more accessible and usable. The task force's starting premise is that "public sector information is a national resource, and that releasing as much of it on as permissive terms as possible will maximize its economic and social value and reinforce a healthy democracy."
Consistent with that view, the Task Force recommends that public sector information should be free, based on open standards, easily discoverable, machine-readable, and freely reusable. Since Australian government data is subject to crown copyright restrictions (much like Canada), the Task Force recommends releasing government data under a Creative Commons attribution licence.
This means that the government will still maintain copyright, but it freely licences the work for re-use with no need for further permissions or compensation (only attribution is required). This approach provides an efficient means of freeing up government works without the need for legislative change.
The British government also made new open government commitments. Noting the success of recent initiatives – an online graffiti reporting site resulted in an 8 percent reduction in graffiti and a 30 percent reduction in complaints – it adopted new public data principles similarly based on the release of public datasets available for reuse at no charge. It now promises to release more public data, including health, weather and traffic datasets, under open licences that enables free reuse, including commercial reuse.
These new initiatives herald a dramatic shift in the way governments use the Internet to make themselves more transparent and useful to their citizens. They may also leave Canadians asking if their government is not prepared to lead, then why not at least follow?
Not A Surprise
Transparency seems to be the enemy of governments these days I’m surprised they agreed to it at all but I guess not following through was their plan all along.
Digital Canada
I certainly hope an open government initiative makes it into the upcoming Digital Canada report! Of course, even if it is, it will probably be anemic, poorly organized, lacking funding, and strangled by Crown Copyright.
Why not open follow?
Because our Harper government refuses to realease any information besides photo opportunities.
Data Sales vs Open Data
I work for a municipal government where we sell data and market our data. How do I argue for open data when that would cut into a revenue stream for the government?
Sales of Public Data
One option would be to release the information to the public under a Creative Commons-Non-commercial license. This would allow everyday citizens to use and re-use your data, but not for commercial gain. While I disagree that any public data should be for sale, you can argue that it will increase the marketability of your data. Some degree of transparency is better than none.
If the data is available to developers, they can start to experiment, to play, to see what is possible. If your dataset is machine-accessible and in a non-proprietary format, and contains useful information, the developers will be able to do things you never dreamed of doing with it. Some of it will have commercial merit, much won’t.
If their idea doesn’t have commercial merit, but is still socially valuable, your community benefits without sacrificing any royalties, while allowing you to showcase how your data is being used. If it does have commercial merit, it will increase the royalties you will be able to collect because these ideas may never have gotten off the ground otherwise.
@Abattoir
Well put. The idea of someone commercializing public data (with no value added) is like someone selling the air we breathe.
Can someone speak to non-domestic users of US government data? I have read somewhere (but can’t for the life of me remember where) an opinion that the US government may still hold something akin to “crown copyright” for foreign uses of the data. They have, and do, restrict access to datasets; for instance just try to get a copy of the STS generated DTED Level-2 data from NGOA or another US government site. Ditto for USFIF info (although the issues surrounding that was the US government was publishing data that was under Australian Crown Copyright if memory serves).
Thanks for blogging about this topic! Although the Feds in the current political regime are definitely still in the mindset of controlling the message (and therefore not going to adapt an open government approach) it’s worth noting that a number of cities are already forging ahead. Vancouver’s at the forefront, with Toronto in second place and Ottawa investigating it’s options as well.
Two thoughts: 1. Signal-to-noise ratio; 2. Copyright and factual data
MG: “The U.S. issued its much-anticipated Open Government Directive, instructing every federal department and agency to take specific actions to open its operations to the public.”
Yes, however keep in mind methodology-shifting http://www.shadowstats.com/ and shrouding techniques such as off-balance-sheet accounting http://online.wsj.com/article/SB124407146605483021.html which the US government itself uses a great deal.
In any case, between a directive and the reality of a normalized, trusted, accessible, meaningful data supply is a significant body of intellectually difficult and time-consuming work. Perhaps a good indicator of what’s underway will be to monitor data to establish whether US government employment/contracting of database analysts and IM professionals goes up or down in FY2010-11. It’s quite specialized work, so if that does not go up, I would get the impression the directive is not having much impact towards improving data quality. Sure more “data” will be disseminated, and some of it will even be useful. But will data which deeply affects lives be cleaned up, mapped across domains, and be suitably documented according to standard? How will data users establish the signal-to-noise ratio?
MG: “Since Australian government data is subject to Crown copyright restrictions (much like Canada)”
A compilation of factual data is not covered by copyright. The boundary line regarding the applicability of copyright law to data was clarified in a 1997 case at the Canadian Federal Court of Appeal (TeleDirect Inc. v. American Business Information Inc.,). Théberge v. Galerie d’Art du Petit Champlain inc., 2002 SCC 34 (CanLII), online:
In his decision, Judge J.A. Denault explained: “Under subsection 5(1) of the (Copyright) Act, copyright subsists not in a compilation of data per se, but in an original work… the selection or arrangement of data only results in a protected compilation if the end result qualifies as an original intellectual creation”. Denault reiterated a United States Supreme Court decision (Feist Publications, Inc. v. Rural Tel. Serv. Co., 499 U.S. 340 (1991), online: http://supreme.justia.com/us/499/340/ ) which found that listings of routine factual data, such as names, towns and telephone numbers in a telephone directory, are uncopyrightable facts, because they are not selected, coordinated, or arranged in an original way.
Micheal, can you share some thoughts about how regular folk inside and outside government might distinguish between data that we should expect to be covered by copyright law, and data that is not likely to be, based on the Denault ruling?
Anon-K
AFAIK, the US federal government is forborne from claiming copyright on any work they created. Certainly there are no licensing restrictions for foreigners on any data published at data.gov.
Open Data
One of the key problems with the open data movement is that advocates have not really pushed the simple idea that the data stored in federal and provincial data repositories are actually owned by the public and not one agency or government. The US government has adopted that simple notion and it has advanced the notion of open data significantly down there.
Unfortunately in Canada, provincial governments and the federal government cling unequivocally to the parochial view that information within their data repositories are governmental property with full copyright/license/ownership. Usually the data is claimed to be owned by government departments rather than the overall ‘crown’.
As an example, provincial departments of health often have stringent policies that force researchers to acknowledge the copyright license of departments when utilizing population based health data. And they place tight restrictions on data access using their ‘custodian’ status as a weapon. They will claim to be supporting ethics through stringent privacy assessments – however they often turn their backs on letting university based research ethics boards to do those privacy assessments – preferring instead to have their own bureaucrats do that work. Which could lead us to ask a question- are they really so concerned about privacy or a loss of information control? Are they worried that if data is released to researchers that embarrassing revelations about government mismanagement may come to light?
The failure to push governments into accepting that they dont actually own, have copyright or singular license to the data they collect is a problem for open government and it also creates a completely dysfunctional policy environment within government. Departments hog data -not just from outsiders and citizens but even from other departments within their civil services.
In a round about way what I’m saying is – the movement for open data is completely roadblocked by boneheaded turf battles between government bureaucrats.
Open Data
I wonder as well if there is a full list of the various data repositories within all government departments across Canada. A basic problem of access is not only the bureaucratic battles but also the simple fact that it is impossible to get access to data sources that are not known to exist!
($)
Here are some of the prices Statistics Canada charges for access to individual data sets:
Road Network and Geographic Attribute File, Canada $2500
http://www.statcan.gc.ca/bsolc/olc-cel/olc-cel?lang=eng&catno=92-155-XWE2006001&Go=Purchase
Postal Code Conversion File, 2006 Census $10,000.00
http://www.statcan.gc.ca/bsolc/olc-cel/olc-cel?catno=92-153-X&lang=eng
Postal Codes by Federal Ridings File (PCFRF) 2003 Representation Order, 2001 Census $2,500.00
http://www.statcan.gc.ca/bsolc/olc-cel/olc-cel?lang=eng&catno=92F0193X
Province of Ontario Ministry of Natural Resources – Land Information Ontario $16/MB +$100 administration fee
http://www.mnr.gov.on.ca/en/Business/LIO/2ColumnSubPage/272487.html