How Open Data Can Strengthen National Statistical Offices
In the last five years, many
national governments have
initiatives, and states and cities have joined in.
Releasing data openly
should make governments more credible. Citizens perceive greater transparency and accountability when they can measure results.
They see the impact of open data on improved service delivery.
Entrepreneurs create value-added applications from open datasets.
Data quality improves when data are well documented and open to public review.
But many national statistical offices, particularly in developing countries, continue to watch the open data movement from a distance, either afraid to act or uninterested. Asked about their concerns, statisticians cite worries about data quality, cost, privacy, and information technology issues. The pinch is that open data is not a fad. The increasing momentum of the open data movement shows that it is here to stay. And the progress already made to open data created in other government ministries and agencies is impressive. So why are many national statistical offices still on the sidelines? They have a lot to offer. After all, they have been in the information business longer than any other government department. If national statistical offices can overcome their worries and address the challenges of open data, the potential opportunities and benefits for everyone are enormous.
What are some of the opportunities for the
national statistical office? Most important
national statistical offices
can and should
respond to the growing demand for data: at home from citizens and
civil society organizations,
businesses and news media, and from the global initiatives such as Rio+20 and the post-2015
sustainable development agenda and other data-intensive
global programs such as
the Global Alliance for Vaccines and Immunization (GAVI), the
Open Budget Initiative, and the
International Aid Transparency Initiative (IATI), among many others.
Another benefit for the
national statistical office is
support improvements on the
including modernization of statistical systems,
IT infrastructure, and
user services. By better responding to demand and improving the efficiency of the supply side,
national statistical offices
audience and raise their profile and influence, which
in turn, help with resource mobilization.
these opportunities have modernized their statistical systems,
improved data coverage, and reached a larger audience.
Good examples are
Britain, Canada, New Zealand, Kenya, and Ghana
the World Bank.
Measuring the benefits of open data
What do we know about the impact and
open data? One of the reasons for interest in Open Data is its potential economic value.
A recent report by
estimated the economic value of open
data at $3 trillion per year, which reflects the commercial value of opening up public data.
Another view of the value of open data comes from the open government community. A survey of country experts who provided assessments for the Open Data Barometer assessed the impact of open government data.
They found the highest impact to be on transparency and accountability of governments. It is interesting to note that impact of open data on economic growth and environmental sustainability is less widely recognized.
Figure 1: The Impact of Open Data
What data are open?
There are many definitions of open data, but they all agree broadly on these five points:
data are free, available to all, accessible, licensed for use and reuse, and well documented.
Figure 2 shows
an example of how the open data definition has
been applied to assessing the openness of government datasets.
The Open Data Index uses 9 criteria and applies them to 10 categories of data in 70 countries.
indexes are popular because they summarize a lot of
information – even if they oversimplify. In this example, the 9 characteristics are applied to datasets providing public transport time tables. The goal is to see how far countries have gone in opening their datasets.
Figure 2: Rating Openness
Source: Open Knowledge Foundation
Much of the focus on the open data movement has been on government data that are not produced by the national statistical
system, such as
maps, election results, legislative records, and the budget. Of
the ten datasets included
the Open Data
Index, perhaps three
customary responsibility of the national statistical office.
included in the Open Data Barometer six or seven are outputs of the national statistical system.
Notably missing from both indexes are labor force statistics, money and banking statistics, price statistics, and measures of poverty and social welfare. (See “Monitoring Data Quality and Openness” for further discussion of the two indexes.) In many cases, national statistical offices already have dissemination programs that make many of these data public, although they may not satisfy all the requirements for open data.
Our goal should be that
major outputs of the national statistical system
national accounts and balance of payments, labor force,
and household and business surveys,
evaluated by such assessment systems.
Whose data are open?
So far the attention of the open government and open data movements has been largely on high-income
and middle-income countries.
Most low-income countries
have been left out.
Open Government Partnership launched in 2011 has been a prominent advocate for open data. There are now 64 countries
that are members of the partnership and have subscribed to its principles. Of these 27 were classified by
the World Bank as high-income (GNI per capita greater than $12,616 in 2012) and 20 as upper-middle-income (GNI per capita between $4,085 and $12,615). There are 12 lower-middle-income members and only 5 low-income (GNI per capita
of $1,035 or
less). The regional distribution of members is highly skewed toward Europe and the Americas. Only 7 members come from Sub-Saharan Africa. The Open Data Index and Open Data Barometer have a similar bias. Both omit more than half the countries classified as low- or middle-income.
Including developing countries is a major challenge for the open government and open data movements.
offices should be at the center of this effort. Even where high-level political commitment is lacking, national statistical offices
provide a natural entry point.
Under the United Nations Fundamental Principles of Official Statistics statistical offices
have a mandate to publish their data under terms that closely parallel standards for open data. National statisticians
participate in international
networks through the UN Statistical Commission, regional economic commissions, and non-governmental bodies such as the International Statistical Institute. But for many poor countries, there is a significant gap between the willingness of
office to provide data and its capacity to do so.
Support for Open Data
The good news is that support for open data is growing.
The World Bank’s Open Data Readiness Assessment tool identifies the steps necessary to create an open statistical system and complements national strategies for the development of statistics.
There are new partnerships around open data, which bring with them access to valuable technical support and knowledge.
The recently formed Partnership for Open Data has resources to promote the benefits of open data in developing countries. So far
financial support from
has been meager, but some are beginning to show interest in open data reforms.
Some trust fund and grant donors seem to be interested in folding open data projects into larger statistical systems improvement projects.
More needs to be done to mainstream open data
national statistical development plans, and
national statistical offices need to increase their efforts to tap into this growing source of knowledge and technical and financial support.
What Can NSOs Do to Open Data?
National statistical offices can play a key role in an open
data initiative. Our advice, based on the experience of those that have gone ahead, is for
national statistical offices to align their work with the national
open data initiative, where possible. And if there is none, go ahead anyway. We
recommend updating the national statistical plans and dissemination policies. We also
encourage statistical offices to
take advantage of partnerships with others who have done it and to reach out to current and future
users and stakeholders.
Open Data is a change management program.
Like any change management program
there is some pain
for the gain.
NSOs have to organize, standardize, and prioritize.
data sites have seen huge
increases in the number of users. For example, the World Bank saw its “hits” increase by 20 times after it opened its databases.
NSOs will also gain better quality data and metadata and
lower costs of dissemination through the investments in the user interface and
bulk download facilities.
the Data Revolution
data can indeed be the key to unlocking the data revolution, helping
national statistical offices
improve their statistical systems and better meet their users’ needs. To mobilize and coordinate a global data revolution including open data as one of its goals,
we need to build a broad coalition in support of
data and we need donors to provide support for open data that focuses on building national capacity (and not just extracting data). As partners we should encourage innovation and sharing of knowledge and tools.
And as the High Level Panel reminded us, we must always be willing to measure impacts and account for our results.
This article is based on a talk given by
Badiee to the Seminar on Emerging Issues “Managing the Data Revolution” on 29 February 2014 in conjunction with the 45th session of the UN Statistical Commission. A copy of the PowerPoint slides from the talk is available on the ODW website.