Supporting Open Data
The call for open data has been closely linked with the global advocacy campaign for good governance founded upon the principles of transparency and accountability. Starting in North America and Europe and now spreading to the developing economies of Latin America, Africa, and Asia, country after country has announced open data policies, mined archives for releasable data sets, and remade websites to highlight data releases. A related--but separate--movement for open government has grown in recent years as well. Open government movements are laudable efforts to increase accountability and transparency through technology and citizen participation. Open government advocates have focused on the release of budget, revenue, and expenditure data needed to monitor the implantation of government programs and to identify instances of waste or abuse. Others have encouraged the release of data with immediate commercial applications, such as map files, transportation schedules, business registries, and even crime statistics.
Sometimes overlooked in all the campaigning for open government and open data are the vast quantities of valuable data produced by the national statistical system: census and survey results, national accounts and other economic and financial data, vital statistics, labor force statistics, educational records, and much more.
While one might expect national statistics offices (NSOs) to be leading advocates for open data, this has not always been the case. In some cases, international or regional organizations are trying to fill this gap. One example is an attempt to provide open data by the African Development Bank. In its web portal "Open Data for Africa", the Bank provides data from countries and international organizations and the site is "open" to anybody to place data in the portal. It does not conform however, to principles of "data openness". For many countries, the government has failed to upload any data to their portal. A country can hardly claim to own an open data portal if the government itself has not uploaded any data to it. This demonstrates even more that the countries themselves need to rally to the call for open data.
With few exceptions, the vast wealth of official statistics produced by national statistical systems remains untapped and unavailable. NSOs and their allied agencies must lead the data revolution for it to provide the knowledge needed to change the course of economic and social development. Careful strategic planning is needed to achieve this goal because opening statistical data is an elaborate, labor-intensive and time-consuming process. A vehicle to strategically plan the development of the National Statistical System is a National Strategy for the Development of Statistics (NSDS).
Creating Country Capacities for Open Data
NSDS was designed to provide a long-term strategy to build statistical capacity across national statistical systems (NSS). Many developing countries have now adopted an NSDS, though implementation has lagged. Recently there have been calls for new, more expansive types of NSDS. An NSDS that covers the entire statistical system as well as the whole process of statistical operations can provide for a more systematic, deeper approach to promoting openness than may be possible through typical efforts to open government. Despite the economic and social benefits of open data, many efforts to update NSDS have treated open data as an afterthought or have not mentioned open data at all. This is true for the recent work by Paris21 on the NSDS Guidelines where discussion about openness is hidden in the last section. Openness should be at the heart of NSDS plans.
An open NSDS should adopt the following features to develop a full commitment to open data:
- Establish the case for open data.
- Seek inputs from all stakeholders in the development of the plan.
- Consult with data users in the formulation of Annual Work Programs.
- Engage prime users with technical discussions in the process of designing major surveys and censuses.
- Establish the legal framework to promote the dissemination of open data, including granting permission for free commercial use and reuse of data.
- Make all raw data collected or compiled by the national statistical office available generally (with appropriate safeguards for safeguarding statistical confidentiality).
- Make macro indicators and social statistics available with accompanying meta-data in non-proprietary formats.
- Include realistic estimates of the resources needed for IT infrastructure (both hardware and software).
- Articulate the step-by-step development of integrated databases and meta-databases.
- Make realistic estimates for staff training and other needs for proper open data dissemination
- Ensure regular updating of the web pages associated with country subscriptions to the IMF's SDDS or GDDS frameworks.
- Conduct periodic audits employing the IMF's DQAF or other quality frameworks.
Open Data Watch is working to develop detailed guidelines for governments to make openness a core component of their national statistical systems through an NSDS. We are interested in working with partners to adapt these principles to the needs of specific countries.