Data governance frameworks for official statistics and the integration of alternative sources

1.Holistic approach to data governance

Several international initiatives have underlined the role of economic and financial statistics in supporting evidence-based policy making in the recent past. This was particularly the case with the statistical response to the Great Financial Crisis (GFC) of 2007–09 in the context of the Data Gaps Initiatives endorsed by the G20 to enhance economic and financial stability [3, 4].

The 2030 UN Agenda for sustainable development [5]33 has also recognised the crucial role that statistics can play to monitor, evaluate and track progresses on economic development. These efforts have been complemented by various regional initiatives. As regards Africa for instance, the implementation of the development Agenda 2063 has been set up as “a results-based approach with concrete targets that are measurable and can be tracked and monitored” [6].

More recently, the Covid-19 pandemic underscored the need for good and reliable statistics [7], including on a wider range of topics that are not properly covered by the “traditional” statistical apparatus, especially on environmental topics (eg climate change) and socioeconomic factors (eg inequalities); cf [8]. It also underlined the importance of high-frequency, well documented and timely indicators to support evidence-based policy, calling for statistical frameworks to become more flexible and granular with the aim of addressing the evolving needs of users and help them monitor fragilities [9]. Another lesson is that the (unexpected) nature of the shock has clearly expanded the range of statistics that public authorities must look at, and highlighted the need to look at less conventional and still untapped sources of alternative information [10].

The renewed interest in the interaction between economic policy and factual evidence underscores the sheer importance of establishing proper governance when dealing with, and using, official statistics.

Yet this concept of data governance can be unclear, reflecting the proliferation of approaches developed over time across institutions and countries. In some places, data governance may be considered in a narrow way, by focussing essentially on data protection and its associated legislation.

But the focus should be broader, covering various intertwined topics from data management to data use. A holistic approach to data governance is thus warranted to consider “everything designed to inform the extent of confidence in data management, data use and the technologies derived from it” [11]. The frameworks put in place to govern corporate data ecosystems will typically include three layers: strategic, tactical, and deployment (delivery); cf Table 1 in the case of the public sector.

2.Governance frameworks for the NSS

What framework should be implemented to follow the holistic approach to data governance as advocated above? In practice, the data governance frameworks adopted by those institutions of the national statistical systems (NSSs), especially National Statistical Offices (NSOs) – or National Statistical Institutes (NSIs) – and central banks’ statistical departments44comprise the various organisational features put in place for dealing with official statistics and cover all the related principles, policies and procedures, structures, roles and responsibilities.

In the European case, for instance, “data governance entails defining, implementing and monitoring strategies, policies and shared decision-making over the management and use of data assets”. In particular, “data policies are a set of broad, high level principles which form the guiding framework in which data assets (…) can be managed. More specifically, data policies govern data management, data interoperability and standards, data quality, data protection and information security” [13].

One main objective of such data governance frameworks is to ensure the “quality” of this information considered as “a strategic institutional asset” [14]. However, it is certainly the case that there is no single definition of “quality”. One starting point, as mentioned in the UN National Quality Assurance Frameworks Manual for Official Statistics (UN NQAF Manual) – a manual developed to guide countries in the implementation of quality assurance frameworks, including for new data sources, new data providers, and for data and statistics of the SDGs [15] – is the definition set up by the International Organization for Standardization’s ISO 8000 global standard for data quality and enterprise master data ((ISO) 9000:2015):55 “quality is the degree to which a set of inherent characteristics of an object fulfils requirements”. In general, data quality will thus comprise the totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs.

Looking more precisely into the area of official statistics, the “object” that needs to “fulfils requirements” is quite large and refers in the UN NQAF Manual to “the statistical output or product, the process, the institutional environment or the whole statistical system”. Moreover, and as stated by the OECD Glossary of Statistical Terms, “the quality characteristics of most importance depend on user perspectives, needs and priorities, which vary across groups of users”. Indeed, the approaches followed by various statistical bodies across the world show that the concept of the quality of the data used to produce official statistics is typically a multi-faceted concept. One can nevertheless identify two main features:

At the operational level, first, the work to ensure quality to statistical products and processes typically follow a relatively detailed, operational approach, by focussing on a number of key attributes of data quality.

A second level is to have a high-level approach complementing the operational guidelines referred to above. The emphasis is on the ultimate goals of the governance framework, in particular to ensure trust and quality of official statistics across the entire NSS, as emphasised by the UN NQAF Manual. Such a “holistic” approach is indeed referred to in many jurisdictions:

As a result, one can define quality as referring to the various elements – including specific operational guidelines but also more general high-level values – that support the Fundamental Principles that have been defined for the perimeter of Official Statistics [25]. In fact, the requirements related to the implementation of the UN NQAF Manual are explicitly “inspired by and consistent with the Fundamental Principles”. These Principles act as general propositions to which statistical bodies are committed and that guide them in meeting their quality-related work [26].

In summary, “quality” will generally be understood as a generic term covering various characteristics sought for official statistics, ie their accuracy and trustworthiness, integrity and security, and that they are documented and easy to find/access. Yet the approach will also cover the user side, for instance to ensure that the data are fit-for-purpose, that their value is maximised, and that they can be traced, reused, and eventually adequately deleted. Perhaps more importantly, it will also include more ethical aspects, such as preserving public trust in official statistics and ensuring that the information collected is not misused and does not undermine personal privacy, confidentiality or democratic principles.

3.Four main steps in the information chain

In practice, a sound data governance framework should encompass the four main steps involved in the information chain supporting the production of official statistics, namely:

Obviously, this is a quite schematic list, and the various steps above can involve many technical issues. The purpose of this paper is simply to focus on those salient points that can represent important challenges in terms of data governance for official statisticians – for a more detailed analysis, cf [27].

As regards first data collection, “traditional” statistical surveys and censuses can increasingly be complemented with new information sources, including administrative records [28]. Big data, described by some as the new oil of the 21st century [29], can also be a great opportunity for those less developed statistical systems, not least considering the high costs associated with setting up standard data collections. Yet there are a number of challenges posed by accessing these new data, for instance in terms of quality and accuracy, privacy, ownership, and reputation. In particular, one issue of key relevance for public statisticians is how alternative information sources (being private commercial data sets or public registers that were not initially set up for a statistical purpose) and the data producers located outside of the NSS feature vis-à-vis the Fundamental Principles of Official Statistics. For example, the production of the statistics in a professionally independent way, based on scientific methods and rigorous quality criteria. Indeed, a major point is that, by using alternative data, authorities would be perceived as endorsing the methodologies applied by third parties; hence, any concerns about the quality of these data could damage the credibility of official numbers, as well as of data-driven public policies. This also raises the question of whether and how national statistical legislation and quality assurance frameworks should be revised accordingly [30].

The recent experience of Slovenia suggests that governance can be a key element to address such challenges. The country has been using different sources since 2011 to produce the census with greater frequency, ie every three years, and more efficiently, since the integration of already collected data sets has not required any new field operation. But this achievement called for lengthy coordination efforts both inside the NSO and within the NSS, as well as with other producers of administrative data. The establishment of a sound governance framework was essential to ensure the smooth functioning of such a complex organisation dependent on so many partners. This allowed in particular to ensure comprehensive data acquisition and management processes in turn supporting a better quality in the provision of official statistics. A key element was to enhance the overall relationship between governance and data quality in the NSS by making owners of information sources aware of the need to improve the quality of their data before they can be used; strengthening the role that official statisticians could play in this endeavour; and identifying ways to organise the related partnerships and processes [31].

Turning to the second area of data management, there are a wide range of governance challenges, for instance in terms of data access, storage, manipulation, security and quality controls. Yet a major point of focus with the advent of the information revolution is the fact that the combination of new and more heterogeneous data types and evolving analytical needs requires more, and sometimes different types of resources (in terms of IT equipment, staff skill mix, budget for acquiring new databases, etc; cf [32]). This may prove particularly difficult in the least advanced countries, where an important issue is how to build and keep sufficient technical expertise in the NSS. In the light of frequent and acute budget constraints, capacity building can be ensured both by joining forces between relevant national bodies and drawing on all international support available. Another important topic for official statistics is related to the implications of globalisation: national compilers have to access information sources that are not locally available, and this can be difficult given budget constraints, technical considerations, and/or data sharing limitations.66

Third, disseminating data also poses clear governance challenges, since, as noted above, a key quality dimension is to ensure that official statistics are fit-for-purpose, address the needs of various user groups, and are well understood especially by citizens. In this area too, important developments have been triggered by the information revolution. On the one hand, new digitalisation techniques allow for easier, almost cost-free access to information, which can be a great opportunity for populations with limited resources, especially in developing countries. On the other hand, the increasing complexity of economic and financial activities puts a premium on statistical education and financial literacy. National financial inclusion policies have been particularly sensitive to this issue, given that measures to facilitate households’ and small firms’ access to credit and payment services can only be effective if economic agents have a good understanding of the related financial aspects including data [34]. Again, this topic is clearly relevant for developing regions, where the promotion of financial inclusion is often a key policy objective.

Fourth, there has been increasing interest globally in using data better for policy purposes, especially when designing, calibrating, assessing and modifying public actions [35]. The experience gained has underlined the importance of having highly granular and flexible data sets. This can facilitate the capturing of distinct, specific dimensions in the data that are relevant for analysing policy actions – in turn supporting ex-ante impact assessment, understanding feedback effects and unintended consequences, and identifying areas of improvement [36]. Yet what is still unclear is how such indicator-based frameworks should be developed so that adequate lessons can be drawn from the actions of public authorities. Moreover, the implications for official statisticians are yet to be fully incorporated, especially as regards the national rules governing data access and data sharing possibilities – for instance, when trying to make use of information collected from supervisory reports [37]. Needless to say, there is also an urgent need to tackle the associated challenges in terms of data protection, privacy and confidentiality – especially when dealing with internet-based data [38].

These various elements clearly underscore the importance of revisiting data governance frameworks for organisations in charge of official statistics – a matter particularly high on the agenda of central banks, given their distinctive dual role of being both producers and users of official statistics [39].

For their part, central banks have indeed worked extensively on issues related to information management – including rapid improvements in technology (the “big data revolution”) – and data dissemination, sharing and standards (eg [40]). This experience has helped to share experiences and highlight existing best practices and potential opportunities, especially to support data-driven policymaking and address priority challenges.

4.Looking forward: Data governance and the evolving role of the NSS

Many principles have been developed in general with a focus on data quality and governance, including outside the perimeter of official statistics. In particular, significant efforts have been devoted to promoting the development of professional values shared by all (public and private) statisticians, for instance with the ISI declaration on professional ethics in 2010 [41]. Its principles are comparable to the UN fundamental principles but with the aim of covering the wide range of the statistical profession working within a variety of economic, cultural, legal and political settings. Moreover, and as reported by [42], a number of private firms involved in the production of alternative sources have also adopted dedicated and transparent mission statements and principles pursuing similar objectives.

This is particularly welcome because of the importance of the growing interlinkages between the governance frameworks in the NSS and those in the rest of the society, reflecting the greater use for official statistical purposes of those data sources located in the public and private sectors with the advent of the information revolution. This highlights the need for a sound and strong legislative base to ensure that internationallya-greed standards governing statistical data are properly followed across the country.

It is fair to say, however, that the international statistical community has been leading the way by establishing clear principles to ensure that the production of official statistics relies on strong ethical considerations. In particular, one of the key objectives enshrined in international codes for public statisticians is to collect and analyse data of the “highest quality possible”, with due respect for privacy and confidentiality [43].

A well-defined data governance framework can clarify how this “data quality” can be ensured at the level of individual institutions participating in the NSS. It would, in particular, document what “high level principles” are followed to secure confidence in data and public trust. For instance, the UK British Academy and The Royal Society have argued that the “overarching principle is that systems of data governance should promote human flourishing” [44]. They recommend following four high-level principles in this endeavour, which are to: (i) protect individual and collective rights and interests; (ii) ensure that trade-offs affected by data management and data use are made transparently, accountably and inclusively; (iii) seek out good practices and learn from success and failure; and (iv) enhance existing democratic governance. An effective data governance framework should also document that adequate methods are applied to preserve confidentiality, clarify data ownership and responsibilities, provide a frame to balance the benefits against the risks posed by using confidential information, and help to promote debates on these issues in the profession and publicly.

A common NSS-wide scheme is however needed to complement such institution-specific data governance frameworks that need to fit into the broader one to ensure the protection and dissemination of the key features of official statistics at the country level. This is especially important to:

In view of this need for a holistic framework, there is a case for clarifying responsibilities related to the evolution of the national data governance landscape as a whole. In the United Kingdom, a dedicated and independent advisory body (the Centre for Data Ethics and Innovation) has been set up to steward the country’s data governance framework and, in particular, connect policymakers, industry, civil society and the public to develop the right governance regime for data-driven technologies. This initiative has been complemented by the set-up of a National Statistician’s Data Ethics Advisory Committee77 to provide transparent and timely ethical advice to official statisticians more specifically. The goal is to ensure that access, use and sharing of public data for research and statistical purposes is ethical and conducted for the public good. This body also reviews the appropriateness of the various projects and policy proposals for using novel alternative data. Turning to Norway, the groups of Norwegian National Research Ethics Committees ([46]) have worked on identifying ethical challenges associated with artificial intelligence and big data tools and sources, especially as regards the issues of human dignity, responsibility, transparency, dissemination, uncertainty, privacy, fair data access and data quality.

At the international level, the Global Partnership for Sustainable Development Data has established an Ethics and Integrity Framework to favour the application of ethical considerations in carrying out the work of the global network of data communities; in addition a clear strategy has been defined to ensure that the new opportunities of the data revolution are used to achieve the SDGs [47, 48]. In particular, a data charter has been set up that emphasises key data governance principles, such as the need to cover all the population, the promotion of data for public use, the need for data granularity and for drawing from all available sources, the accountability of data collectors, and human and technical capacity requirements.

Another important initiative has been the call by the World Bank to build on the changing data landscape to foster development goals. In particular, the World Development Report 2021 underlined the sheer importance of proper data governance arrangements to support the generation and use of data in a safe, ethical, equitably and secure way [49]. It calls in this context for a new social contract that enables the use and reuse of data to create economic and social value, ensures equitable access to that value, and fosters trust that data will not be misused in harmful ways, and also emphasises the importance of working toward an integrated national data system.

NSS organisations and in particular NSOs and central banks are playing a driving role in these developments to reinforce the value of official statistics at national, regional and global levels. This suggests a fundamental evolution in their mission, given that their traditional function of “data collectors” is diminishing (at least in relative terms) with the growing availability of alternative sources. As highlighted by Križman [50], official statisticians have to adapt to a range of new challenges. These include competencies (impact of IT innovation, understanding of new big data sources and tools), legal frameworks (to get access to data but also coordinate the compilation of statistics and ensure quality assurance including for data collected from outside the NSS), and political support (to secure public commitments and the provision of adequate resources).

NSS bodies have therefore a key role to play as reference custodians of the quality of the data used by society. In other words, they need to become “data curators”,88 by developing the use of (external) secondary data, providing managerial, methodological and technical standards and guidance, and ensuring trust in all the stakeholders involved in statistical compilation. They could therefore play a leading role in favouring the integration of national data systems with the view of addressing the growing challenges posed by the information revolution in terms of excessive data collection, insufficient governance of data held by private firms, and inadequate protection of personal data [52].

One view is that NSS bodies should from this perspective get an expanded mandate to provide assurance on the quality of statistics derived from big data sources. In the absence of internationally accepted standards, a related certification process could be established when designing national quality assurance frameworks. For instance, it has been argued that the legal framework for official statistics should be enhanced to include the new roles of NSOs as “data stewards” that should, in particular, becoming more user centric; playing a larger role in governance of data (eg by advising the government about strategic issues related to statistics and data and providing expertise for developing data hubs possibly also outside government); and playing a bigger role in ensuring the quality of information used in decision-making [53].

In this perspective, the international statistical community has recently reinforced the relevance of the Fundamental Principles for official statistics to navigate current times of significant change – including the proliferation of data sources being used, the new technologies being harnessed, and the expanding need for data stewards who can curate data from a range of other sources [54]. Whatever the specific institutional arrangements considered, the main objective should be to strengthen the national statistical infrastructure by enhancing the relationship between the NSS and external data compilers. Needless to say, establishing sound data governance frameworks would be an essential element in developing this “data curator approach”.

5.Conclusion

Official statistics can play a decisive role in measuring, monitoring and evaluating the implementation of major international initiatives supporting development. But the advent of the information revolution is posing important challenges when dealing with, and using, official data. To address these challenges effectively, a key point is to have an all-encompassing approach to data governance when collecting, managing, disseminating and making use of official statistics. This should cover all the related principles, policies and procedures, structures, roles and responsibilities. Another aspect is the importance of proper data governance frameworks for NSSs’ organisations, to reap the full benefits of the ongoing “data revolution”. Such governance frameworks should cover the entire organisations and be an integral part of their strategic plans.

As regards data collection, traditional statistical surveys and censuses can be usefully complemented with new types of information, eg alternative data sources including administrative registers and web-based records (big data). This can be a great opportunity for those less developed statistical systems, not least considering the high costs associated with setting up and maintaining standard exercises [55]. One risk from this perspective is the hoarding of the vast data resources collected outside the official perimeter of statistics if one fails to incorporate them properly to support the measurement of economic indicators.

Second, there are clear challenges related to NSSs’ management of the evolving data ecosystem. In particular, what is unclear is how to deal with “organic” information sources, whether private commercial data sets or public registers that were not initially set up for a statistical purpose, and which may not pass the test of time. Sticking to long-established and internationally agreed practices and standards, preserving sufficient “traditional” statistical capacity in the NSS and favouring a complementary use of both traditional and alternative data sources are central to maintaining well-founded trust in official statistics.

Third, turning to data dissemination, digitalisation techniques allow for easier, almost cost-free access to information for the public. However, the increasing complexity of economic and financial activities in a data-rich world puts a premium on statistical education and financial literacy. In addition, official statistics are essential to provide reference, objective information and in turn support economic development and well-being.

Fourth, there has been a growing interest globally for the better use of data for policy purposes, especially when designing, calibrating, assessing and modifying public actions. But the development of such indicator-based policy frameworks is facing important obstacles, reflecting existing limitations to effective and seamless data access and the sharing of official statistics. One way to go is to promote the exchange of experience among institutions and countries in addressing these challenges in an effective and practical way.

Looking forward, well-defined data governance frameworks can be instrumental in supporting official statisticians’ task to collect and analyse data of the highest quality possible. However, an institution-level approach to data governance should be complemented by a broader focus covering the entire production and use of statistics, including alternative sources. For instance, ensuring the following of adequate Codes of Principles by private data providers, clarifying the responsibilities in the national governance landscape, and establishing proper international guidelines and cooperation mechanisms.

Perhaps more importantly, while NSS organisations are facing a decline in their traditional function of “data collectors”, they have a key role to play as reference custodians of the quality of the data used by society. Needless to say, establishing sound data governance frameworks can be a central element in supporting this “data curator approach”.