Special section: Understanding and assessing the value of official statistics

“It is a truth universally acknowledged, that official statistics are uniquely valuable to society”. So wrote Jane Austen, give or take a few words.

Internationally-convened discussions, debates, task forces and seminars on the topic of ‘the value of official statistics’ have proliferated over the past decade. As far back as 2014, the Conference of European Statisticians (CES) held a seminar for chief statisticians https://unece.org/info/events/event/19645during its 62nd Plenary Session to discuss the questions, “What is the value of official statistics and how do we communicate that value?”

The title of this seminar serves as a clear example of a crucial fact. Almost all of these high-level intergovernmental discussions take as a basic premise that this value exists – usually endowed by the unique features of official statistics driving from the Fundamental Principles of Official Statistics – and simply needs to be better promoted and communicated. That we in the official statistics world (like most statisticians, let’s be honest) are good at what we do, but not so great at explaining it to others. That we have an image problem, not a product problem.

The fact is, whether in these task forces and seminars or in anything else we do, we in the official statistics community tend to take our unique value so completely for granted that we never question it. We know in our hearts that official statistics are the cornerstone of informed decision-making for governments; that they allow businesses and individuals to make evidence-based decisions about things that matter to them; that they empower the public to hold their leaders to account, armed with independent facts about the economy, society and the environment.

We realize that the ability of official statistics to fulfil these roles stems from the Fundamental Principles of Official Statistics which, universally adopted by all the world’s countries via the United Nations General Assembly, enshrine the political independence, scientific rigour and ethical assurance of our work.

To borrow the words of some great voices from the past, we hold these truths to be self-evident. So certain are we of all these things, in fact, that we never stop for a second to ask whether they’re actually true. Instead, we focus our efforts on how better to get the message across to users or potential users that they ought to use our official figures. The intergovernmental discussions tend to coalesce around conclusions that, as an industry, we need to keep producing the same things but work a bit on our messaging.

As chair (Potter) and secretary (Willis-Núñez) to a CES task force that ran from 2018 to 2022, we had the privilege of spending time with a range of experts from around the world, to totally pick these assumptions to pieces. Our paper in this issue summarizes the thrust of our conclusions, and argues that if we as an industry and as a field continue to operate in this rather self-satisfied way, we risk becoming irrelevant.

The ideas that shaped the work of the CES task force, and which underlie the arguments presented in our paper in this issue, are the result of collaborative thinking among a group of people spanning a range of countries and backgrounds. The list is too long to name them all here, but all are acknowledged in https://unece.org/sites/default/files/2022-11/VOS_ReportforWeb_withCovers.pdf the 2022 report. Some were instrumental in developing the arguments we present. Giles Sullivan, an independent strategy expert in New Zealand, whom we could call the ‘Value Evangelist’, is responsible for converting us to this way of thinking in the first place. It was he who made us realize that we, as a group, were initially guilty of thinking in exactly the way that we now critique: taking the value of official statistics as an unexamined given and simply trying to find easy metrics to reassure us of what we already thought we knew. Andrea Ordaz-Németh, with her meticulous linguist’s approach to clarifying concepts and organizing ideas, examined the literature and developed the typology of value which led us to see the distinction between production-based and consumer-based understandings of value. Juan Ignacio de Anda, Andrea Fernandez, Angéla Kátainé Marosi, Olga Świerkot-Strużewska and many others played key roles in developing the thinking that brought us to this paper and to this special issue of the SJIAOS. Nevertheless, any errors or ommissions are ours alone.

The variety of papers received for this special issue shows just how broad-ranging the idea of the ‘value of official statistics’ is. When we issued the call for contributions, we invited papers that looked at how statistical organizations define their intentions to add value to society; how they discern what value they are adding, and for whom; how they use this information to prove, prioritize or quantify benefits; and, crucially, what could be done with an improved understanding of value. Each of the contributions received examines a different angle among these.

Ken Roy takes us on a tour of the ideas encompassed in the value of official statistics, seen from the perspective of the official mission statements, strategy documents or business plans of national statistical offices (NSOs) and systems. Making a formal statement about the value that the NSO intends to create is a crucial first step in being able to assess whether that value is being delivered, as we argue in our presentation of the Results Map approach in our own paper. Thus, Roy’s review of the stated value propositions of select NSOs, looking at the types of value they aim or claim to create (value to citizens, value to governments, value for the economy, value for public debate, etc.), is illuminating. It leads us to wonder whether this approach could usefully be applied systematically across a wider range of countries, and how this might help us, as an international community, to get a better grip on what we’re collectively trying to achieve and how we’re doing.

Sofi Nickson continues this theme of examining the stated missions of national statistical systems, in her case honing in on the United Kingdom, in which the stated goal of official statistics has been legally elevated from merely being a public good, to serving the public good. Her thought-provoking paper demonstrates the kind of humility that is so sorely needed in this arena, framing the exploration as an open field full of questions for which the answers still need to be sought. What is the public good, anyway? Rather than trying to answer this internally, Nickson and her colleagues have actively sought the views of the public in their country to see how this aspiration is understood, reaching the initial conclusions that two of the main ways in which statistics serve the public good are by reflecting and recording information about the world, and by providing material (information) that can be used to create impact. Like Roy, Nickson distils some common core goals that we see stated explicitly by statistical organizations and systems across the world: enabling improved decision-making, formulating and monitoring policy, underpinning public debate and enabling citizens to hold government to account.

This last theme, of official statistics as a necessary means of fostering democracy, is the centrepiece of the next paper, that of Luca di Gennaro. Di Gennaro’s novel contribution takes the delightfully bold step of trying to examine one of these truths that we too easily hold to be self-evident: the claim that official statistics underpins democracy. Indeed, he goes one step further and in fact tries to quantify this relationship. While there may be a long way to go before we can claim to have proof of such a quantitative link, di Gennaro is to be commended for his determination to question assumptions that so many of us tend to leave unexamined. Furthermore, as the 2022 CES report on value said, “we of all people, as statisticians, should not fall into the trap of saying that if something is hard to measure then we won’t even try”. For this reason, di Gennaro deserves credit for starting the conversation on a very challenging and important task.

Arman Bidarbakht-Nia picks up this theme of the essential role of public statistics in a democratic society, and takes it in an altogether different direction. Bidarbakht-Nia argues for a broadening of our perspectives, from data to evidence, and from decisions to actions, so that ‘evidence-based decision-making’ is replaced by a more holistic view of the relationship between information and society. His framework reframes our questions about the value of official statistics into, more broadly, ‘what are official statistics for?’. His wide-ranging examination of this question makes clear that statistics are not merely a neutral reflection, a mirror on reality, but that they play an important role in shaping that reality, our understanding of it and our discourse around it. What we choose to measure, how we measure and classify things, are both reflective of complex negotiations among states, institutions and society, and liable to directly influence how we see and understand the world around us. This fascinating paper asks us to interrogate the idea that national statistical systems are meant “to provide official statistics for evidence-based decision-making”, potentially replacing it with the idea that they are supposed to provide “the informational basis necessary for the democratic and objective co-construction of social realities” – something which, he argues, they are not currently equipped to do, as long as they continue to treat citizens and other non-state actors as mere ‘stakeholders’, contributors, users or supporters rather than as equal and active agents in a system of public statistics.

The final conclusion of Bidarbakht-Nia’s article, that “Official statisticians can’t run the global agenda for statistical development single-handedly and expect the systems to work for a broader society” is, like Nickson’s, another welcome and rare echo of our own thinking, an exhortation to look beyond our own bubbles and attempt to see the world from perspectives other than our own.

When times get tough, when budgets are cut and resources are scarce, national statistical offices (NSOs), like any other kind of organization, understandably tend to pare things down to the basics, the things we know work, the things that are not experimental or exploratory. Cutting-edge work to try and better understand our place in the world, (our ‘social function’, as Bidarbakht-Nia calls it) thus risks being left by the wayside as NSOs dig their heels in. Yet every one of the papers in this special theme on value has made abundantly clear that there is still a massive amount of thinking that needs to be done, and it cannot stop here for lack of time or resources. The CES work gathered a wealth of case studies, yet in so doing revealed that very few countries can genuinely claim to be trying to assess the value of what they do. That no contributions have been received for this theme illustrating any NSO’s attempts to understand or quantify value is in itself telling. Does this mean there are no such efforts going on? All of the papers here call for more thinking, more self-critique, and more cross-fertilization of ideas. We hope that readers will be prompted to share more on this topic via this journal, to keep this crucial discussion alive. In the meantime, we hope you will find this selection as thought-provoking as we do.