Is there a quantitative relationship between democracy and official statistics?

1.Introduction

The vital nexus between official statistics and democracy is highlighted in this article by quantitative analysis. The capacity of national statistical systems is correlated with political regimes. The main contribution of this paper is to find the quantitative relationship between democracy and official statistics. This relationship is usually addressed qualitatively: [1, 2, 3, 4, 5, 6, 7, 8] and [9]. If it is true that a well-functioning official statistical system is the foundation of a democratic society, it may be useful to point to quantifiable evidence. I am convinced that democracy is based on reliable official statistics. Nevertheless, the aim of this paper is to introduce the quantitative analysis of the topic and to explore the data, rather than their causation. The question I want to address here is: Is there a quantitative relationship between democracy and official statistics?

It greatly depends on the format of the statistics, though it is generally accepted that statistics are a sort of public good. No government, political party, or policymaker should influence these statistics. However, the reliability and independence of official statistics can be compromised in both authoritarian and democratic regimes. Political pressure,11 censorship, and manipulation may distort the accuracy of data, undermining its credibility and utility. Therefore, ensuring the integrity and the impartiality of official statistics is essential for maintaining public trust and for promoting effective governance within political regimes, regardless of their nature.

National Statistics Offices (NSOs) are expected to be organisations operating independently of government. In the complex society we live in, knowledge of the population structure, the economy data, and the depth of phenomena (like tourism, immigration, energy, etc.) are crucial components. How can we solve a problem if we do not know the problem? No data, no problem! Knowledge is the first step to problem solving. High-quality data is absolutely necessary in order to understand society and the consequent actions of policymakers. Data – as open as possible and as closed as necessary – are essential for the population to understand Government policies and their outcomes. The whole cycle of public knowledge, policy decisions, implementation, and accountability derive from data.

‘In a democratic society, the independence of official statistics has the same status as the freedom of speech for the citizens.’ (Jeskanen-Sundström [14]). The Fundamental Principles of Official Statistics state that ‘Official statistics provide an indispensable element in the information system of a democratic society, serving the government, the economy, and the public with data about the economic, demographic, social, and environmental situation’, [15]. In addition to this, [9] stated, ‘Official statistics are fundamental to democracy. With increasing demands for more relevant, frequent, and rich statistical information and declining resources, NSOs are continually looking for more cost-effective ways to produce official statistics’. Those statistics are important for the citizen to evaluate the government of the day and other opposing political parties in waiting. For this reason, the production and dissemination of statistics must be completely and entirely independent.

What follows summarizes the current qualitative work on the role of national statistics offices in democracy (Section 2); Section 3 explains and discusses the variables and data used. Section 4 presents the empirical results. Section 5 I triggers the discussion about the present research and the challenges, limitations, and opportunities lying ahead. Section 6 provides the conclusion.

2.Background

There is no single definition of democracy in academic literature. There are hundreds of different definitions of democracy in use, and we could categorize them into different groups. Nevertheless, this is not the aim of this paper. I distinguish two broad groups of definitions of democracy: minimalist and maximalist. The minimalist definition of democracy is also known as electoralist’, among others [16] and [17]. Elections are at the centre of this definition. The maximalist definitions of democracy take into consideration more aspects such as the different powers, the parliament, government, justice system, supreme court, freedom of speech, free press, freedom of assembly, statistical system, etc. In this paper, I consider democracy to be a complex concept, not limited to election. Democracy is not only a continuous balance within powers – the legislature, executive, and judiciary – but also a broader ecosystem of individuals and organisations both within and around those institutions, all of which play a role in ensuring power does not accrue in one place. One of these institutions is the National Statistics Office and the official data. For instance, in the UK, the work [3] considers the UK Statistics Authority one of the UK constitutional guardians. Nevertheless, I would like to stress that even if I consider the minimalist definition of democracy, the official statistics remain crucial. How can a country organise an election without official statistics? The government would say that the economy and social indicators were positive, and the opposition would say the opposite. The voters would not have any trustworthy data. The absence of reliable and independent data during the campaign would leave some voters without reference points on economic and societal trends. A discussion of the potential conflict between the election and official statistics is available in [1]. In political disputes and electoral campaigns in a full democracy, the data of the NSOs should remain substantially uncontested. Perhaps the interpretation of the data is forced by political rhetoric; nevertheless, it should not be disputed. In Ecuador in 2021, an unforeseen resignation by the NSO’s (INEC in Ecuador) Director took place just one week before the election day. Every NSO has a statistical calendar, normally pre-established and disseminated, usually one year in advance. In Ecuador, the unemployment rate was fiddled with one week before the election. Consequently, the INEC Director resigned the day before the dissemination of the unemployment rate.22[18] analyses the relationship between democracy and transparency. In this short, clear and sharp paper, the country’s transparency is measured by the presence of data on economic policy and debt in the World Development Indicators by the World Bank. Paradoxically, this paper never mentions official statistics.

As democracy cannot be limited to the electoral process, so official statistics is not just confined to reliable data. Measuring the level of official statistics in a country involves assessing various aspects of data quality, accessibility, relevance, and independence. Having a good national statistical system means so many things: from statistical literacy to the quality of data released, from disaggregated data to national statistical experts, from the richness and openness of online access to an extensive use of data sources and censuses, from appropriate statistical legislation to international statistical standards and methods. Citizens and decision-makers demand high-quality and trustworthy data. The statistical performance of countries is a complex and latent variable. Relevance refers to the extent to which statistics meet users’ needs and address key policy questions. Accuracy involves the degree to which statistical data reflect the true characteristics of the phenomena being measured, including the reliability of data sources and methods of estimation. Timeliness assesses the speed at which statistics are produced and disseminated, ensuring their relevance for decision-making. Accessibility measures the ease with which users can access and understand statistical information, including the availability of data in different formats and languages. Coherence examines the consistency and comparability of statistical data across different sources and time periods. Additionally, the institutional framework supporting official statistics is crucial, including the independence and autonomy of statistical agencies, adherence to international standards and best practices, and the existence of legal and regulatory frameworks protecting the integrity of statistical processes. Evaluating these dimensions provides a comprehensive understanding of the level of official statistics in a country, enabling comparisons across nations and identifying areas for improvement.

In [6], it is stated: ‘Since its beginnings in the second half of the 19th century, official statistics has made enormous advances, both quantitatively and qualitatively. It plays an essential role in the development of democratic societies. No real democracy can exist without the National Statistical Institute (NSI) producing independent information about the state and how it is changing. This may appear to be a paltry observation: we are used to ready-free access to important information we need and generally get it. However, on reflection, we have to accept that nothing is free forever and that institutions like an NSI are fundamental for humankind. They require continual investment by statisticians, by politicians, and by society as a whole’.

Before the Second World War, statistics mostly came from the different ministries or cabinets of the government. Only after the Second World War, thanks to Stone and Keynes, among others, [19], the United Nations Statistical Commission highlighted the need for international statistical standards for the compilation and updating of comparable statistics in support of a large array of policy needs. Also, Fisher [4] stated, ‘In democratic countries the emphasis has long fallen upon the important task of providing public information.’

The evolution of official statistics in society is well described in [5]: ‘In the past, statistics were used as an instrument of government control over the population, but over time they have become a tool for citizen participation, helping to ensure the proper functioning of democratic societies. Official statistical institutes took on a decisive role in this change’. This work – [5], analyses the changes in history, from a sociological point of view, to the relationship between the state and statistics. More so and generally speaking, the article attempts to position statistical science and official statistics as a key sociological factor in the construction and consolidation of modern democracies: statistical information ‘trustable and available to all’ is one of the foundations on which modern democratic states are built. Thus, NSOs are essential democratic institutions.

3.Data description

Democracy and the national statistical system are all-encompassing concepts with many different aspects. Measuring them is a complex and challenging operation. About democracy, there are many different indexes.33 I checked and analysed some of them: Freedom House, Polity, V-Dem Democracy Indices, and Democracy Index. Freedom House dataset by a U.S. non-governmental organisation Freedom House is a prestigious dataset in democracy. It stresses more freedom than democracy, according to the Freedom House culture organisation. Polity by the Political Instability Task Force has a long and historical dataset started in 1800 by a U.S. government-sponsored research project. V-Dem Democracy by V-Dem by the Institute University of Gothenburg (Sweden) indices is a comprehensive list of hundreds of different indexes; nevertheless, there is not an all-encompassing index on democracy. The Democracy Index (DI) by the UK group Economist takes into consideration different categories and started recently. Figure 1 shows the decrease of the Democracy Index in the last few years.

When it comes to measuring the national statistical system, there are not many possibilities. In the Word Bank, there are the Statistical Capacity Indicator and Statistical Performance Indicator. The Statistical Capacity Indicator is done only for 146 development countries. The Statistical Performance Indicator (SPI) has been done for all the countries since 2016. Both indexes are made with similar methodology, but they take into account different aspects of the statistical system. SPI tries to estimate the different characteristics of national statistical systems. Statistics have value only if they are used widely and frequently. Therefore, it is necessary to connect data users with producers. Statistical data and products must be accurate, timely, frequent, comparable, and disaggregated. Also so as to produce indicators related to the 17 Sustainable Development Goals. By now, in order to create useful products, the statistical system must draw on several sources, in addition to typical censuses and surveys, administrative and geospatial data, as well as data generated by private businesses and citizens. A mature statistical system has well-developed hardware (legislation, governance, standards) and soft (expertise, partnerships) infrastructure and necessary financial resources. Open Data Watch produced two indexes: the Open Data Inventory (ODIN) and the Gender Data Compass (GDC). Both indexes are focused on the webpages of the national statistics offices around the world. The first one focuses on open data, and the second one disaggregates data. The SPI included the ODIN data. The GDC just started in 2023. Figure 2 shows the Statistical Performance Indicators over time. We can appreciate the increasing index.

Figure 1.

Aggregate Democratic Index over the year 2016–2022. Note – This figure presents the Democratic Index, in the figure ‘dem’ by The Economist and the years 2016–2022, in the figure ‘date’. It highlighted the number of countries (n) per year. In the horizontal axis is represented by the Democratic Index, and in the vertical axis is represented by the date.

Figure 2.

Aggregate Statistical Performance Indicators over the year 2016–2022. Note – This figure presents the Statistical Performance Indicator, in the figure SPI.INDEX by the World Bank and the year 2016–2022 in the figure ‘date’. It highlighted the number of countries (n) per year. In the horizontal axis is represented by the Statistical Performance Indicator, and in the vertical axis is represented by the date.

My ideal index has reliable data, a complete frame of reference, and the result is a scale of values (not a binary value). The data must be reliable and come from experts or surveys from reliable organisations. Democracy, like the statistical system, are complex concepts that must be measured by trying to measure all factors, so there is a need for a complete measurement framework. It is interesting to have a value scale to have the ability to do different analyses and to appreciate the variability of the index. The scale is useful for producing analyses.

Following this criteria, the two main indicators were employed to calculate the correlation between democracy and official statistics: the Democracy Index44 by the Economist and the Statistical Performance IndicatorsIndicators55(SPI) by the World Bank. Tables 1 and 2 show the main statistics of the indices.

Table 1 recaps the main features of the indexes: the number of observations (N), the number of countries, the range of years, and the organisation that disseminates the index. Table 2 presents summary statistics for Statistical Performance Indicator (SPI) and the Democracy Index (DI): mean, standard deviation, minimum and maximum.

The Democracy Index assessed, from 2006 onwards, the democracy of over 167 countries. This was based on five categories – electoral process and pluralism, civil liberties, functioning of government, political participation, and political culture – thanks to experts’ assessments and the public’s opinion from multiple significant surveys. The index’s values range from 0 to 10, and these countries were within one of four types of regimes: ‘Full democracies’, ‘Flawed democracies’, ‘Hybrid regimes’ and ‘Authoritarian regimes’.

The SPI by the World Bank assessed the performance of national statistical systems from 2016 to 2022 in over 174 countries. This data came from the most important international organisations, including the Open Data Inventory (ODIN) by Open Data Watch. The SPI is a framework of five pillars (data use, data services, data products, data sources, and data infrastructure) and 22 dimensions. Nevertheless, there are currently 14 dimensions that have been proven by their established methods, whereas the other 8 dimensions have no measurable indicators.

Some well-meaning stakeholders may object to the fact that those indicators are not always trustworthy. About the SPI, several limitations are highlighted in [20]. For instance, high-income countries do not often collect data on certain indicators that are more relevant for poorer countries. Also, we need to notice that the World Bank is an international institution, and they need to take into consideration the different national sensibilities. Sometimes the ranking of some national statistical systems is surprising. Eight of the 22 dimensions of SPI are not populated by data. Updates and changes are expected, and the SPI could be one of the Sustainable Development Goal indicators (17) relating to the national statistics system. Statistics are vital if improvements to statistical systems are to be made.

Some stakeholders with good intentions can argue that the Democratic Index lacks transparency and accountability beyond the numbers. Indeed, the final report does not indicate the characteristics of the experts who produce the index. The Economist Group is a reputable and trustworthy institution, and it is outside the statistical world.

The computation of an indicator is always complex and is always done with a certain degree of subjectivity. However, those indicators – the Democracy Index and the Statistical Performance Indicators – are disseminated by reputable institutions, produced with sound and valid methodology, and have suitable output for this research. Although the position of some countries for some years may seem suspicious or absurd to us in an index, using (inovatively) the indexes globally and in their entirety, the error of a single country may be negligible. Last but not least, at the moment, there are no better choices.

4.Exploratory data analysis

The vital link between official statistics and democracy is highlighted by quantitative data. In this first paper on the subject, I have dwelt on an exploratory analysis of the data, but one that I consider critical and, in part, exhaustive. In the hope that it can be continued by myself and others. Good performance of statistical systems is more likely in a democratic country. The political regime can create a good or bad statistical environment.

Figure 3.

Scatter plot of the Democratic Index and Statistical Performance Indicator 2022. Note – This scatter plot represents the two main variables used in our analysis: the Statistical Performance Indicator (SPI) by the World Bank and the Democratic Index (DI) by The Economist for the year 2022 for 160 countries. In the horizontal axis is represented by the Statistical Performance Indicator SPI-INDEX-2022, and in the vertical axis is represented by the Democratic Index ‘dem-2022’. At present, 2022 is the last year available for the two indices simultaneously.

Figure 3 shows the scatter plot of democracy and statistical performance in 2022. The plot confirms our intuition and the quantitative analysis. There is a strong relationship between democracy and statistical performance. The closer the data points to the line, the higher the correlation between the two indices. On the right-top, there are the countries with a high value in the democracy index and a high value in statistical performance. On the left-down part of Fig. 3, countries have a low level of democracy and statistical systems.

In Table 3, the correlation between the Democracy Index and statistical performance indicators for the period 2016–2022 is presented. The interval of years 2016–2022 is the only overlapping interval of the two indices.

The simple correlation between democracy and official statistics has been around 70% over the years. There was a small decrease over the year, which can be interpreted as a good statistical system in 2022 being slightly less correlated than in 2016. A similar analysis can be made with the Open Data Inventory66 (ODIN) assesses the coverage and openness of official statistics by Open Data Watch (after 2018, the data is reported biennially). Same decreasing trend in the correlation between the Democracy Index and ODIN data: 57 percent (2016), 62 percent (2017), 62 percent (2018), 56 percent (2020), and 55 percent (2022). The ODIN data are included in pillar 2, Data Services, of SPI in the dimension ‘Richness and Openness of Online Access’. Pillar 2 is composed of 4 dimensions.

The closer the data points to the line of the scatter plot of the Fig. 3 anticipated, the higher the correlation between the two indices. In the Table 4, I am analysing the SPI more deeply. As aforementioned, SPI is composed of 5 pillars: data use, data services, data products, data sources, and data infrastructure,77 more details [20]. Data Use (Pillar 1) evaluates whether the produced data from the statistical system is widely and frequently used by different institutions. Data Services (Pillar 2) assessed the connection between data users and producers. Data Products (Pillar 3) reckoned the data accuracy, timeliness, frequency, comparability, and levels of disaggregation. Data Sources (Pillar 4) estimated the variety of data sources, especially, administrative and geospatial data as well as data generated by private firms and citizens. Data Infrastructure (Pillar 5) considers hard infrastructure (legislation, governance, standards) and soft infrastructure (skills, partnerships). Every pillar includes 4 or 5 dimensions; not all the dimensions are populated. In the Table 4, three pillars have a very strong correlation (more than 60%) with democracy: Data Services, Data Sources, Data Infrastructure.While two pillars have a lower correlation, between 30% and 40%: Data Use, Data Products. The five pillars are stable over time, especially Data Sources and Data Infrastructure.

From the analysis, I observed that high-income countries have a high level of Democracy Index and Statistical Performance Indicator. The indicators register ‘Full Democracy’ and a high score (top 20 percent) of statistical performance in Western Europe and North America. Western European countries and North American countries (US and Canada) have high scores both in democracy and in the statistical system. Norway has the highest score in the democratic index as well as in the statistical performance indicator. Similar stories exist for Iceland, Sweden, New Zealand, and Finland. If we consider the only 24 ‘Full Democracy’ countries of DI, 14 of them are also in the best 24 countries of the SPI index (Taiwan is in the top ‘Full Democracy’ of DI, but there is no value for SPI). Table 5 lists the top 10 countries for SPI and shows their position in the democratic index. Among the top 10 countries for SPI, there are five top 10 democracies in the DI (check column DI Rank), and six of them are flagged ‘Full Democracy’ and the rest are ‘Flawed Democracy’ (Taiwan is in the top 10 of the DI, but there is no value for SPI).

Sometimes, I registered exceptions; ‘Authoritarian’ regime countries register high statistical performance. For instance, Belarus, Russia, Kazakhstan, Kyrgyzstan, Palestine, and Egypt are not considered a ‘Full Democracy’ but they have very high statistical performance. In Africa, Mauritius is a ‘Full Democracy’ with the highest score of the DI, and the SPI has the highest score in the continent. There are similar stories with Ghana, South Africa, Cape Verde, and Botswana, as they registered both high scores in the Democratic Index and the Statistical Performance Indicators. On the other hand, there is a different story for Egypt, which registers a very high score in statistics but not in democracy. In the region of Asia and Australasia: South Korea, New Zealand, and Australia are ‘Full Democracies’ and have high statistical performance. Central and Eastern European countries are atypical; the level of these countries’ statistical performance is high, whereas the level of democracy is quite low. In Latin America, only Chile, Costa Rica, and Uruguay are a ‘Full Democracy’ according to the DI classification. These three countries registered the highest scores on Statistical Performance Indicators. Asia and Latin America have many emerging democracies, and it will be interesting in the coming years to see if they strengthen their statistical systems.

5.Discussion: Limitations and future work

To begin with, this study does not aim to overuse rankings. Indeed, the analysis is mostly based on the value scale. Neither it wants to just point out only negative countries performance in democracy or/and statistical performance. I have included a list of potential limitations below, which could possibly be a topic for further research. I like this idea that a limitation can also be an opportunity.

Secondly, some well-meaning stakeholders may object that the paper’s strongest point is also its greatest limitation. The quantitative analysis is based on the indexes of the Democratic Index and Statistical Performance Indicators. Generally speaking, indexes are based on subjective assumptions and can be manipulated deliberately or not. Indices always arise from a conceptual paradigm of interpreting reality, and I realise that they may not be accepted by everyone. As I showed above, those two indexes can have several limitations. This limitation can be an incentive to create new data on the topic or find other sources. To overcome this limitation, it may be possible to cross-check more indices. Above, I mentioned some of them. Using several indexes for democracy and statistical performance can show the consistency of results and also be an indirect proof of causality. From an initial, unpublished exploration, it appears that the results are consistent with those presented here.

Thirdly, correlation does not mean causation. I will continue to collect data on these issues. I am aware that this data analysis is only a starting point; in this area data analyses could be a real mine from a research point of view. I leave causation analysis for future exploration; nevertheless, the data are perhaps barely sufficient for an analysis of the panel data. Indeed, their results are consistent with the intuitively observed reality of democracy and statistical perspectives. Perhaps it is a naive or wrong idea, but I am convinced that if two variables are highly correlated and have a logical relationship and clear causality, such as wages and consumption, there is no need to search for a quantitative measure of causality. Unfortunately, there is no easy recipe to establish causality. I think that one way would be to look for exogenous shocks to either democracy or the goodness of the statistical system and use that for difference-in-differences. Panel data analysis can be useful in causality analysis; nevertheless, the time series is not long but may be sufficient.

Fourthly, another limitation is that the correlation cannot detect the presence or effect of other variables outside of the two being explored. When two variables seem to be correlated but the relationship is actually the result of chance or a third component, this is known as spurious correlation. For instance, the better democracies are normally in the wealthier countries, as well as the better statistical performance.

Lastly, I do not believe that democracy is the panacea for everything we dislike. I am aware that considering democracy at the centre of everything is just one way of looking at things. For instance, checking the relationship between governability and official statistics, I realised that this relationship is even stronger than what I presented here. Future comparisons by region, income, and regime can bring new insights.

6.Conclusions

In the quest for democratic development, the pivotal role of a strong statistical systems in the world cannot be overstated. The present article emphasizes the importance of quantitative elements in supporting the relationship between democracy and the statistical system. Political regimes influence the performance of the statistical system. The correlation, I calculated (see Table 3), between the Democracy Index and the Statistical Performance Indicators for 2016-2022 for around 160 countries, is 71 percent (2016) and 68 percent (2022).

According to the Democracy Index, only 24 countries can be considered ‘Full Democracies’, 14 of them are also among the top 24 countries in the Statistical Performance Indicators. Among the top 10 countries for Statistical Performance Indicators (see the Table 5), five are also among the top 10 for DI statistical performance.

In the analysis of the five Statistical Performance Indicators pillars, there is a very strong correlation (more than 60 percent) with democracy: Data Services, Data Sources, Data Infrastructure (see Table 4). While two pillars have a lower correlation, between 30 and 40 percent: Data Use, Data Products. The five pillars are stable over time, especially Data Sources and Data Infrastructure.

Here, for the first time, the strong quantitative relationship between democracy and official statistics is taken into consideration. It is achieved through the innovative use of indices of democracy and official statistics. The indices are used in their entirety and for the entire applicable historical series, not for a single country. The official statistics produced and disseminated by the NSOs are crucial for democracy in a modern society. In a democratic society with an overwhelming availability of information, the credibility of the data source is a central factor in assessing the usefulness of statistics.

Lastly I would like to conclude with a quotation that appears to summarize perfectly the intentions of the present research: ‘In a democratic society, the independence of official statistics has the same status as the freedom of speech for the citizens.’ [14]. Of course, we must be aware that, without Democracy, Official Statistics Dies in Darkness.