Success, failures, challenges, and opportunities for official statistics in the development and implementation of the SDG Indicator framework

1.1The Cape Town Global Action Plan for Sustainable Development Data

The Sustainable Development Goals (SDGs) were adopted by the United Nations in September 2015 as a universal call to action to end poverty, protect the planet, and ensure that by 2030 all people enjoy peace and prosperity. The United Nations Statistical Commission (UNSC) agreed to establish the High-level Group for Partnership, Coordination and Capacity-Building for the 2030 Agenda (HLG-PCCB), comprising Chief Statisticians from 23 national statistical offices representing other countries in their respective regions. The HLG-PCCB was tasked to promote national ownership of the 2030 Agenda monitoring system and fostering statistical capacity building, partnerships and coordination. NSOs must coordinate their implementation at the country level. The UNSC, also established the Inter-agency and Expert Group on SDG indicators (IAEG-SDGs) to conduct the work necessary to identify the indicators and ensure the full implementation of the related data development programs. In March 2016, at its 47th session, the UNSC agreed, as a practical starting point, with the proposed global indicator framework as developed by the IAEG-SDGs. In June 2016, ECOSOC took note of the report of the UNSC and adopted its decisions, including the global indicator framework. With this endorsement, the UNSC was mandated to develop a global indicator framework for the follow-up and review of the 2030 sustainable development agenda. Following this, in March 2017, the UNSC endorsed in its 48th session the Cape Town Global Action Plan (CTGAP) for Sustainable Development Data. This action plan is based on inputs from the statistical community, including national statistical systems, and other stakeholders, and is the main guidance for the SDG work in official statistics.

Since 2017 a lot of work has been done by the AEG, international organizations as custodian agencies for the SDGs as well as the national statistical offices. Many other organizations and institutions have also been involved, and many events and conferences were dedicated to the SDG framework, to its governance and financing, and to individual indicators for specific SDGs. The successful positioning by the UNSC as the global statistics network with UNSD at the helm, as best suited to manage the development and Implementation of an indicator framework for the SDG indicators was a game-changing event for global official statistics. The official statistics community fully realized from the beginning that this indicator development would be a difficult and complex task, a big challenge and that a failure would be a serious debacle but on the other hand, success would be rewarding and respected.

The world has undergone many very striking developments since 2015. In this context, a report by the WorldBank/UNSD/Paris2111concludes that ‘cascading and interlinked crises are putting the 2030 Agenda for Sustainable Development in grave danger, along with humanity’s very own survival’. Also, the work on the Global Action Plan (globally as well as for individual countries and regions) has severely suffered from these crises. At the beginning of 2023, some six years later, and two years still to go for the next (2025) comprehensive review of the action plan, many conceptual discussions on the initial list of indicators have been concluded with concrete measurement recommendations, successful operationalizations have been agreed upon, as well methodological exercises and implementation projects successfully finalized. Though many achievements can be registered, the monitoring reports also conclude that there are substantial delays and failures, in development planning instruments, coordination between NSOs and partners within and outside the system, financing, capacity development, implementation and dissemination, regional coverage, and subregional data, to just name a few areas. A clear picture of the mixed progress is given by the report ‘Survey on the Implementation of the Cape Town Global Action Plan for Sustainable Development Data’,22 This report sketches that ‘despite gradual improvements, the transformative change in national statistical capacities proposed in the Global Action Plan has yet to be realized, particularly in low- and middle-income economies’.

In conclusion, one can say that on the one hand, official statistics have greatly benefited from this Global Action Plan for the relatively fast development of indicators and related methodologies. The SDG discussion also reinstalled the structure of the global statistical system with an important role for the ISO as custodian agencies of the indicator methodology and the Member States as the owners (and responsible) of the results per indicator. Surely, also on the national level in many countries, the coordination role of the SDGs has strengthened the role of NSIs. On the other hand, official statistics are confronted with unrealistic expectations concerning several other indicators, the impossibility of finding/collecting required data and their implementation in general and regional coverage.

The 15th SJIAOS discussion33 ‘Success, failures, challenges and opportunities for official statistics in the development and implementation of the global indicator framework for the Sustainable Development Goals and targets of the 2030 Agenda for Sustainable Development’ invites readers to react to the statement that official statistics, via endorsing the Cape Town Global Action Plan for Sustainable Development Data at the UNSC in 2017, on the one hand, have greatly benefited in strong support for developing new methodologies and indicators, but the other hand is confronted with unrealistic expectations concerning several other indicators, the required data, their implementation in general and regional coverage.

The leading question to the discussion is ‘did the challenge of developing the indicators for the Sustainable Development Goals give a boost to Official Statistics or was it a burden?’ Detailed questions will focus on the successes, failures and more specifically the challenges, obstacles and opportunities. The readers are invited in the 15th discussion, launched on the SJIAOS discussion platform (www.officialstatistics.com) to either react with supporting or disagreeing arguments to this reflection, examples of successes, failures, and suggestions on how to tackle challenges.

1.2SDG oriented manuscripts in SJIAOS

As incoming Editor in Chief in 2019, I formulated a strategy for the Statistical Journal. This strategy aims to support the role of the IAOS to be at the forefront of important developments in official statistics. Surely, one of these important developments is the implementation of the SDG indicator framework. I included as an objective in this strategy to encourage the discussion of SDGs indicators and emerging issues in the SJIAOS by soliciting and publishing articles on the SDGs and their implementation. In discussing the progress of the action plan, it is, useful to reflect on the extent to which SJIAOS has contributed. A quick count of the number of manuscripts in the journal, for which the keywords and/or the abstract directly mention the SDGs in general or a specific SDG, reveals that over a period of six years (2017–2022), some 70 manuscripts have been published that fulfill this criterium. Clearly, this is a rate (15%) that could be improved. Obviously, many manuscripts indirectly relate to the governance and methodologies in official statistics and might also contribute. The Advisory Board of the Journal44 concluded in its annual meeting in November 2022, that some extra effort soliciting manuscripts on SDGs could be part of the 2023–2025 strategy for the Journal.

1.3The United Nations SDG Publishers Compact

In December 2020, IOS Press, the publisher for SJIAOS, became a signatory of the United Nations SDG Publishers Compact55 and in doing so committed to developing sustainable practices and acting as champions of the Sustainable Development Goals (SDGs). This commitment does not only relate to publishing books and journals that will help inform, develop, and inspire action in that direction but also aims for the Journals’ structure and governance to be as compliant with the SDGs as possible, especially on goal number 5, Gender Equality, and goal number 10, Reduced Inequalities.66 For Gender Equality the performance of the Journal with respect to gender distribution in the editorial board is measured. With currently a rate of only 34% of the emphasis and general editors being female some work is needed on the gender balance in the Editorial Board. With respect to the gender balance of the authors of manuscripts, the Journal does better with 45% female, but also here some gains can be achieved.

On the Geographical Distribution of Authors and the Editorial Board, it is clear there are some steps to be made before the Journal is a truly global Journal. In the Editorial Board, the Journal’s traditional core region (Europe, North America, Australia and New Zealand) is with 75% still strongly overrepresented. The rate of authors from the non-core regions (Africa, Asia, Latin America and the Caribbean) shows over the last couple of years a promising increase to around 25% of all contributions.

1.4The themes in this issue

This issue of the Statistical Journal of the IAOS contains 20 manuscripts covering four specific themes. The first theme COVID Impact and Innovation during the COVID-19 Crisis contains one manuscript focussing on adaptations in Short Term Statistics due to the economic downturn due to the lockdown. The second theme contains five manuscripts on Governance in Official statistics. The manuscripts by Hans Viggo Sæbo and Marit Hoel, two manuscripts on trust in statistics. Hernan Munoz and Julien Dupont described in detail the receiver of the Argentinian statistical system. The final contribution in this section is a description of the French National Council for Statistical Information; Lessons from a Formal Dialogue between Producers and Users on Sustainable Development

The third theme has four domain-specific manu-scripts, on respectively Environmental Statistics in Lao PDR, Imputation methods for sub-municipal data in the Italian Housing Census, time series of temporary employment in the Netherlands, and results for a new Tourism Sectoral Account for Greece. In the fourth theme, seven manuscripts focus more specifically on Data, Methodology and Techniques. It covers an overview of Big Data for Official Statistics in Latin America and the Caribbean, combining data from multiple probability samples, web scraped data, the transition from telephone to online and mixed mode data collection for employee panel survey data, oversampling techniques for imbalanced data in poverty classifications and methodologies for variance estimation in labor force indicators

The issue ends with a regular update on the open discussions on the SJIAOS discussion platform www.officialstatistics.com, and a brief look into the next two issues of the Journal.

2.1COVID Impact and Innovation during the COVID-19 Crisis

Arnout van Delden, Martijn de Winter and Koert van Bemmel, (Statistics Netherlands) describe in ‘Identifying and evaluating COVID-19 effect on short-term statistics’ if due to the COVID-19 lockdown measures and resulting economic downturn any adaptations to the short-term statistics were needed to ensure accurate and relevant output. In the paper the authors give an overview of the anticipated effects, and the subsequent measures for each of the stages in the production process and how Statistics Netherlands anticipated a number of potential lockdown effects. The authors evaluate to what extent the anticipated effects occurred in practice and mention some unforeseen effects.

2.2Governance of Official Statistics

In the editorial to the December 2022 issue (Volume 38/4) I reflected on the current discussions in official statistics on how to go forward. A discussion on the future position of official statistics in the changing landscape due to new data sources and actors etc. In ‘Official Statistics – Quo Vadis?’ Hans Viggo Szabo and Marit Hoel (Statistics Norway) question if official statistics will survive as a brand, or if such statistics drown in the flow of data and statistics from new sources and actors, including misused statistics and fake news. They argue in favor of the need for official statistics from the point of their quality and the value such statistics have for the users as a basis for – and supplement to – other statistics and information. Though this should not lead to a conservative approach but importantly also to implement new developments to improve and keep up the relevance of official statistics. The paper addresses issues such as statistical legislation, quality frameworks and core values: the defining requirements for official statistics, in the light of trends in official statistics since the UN Fundamental Principles of Official Statistics were formulated about 30 years ago. Quality challenges for statistics and dilemmas in defining the roles of statistical institutes are considered. The paper includes examples from Statistics Norway.

Building on the changing environment and new roles for Official Statistics, Yolanda Gómez, Ana Cánovas and Ana Carmen Saura (INE, Spain) in their manuscript ‘Trusted Smart Statistics and the need for new ethical principles’ focus on horizontal issues that are essential to address properly this changed environment. The first issue is access to Big Data (including the Internet of Things – IoT) and the legislative framework and ethical principles related to such access. The second issue is how to communicate these principles to the citizens and inform them about the statistical treatment of the data from these new sources. With respect to ethical issues, they argue, that the 2017 updated European Statistics Code of Practice due to this changing environment is demanding new rules and principles that could be incorporated in the Code or even in an amended Regulation 223/2009 on European Statistics. The authors analyze the existing ethical principles for Big Data uses (in a broad sense) and under different scenarios and compare them with the current statistical principles.

Serge Allegrezza, Wolfgang Langer and Majlinda Joxhe (Statec Luxembourg) report in ‘Trust in official statistics across Europe: Evidence from two waves of Eurobarometer using multilevel models’ on the results of an analysis of data on trust for the years 2007 and 2015. They distinguish within-country and between-country variations in individual trust in official statistics from the 28 countries and 2 regions in Europe. The within-country variation is mainly explained by individual-level statistical literacy and education, as well as occupational status. With respect to the variation between countries, they show that neither the level of GPD nor the index of inequality is important in explaining cross-country variation. Instead, EU membership history, i.e. the ‘acquis communautaire’, is the main macro variable that explains the increase in trust for the official statistics across Europe.

Focusing on the French National Council for Statistical Information (Cnis), the paper ‘The French National Council for Statistical Information: Lessons from a formal dialogue between producers and users on sustainable development’ authored by Isabelle Anxionnaz, Cristina D’Alessandro*, Arnaud Montus and Stéphane Tagnani provides an overview on the value and strength of the dialogue between producers and users of official statistics in France. Highlighting its specific contribution to shaping official statistics in relation to sustainable development, it presents its role and original value added in the French context on a critical issue for the post-COVID era. If this case study proposes lessons to learn from this experience, its history, and transformation over time, it wants also to point out promising evolutions and paths for the future. In the context of crisis and pandemic, the need to produce quality and timely statistics adapted to the situation has shown its capacity to adapt and to confirm its critical role.

From 2007 to 2015, the National Institute of Statistics and Censuses (INDEC) of Argentina underwent damaging political interventions, which undermined the institution and the quality of its products and services, leading to a widespread distrust of the official statistics of the country. In January 2016, a presidential Decree declared a state of administrative emergency in the National Statistical System (NSS), allowing the recently appointed Director General of INDEC, to reorganize the agency and the NSS. To this end, INDEC authorities implemented several strategic, legal and operational actions in order to recover and develop the statistical capacity. The modernization of the statistical legislation represented an important objective of this process and a draft law was prepared drawing especially on the Generic Law on Official Statistics (UNECE), the Fundamental Principles of Official Statistics (UN), and the OECD Recommendation on Good Statistical Practice. This important documentary paper ‘Rebuilding the National Statistical System of Argentina. Some lessons Learned’ by Hernan Daniel Munoz and Julien Dupont describes the main actions taken to recover the Argentinian statistical system and derives the lessons learned.

2.3Domain-specific manuscripts

Perig Leost and Salika Chanthalavong (Luxembourg) in ‘Environment statistics in Lao PDR – Case study on multiple data sources integration for a new statistical domain’ describe how the Lao Statistics Bureau launched the development of a Statistical Information System on the Environment. In the Lao PDR strategy for the development of environmental statistics, as well as for the mainstreaming of their use in policy processes, the System of Environmental-Economic Accounting (SEEA) plays a central role. The authors show that routinely producing environmental-economic accounts at the national level by the national statistical office requires addressing the technical and institutional barriers to the integration of several different data sources. The manuscript illustrate the challenges faced to integrate data from multiple sources into one single database (these are: the number and diversity of stakeholders and domains and the consequent dispersion of data, the difficulty of access, and the often poor quality of the data (linked notably to the relative novelty of the statistics involved) etc.) and provide feedback on the solutions applied to overcome the challenges.

Population and Housing Censuses have always ensured the availability of sub-municipal data useful for social, economic, and environmental decision-making processes. The new Italian Permanent Census focuses heavily on the integration of administrative and sample data and plans to provide more stable and consistent statistical data at the various territorial levels every year. Giancarlo Carbonetti, Marco di Zit, Davide Fardella, Raffaele Ferrara, and Fabio Lipizzi (Istat, Italy) describe in ‘Enumeration area imputation methods for producing sub-municipal data in the Italian permanent population and housing census’ how sub-municipal data are derived within this framework from the integration of the Base Register of Individuals and the Base Register of Places. Obviously, data accuracy depends on the quality of the registers and the procedures adopted to integrate and process the input data. In this regard, Istat is working to improve geocoding information and linking procedures. In their manuscript, the authors show the results of an experimental study to assess the quality of the imputation procedure that integrates deterministic and probabilistic approaches to assign the enumeration area code to non-geocoded units.

Nino Mushkudiani and Jeroen Pannekoek (Statistics Netherlands) investigate in ‘Estimating a time series of temporary employment using a combination of survey and register data’ the application of macro-integration methods to combine two sources of labor force statistics: a survey and an administrative source. They aim, in particular, to arrive at a single estimate of the time series of temporary employment that efficiently combines the information from both sources. Out of four different macro-integration models they conclude that a model that treats neither of the sources as fixed and uses multiplicative adjustments obtains the most plausible results. Compared to the results of previous research where a latent Markov model was used to estimate the same time series, the new approach does not lead to very different estimates of the time series of temporary (or permanent) employment contracts but results in smaller estimates of the proportion of “movers”, persons that change contract status from temporary to permanent or the other way around. The model-based approach also provides estimates of the measurement errors in each of the sources. On the other hand, the macro-integration approach is less restrictive because it does not impose a Markov property of the integrated times series of proportions and is easier to implement.

In many countries, early attempts in implementing Tourism Satellite Accounts (TSA) Tables suffer from limitations in available data. For Greece, pilot TSA Tables 1 to 6 were implemented in an EU-funded project for the reference year 2015. However, TSA Table 6 could not be implemented in all details. TSA Tables were not implemented for the years after 2015, partly due to the need for improving the TSA-related database. The manuscript ‘Tourism GDP for 2010 to 2020: New TSA results for Greece’ by Peter Hackl (Austria) and Stavros Hatzimarinakis (Greece) presents methods for implementing the TSA Table 6 given only limited data on tourism expenditures and Supply and Use Tables and provides estimates of the Tourism Direct GPD for 2010 through 2020. Besides the methodological approach, their results show a rather pleasing picture of Greek tourism. In the years 2010 to 2019, Tourism Direct GDP was steadily growing and increased from 7,026 to 10,994 million Euros, an average annual growth rate of 5.1%; the GDP decreased annually by 2.2% on average. The TDGVA ratio increased from 3.1% to 5.5% and the TDGDP ratio from 3.1% to 6.0%, the average annual growth rate of the Tourism Direct GVA and GPD ratios amounting to 6.7% and 7.5%, respectively.

2.4Data, Methodology and Techniques

In the first manuscript in this section, Andrea Diniz da Silva, Beatriz Menezes Marques de Oliveira, Ísis Gonçalves Peixoto, and Lidiane Braga Sales de Souza (Brazil) present an ‘Overview of the Use of Big Data for Official Statistics in Latin America and the Caribbean’. Part of the success of the use of big data in National Statistical Offices is due to the support of the United Nations through the UNBigData initiative, specifically the 4 Regional Hub for Big Data. The manuscript reports on a consultation with the NSOs to learn the extent of the use of big data for official statistics in Latin America and the Caribbean. A very positive and promising scenario regarding the use of big data from satellite imagery, web scraping and other sources, for applications such as the production of price statistics, coverage and land use and migration, was found.

In ‘Approaches for combining data from multiple probability samples’ Marcel Vierra (Brazil), Loveness N. Dzikiti (South Africa) and Brendan Girdler-Brown (St. Kitts and Nevis) evaluate simulation results of different methods for combining survey data, in the context of simple random sampling, stratified random sampling and two-stage cluster random sampling from finite populations generated from a normal distribution super- population model. The simulation results suggest that super population variance does not influence the choice of the weighting method, though, the population size does. Combining samples improved the precision of estimates regardless of the weighting method used for data collected under all considered sampling techniques, with stratified sampling being more precise than simple random sampling and two-stage random cluster sampling.

Elena Catanese, Mauro Bruno, Luca Valentino and Monica Scannapieco (Istat, Italy) in ‘Italian sentiment analysis on climate change: emerging patterns from 2016 to today’ describe the use of social media websites as a data source for mining public opinion on a variety of subjects including climate change. In the article, the authors focus on Twitter, in particular, as this allows for the evaluation of public opinion across time. The sentiment analysis of Twitter textual data on climate topics from a large dataset of Italian tweets between 2016 and 2022, provides valuable insights into the climate discussion and is considered by the author as representative of the rising climate movement. Based on a set of keywords related to climate change daily volume and sentiment of tweets series have been analyzed. The first series allows assessing the Italian participation in the climate debate, while the latter provides useful insights into the overall evolving mood during these years.

Peter Knizat demonstrates in ‘Web scraped data in consumer price indexes’ different methods for aggregating the web scraped daily price data to determine monthly prices that are normally used in the estimation of consumer price indexes. Moreover, various economic approaches for estimating indexes, which capture price dynamics of product items such as replacements and missing prices, are presented and applied to the observed web-scraped data.

Interviewer-administered panel surveys, historically considered the gold standard form of data collection, are facing high costs and nonresponse issues that threaten their sustainability and inferential capabilities. Supplementing interviewer administration with online data collection is a popular method of reducing costs and may improve contact ability and reduce nonresponse in employee surveys. Jan Mackeben and Joseph W. Sakshaug (Germany) address in their manuscript ‘Transitioning an employee panel survey from telephone to online and mixed-mode data collection’ the effects of introducing online data collection in an ongoing panel survey of the employed population by analyzing a mode design experiment embedded in the German employee panel survey. Individuals were randomly assigned to the standard telephone-only design or a sequential web-telephone mixed-mode design. An invitation letter experiment was also conducted to test the effect of mentioning telephone follow-ups in the web survey invitation. They report that introducing the mixed-mode design led to a higher response rate (59.9% vs. 50.1%), similar levels of nonresponse bias, and lower costs compared to the single-mode design. Mentioning the telephone follow-ups had no effect on participation in the web starting mode or the full mixed-mode design.

A solution for solving biases in the estimation results and prediction errors in the classification of poor people recorded yearly is using Synthetic Minority Over-sampling Technique (SMOTE). In their manuscript ‘Synthetic Minority Over-sampling Technique (SMOTE) for handling imbalanced data in poverty classification’ Firza Refo Adi Pratama and Siskarossa Ika Oktora (Indonesia) evaluate the inference and classification quality using the binary logistic regression model without and with SMOTE. It was concluded that the model with SMOTE approach was better at inference and classifying the results.

In the final article in this section Lida Kalhori, Fahrad Mehran, Mohammed Reza Reyhani and Roshanak Aliabari Saba in ‘Adaptation of Statistics Canada and Eurostat methodologies for variance estimation of changes of the main labor force indicators in Iran’ examine the procedures for estimating the variance as used by Eurostat and Statistics Canada. Beyond many similarities, in terms of the periodicity of the survey, the rotation pattern as well as the unit of rotation, and the possible existence of non-response among the primary sampling units the survey in Iran differs from those of Statistics Canada and Eurostat. Among the four methods examined, the bootstrap methodology of Statistics Canada is found to be especially suitable for application in Iran. The proposed methodology can, particularly well, take into account the impact of the various steps of weight calculations on the variance estimates of change of the main labor force indicators.

I wish you pleasant readings of these interesting articles

Launch of the 15th discussion:

The 15th SJIAOS discussion ‘Success, failures, challenges and opportunities for official statistics in the development and implementation of the global indicator framework for the Sustainable Development Goals and targets of the 2030 Agenda for Sustainable Development’ invites readers to react to the statement that official statistics, via endorsing the Cape Town Global Action Plan for Sustainable Development Data at the UNSC in 2017, on the one hand, have greatly benefited in strong support for developing new methodologies and indicators, but the other hand is confronted with unrealistic expectations concerning several other indicators, the required data, their implementation in general and regional coverage.

The leading question to the discussion is ‘did the

challenge of developing the indicators for the Sustainable Development Goals give a boost to Official Statistics or was it a burden?’ Detailed questions will focus on the successes, failures and more specifically the challenges, obstacles and opportunities. The readers are invited to the 15th discussion, launched on the SJIAOS discussion platform (www.officialstatistics.com) either react with supporting or disagreeing arguments to this reflection, examples of successes, failures, and suggestions on how to tackle challenges.

The discussion will be opened around mid-March on the SJIAOS discussion platform (www.officialstatistics.com).

4.1The next issue: June 2023, Volume 39 (2)

For the June 2023 issue (Vol39 (2)) some 30 manuscripts on varied topics are in preparation, many stemming from papers presented at the 2022 European Quality on Statistics Conference (Vilnius) and the 2022 IAOS Conference (Krakow). For this issue, a special section with some ten manuscripts is expected from the statistics on Governance, Peace, and Security, as covered also by the work of the so-called Praia group on Governance Statistics.

The Special issue on the ‘History of Official Statistics’, is still in preparation but will not make it before June 2023. The guest editorial team is still in search of more authors and relevant manuscripts, so, do not hesitate to inform me when you have a ready manuscript or an idea for a manuscript for this Special. ([email protected]).

Of course there are always slots for other manu-scripts; authors are kindly invited to submit their manuscripts via the submission channel: https://officialstatistics.com.

Pieter Everaers

Editor-in-Chief

January 2023

Statistical Journal of the IAOS

E-mail: [email protected]