Assessing compliance with the United Nations Fundamental Principles of Official Statistics: A maturity model for continuous improvement1
Abstract
In 2019 the United Nations Statistical Commission (UN STATSCOM) celebrated the twenty-fifth anniversary of their adoption of the United Nations Fundamental Principles of Official Statistics (UNFPOS). To mark this milestone and drive further adoption of the principles, a UN STATSCOM Friends of the Chair Group on the Fundamental Principles of Official Statistics (FoC) was convened to work on selected aspects of implementing the UNFPOS. One aspect the group worked on was development of criteria for assessing compliance with the UNFPOS. The group achieved this by developing a maturity model that both assesses compliance with the UNFPOS and identifies areas for continuous improvement. The Maturity Model was endorsed at the 51
1.Introduction
The UN Fundamental Principles of Official Statistics (UNFPOS), also referred to as the Fundamental Principles, enshrine the basic guiding tenets of official statistics produced by national statistical offices (NSOs) and all entities that are part of national and global statistical systems.
Table 1
UN Fundamental Principles of Official Statistics [4]
| Principle 1. Official statistics provide an indispensable element in the information system of a democratic society, serving the Government, the economy and the public with data about the economic, demographic, social and environmental situation. To this end, official statistics that meet the test of practical utility are to be compiled and made available on an impartial basis by official statistical agencies to honour citizens’ entitlement to public information. |
| Principle 2. To retain trust in official statistics, the statistical agencies need to decide according to strictly professional considerations, including scientific principles and professional ethics, on the methods and procedures for the collection, processing, storage and presentation of statistical data. |
| Principle 3. To facilitate a correct interpretation of the data, the statistical agencies are to present information according to scientific standards on the sources, methods and procedures of the statistics. |
| Principle 4. The statistical agencies are entitled to comment on erroneous interpretation and misuse of statistics. |
| Principle 5. Data for statistical purposes may be drawn from all types of sources, be they statistical surveys or administrative records. Statistical agencies are to choose the source with regard to quality, timeliness, costs and the burden on respondents. |
| Principle 6. Individual data collected by statistical agencies for statistical compilation, whether they refer to natural or legal persons, are to be strictly confidential and used exclusively for statistical purposes. |
| Principle 7. The laws, regulations and measures under which the statistical systems operate are to be made public. |
| Principle 8. Coordination among statistical agencies within countries is essential to achieve consistency and efficiency in the statistical system. |
| Principle 9. The use by statistical agencies in each country of international concepts, classifications and methods promotes the consistency and efficiency of statistical systems at all official levels. |
| Principle 10. Bilateral and multilateral cooperation in statistics contributes to the improvement of systems of official statistics in all countries |
The need for such principles emerged at the end of the 1980s when countries in Central Europe began to transit from centrally planned economies to market-oriented democracies [1]. The Conference of European Statisticians responded by developing and adopting the Fundamental Principles of Official Statistics in 1991 (CES/702) and subsequently, at the ministerial level, in 1992 (UNECE decision C(47)) [2]. Realising the Fundamental Principles were of much wider global significance, a milestone in the history of international statistics was reached when the United Nations Statistical Commission adopted, at its Special Session of 11–15 April 1994, the very same set of principles – with a revised preamble – as the United Nations Fundamental Principles of Official Statistics [3]. The Fundamental Principles and updated preamble were subsequently endorsed by the UN General Assembly in 2014. This marked recognition beyond the statistical community of the importance of appropriate and reliable data that adheres to certain professional and scientific standards [4].
The Fundamental Principles comprise ten principles (see Table 1) which collectively guide the production and availability of official statistics and recognise the value of coordination and international cooperation. They are of such a general and basic nature they may be referred to as the ‘basic law’ or the ‘constitution’ of official statistics.
Designed to be universal across cultures, political systems and time, the Fundamental Principles express the conviction and commitment that official statistics need to adhere to well-defined professional and scientific standards in order to provide truthful information on which to base decisions at national, regional and global levels. They define the value system to which the professional community of official statisticians should aspire.
The Fundamental Principles form a solid basis for other documents which guide good statistical practice throughout the world. The European Statistics Code of Practice [5] and the OECD Recommendation for Good Statistical Practice [6] are two of such examples. Whilst such documents are complementary, they can also introduce differences. For instance, the OECD Recommendation is a legal instrument and it only applies to OECD members (although non-OECD members are welcome to adhere also). Further information on the OECD Recommendation is provided in Annex I.
As many countries develop national approaches such as governance frameworks and codes of practice, or when they update their own national legislation for official statistics, alignment with the Fundamental Principles is recognised as both essential and good practice [7].
Like other ‘constitutions’, the Fundamental Principles come to life and receive meaning only through their implementation and application. They have been supported by a variety of tools at the global and national level, including a good practices database [8], implementation guidelines [9] and the Handbook on Statistical Organization [10]. However, until now no tool was available for actors in a national statistical system to assess compliance with the Fundamental Principles.
In the next section, we explain why compliance is important before outlining a maturity model framework as a tool for assessing compliance with the Fundamental Principles. We then present an example of the application of this maturity model to a hypothetical statistical organisation and conclude with a brief discussion on the importance of applying the maturity model to national circumstances.
2.Why is compliance important?
Official statistics are a recognised part of the knowledge infrastructure for a country. Official statistics:
• underpin decisions by governments, businesses, communities, citizens and the international community;
• influence public and private sector investment choices;
• monitor progress in policies and development agendas, including the 2030 Agenda for Sustainable Development; and
• inform policy development and measure its degree of success.
Official statistics are also integral to good governance and meeting citizens’ expectations of authoritative and trustworthy information about their country.
In this era of ‘fake news and alternative facts’, official statistics are one source which should be able to be trusted to provide ‘authentic information and real facts’. Compliance with the ‘basic law’ or ‘constitution’ represented in the Fundamental Principles is a powerful assurance of the impartiality, rigour and best possible accuracy of official statistics.
There are significant benefits to implementing and achieving compliance with the Fundamental Principles. In the 1990’s, the development of the principles responded to a need to ensure national statistical systems would be able to produce appropriate and reliable data that adhered to certain professional and scientific standards [1]. The need to ensure appropriateness and reliability of official statistics remains today, but more recently emerging factors such as increasing contestability of ‘facts’ and availability of new data sources are reinforcing both a need to assure compliance of official statistics with the Fundamental Principles, and to identify and guide the continuous improvement priorities for producers of official statistics.
In 2015, the UN STATSCOM gave attention to country compliance with the Fundamental Principles and explored ways to effectively address perceived non-compliance [11]. Discussion at the time noted the absence of definitions of compliance and non-compliance and that such absence left these definitions open to interpretation across different national and regional contexts. The need for definitions was left unresolved.
In 2017, the UN STATSCOM established a Friends of the Chair Group on the Fundamental Principles of Official Statistics (FoC) to work on selected implementation aspects of the Fundamental Principles in the context of preparations for twenty-fifth anniversary celebrations since their adoption by UN STATSCOM in 1994 [12]. This work comprised a number of activities including:
• a global assessment survey on implementation of the Fundamental Principles to assess the current state, to provide insights into good practice and to identify where there was further room for improvement;
• development of supplementary chapters to the existing Fundamental Principles implementation guidelines to support use and adoption, including when using new data sources for the production of official statistics; and
• case studies to support learning and good practices.
The FoC presented progress with their work at the UN STATSCOM at its 50
• understanding and appreciation of the principles among members of National Statistical Systems beyond the NSO, and in particular among those line ministries or governmental departments to which the NSO reports;
• the need to strengthen the integration of Fundamental Principles into legal frameworks; and
• knowledge about Principle 2 on professional standards, scientific principles and professional ethics for those working on the production of statistics among line ministries [14].
The FoC also recommended a maturity model approach to compliance with the Fundamental Principles [13].
In welcoming the work of the FoC, the UN STATSCOM requested the group to complete its work and submit two supplementary chapters to the implementation guide for the Fundamental Principles to the UN STATSCOM at its fifty-first session in 2020:
• a supplementary chapter with guidance on evaluation criteria and recommended actions to assist countries in transitioning their official statistical systems towards compliance; and
• guidance on implementation of the Fundamental Principles when using new data sources for the production of official statistics.
Evaluation criteria and recommended actions towards compliance are the subject of this paper, while guidance when using new data sources can be found in Rozkrut et al. [15].
3.Assessing compliance – a maturity approach
The concept of “maturity” rather than the more binary concept of compliance is more helpful for focussing improvement and thereby, ultimately achieve compliance.
More specifically, maturity models can:
• serve as a diagnostic tool that enables organisations to assess where their current capabilities lie in terms of the Fundamental Principles and therefore, their current maturity level (descriptive);
• be used to identify desirable maturity levels and suggest actions to reach a desired maturity level (prescriptive); and
• serve as a comparative tool since it allows for external benchmarking among other organisations (comparative).
The maturity model approach is structured as a series of levels towards full compliance. The approach begins with an assessment of the level of current compliance. Once the current level is determined, the next level sets out what activities need to be prioritised to move to the next level and/or maintained to stay at the current level.
The purpose of a maturity model should not be to ‘tick the box’ and comply with the description of the highest maturity level, but to use it as a tool for achieving greater maturity over time. In other words, an organisation should focus on identifying ways to continuously improve rather than on attaining the highest maturity level – particularly as statistical organisations have limited resources and therefore need to prioritise their efforts to mature.
Maturity models are becoming prevalent in the world of official statistics. Recent examples include the Modernisation Maturity Model [16] and the Communications Function Maturity Model [17].
4.The UN Fundamental Principles of Official Statistics Maturity Model
The UN Fundamental Principles of Official Statistics Maturity Model (Annex II) was endorsed by the UN STATSCOM in March 2020 [18]. As noted, the Maturity Model is a diagnostic tool to aid NSOs and national statistical systems in identifying actions to improve compliance with the Fundamental Principles.
The Maturity Model has eleven dimensions and three levels.
The eleven dimensions represent the ten principles with Fundamental Principle 1 comprising two sub-dimensions: ‘relevance’ and ‘impartiality and equal access’.
For each dimension or sub-dimension, there are three maturity levels – Developing, Practising and Leading. ‘Developing’ is the least mature level and ‘Leading’ is the most mature.
Descriptions of the maturity levels were drawn from existing resources, including the Fundamental Principles Implementation guidelines [9], the Handbook on Statistical Organization [10] and Global Assessment Survey [14]. They serve as familiar material, repackaged to provide guidance about maturity/compliance with the Fundamental Principles.
Within each maturity level, criteria are given for an organisation to assess their maturity. For example, in sub-dimension 1 (relevance), four criteria are presented: involvement of users and partners in the development of work programmes; accessibility of data; the ability to download data; and the existence of a release schedule for statistical products and services.
A maturity description is given within each criteria. For example, an organisation’s maturity is ‘developing’ if users have limited access to online data, is ‘practising’ if users have access to online data for exploration and is ‘leading’ if users have full access to data for exploration and additional/secondary uses.
In total, 42 categories are presented across the eleven dimensions/sub-dimensions and three maturity levels. To establish the overall current state of maturity, an organisation’s maturity is assessed against the 42 categories.
Once the current state of maturity is established, organisations can use the Maturity Model to identify activities for self-improvement. A range of reference material is available to support self-improvement and attainment of a higher level of maturity against the dimensions and sub-dimensions. The material is available in the full document endorsed by the UN STATSCOM [19].
4.1A hypothetical example of the UN Fundamental Principles of Official Statistics Maturity Model in practice
In this hypothetical example, the application of the Maturity Model is presented for illustrative purposes only. The organisation chosen is a NSO in a developing nation without the existence of statistical legislation. Fundamental Principle 1 is chosen to illustrate the application of the model across dimensions and sub-dimensions.
Table 2
Maturity assessment against Fundamental Principle 1 for hypothetical statistical organisation
| Dimension | Evaluation against the criteria | Maturity level assessment | |
|---|---|---|---|
| Relevance | Work programmes are developed with users and signed off with the Government. The programme is publicly available. | Leading | The NSO is leading in its engagement with users when developing the work programme and are |
| Users can explore official statistics and in some cases access microdata files on the www. | Practising | transparent with statistical release schedules and plans. Areas for improvement include data | |
| Data downloads are limited to pdfs and printed publications. | Developing | accessibility including downloads of data in | |
| Release dates and times are pre-announced on the organisation’s website via an advance release calendar. | Practising | machine readable format and use of APIs. | |
| Impartiality and equal access | Rules for the appointment and dismissal of the Chief Statistician are issued by the Government of the day and not enshrined in legislation. | Developing | The statistical organisation has mature practices for the compilation of official statistics, sourcing data, methods and procedures, and training of |
| Pre-release of statistics is given to the Central Bank and Finance Minister only but there are no conditions imposed. | Practising. | technical and managerial staff. The organisation is practising in two areas (pre-release and | |
| Most data services are charged as the organisation operates under a cost-recovery model. | Developing | transparency of revisions to statistical methods and classifications. Areas for development | |
| Statistics are compiled on an objective basis by qualified professionals employed by the statistical organisation and with statements issued by the statistical organisation. | Leading | include rules for the appointment and dismissal of the Chief Statistician and more objective recruitment and promotion practices, free access | |
| Data sources are chosen based on respondent load, cost, availability and relevance to statistical considerations. Methods and procedures are guided by international recommendations of the UN Statistical Commission wherever practical. | Leading | to data by all and procedures and policies for error detection and correction. | |
| Changes and revisions to statistical methods and classifications are made known to users in advance as much as possible, but this is not always possible. | Practising | ||
| Procedures and policies are not in place for the reporting and correcting of errors. | Developing | ||
| Recruitment and promotion of staff is mostly objective, but there are times where appointments are made at the discretion of our Minister or Ministry. | Developing | ||
| Training programmes are in place for all staff, both technical and managerial. | Leading | ||
Fundamental Principle 1: Relevance, impartiality and equal access
Description of dimension
Official statistics provide an indispensable element in the information system of a democratic society, serving the Government, the economy and the public with data about the economic, demographic, social and environmental situation. To this end, official statistics that meet the test of practical utility are to be compiled and made available on an impartial basis by official statistical agencies to honour citizens’ entitlement to public information.
Maturity Assessment (Table 2)
The statistical organisation has mature practices in place for engagement with users when developing the work programme and is transparent with statistical release schedules and plans. It is also mature when it comes to the compilation of official statistics, sourcing data, methods and procedures, and training of technical and managerial staff.
The organisation is maturing its practices in two areas: pre-release practices and transparency of revisions to statistical methods and classifications.
Areas for improvement
Areas of improvement the NSO may wish to consider are:
• data accessibility, including dissemination practices (for example downloads of data in machine readable format, the use of APIs and free access to data by all);
• more objective recruitment and promotion practices including rules for the appointment and dismissal of the Chief Statistician; and
• development and publication of procedures and policies for error reporting and correction.
5.Assessing compliance can take many forms
The Maturity Model provides an assessment tool that can be used to assess current maturity against a standard framework. The tool can be used as part of self-assessments or peer reviews.
5.1Self-assessment
Self-assessment has an important role in bringing compliance challenges into the open, promoting good practice and detecting non-compliance [11]. The FoC recommended self-assessment should form the basis for evaluating the level of compliance and this self-assessment should be primarily through the regular Global Assessment Review which assesses the implementation of the Fundamental Principles [19].
The approach of encouraging self-assessment against good practice models is widely used. For example, guidance on self-assessment forms part of the toolkit for the OECD Recommendation on Good Statistical Practice 2̧0 and, as of June 2020, twelve countries have publicly released a self-assessment: nine OECD and three non-OECD members [20].
5.2Peer review
A peer review programme can complement self-assessment. Peer reviews can assess a country’s capacity to produce and disseminate official statistics and also assess their adherence or alignment to established standards/principles, including the Fundamental Principles.
The FoC noted establishing a global peer review programme is impractical and a more manageable option is for countries (or groups of countries) to invite independent peer review on a voluntary basis [19]. Voluntary peer reviews can then be helpful in supporting countries to understand how they might improve their practices to build compliance, and in highlighting good practice in specific cases.
In official statistics, voluntary peer reviews have led to quality improvements in the institutional environment, statistical processes and statistical outputs of a number of countries. For example, a 2014 peer review of Mongolia’s NSS boosted policy, tools and technology for dissemination of official statistics [21] and today, Mongolia’s NSO is known and well-respected in the Asia-Pacific region for their state-of-the-art dissemination platform.
Peer reviews can also contribute to improving the standing of official statistics as a useful and credible public good and be used to enhance the independence, integrity and accountability of the statistical organisation [22].
Peer reviews also enable the direct sharing of experience and good practices. Such processes have been used effectively in the European Union [23] and in a range of regions around the world including the Asia-Pacific and the Caribbean Community. Mongolia, for instance, ‘paid it forward’ by contributing to a peer review of the NSS of the Maldives in 2017 which resulted in a new regulatory framework for official statistics [24].
The FoC also recommended the outcomes of peer reviews be published publicly [19]. This is the practice of the peer review examples mentioned earlier.
For a peer review process to work effectively, there also needs to be an open and transparent process to appoint peer review teams and agree on terms of reference for the remit of such reviews. Support is available for such processes. The UN Economic and Social Commission for Asia and the Pacific, for example, supported voluntary peer reviews. In the Europe region, the UN Economic Commission for Europe provides similar support [25]. The OECD’s Partnership in Statistics for Development in the 21
Peer review is part of the toolkit for the OECD Recommendation on Good Statistical Practice [20] and as of June 2020, thirty countries (twenty-seven OECD and three non-OECD members) have publicly published the outcomes of their peer review assessment [20].
5.3Accreditation
A third option for improving the implementation of the Fundamental Principles recommended by the FOC might be some form of certification or accreditation conferred on producers that exemplify sustained and consistent application of some or all the principles [19]. Certification would usually be based on the assessment by one or more independent third parties.
The FoC noted practical challenges, especially time and cost, to this option and, as such, recommended implementation of an accreditation system be reviewed in the future if there is a recognised need and it is supported by countries. The process would require proper development of accreditation methodology and processes, including more formally agreed definitions of what is required to have reached the various maturity levels against each of the principles.
6.Conclusion
The Fundamental Principles enshrine the basic guiding tenets of official statistics produced by all entities that are part of national and global statistical systems including national statistical offices. The Fundamental Principles come to life and receive meaning only through their implementation and application, and a new Maturity Model is now available to enable this.
The Maturity Model is based on the notion of self-improvement. It is a diagnostic tool that can aid organisations identify actions to improve compliance with the Fundamental Principles.
The Maturity Model can also guide future global assessment surveys on implementation of the Fundamental Principles to provide a better understanding of the overall level of global compliance.
The tool is easy to apply and written guidance is available. However, it would be of real value to NSO’s and national statistical systems to have more examples of the application of the tool in practice. This paper has presented an illustrative example for a hypothetical organisation – and for only one Fundamental Principle.
Through application of the Maturity Model and refinement of the processes to undertake self and other forms of assessment, compliance with the Fundamental Principles can be assessed nationally, regionally and globally. The Maturity Model can then become a valuable part of the fabric of standards, classifications and frameworks available to support the improvement of, and increase the value of, statistical systems worldwide.
Adoption of the Maturity Model by the UN STATSCOM in 2020 is timely and joins an increasing number of calls for the Fundamental Principles to receive meaning through implementation and application. Georgiou, for instance, argues whilst progress has been made to promote the Fundamental Principles, monitoring is limited and an international system of review and assessment of implementation is needed [27]. Some systems are emerging, for example the establishment of the United Kingdom Office of Statistics Regulation22 or at the European Level by the European Statistical Governance Advisory Board33 to defend the integrity of official statistics and the independence of official statisticians when they are endangered. The Maturity Model is another step in this direction.
The Maturity Model can also be used to guide wider engagement with key advisory bodies such as National Statistical Councils. The Maturity Model considers user consultation, data access and downloads and transparency of release schedules for relevance principle. An assessment of the maturity level against each of these criteria can be used as the basis for self-improvement as well as for engaging with National Statistical Councils in ensuring statistical programs effectively meet the needs of all users.
The Maturity Model also provides a tool to guide both implementation and engagement on other principles, such as Fundamental Principle 4. Principle 4 entitles statistical agencies to comment on erroneous interpretation and misuse of statistics and the Maturity Model considers interventions, commentary and resources and training as assessment criteria. There are an increasing number of high profile examples of implementation of principle 4 (Georgiou [28] documents examples from Greece, Canada and Argentina amongst others) which can be used to inform national self-improvement efforts. However, as many of these examples highlight, serious consequences can emerge such as the removal of the Chief Statistician from office.
The IAOS, as the professional body of the official statistics community, has convened multiple discussions to bring attention to the essence of Fundamental Principle 4 [29, 30] and several proposals for action are emerging. Martine Durand, OECD Chief Statistician and Director of Statistics and Data until the end of 2019, proposes, for instance, a new Code of Ethics and the establishment of an independent body [31]. The possible need for a global data convention was also noted by the Committee for the Coordination of Statistical Activities in its report to the 52
The Maturity Model, including its implementation for Fundamental Principle 4, is therefore a timely contribution to a possible code of ethics or data convention which may emerge as a contemporary set of guiding tenets for the official statistical community nearly 20 years after the adoption of the Fundamental Principles in 1991 by the Conference of European Statisticians and subsequently in 1994 by the UN STATSCOM.
Notes
References
[1] | United Nations Statistics Division. UN Fundamental Principles of Official Statistics Background. Accessed at https://unstats.un.org/fpos/. Last viewed 21 January (2021) . |
[2] | United Nations Statistical Commission and United Nations Economic Commission for Europe. 50 years of the Conference of European Statisticians. 2003. Accessed at https://unece.org/DAM/stats/publications/50yearsCES.pdf. Last viewed 13 March (2021) . |
[3] | United Nations Statistical Commission. Report on the Special Session (11–15 April 1994) Economic and Social Council Official Records, 1994 Supplement No.9. 1994. Accessed athttps://unstats.un.org/unsd/statcom/1994-special-session/documents/statcom-1994-special-report-E.pdf. Last viewed 21 January (2021) . |
[4] | United Nations. Resolution adopted by the General Assembly on 29 January 2014 Fundamental Principles of Official Statistics A/RES/68/261. 2014. Accessed at https://unstats.un.org/unsd/dnss/gp/FP-New-E.pdf. Last viewed January (2021) . |
[5] | European Union. European statistics code of practice. Accessed at https://ec.europa.eu/eurostat/web/quality/european-statistics-code-of-practice. Last viewed 10 February (2021) . |
[6] | Organisation for Economic Co-operation and Development. Recommendation of the Council on Good Statistical Practices. Accessed at https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0417. Last viewed 21 January (2021) . |
[7] | United Nations Economic Commission for Europe. Modernizing Statistical Legislation ECE/CES/STAT/2018/3. E.17.II.E.17. (2018) . |
[8] | United Nations Statistics Division. Good Practices Database. 2014. Accessed at https://unstats.un.org/unsd/dnss/gp/searchgp.aspx. Last viewed 21 January (2021) . |
[9] | United Nations Statistics Division. United Nations Fundamental Principles of Official Statistics: Implementation Guidelines. 2015. Accessed at https://unstats.un.org/unsd/dnss/gp/impguide.aspx. Last viewed 21 January (2021) . |
[10] | United Nations Statistics Division. United Nations Handbook of Statistical Organisation Third Edition. The Operation and Organisation of a Statistical Agency. 2003. Accessed at https://unstats.un.org/unsd/dnss/hb/searchhb.aspx. Last viewed 21 January (2021) . |
[11] | United Nations Statistical Commission. Report of the Friends of the Chair Group on implementation of the Fundamental Principles of Official Statistics E/CN.3/2015/18. 2015. Accessed at https://unstats.un.org/unsd/statcom/doc15/2015-18-FP-E.pdf. Last viewed 21 January (2021) . |
[12] | United Nations Statistical Commission. Report of the forty-eighth session (7–10 March 2017) E/2017/24-E/CN.3/2017/35. 2017. Accessed at https://unstats.un.org/unsd/statcom/48th-session/documents/Report-on-the-48th-session-of-the-statistical-commission-E.pdf. Last viewed 21 January (2021) . |
[13] | United Nations Statistical Commission. Report of the fiftieth session (5–8 March 2019) E/2019/24-E/CN.3/2019/34. 2019. Accessed at https://unstats.un.org/unsd/statcom/50th-session/documents/Report-on-the-50th-session-of-the-statistical-commission-E.pdf. Last viewed 21 January (2021) . |
[14] | United Nations Statistics Division. Assessment survey of the implementation of the Fundamental Principles of Official Statistics. 2019. Accessed at https://unstats.un.org/unsd/statcom/50th-session/documents/BG-Item3b-FPOS-survey-results-E.pdf. Last viewed 21 January (2021) . |
[15] | Rozkrut D, Świerkot-Strużewska O and Van Halderen G. Mapping the united Nations Fundamental Principles of Official Statistics against new and big data sources. Statistical Journal of the IAOS 37 No. 1, forthcoming. (2021) . |
[16] | United Nations Economic Commission for Europe. Roadmap for Implementing Modernstats Standards: Modernisation Maturity Model. 2017. Accessed at https://statswiki.unece.org/pages/viewpage.action?pageId=129172266. Last viewed 21 January (2021) . |
[17] | United Nations Economic Commission for Europe Conference of European Statisticians. Strategic Communications Framework for Statistical Institutions. 2019. Accessed at https://unece.org/fileadmin/DAM/stats/documents/ece/ces/2019/7_Strategic_commmunication_framework_for_consultation.pdf. Last viewed January (2021) . |
[18] | United Nations Statistical Commission. Report of the fifty-first session (3–6 March 2020) E/2020/24-E/CN.3/2020/37. 2020. Accessed at https://unstats.un.org/unsd/statcom/51st-session/documents/2020-37-FinalReport-E.pdf. Last viewed 21 January (2021) . |
[19] | United Nations Statistical Commission Friends of the Chair Group on the Fundamental Principles of Official Statistics. Supplementary chapter to the Fundamental Principles of Official Statistics Implementation Guidelines 2020. Accessed athttps://unstats.un.org/unsd/statcom/51st-session/documents/BG-Item3q-Supplementary_chapter-E.pdf. Last viewed 21 January (2021) . |
[20] | Organisation for Economic Co-operation and Development. Good Statistical Practice Toolkit. Accessed at https://www.oecd.org/statistics/good-practice-toolkit/. Last viewed 21 January (2021) . |
[21] | National Statistics Office of Mongolia. Global Assessment Report National Statistical System of Mongolia. 2014. Accessed at https://www.en.nso.mn/page/44. Last viewed 10 February (2021) . |
[22] | Martín-Guzmán P and Aguilera M. Statistical governance in the Latin American and the Caribbean Region: Achievements and challenges. Statistical Journal of the IAOS 31, 655–659. (2015) . |
[23] | European Union. Eurostat. Quality. European statistics code of practice. Monitoring: Peer Reviews Accessed at https://ec.europa.eu/eurostat/web/quality/peer-reviews. Last viewed 21 January (2021) . |
[24] | United Nations Economic and Social Commission for Asia and the Pacific. Report of the Review of the National Statistical System of the Maldives. 2017. Accessed at http://statisticsmaldives.gov.mv/nbs/wp-content/uploads/2018/05/NSS-Review-Report-Maldives-FINAL.pdf. Last viewed 10 February (2021) . |
[25] | United Nations Economic Commission for Europe. Global assessments and sector reviews. Accessed at https://unece.org/statistics/global-assessments-and-sector-reviews. Last viewed 10 February (2021) . |
[26] | Partnership in Statistics Development in the 21 |
[27] | Georgiou A. Towards a global system of monitoring the implementation of the UN fundamental principles in national official statistics. Statistical Journal of the IAOS 33 No. 2, 387–397. (2017) . |
[28] | Georgiou A. The manipulation of official statistics as corruption and ways of understanding it. Statistical Journal of the IAOS 37 No. 1, forthcoming. (2021) . |
[29] | International Association of Official Statistics. Misuse of Statistics: Time to Speak Out. Side event at the virtual United Nations World Data Forum. 2020. Accessed at https://www.iaos-isi.org/index.php/latestnews/278-misuse-of-statistics-time-to-speak-out. Last viewed 13 March (2021) . |
[30] | International Association of Official Statistics. Misuse of Statistics: Time to Speak Out. Side event at the 52 |
[31] | Durand M. Presentation at a Misuse of Statistics: Time to Speak Out. Side event at the 52 |
[32] | United Nations Statistical Commission. Report of the Committee for the Coordination of Statistical Activities E/CN.3/ 2021/6. 2021. Accessed at https://unstats.un.org/unsd/statcom/52nd-session/documents/2021-6-CCSA-E.pdf. Last viewed 13 March (2021) . |
[33] | World Bank. World Development Report 2021: Data for Better Lives. 2021. Accessed at https://www.worldbank.org/en/publication/wdr2021. Last viewed 25 March (2021) . |
Appendices
Annex I: OECD Recommendation of the Council on Good Statistical Practice
The OECD Recommendation was developed to ensure the quality of evidence-based analytical work and the quality of statistical publications and databases produced by the OECD. It was also developed to provide specific guidance and indications of good statistical practices (noting while the UNFPOS provide a useful reference point, they are general in nature). Its release was accompanied by a toolkit comprising a set of good practices based on existing international and national guidelines, a self-assessment tool, completed country assessments and links to other international guidelines for official statistics including the Fundamental Principles.
The OECD Recommendation on Good Statistical Practice cover institutional, legal and resource requirements for statistical systems; methods and quality of processes of statistical production; dissemination; coordination and co-operation and statistical innovation.
| OECD Recommendation of the Council on Good Statistical Practice Recommendation 1. Put in place a clear legal and institutional framework for official statistics. Recommendation 2. Ensure professional independence of National Statistical Authorities. Recommendation 3. Ensure adequacy of human financial and technical resources available to the National Statistical Authorities for the production and dissemination of official statistics. Recommendation 4. Protect the privacy of data providers (including individuals, households, enterprises, administrations, and all level of government), and guarantee by law the confidentiality of the individual information provided and its use for statistical purposes only. Recommendation 5. Ensure the right to access administrative sources to produce official statistics. Recommendation 6. Ensure impartiality, objectivity and transparency of official statistics, through the development, production and dissemination by the National Statistical Authorities of statistics respecting scientific independence put in place in an objective, professional and transparent manner in which all users are treated equitably. Equitable treatment implies in particular, equal access to data by all users. Recommendation 7. Employ sound methodology and commit to professional standards used in the production of official statistics. Recommendation 8. Commit to the quality of statistical outputs and processes, in particular with key quality dimensions as defined in national and international quality assessment frameworks, for instance in the Quality Framework and Guidelines for OECD Statistical Activities: timeliness and punctuality (statistics are released in a timely and punctual manner), relevance (statistics meet the needs of users), accuracy (statistics accurately and reliably portray reality), credibility (confidence placed by users in statistical products), coherence and comparability (statistics are consistent internally, over time and in space and it is possible to combine and make joint use of related data from different sources), and interpretability and accessibility (see Recommendation 9). Recommendation 9. Ensure user-friendly data access and dissemination, so that statistics are presented in a clear and understandable form, released in a suitable and convenient manner, including in machine readable form (‘open data’), can be found easily, and are available and accessible on an impartial basis with supporting metadata and guidance This also entails a commitment to respond to major misinterpretations of data by users. Recommendation 10. Establish responsibilities for coordination of statistical activities within the NSS. Recommendation 11. Commit to international cooperation. Recommendation 12. Encourage exploring innovative methods and new and alternative data sources as inputs for official statistics, and in particular, encourage statistical agencies to actively explore possibilities to use new data sources (including large datasets owned by the private sector) or to combine existing and new data sources as input for official statistics. At the same time, these opportunities are weighted against the limits of using modern information technologies and the need to maintain the quality of official statistics. |
Annex II: UN Fundamental Principles of Official Statistics Maturity Model
| Developing | Practising | Leading | |
|---|---|---|---|
| Principle 1: Relevance | Awareness of the importance of user consultation (e.g. investigate and document user needs, measure user satisfaction) to ensure that the statistical work program achieve that. | Regular consultation ensures user needs are considered in statistical work program development. This might include e.g. regular user meetings or use of online collaboration tools. | Statistical work programs are developed in partnership with users and reflect user needs. This might include e.g. sign-off of work program by a user council or similar external bodies. |
| Users have limited access to data including online channels. | Users have access to data for exploration including online channels. | Users have full access to data for exploration and onward use including online channels. | |
| Data downloads are limited. Data are mostly released through print publications, online PDF files etc. | Data downloads available in propriety formats (Excel, Access, SAS, Stata, SPSS). | Data downloads available in open machine-readable formats (CSV, XML, JSON). Online APIs allow users to interact directly with statistical outputs. | |
| Users are not informed of upcoming data releases. | Release dates and times are pre-announced via a public advance release calendar. | Release dates and times are pre-announced via a public advance release calendar. Changes to release dates are published and the reasons for the change are documented and made public. | |
| Principle 1: Impartiality and Equal Access | There is limited clarity about the rules for the appointment and dismissal of the Chief Statistician. | There are clear rules for the appointment and dismissal of the Chief Statistician. | Rules for the appointment and dismissal of the Chief Statistician include specifics such as qualifications, selection procedure, length of appointment, reasons for dismissal. |
| Pre-release of statistics occurs. | Pre-release of statistics to some user groups occurs. | There is equal access to statistics in principle. Pre-release of statistics is limited, controlled and transparent. | |
| Users have limited access to data free of charge. | Other than special analyses, data are free of charge. | Other than special analyses, data are free of charge. For statistical services which are charged, there is a clear pricing policy that is public. | |
| Statistics are not always free from political interference. | Statistics are compiled on an objective basis. | Statistics are compiled on an objective basis. Statistical releases and published statements including press releases are impartial and objective. | |
| The choice of data sources and statistical methods are not always free from political interference. Information about methods and procedures is not always available. | The choices of sources and statistical methods as well as dissemination of statistics are informed only by statistical considerations. Information on methods and procedures is publicly available. | The choice of sources and statistical methods as well as dissemination of statistics are informed only by statistical considerations and draw on recognised good practice, both nationally and internationally. | |
| Changes and revisions to methods or classifications are not publicized. | Changes and revisions to methods or classifications are made public, but advance notice is not given. | Information on methods and procedures is accessible, freely available to the public and written to support user understanding. Advance notice is given about changes and revisions to methods or classifications. Users are supported with clear advice on how changes and revisions feed through into effective use of the statistics. | |
| Internal procedures for error reporting and correcting are limited or not in place. | There are internal procedures for error reporting and correcting in place. | A clear revision policy is in place and published. Errors are corrected are as soon as possible and publicized. | |
| Recruitment and promotion of staff is not always on an objective basis. | An effective, objective HR system is used to manage the appointment and promotion of staff in place. | Recruitment and promotion of the staff responsible for statistical information is through professional and open processes, based on aptitude and expertise. | |
| Training provision for staff is limited. | Training is available for all staff. | Training programs are in place to identify, maintain and develop staff so that they have the skills they need to perform their tasks. |
| Developing | Practising | Leading | |
|---|---|---|---|
| Principle 2 | Awareness of need to align statistical methodologies with international best practice where they exist. | Most statistical methodologies use internationally recommended standards and methods where they exist. | Statistical methodologies use internationally recommended standards and methods where they exist. |
| Guidelines on professional ethics exist (for example in the statistical legislation or in organizational policy). | Training on professional ethics exists. | Regular training and reminders on professional ethics are provided to staff. There is published guidance on the ethical production and use of statistics. There is independent assessment of ethical use of data and statistics and findings are published. | |
| Awareness that peer or expert reviews will add value. | Actively seeks peer or expert reviews. | A group of experts (for example an independent methodological council) advises and/or endorses methods used. | |
| Awareness of the benefit of consistent use of standards across national statistical system. | Promotes consistent use of standards across national statistical system. | Assists the national statistical system with use of standards and methods (for example, external monitoring or auditing of practices, approving questionnaires). | |
| Principle 3 | Explanatory text or methodological notes are provided with data. | There are frameworks and standards for statistical production and quality reporting and these are publicly available. | There are clear processes and policies for reporting on quality and methods. They are publicly available and align with international best practice. |
| Information about statistical quality is limited or unavailable. | Quality information is provided as part of metadata. | Quality reports are always available with data releases. Guides supporting users to help them interpret data are available. | |
| There is limited information about changes in methods or corrections. | Users are notified of major methodological changes and error corrections. | Users are always notified about methodological changes and error corrections. There are clear processes and policies for reporting and these are publicly available. | |
| Principle 4 | The National Statistical Office has no policy for intervention on erroneous interpretation and misuse of statistics. | The National Statistical Office or equivalent body has a policy on how it will intervene on erroneous interpretation and misuse of statistics. | The National Statistical Office or equivalent body has a public policy on how it will intervene on erroneous interpretation and misuse of statistics. |
| The National Statistical Office does not comment on erroneous interpretation and misuse of statistics. | The National Statistical Office may comment on erroneous interpretation and misuse of statistics. | The National Statistical Office or equivalent body comments publicly and regularly on erroneous interpretation, misuse or unclear presentation of statistics. This might include impartial cooperation with the media, independent fact checking organisations or similar. | |
| Resources on the responsible use of statistics are limited and training is infrequent. | There is some training and guidance on the responsible use of statistics to prevent erroneous interpretation and misuse. | There is timely comprehensive training and published guidance on the responsible use of statistics to prevent erroneous interpretation and misuse. | |
| Principle 5 | Data is drawn from a limited set of sources. Little use is made of relevant administrative or commercial data. | Administrative records, statistical registers and third party data sources are used where appropriate. | Administrative records, statistical registers and other data sources are used whenever appropriate to improve the quality and coverage of official statistics. Appropriate governance and data policies are used to manage data use. |
| There is limited or no liaison with data suppliers outside the National Statistical Office. | Relationships with data suppliers are in place. Activities with data suppliers include advice on amending the composition and classification of administrative datasets, and feedback when errors are detected. | New or alternative data sources are actively investigated for opportunities to improve the quality and coverage of official statistics. | |
| Coherence and consistency across statistics from different agencies and offices are not addressed. | Coherence and consistency of statistics is coordinated across some departments and agencies. | Coherence and consistency are recognized priorities for the statistical system. There is strategic oversight of coherence and consistency across statistical data sources. Governance is in place to promote and ensure coherence. |
| Developing | Practising | Leading | |
|---|---|---|---|
| There is limited or no oversight of how statistical concepts and resources are used across agencies and departments. | The importance of harmonized standards is recognized. There is some guidance on harmonization. | Statistical concepts and resources are harmonized across producers wherever possible and there are central resources to support producers in harmonizing their outputs. | |
| Principle 6 | A confidentiality policy is in place. | A confidentiality policy (including guidelines and instructions) is in place. | A comprehensive confidentiality policy (including guidelines and instructions) is in place. |
| The confidentiality policy is provided to staff. | The confidentiality policy outlines mechanisms to guarantee the privacy of data. The confidentiality policy is provided to staff and is made publicly available. | The confidentiality policy outlines mechanisms to guarantee the privacy of data at each stage of the statistical process – from the preparation of surveys up to the dissemination of statistical products. The confidentiality policy is provided to staff and is made publicly available. | |
| There are limited additional requirements for staff with access to confidential or personal data. | Staff with access to individual or confidential information must sign a confidentiality commitment on appointment. | Staff with access to individual or confidential information must sign a confidentiality commitment on appointment. There is training in place to ensure responsible access to confidential and personal information. | |
| Conditions apply to users accessing statistical microdata for research purposes, but they are not consistently applied. | Established appropriate confidentiality procedures and processes take place before researchers have access to microdata. | Strict conditions apply to users accessing statistical microdata for research purposes. The conditions and processes required are publicly available. Researcher’s use of microdata is monitored in order to immediately apply corrective actions. | |
| Basic physical, technological and organizational provisions are in place to protect the security and integrity of statistical databases. | Physical, technological and organizational provisions are in place to protect the security and integrity of statistical databases. These draw on relevant international best practice. | Sophisticated and modern physical, technological and organizational provisions are in place to protect the security and integrity of statistical databases. These follow relevant international best practice and are independently audited. | |
| Principle 7 | A general statistics law exists. | The statistical law and associated regulations address most of the following elements:
| The statistical law and associated regulations address all of the following elements:
|
| Laws, regulations and measures for statistics are not easily discoverable. Public availability is limited. | The laws, regulations and measures are easily discoverable and available to the public. | The laws, regulations and measures are easily discoverable and available to the public. | |
| Principle 8 | Awareness that statistical coordination is required, but limited scope to achieve it. | There is some coordination across statistical producers. | Statistical coordination is cross-system, so that all national producer bodies work together to ensure coherence and take a unified approach.There are frameworks in place to formalise coordination. |
| Developing | Practising | Leading | |
|---|---|---|---|
| Governance for statistics is split across different organisations. | Statistical governance is coordinated, but there is limited or no centralization. | The National Statistical Office is recognized and valued as the lead body for strategic decisions about statistical coordination. There are agreed frameworks in place that set out how statistical governance should operate across the system. | |
| Principle 9 | The main international concepts, classifications and methods for the development, production and dissemination of official statistics are used. | International concepts, classifications and methods for the development, production and dissemination of official statistics are used and new versions are implemented where appropriate. | International concepts, classifications and methods for the development, production and dissemination of official statistics are used and new versions are implemented where appropriate. |
| Awareness that use of international standards should be promoted in the national statistical system, but limited scope to achieve it. | Use of international standards are promoted in the national statistical system. | Use of international standards are actively promoted in the national statistical system. Participation in the development of international standards is supported. | |
| Principle 10 | Participation in the main international and regional statistical discussion forums. | Active participation in international and regional statistical discussion forums. | Continuous improvement of statistics at the international level is recognised as a priority and resources are available to support this. Donor coordination mechanisms or basket funds for statistics exist (among countries receiving support from donors). |
