Updating the paradigm of official statistics: New quality criteria for integrating new data and methods in official statistics

Article type: Research Article

Authors: De Broe, Sofie^* | Struijs, Peter | Daas, Piet | van Delden, Arnout | Burger, Joep | van den Brakel, Jan | ten Bosch, Olav | Zeelenberg, Kees | Ypma, Winfried

Affiliations: Statistics Netherlands, The Netherlands

Correspondence: [*] Corresponding author: Sofie De Broe, Statistics Netherlands, P.O. Box 4481, 6401 CZ Heerlen, The Netherlands. E-mail: [email protected].

Abstract: This paper aims to elicit a discussion of the existence of a paradigm shift in official statistics through the emergence of new (unstructured) data sources and methods that may not adhere to established and existing statistical practices and quality frameworks. The paper discusses strengths and weaknesses of several data sources. Furthermore, it discusses methodological, technical and cultural barriers in dealing with new data and methods in data science; cultural as in the culture that reigns in an area of expertise or approach. The paper concludes with suggestions of updating the existing quality frameworks. We take the position that there is no paradigm shift but that the existing production processes should be adjusted and that existing quality frameworks should be updated in order for official statistics to benefit from the fusion of data, knowledge and skills among survey methodologists and data scientists.

Keywords: Non-probability sources, selectivity, model-based estimates, quality dimensions, input-driven approach

DOI: 10.3233/SJI-200711

Journal: Statistical Journal of the IAOS, vol. 37, no. 1, pp. 343-360, 2021