Affiliations: [a] Vrije Universiteit Amsterdam, The Netherlands | [b] Statistics Netherlands, The Netherlands | [c] Utrecht University, The Netherlands
Correspondence:
[*]
Corresponding author: Paulina Pankowska, Department of Sociology, Faculty of Social Sciences, Vrije Universiteit Amsterdam, de Boelelaan 1105, 1081 HV Amsterdam, The Netherlands. Tel.: +31 20 59 83178; E-mail: [email protected].
Abstract: National Statistical Institutes (NSIs) often obtain information about a single variable from separate data sources. Administrative registers and surveys, in particular, often provide overlapping information on a range of phenomena of interest to official statistics. However, even though the two sources overlap, they both contain measurement error that prevents identical units from yielding identical values. Reconciling such separate data sources and providing accurate statistics, which is an important challenge for NSIs, is typically achieved through macro-integration. In this study we investigate the feasibility of an alternative method based on the application of previously obtained results from a recently introduced extension of the Hidden Markov Model (HMM) to newer data. The method allows a reconciliation of separate error-prone data sources without having to repeat the full HMM analysis, provided the estimated measurement error processes are stable over time. As we find that these processes are indeed stable over time, the proposed method can be used effectively for macro-integration, to reconciliate both first-order statistics – e.g. the size of temporary employment in the Netherlands – and second-order statistics – e.g. the amount of mobility from temporary to permanent employment.
Keywords: Hidden Markov Model, register data, survey data, data quality, labour market transitions, measurement error, administrative data
