Estimation from contaminated multi-source data based on latent class models

Article type: Research Article

Authors: Guarnera, Ugo^* | Varriale, Roberta

Affiliations: Italian National Statistical Institute ISTAT, via Cesare Balbo 16, Roma

Correspondence: [*] Corresponding author: Ugo Guarnera, Via Tuscolana 1788, 00173 Roma. Tel.: +39 0646736637; E-mail:[email protected]

Abstract: Recently, many statistical institutes have been moving from traditional estimation approaches based on sample survey data to new approaches that try to exploit the increased availability of administrative data, due to the need of reducing the response burden and providing users with more reliable statistical information. In this context, problems concerning the use of multiple sources for estimation purposes have been receiving an increasing attention in Official Statistics. A commonly adopted strategy is to rely on a ``hierarchy'' of the sources, based on preliminary analyses of the data quality of each source. In this work, we propose an alternative approach based on the concept of latent variables, where one takes advantage of the simultaneous availability of information from different sources. The true values of the target variable are viewed as realizations from a latent (unobserved) variable and the distinct (possibly coinciding) observed values from different sources are considered as imperfect measurements of this latent variable. According to this approach, all the available information is used and ``weighted'' according to its reliability, and a prediction of ``true'' values of some numeric variable of interest is obtained conditional on all the available information.

Keywords: Multi-source data, data integration, contamination models, latent variables

DOI: 10.3233/SJI-150951

Journal: Statistical Journal of the IAOS, vol. 32, no. 4, pp. 537-544, 2016