Standardized calibration adjustment index (SCAI): A new measure of survey data quality

Article type: Research Article

Authors: Levine, Burton^* | Lewis, Taylor | Nonnemaker, Jim | Farrelly, Matthew

Affiliations: RTI International, Research Triangle Park, NC 27709, USA

Correspondence: [*] Corresponding author: Burton Levine, RTI International, 3040 E. Cornwallis Rd., Research Triangle Park, NC 27709, USA. E-mail: [email protected].

Abstract: In this article, we introduce a new indicator of survey data quality called the standardized calibration adjustment index (SCAI). The SCAI quantifies the difference in the distributions of variables used in the calibration of survey respondents to the target population, accounting for the study design. It does so by a function of the change in respondent-level weights developed to calibrate the survey data to known population totals. A key feature of the SCAI is that it does not require auxiliary information to exist on the sampling frame. The SCAI can be used as a survey data quality metric in both probability and nonprobability sample settings, which we show through example applications with an outbound dual-frame random digit dialing telephone survey, an address-based sample survey, and a redirected inbound call sampling survey.

Keywords: Nonresponse, missing data, weighting, nonprobability sample

DOI: 10.3233/SJI-200719

Journal: Statistical Journal of the IAOS, vol. 37, no. 2, pp. 603-610, 2021