Use of administrative records for non-response in the New Zealand 2018 Census

1.1Longer-term census transformation based on administrative data

Stats NZ’s Census Transformation programme is exploring the feasibility of a census based largely on administrative data, and supported by sample surveys. Bycroft [5] describes the administrative data landscape in New Zealand, and progress producing an administrative resident population to that point.

Stats NZ reports to government periodically, outlining progress and seeking decisions on future direction. In a 2015 Cabinet paper government agreed that Stats NZ actively work towards a future census based primarily on government administrative data, supported by redevelopment of its household surveys. The linked administrative data sources available in Stats NZ’s Integrated Data Infrastructure (IDI) have been the basis for this research. The IDI provides safe access to anonymised linked microdata for research and statistics in the public interest. The basic structure of the IDI consists of a central population list or ‘spine’ to which a series of data collections are linked. The IDI continues to expand as more data sources are added, and the number and range of research projects is rapidly increasing.

For an administrative-based census, we aim to derive a list of people who are resident within New Zealand at a given point in time, without relying on the full field enumeration of the traditional census. The progressive development of a ‘signs of life’ approach is described in [6, 7, 8]. A wide range of events-based interactions with government agencies are available in the IDI including: tax payment, benefit receipt, school and tertiary education enrolment, health care (visit to the doctor, hospital admission, pharmaceutical prescription), accident insurance claims, and visa applications. Activity in New Zealand as reflected in at least one of these administrative data sources during a two-year window is used to indicate an individual’s presence in New Zealand. Anyone who had died or migrated overseas before the reference date is removed. We also need to determine where in New Zealand these individuals live. Geographic location is derived from address information sourced from multiple agencies. The resulting New Zealand admin resident population derived from the IDI is called the IDI-ERP, and is currently implemented in the IDI for use by researchers. This IDI-ERP population is also the source of administrative records for inclusion in the 2018 Census file.

In 2016, we released an experimental data series [7, 8] of national-level administrative population estimates, with further releases for a subnational geography time-series in 2017, and inclusion of ethnic groups in 2018. The experimental data series include estimates at 30 June from 2006 to 2016. Online tables compare the IDI-ERP with official population estimates over the same period. These comparisons are largely encouraging. Often there is close agreement with official figures, and consistency has increased steadily over time. However, there are still marked differences for some age groups and local areas.

While the IDI-ERP is a good approximation of the New Zealand resident population, it includes an unknown group of erroneous inclusions, and also misses some people. These coverage errors make it more challenging to derive an admin-based population estimate sufficiently accurate for official statistics, to low levels of geography. We are developing new population estimation models combined with a single coverage survey that will adjust for over-coverage in administrative sources as well as for under-coverage. We are also developing methods to adjust for the mis-classification of admin-based location information. However, neither of these methods were sufficiently developed for the 2018 Census application.

The Census Transformation programme has also undertaken a structured quality assessment of census variables derived from linked administrative data sources, which has shown where administrative data has most to offer the census. Stats NZ continues to publish results of investigations that compare 2013 Census variables with their counterparts derived from administrative sources. These include papers on ethnicity, country of birth, Māori population identifiers, income, educational qualifications, number of children born, households and families, and housing variables.

1.2The 2018 Census: Modernising the full field enumeration census

The 2018 Census strategy [9] set out an ambitious modernisation programme across all components of census taking. This followed several censuses of minimal content change, and limited innovation. In a reversal of previous censuses, the collection was designed to be predominantly on-line, with paper forms in a supporting role. Mail-out of census material, including online access codes, to 80 percent of dwellings was achieved through an address frame based on administrative sources.

The 2018 Census strategy also included use of alternative sources to improve data quality in the context of missing data for census questions, based on the findings of the longer-term administrative data research. Improvements to adjust for missing census characteristics data were signalled in the 2018 Census strategy. These included the use of alternative sources (the previous 2013 Census and good quality administrative sources), and more extensive use of statistical imputation.

While some aspects of the 2018 modernisation have been successful, major challenges were faced when implementing the new collection model. Some 4.2 million census forms were received through the census field collection. This level of response was lower than expected and was seen across all regions. However it was clear that non-response was more concentrated in some geographic areas, and that groups which are typically harder to count in the census (younger adults, Māori and Pacific ethnic groups) were disproportionately affected in 2018.

Previous censuses have included an adjustment for non-response, through inclusion of ‘substitute’ records, a form of unit imputation [10] and a similar approach had been planned for 2018. Once these collection issues became clear in mid-2018, Stats NZ embarked on development of new methods for including people in the 2018 Census dataset based on administrative data. The new methods have built on the research undertaken by the longer-term census transformation investigations and fast-forwarded their development for use in the context of a full field enumeration census. The final 2018 Census dataset consists of 89 percent census responses and 11 percent of people counted from administrative records. These admin enumerations completely replace unit imputation as a mitigation for unit non-response to the field collection. The use of alternative sources for census characteristics has taken on greater significance than originally anticipated given the contribution of administrative records to the final census dataset and an increased number of partial census responses.

The inclusion of administrative records in a traditional full field enumeration census is unusual internationally. In their 2011 Census, Northern Ireland added records obtained from administrative health data to count people who had been missed by the field collection [11]. Statistics Canada used administrative records in their 2016 Census when major wildfires led to the evacuation of the Fort McMurray area and the suspension of census collection activity very close to census day. Of most relevance here is the United States Census Bureau work investigating the use of high-quality administrative records in the United States 2020 Census as a means of improving efficiency in their field collection process [12].

While the New Zealand census in 2018 was planned as a full field enumeration census, with the inclusion of more than 10 percent of records from administrative data, we consider the final census dataset to fit within the United Nations ‘combined census’ definition [13].

As well as methodological issues (‘can we’ do this), we also needed to consider the use of administrative data in the census from a legal and ethical perspective (‘should we’). In New Zealand, the Statistics Act 1975 and Privacy Act 1993 together provide the legal basis for the use of administrative data in the 2018 Census. The primary reason for this is that the Statistics Act requires that the same level of confidentiality is applied to statistics or research as is required by the Privacy Act. Using data with inherent biases can raise significant ethical issues because it can cause disadvantage and harm to particular groups. The use of administrative data to improve the quality of the census is an ethical approach from this perspective. A privacy impact assessment of combining administrative data and data from census forms concluded that this use of administrative data is lawful, safe, and beneficial to New Zealanders [14].

The following sections describe the methods for determining which administrative records to include in the 2018 Census dataset, followed by key results. The focus is on the key census demographic counts by age, sex, and ethnicity, and for geographic areas. We conclude with a discussion.

2.1Linking the census and administrative data

Received census forms always take priority. We link 2018 Census respondents to the same person in the IDI spine, so that we can remove them from the IDI-ERP, leaving only those who did not respond. This requires a high linkage rate and accurate linkages. Since New Zealand does not have a common identifier, probabilistic linkage methods were applied. Variables used to compare records were first and last names, date of birth, age, sex, country of birth, meshblock and address ID [17]. The overall linkage rate of 97.7 percent is high in the New Zealand context.

Census respondents who have not been linked to the IDI spine are a mix of those who:

The rate of missed matches for individual forms is estimated as 1.21 percent. If we have not linked a census record to the IDI spine when we should have, the same person could be included in the final census dataset twice. Below we describe an adjustment made to account for missed linkages.

False positive matches (when different people are incorrectly linked) are estimated as being less than 1 percent of the links made. An incorrect link removes the administrative record from the possibility of inclusion in the final census dataset.

2.2Admin enumerations in dwellings

The field enumeration census is designed to count people within dwellings, and previous censuses adjusted for non-response by adding imputed records within dwellings where responses were thought to have been missed. Following this traditional approach, the first and most demanding use of administrative data is the placement of groups of people within a dwelling to form households. The census dwelling frame provides a list of addresses for private dwellings where no census responses have been received. A combination of two statistical models were used to predict which households constructed from administrative records are likely to have reliable data.

The approach is based on methodology developed by the US Bureau of the Census who have a planned strategy to use admin enumerations in the non-response follow-up phase for their 2020 Census [12]. While the focus of the US approach is to reduce contacts in their field operations, similar approaches can be used after data collection is completed to use administrative records in lieu of imputation.

The ‘person-place’ model assesses the probability that the admin address reflects the true address for an individual; the ‘household composition’ model assesses the probability that the admin derived household members reflect the true household composition at a given address. A composite distance score for an address combines the probabilities from each of these models. This distance score represents how reliable the administrative data is for representing the entire household in a given dwelling, and a cut-off determines which of those households will be added to the census dataset.

The required probabilities are estimated through logistic regression. Most of the covariates used relate to information about the addresses, such as the number of admin sources that report the same address, or the amount of time living at the current address. Age was the only demographic variable determined to be an important predictor. Information about whether relationships were identified to other individuals at the same address was also used. 2018 Census responding households are assumed to represent the truth when training and assessing the models.

The distance cut-off has been set as a balance between strict criteria of obtaining exactly the same people in the household as we observe in the census, and including admin households that reflect similar adult-child patterns as the census, even if we cannot guarantee that all household members are the same. Making the trade-off in this way means we include relatively more large or complex households than if we had set a more conservative cut-off, and makes some allowance for errors in census responding households.

Of the 89,355 admin households that could be derived for non-responding census dwellings, 57,609 (64 percent) were above the cut-off and the 141,411 members of those households were added to the census dataset as admin enumerations. [18] provides a full description of the statistical models and further results.

While households where we have received some census responses may still be missing people, we have not developed a model to predict when administrative records ostensibly for the same address should be placed within those responding households. Some administrative records are included in responding private dwellings, but only within strict conditions. We require strong evidence from the received responses that a person who usually lives there has been missed, and the address and relationship characteristics must be mapped to an eligible administrative record. A small number of administrative records have also been included in two types of non-private dwelling – prisons and defence establishments.

2.3Admin enumerations in meshblocks

It became apparent that the traditional census approach of only placing people into dwellings where they had been missed would not be sufficient, given the scale of non-response. The next phase places administrative records into meshblocks, the smallest New Zealand geographic area. The administrative address ID provides an x, y location coordinate, but the person will be excluded from census households.

The remaining IDI-ERP population eligible for inclusion in meshblocks are those who have not been linked to a census respondent, and are not already included at a dwelling. We then explicitly adjust for potential coverage errors that could result from over-coverage in the IDI-ERP, and for duplication caused by missing linkages between the census and the IDI. We draw on new methods Stats NZ have developed to remove over-coverage and adjust for missed matches in the context of the new Dual System Estimation (DSE) population [19]. DSE is a well-established methodology used in population estimation, often applied in the context of a full enumeration census and a coverage survey. Over-coverage is typically avoided by design of the coverage sample, and extensive checks are made to meet the DSE assumption of no linkage error. Neither of these approaches were possible in this situation.

3.1Contribution of admin enumerations to the final 2018 Census dataset

Overall, census responses make up 89 percent of the census dataset, and admin enumeration 11 percent. Table 1 shows the number of administrative records included from each step. The admin meshblock enumerations make up 8 percent of the final 2018 Census usual resident population count, admin household enumerations make up 3 percent, with the other dwelling-based admin enumerations contributing 0.6 percent.

The age distributions of the admin enumerations are markedly different from received census responses (Fig. 1). Both the admin household and meshblock enumeration groups tend to include relatively more children and young adults, and fewer people in the older age groups. This trend is most observable for the meshblock enumerations, with 34 percent of these individuals aged 20–34, compared with 20 percent of respondents. This suggests young adults are less likely to have responded to the census than other age groups and are also less likely to have been part of high-quality admin households. In contrast, 16 percent of census respondents were aged 65 or older, compared with 7 percent for both the admin household and meshblock enumerations, indicating higher census participation for older age groups.

Figure 1.

Age distribution of census usually resident population count, by unit record source.

Figure 1 also shows the age distribution of administrative records that have been excluded because their admin meshblock was predicted to be low quality. These excluded records are overwhelmingly in the 18-to-24-year age group. This feature is driven by the meshblock prediction model, as this age group tends to have a lower probability of a correct admin meshblock due to poor-quality address information.

The admin enumerations also had considerably higher proportions of people identifying with Māori and Pacific ethnicities than for census respondents, and a lower proportion of people identifying with the European ethnic group. This indicates lower participation for people of Māori and Pacific ethnicity in the 2018 Census.

As would be expected from these results, admin enumerations make up a greater proportion of the final census dataset for young adults, and for Māori and Pacific ethnic groups. Almost a quarter of the Māori and Pacific populations are admin enumerations, compared with 8 percent for the European ethnic group (Table 2). These populations have been associated with higher levels of undercoverage in previous censuses [2]. The prevalence of these groups among the admin enumerations is an indication that administrative records do include those who are typically harder to count in a census field enumeration. A key advantage of the admin enumeration approach is that these populations are better represented than they would be based only on imputation. Administrative data is generally of high quality for key demographic variables and so we can be confident the different distribution of admin enumerations compared with census responses is likely to represent an improvement to census counts.

3.2Comparison of final 2018 Census dataset and DSE benchmark populations

The DSE benchmark population is the most suitable estimate of the true census usual resident population available at this time. Comparison with the DSE provides an indication of the coverage patterns for the 2018 Census after admin enumerations have been included. DSE benchmarks are available by single year of age, for Māori, Pacific, and Asian ethnic groups, and for TALB geographies. Further breakdowns are not available due to constraints of the estimation method [19].

The DSE population total is 4,768,600, compared with the total census count of 4,699,755, 1.4 percent smaller than the DSE. Until the formal post-enumeration survey undercount measurement is available, this 1.4 percent net undercount compared with the DSE provides our best indication of census overall coverage. It compares with a 2.4 ± 0.5 percent undercount for the 2013 Census [2].

Table 3 shows the DSE population benchmarks compared with the census counts for several key groups. Despite admin enumerations having significantly boosted populations for Māori, Pacific people, and young adults, the undercount is still higher for these groups than the overall average.

Figure 2 compares the census and DSE distributions by single year of age, for males and females. The census follows the DSE very closely for children and for older adults, while there is some undercount apparent for younger adults. The gap is more marked for younger males than for females. These patterns are consistent with younger adults being more likely to be excluded from the admin enumerations due to lower-quality administrative address information.

Figure 2.

2018 Census and DSE benchmark counts, by sex and single year of age.