The usability of administrative data for register-based censuses

1.1Data collection at Statistics Netherlands

At Statistics Netherlands, the data collection has been centralised in one input division. In this division, both register and survey data are collected. This change has been made to be able to work more efficiently and professionally. Now Statistics Netherlands uses the same infrastructure for social and economic statistics as much as possible. In its collection strategy the preferred order is to first look at register data. If variables or categories are missing and cannot be derived from register information, then sample surveys are conducted. In the Census 2011 Statistics Netherlands only used already existing sources. Therefore, questionnaires for the Census 2011 were not necessary.

1.2History of censuses in the Kingdom of the Netherlands

Following the withdrawal of Napoleon, the Netherlands became a kingdom and a census was held once every ten years. The first census of the Kingdom of the Netherlands was taken in 1829. Before the official institution of a central bureau of statistics, the Ministry of Home Affairs organised another six censuses (in 1839, 1849, 1859, 1869, 1879 and 1889). In 1899 the Central Bureau of Statistics (nowadays called: Statistics Netherlands) was established, and was put directly in charge of the eighth census. Six more traditional censuses were carried out in the twentieth century: in 1909, 1920, 1930, 1947, 1960 and 1971. During the 1971 Census enormous privacy debates took place and thereafter the government decided to stop the traditional censuses in the Netherlands. In taking this decision also the growing unwillingness to participate played a role, but the main aspect was the privacy issue [1]. The four most recent censuses (1981, 1991, 2001 and 2011) were not based on a complete enumeration but on registers and surveys available at Statistics Netherlands. Until now, 18 censuses have been held in the Kingdom of the Netherlands.

The Dutch data compiled for 1981 and 1991 were much less detailed than the sets of tables compiled for the 2001 and 2011 Censuses. Moreover, they were largely based on a register count of the population in combination with the then existing surveys on the labour force and housing conditions. The amount of information that has become available in the 2011 Census is again larger than that of the 2001 outputs. This is mainly the result of the introduction of so-called hypercubes (high dimensional tables): more variables have now been cross-classified than ever before.

In 1991 the Census Act was rescinded, officially terminating Statistics Netherlands’ obligation to conduct a census every ten years [2]. For the 2001 Census there was no obligation to produce census data, but Statistics Netherlands compiled a set of tables based on a gentlemen’s agreement between the European Union and its Member States at that time. The 2011 Census has its legal basis in the 2008 European Census Act [3] and three implementing regulations [4, 5, 6]. Eurostat has a coordinating role in collecting harmonised data on the EU and a duty to make international comparisons of the outcomes.

Originally, censuses had two aims. First, to correct errors in the municipal population registers. Second, to obtain extra information about socio-economic phenomena in the country. As the Netherlands conducts a register-based census, the first aim no longer exists. In addition, the quality of the central population register (PR), which unites all municipality population registers, has improved considerably: as central government funding is allocated based on population size, it is in the municipalities’ own interest to keep their population registers up-to-date. Another reason for the improvement is that it is extremely difficult to function in Dutch society if you are not in the PR. Both municipalities and citizens, therefore, have enough incentive to maintain a high quality PR. The second aim is still valid and many census results are published in a historical or international context. At present, census data are also popular for comparisons between countries.

1.3Preparations for the Dutch Population and Housing Census 2011

Census experts at Statistics Netherlands started pre- parations for the 2011 Population and Housing Census in 2008. In 2009, they started work on the data collection procedures required to collect the census information about the 16,655,799 people living in the Netherlands on 1 January 2011.

Data from different sources were combined to produce the 2011 Census tables. These data were not obtained by interviewing inhabitants in a complete enumeration, as in traditional censuses in most other countries, but by using data from registers and sample surveys that were already available at Statistics Netherlands. The main sources were:

This approach of combining different sources has a number of advantages and disadvantages that will be explained in the last section of this chapter.

1.4The Dutch Population and Housing Census 2011

A number of analyses based on the Dutch Population and Housing Census 2011 can be found in the book published about this census [8]. A number of census variables have been harmonised in Europe. According to [4] 35 variables are mandatory: 16 person variables, 7 household variables and 12 housing variables. An overview of these variables is given below.

Person variables

10

• 1. Place of usual residence

• 2. Location place of work

• 3. Locality size

• 4. Sex

• 5. Age

• 6. Legal marital status

• 7. Current activity status

• 8. Occupation

• 9. Industry

• 10. Status in employment

• 11. Educational attainment

• 12. Country/place of birth

• 13. Country of citizenship

• 14. Year of arrival in the country

• 15. Place of usual residence one year before

• 16. Housing arrangements

Figure 1.

Population in the Netherlands by current activity status. Source: Census 2011.

1.5Advantages and disadvantages of the Dutch register-based census

One of the advantages of this approach is a much lower census bill for Dutch taxpayers. A traditional census in the Netherlands would cost a few hundred million euros, while with this method it costs ‘only’ around 1.4 million euros. This bill includes the costs for all preparatory work, such as extending the methodology and updating and developing accompanying software, as well as the analyses of the results. It does not include the costs of the registers, as these are not kept for censuses, but primarily for other purposes. In addition, under Dutch law, Statistics Netherlands may access government registers free of charge. This low-cost census approach is only possible for countries with sufficient register information. A register-based census costing less than 1 percent of a traditional census is not exceptional [9]. Today, the huge costs of traditional censuses are often justified by pointing out the enormous implications of the census results for regional funding distribution. But a register-based census would be impossible in many countries, because of the lack of sufficient register data and access restrictions to the existing registers.

Apart from the financial aspect, there are also other important differences between a traditional census and the register-based census conducted in the Netherlands. A well-known problem with traditional censuses is that participation is limited and selective. In spite of the mandatory character of a traditional census, part of the population will not participate at all (unit non-response) and those who do will not answer all questions (item non-response). Although correcting for non-response by weighting and imputation techniques is worth trying, traditional correction methods are inadequate to obtain reliable results. The last traditional census in the Netherlands, in 1971, met with many privacy objections against the collection of integral information about the population living in the Netherlands. This increased the non-response problem, and non-response was expected to be even higher if another traditional census were to be held in the Netherlands [2]. There are almost no objections to a register-based census in the Netherlands and the non-response problem only plays a role when survey microdata are reused.

Figure 2.

Number of inhabitants and mean household size in the Netherlands, 1829–2011. Source: Statistics Netherlands.

Another advantage of the register-based census is the short production time. The register-based census in the Netherlands got off to a later start than traditional censuses in other countries. It would have been pointless to start the production phase of the 2011 census project before all sources were available, and some registers became available relatively late. In spite of this delay, Statistics Netherlands compiled its census tables faster than most other countries in the 2011 European Census Round. In fact, the Netherlands had one of the shortest production times for the complete set of tables required by Eurostat. Statistics Netherlands had the advantage that no incoming census forms had to be checked and corrected.

A disadvantage of the Dutch census is that for some variables only sample information is available, which meant it was impossible to meet the level of detail required in some census hypercubes. At the moment, however, the Netherlands perceives the advantages of the register-based census in terms of cost and non-response problems to amply outweigh the loss of some detail compared with a traditional census.

Statistics Netherlands is not the only country that uses registers to produce census information. Four Nordic countries (Denmark, Finland, Norway and Sweden), Austria and Slovenia have more variables available in registers than the Netherlands, and the problem of insufficient detail in the outcome does not play a major role there. Most of the other register-based countries are in a similar position to the Netherlands: not all variables relevant for the census can be found in registers. They are therefore very interested in the Dutch approach of combining registers and existing sample surveys and using modern statistical techniques to compile the hypercubes. Obviously, it is essential that statistical bureaus are permitted to make use of registers that are relevant for the census. For Statistics Netherlands this is laid down in the statistical law that came into force in 2004 (see https://www.cbs.nl/en-gb/about-us/organisation for an English translation of the most recent version of the Statistics Netherlands Act). Nevertheless, Statistics Netherlands will have to maintain the good contact it has established with register holders over the last 25 years. For the census (and all other register-based statistics) Statistics Netherlands has become very dependent on the register holders. Timely deliveries with relevant variables for Statistics Netherlands are crucial for official statistics production.

All tables produced in the Dutch Census 2011 are numerically consistent. The results can be compared over time as Fig. 2 illustrates. In the period 1829–2011 the population has grown continually from 2.6 to 16.7 million people. The average private household size in the Netherlands has decreased since the beginning of the twentieth century. Together with the growth in prosperity in the post-war period, the average size diminished rapidly. Not only has the number of children per couple decreased since the 1960s, but children are leaving home earlier and live alone longer than half a century ago. From 4.8 persons in 1899 the average private household size has dropped to 2.2 in 2011. Specifically worth mentioning is the decrease since 1947, when the average size was still 4.0 persons.

In addition, international comparisons between census results are possible and the earlier mentioned Census Hub is facilitating such comparisons between European countries. Finally, Dutch census data are continuously available: there are no restrictions to official census years as census tables can easily be updated every year (see also Chapter 4).

2.1Census practices in the 2010 census round

As mentioned in the previous chapter, a number of countries conduct register-based censuses. As it is interesting to know how countries conduct their censuses, in 2013 the United Nations Economic Commission for Europe (UNECE) conducted an online survey among its members to collect information about national census practices in the 2010 Census Round, and about plans for the 2020 round. All EU and EFTA countries are also members of the UNECE, which also includes Canada, the Russian Federation and the United States, among others. Response to the UNECE questionnaire was high and the results of fifty countries on important methodological issues were analy- sed [10].

As expected the countries used different methods, and some countries reported a different method for the population than for the housing census, often connected with the availability of registers for these domains. Using registers to produce official statistics reduces costs and bypasses the problem of declining survey response rates. Three main types of census method can be distinguished: the traditional census, the combined census, and the register-based census.

2.2Traditional censuses

The traditional census approach collects basic characteristics from all individuals and housing units (full enumeration) for a specific point in time. More detailed characteristics can be collected either from the whole population or on a sample basis. Collection modes include personal interviews, self-completed paper questionnaires, and data collection by telephone and the internet. Across the world, this is still the most common approach to census taking. Most UNECE countries with a traditional census use personal face-to-face interviews with paper questionnaires as their main approach. However, in the Czech Republic, France, Ireland, Italy, Luxembourg, Slovakia, the United Kingdom and the United States, the main method is self-completion of paper questionnaires by respondents. In Canada most respondents participate online (CAWI), while in Portugal self-completed paper questionnaires and online response were equally popular.

Just as in the census round of 2000, full field enumeration without register information (traditional census) is still the most popular method in the UNECE region in this census round. Almost two-thirds of countries collected data using ‘traditional’ methods. But although it is still the most common general approach in the region, it is less so than in the 2000 round, when four-fifths of countries used this approach. A substantial minority (33 percent) of the full field enumeration countries used information from registers only as a frame or control. The United States was alone in using traditional enumeration with yearly updates of characteristics on a sample basis. France used another alternative approach to the traditional model: the rolling census. This is a cumulative continuous survey covering the whole country over a period of time rather than on one particular day.

2.3Combined censuses

Four countries (Estonia, Latvia, Liechtenstein and Lithuania) used a combination of register data with complete field data collection for selected population census variables, and six countries (Germany, Israel, Poland, Spain, Switzerland and Turkey) used a combination of register data with ad-hoc sample data collection for selected population census variables.

Figure 3.

Census methods in UNECE countries. Source: [10].

2.4Register-based censuses

A growing number of EU and EFTA countries have switched to methods without field data collection, relying on registers for their 2011 Population and Housing Censuses, and skipping census questionnaires completely. Some of these countries ‘recycled’ information from their Labour Force Surveys, combining it with register data (Belgium, Iceland and the Netherlands). Lastly, some countries used registers only in this census round (Austria, Denmark, Finland, Norway, Slovenia and Sweden). All nine register-based countries collected census information relating to housing entirely from registers.

2.5Overview

The map of the UNECE area reveals interesting east-west and north-south tendencies in census methods (see Fig. 3). Three main categories are distinguished on the map: traditional (31 countries), combined (10 countries) and register-based (9 countries). Register-based censuses are becoming increasingly popular in northern Europe, combined censuses are more often found in central Europe. Traditional censuses continue to be more popular in English-speaking and Commonwealth of Independent States (CIS) countries. All UNECE countries outside Europe conduct traditional censuses. Only Uzbekistan did not conduct a census in this round, and had no plans to do so.

2.6Essential features of a population and housing census

One may wonder whether register-based censuses still meet the well-known United Nations essential features of a population and housing census. According to the Conference of European Statisticians Recommendations for the 2020 Censuses of Population and Housing [11] the set of essential features that makes a population and housing census unique is as follows.

2.7Necessary conditions for a successful transition to a combined or register-based census

If countries want to move from a traditional census approach towards a combined or register-based census, a number of conditions should be fulfilled before information from registers can successfully be used in the census. A number of these necessary conditions can present some challenges, and these are discussed briefly in the following paragraphs.

As a huge interest exists in possibilities for a successful transition to a combined or register-based census, many presentations and courses are given about this topic (see e.g. [12]). In addition, country visits to countries planning a change in census methodology are often helpful to have a successful transition.

3.1Introduction

The current census results in the Netherlands refer to 2011. The backbone of the Dutch census is the central population register (PR), which combines all the municipal population registers. PR data for 1 January 2011 were used as the basis for the set of hypercubes. The hypercubes focus on frequency counts, not on quantitative information. Data not available or derivable from the PR were taken from other registers. All register variables are now available from Statistics Netherlands’ system of social statistical datasets (SSD), and their quality has been improved by applying micro-integration techniques. More information about the SSD can be found in [13]. Micro-integration entails checking the data and adjusting those that are incorrect. It is widely assumed that micro-integrated data provide more reliable results, as they are based on a maximum amount of information. They also provide better coverage of subpopulations: if data are missing in one source, another source can be used.

3.2System of base registers

In the past micro-integration played a very important role in the SSD and in particular in censuses. However, in the future the role of micro-integration will be more limited, because in the Netherlands the new so-called system of base registers is better integrated than the set of registers that Statistics Netherlands received before. The system of base registers will ultimately consist of 13 registers on population (residents and non-residents), addresses and buildings, enterprises, real estate (boundaries, ownership, value, etc.), topography (maps: land, water, roads), motor cars (model, colour, ownership, etc.), taxable income, labour (wages, employers, social benefits, etc.) and subsoil (sewerage, cables, etc.). The system of base registers is based on legislation. For each base register one cabinet minister is responsible. The Minister of Home Affairs is responsible for the system of base registers.

All variables in the system of base registers are kept in only one register. For example, addresses are kept in the base register for addresses and other registers that need address information copy this information exactly from the base register for addresses. The data linkage is through identification numbers. All objects (persons, enterprises, addresses, etc.) have a unique identification number. The use of data from base registers is compulsory for governmental agencies. The objective is that all users of the system contribute to the quality of the data. Therefore, users are obliged to notify the holders of base registers if they have alternative data that are considered to be of better quality. However, due to legal restrictions Statistics Netherlands does not notify the holders of base registers.

Users of base registers can rely on their validity. Statistics based on base registers demand only a limited amount of data editing. As base registers are linked to one another (through identification numbers), it follows that statistical data are also coherent. Contrary to sample survey data, no estimation methods are needed for small area statistics and non-response problems do not exist. However, some data are delayed, for example information about self-employed persons. One should not forget that registers are not developed for statistical purposes and do not always contain the statistical concepts required. In some situations different statistical concepts are acceptable. In other situations one tries to derive information about the required statistical concept. If such derivations are not satisfactorily, one could finally decide to fall back towards sample surveys with the required statistical concept, but with more limited publication possibilities compared to register data.

It is important to manage the base registers properly. Therefore, each base register has its own project board. All groups of stakeholders are represented in these project boards. Project boards operate within the legal framework and see to it that the register data fulfil the legal requirements (quality, completeness, etc.) and that the data are correctly applied. Project boards act as advisory boards to the responsible cabinet ministers and meet a few times per year.

3.3Survey variables

In the Census 2011, only two variables were not taken from a register: ‘occupation’ and ‘educational attainment’. Records from the Labour Force Survey (LFS) in a three year period around the enumeration date (1 January 2011 was the so-called Census Day) were used to estimate values for these two variables. Table consistency was guaranteed by using repeated weighting for tables including these variables. Repeated weighting is a method for the consistent estimation of multiple frequency tables from registers and sample surveys. The method of repeated weighting, described extensively in [14], is based on the repeated application of the regression estimator, generating a new set of weights for each table estimated. The weights of the records in the microdata are adjusted in such a way that a new table estimate is consistent with all earlier table estimates.

3.4Repeated weighting

Statistics Netherlands had already used repeated weighting for the compilation of the Census 2001 tables. However, the set of tables for the Census 2011 was larger and much more detailed and therefore the application of the method of repeated weighting to the Census 2011 was not without problems. If cells were covered by only a few (or even no) observations, estimation became problematic. New additional solutions were developed (see Chapter 7 of [8]) and introduced for 23 hypercubes that were to be estimated. In [15] existing solutions for methodological problems are reviewed and new solutions are proposed when necessary. The problems and solutions mentioned are illustrated with experiences of the Dutch Census 2011 compilation. The general message of [15] is that repeated weighting can be applied to very complex estimation problems, although it still has its limitations.

Statistics Netherlands developed and used its own software for applying repeated weighting. Under the condition of small, independent samples, variances of table values can also be estimated. Such estimated variances were used to set publication rules for cells and to calculate variation coefficients for the quality hypercubes, which serve as a quality assessment of the census hypercubes.

To maximise accuracy, all estimates are based on the largest possible number of records. Tables containing only register variables are counted from the registers. Tables with at least one variable from the LFS are estimated from the largest possible combination of register and survey data. Initial weights from the LFS were available for these estimations.

As part of the Census 2011 was compiled based on sample data, margins of inaccuracy had to be taken into account for some results. A rule of thumb was applied for cell values based on a sample from the census population: only estimated table cells based on a minimum number of persons are published. In addition, rare categories have been made confidential to prevent disclosure of individual information.

4.1Introduction

The work on the Census 2011 has been finished at Statistics Netherlands and the Census 2021 project has not yet started. The 2008 European Census Act [3] will also be the legal basis for the European Census 2021. To prepare the European Census 2021 Statistics Netherlands actively joined the discussions on the future of the European censuses based on UNECE recommendations [11]. So far, these discussions have already led to three new implementing regulations [16, 17, 18]. Moreover, a number of preparatory projects for the Census 2021 have been initiated at Statistics Netherlands that are described briefly in the following paragraphs.

In 2016 and 2017, six countries (Finland, France, Germany, Hungary, Slovenia and the Netherlands) worked together in a European project (see https://ec.europa.eu/eurostat/cros/content/harmonised-protection-census-data_en) in which new methods for the protection of census data were developed that should lead to better comparable safe outputs between countries in the European Census 2021. A choice was made in this project for a combination of the pre-tabular method of data swapping and the post-tabular cell key method in which some random noise is added to the results. In 2018 and 2019 a further European project is foreseen to make the software for this combination of methods available in both R and software packages for Statistical Disclosure Control (e.g. the data swapping method in μ-ARGUS and the cell key method in τ-ARGUS). Then, it will be much easier for statisticians involved in the European Census 2021 to produce safe (according to their national criteria) and well comparable census tables (including the new grid squares).

4.2Educational Attainment File (EAF)

In this section, some more information is given about the Educational Attainment File (EAF) that will be an important new source for the Dutch Census 2021.

A complex integration process of microdata from the LFS and examination registers has led to a new source for Statistics Netherlands: the EAF. Early versions of the EAF covered a smaller part of the population and missed some kinds of education and therefore they were not considered to be of good enough quality for the Census 2011. However, the new version, that is containing also information on private education institutions, is available since 2016. In this new version about 60 percent of the records have information on highest level of education attained. This huge set of records can be weighted to known marginals of the population for statistics on level of education. In such a way better quality and more detailed education tables can be produced than before when only LFS information was used.

4.3Imputing the EAF

Statistics Netherlands launched a special project (subsidised with a European grant) to find out how to impute the EAF for the Census 2021 so that education hypercubes can be produced based on this source. In this section, the different parts of this special project are described.

First, a good imputation model had to be found for the EAF. The choice was made for a logistic regression model. Then a set of hypercubes of the Census 2011 was produced again, now by using the imputed EAF. By comparing the newly produced hypercubes with their official variants of the Census 2011, the differences were studied. These differences are currently (last quarter of 2017) being studied. As about 40 per cent of the records is imputed, it is clear that one cannot make too detailed publications based on the EAF. However, more detailed publications than the official publications of the Census 2011 should be possible. The differences found are used to decide on a criterion how detailed the future Census 2021 education hypercubes based on the imputed EAF will be. Finally, in the first quarter of 2018 a plan, that is possibly also of interest for other countries, will be made for the variable highest level of education attained in the Dutch Census 2021.

4.4Yearly census results

As mentioned at the end of Chapter 1, in the Netherlands there are no restrictions to official census years as census tables can be updated every year. In this section both the current national yearly tables and the future European plans are described.

Statistics Netherlands publishes over 100 census tables on a yearly basis about persons and households (not about housing) based on the SSD. Some extra variables and specific categories are included in these yearly tables, but it concerns lower dimensional tables than the census hypercubes produced every ten years. Examples of extra variables are:

Examples of extra specific categories are:

4.5Future work

The current situation in Europe leads to outdated census publications (the European Census 2011 results were published at the end of 2014). This has caused users to look for other sources than the census to meet their data requests. Now that these requests can often be met (based on existing register and survey information), census budgets are no longer justified and are shrinking because of government budget cuts.

It is clear that additional publication of yearly tables will make the census information much more useful. The level of detail in such yearly European census tables should be more limited than in the current tables published every ten years. It is important that all countries can participate, either by using registers or by yearly updates (as the United States Census Bureau currently does with its American Community Survey). To make such a European transformation a success, time is needed to come to a change backed by new European legislation. The strategy (see https://circabc.europa.eu/sd/a/18e61999-517a-45ed-b348-aeb08e44c321/CENS(2017)07%20Post-2021%20census%20strategy.pdf) first includes the European Census 2021 (the results of which will not be published before 2024) and thereafter yearly updates (from 2025 onwards with reference years one year earlier). In the yearly updates current output from censuses, demographic statistics and migration statistics as well as information on small areas could be combined to prevent an extra burden for the Member States of the European Union.

It is clear that a more flexible European legal framework is needed to accommodate future requests by users. After some experiences under the new framework, it will be decided which extra census information will be necessary for the European Census 2031. But first the challenge for the European Union and its Member States is to work out a new and flexible European legal framework to be able to meet future census requests in a timelier manner.