An index-based approach to determine partnership in a register-based census

1.Introduction

Estonia has announced that the next census in 2021 will be based on register data. We are in a good position to perform the next census relying on administrative sources: our registers cover a variety of census characteristics. People, addresses and businesses can be identified by unique codes across the registers, and linking different sources is straightforward. The preparation for the register-based census began before the last census (a combined one), in 2010 [1, 2].

The first pilot census was conducted in 2016 (PC2016), with census date on December 31st, 2015. It showed that most of the census characteristics were at least of satisfactory quality. One of the most problematic variables was the place of usual residence [3]. The proportion of people who have not registered to their actual place of residence has been estimated to vary around 20–25%. This is despite the fact that Population Register Act obliges a person to ensure the correctness of his or her residential address in the Population Register [4].

In the register-based census, a household is defined by a household-dwelling concept, as opposed to housekeeping concept [5]. The inaccuracy of place of residence also obscures the composition of households and families, as was seen in PC2016 [3]. The number of lone parents was 67% greater in the PC2016 compared to the Population and Housing Census 2011 (PHC2011). Also, the number of registered or cohabiting partners decreased by 26%. Although this gap can partially be attributed to changes in the family structure over five years as well as to the transition to a dwelling-based definition, the main cause of discrepancy is inadequate place of residence data [6].

In 2015, questions about registering place of residence were added to the Estonian Labour Force Survey. The actual living place coincided with the registered place of residence in 88% of 15–74-year-olds. The main reasons for not registering actual place of residence were because (a) it was not considered necessary, (b) the place of residence was temporary or (c) some local benefits were involved (e.g. free public transport, place in kindergarten or school) [7].

Inaccurate place of residence data also complicates determining the true population of Estonia. The Population Register is over-covered, because people moving abroad do not always register their leaving (and returning). In a register-based census, residents cannot directly be asked about their status, so the over-coverage has to be addressed indirectly. Statistics Estonia developed a methodology to estimate the probability that a person is residing in Estonia based on his or her activity in a wide range of registers. The residency index on a current year is a function of last year’s residency status and weighted sum of signs of life (SOLs) – binary variables showing activities in registers. In 2017, 29 SOLs from 16 registers were used. The weights depend on individual SOL’s capability of discriminating between residents and non-residents and are allowed to be negative, if they tend to be more prevalent in non-residents. The residency index has been used to estimate the population size in Estonia since 01.01.2016 and for migration flows since 2015 [8, 9].

This paper introduces a potential solution to improve the accuracy of household data by using signs of partnership (SOP) from various registers. Our special interest is to reunite partners who appear as lone parents in a Population Register but in reality live together.

2.Signs of partnership in registers

The index-based method for matching partners in Estonia was proposed by Tiit and Vähi in 2017. The idea is to collect data on SOPs – indicators of presence or absence of partnership between potential partners, typically men and women. Each piece of data, e.g. marriage, co-ownership of property or a mutual child, defines a quasi-couple. In principle, each person may appear in several quasi-couples. It is reasonable to assume that real partners have more in common and share more SOPs [10].

In the current version, we have excluded same-sex couples and people under age of 18 on the pilot census day. We have not set an upper bound for age. Although the vast majority of parents were younger than 50 years, there is no theoretical upper limit for a partner of a lone parent. Using parental data from the Population Register [11] and PHC2011, we also excluded quasi-couples formed by close relatives: parents and children, siblings, grandparents and grandchildren, cousins, uncles, aunts, nephews, nieces.

Table 1 lists the signs of partnership obtained from various administrative data sources, along with their prevalence.

The most obvious sign of partnership is marriage. In principle, married couples should be easily extracted from the Population Register. However, some records are conflicting, e.g. record for person A shows marriage with B, but B is not married to A. Rather than excluding inconsistent data entirely, we treat these cases as a separate, possibly weaker SOP – “half-marriage”. “Half-marriages” constitute less than 1% of marriages.

In Estonia, married couples can optimize taxes by submitting declaration of income together. Also, it is possible to receive tax benefits on housing loan interests. Housing loans are often taken jointly by partners, making it an excellent SOP for detecting cohabiting partners.

Co-ownership of real estate property (apartment, house, land etc.) can be obtained from the Land Register. Most people in Estonia are owners of their homes [12]. We assume that co-ownership is informative SOP only if the number of co-owners of a property is sufficiently small. In this paper, we consider properties of up to six co-owners or exactly two co-owners (i.e. the SOP is defined as an indicator that a quasi-couple has a property that belongs solely to them).

The place of residence also serves as an important SOP. Unique codes for addresses allow to link data in Land Register and Population Register, making it possible to detect couples where one quasi-partner’s place of residence is in other quasi-partner’s property.

When applying for monthly subsistence benefits, the applicant is required to list members of his or her household. Quasi-couples are formed by men and women who received subsistence benefit within the same household on at least one month in 2015.

Parents of underage children (as of pilot census day) formed quasi-couples. This SOP also includes stillbirths from 2012–2015 because stillbirths are too rare – only 170 quasi-couples have a history of stillbirth – to form a separate SOP.

The listed SOPs are all positive i.e. we expect that their presence makes real partnership more likely. We have also defined some SOPs that influence the probability of a partnership in negative direction: divorce (and half-divorce for inconsistent records) and maintenance for children. The data on alimonies is not complete, as we did not have data for maintenance with legal action involved for 2015.

To optimize the parameters for partnership index, we also need to know the true partnership status for a representative sample. For that purpose, we used data from the Estonian Labor Force Survey and Estonian Social Survey – both collect data about household members of an individual.

Altogether, 536,127 quasi-couples with 387,367 men and 412,022 women were formed. For comparison, the Estonian population was 1,315,944 in the PHC2016, with 498,464 adult men and 603,133 women [13].

About half of the quasi-couples share place of residence or have one partner living in the other’s property (Table 1). Ten percent of quasi-couples with at least one quasi-partner classified as a lone parent in PC2016 have common place of residence. That seems to be in conflict with the definition of a lone parent under the dwelling-based household concept. The contradiction is explained by the fact that the criteria for being a couple in PC2016 also required a reasonably small age difference (up to 16 years) and no other potential partners in dwelling, whereas none of those restrictions are applied here.

More than a third of quasi-couples are married spouses. Also, 23% of quasi-couples with lone parents are married, which is evidence that in some cases, family members are registered to different dwellings. All quasi-couples with lone parents do not have a mutual child (73% have) as lone parents can also form quasi-couples with others rather than their children’s parents.

Other SOPs are less frequent; half-marriage, half-divorce, subsistence benefits and child maintenance can be considered rare (less than 0.5% of quasi-couples).

Table 2 describes men and women who form quasi-couples. The majority of people are 30–74 years old. In Tiit et al. [6], the comparison of the age distribution between quasi-couples data and the whole population showed that young men have few SOPs, but the probability of having at least one SOP increases with age, exceeding 50% in their thirties. Most people appear in one or two quasi-couples, 1.8% of men and 1.2% of women have three or more possible partners. Over 90% of people were considered Estonian residents at the moment of the pilot census. Three percent of men and 15% of women were classified as lone parents based on place of residence.

When collecting SOPs from registers, we also looked for meaningful dates confirming the existence or non-existence of partnership on that moment. We found dates for both positive events – marriage, most of half-marriages, children’s birthdays, submitting application for subsistence benefit, and negative events – divorce and child maintenance. For the analysis, we use the time between the last event and the pilot census moment. Eighty percent of survey partners and 21% of non-partners had at least one date for a positive event (Table 3), whereas dated negative events were scarcer: 1.5% partners and 29% non-partners had any.

We also calculated the age difference between quasi-partners. Mostly, quasi-partners were of similar age, while large differences were distinctive to non-partners.

3.Model for partnership status

Various models for the partnership index on an earlier version of the dataset are presented in Tiit et al. [6]. The tested models include logistic regression, linear discriminant analysis, weighted sum with frequency ratios or logarithms of frequency ratios as weights, and a combination of all of them. The models give correct answers on 84–86% of quasi-couples.

In this article, we add subsistence benefit and stillbirth data and also propose a general solution for partner selection. As we add more SOPs to the data, it becomes more likely that a person has multiple partners to choose from. Any person can potentially appear in several quasi-couples. Less likely but not impossible is the event that a person has a strong relationship with several quasi-partners. It is natural to select the quasi-partner with the highest index. However, the matter becomes more complicated if the potential partners also have multiple options to choose from. Also, situations may arise where the best potential partner of an individual does not prefer him or her.

Men and women can be viewed as vertices in a graph G, belonging to the same quasi-couple defining the edges. Since each edge connects a man and a woman, and the sets of men and women are disjoint and independent, the graph G is bipartite. Then, finding partners for people is like solving a matching problem on a graph.

A stable marriage problem in mathematics looks for a stable matching between the sets of n men and n women. That is, each person has an order of preference for the other sex, and men and women are married in a way that no two people would both rather have each other than their current partners [14]. An algorithm to solve the problem and make all marriages stable was presented in 1962 by Gale and Shapley [15]. The algorithm is easily generalized in the case of unequal number of men and women.

In our case, the order of preference is created by the partnership index values. The Gale-Shapley algorithm is implemented in different software packages, for the purpose of this paper we used the matchingR package (v1.2.1) in R 3.3.3.

As a partnership index we used the prediction from the logistic regression model for partnership status. Only records of quasi-couples with known partnership status were included; 20 records with missing values in covariates were also excluded. We used 80% of the remaining 19,133 records to estimate the parameters of the model, and used the other 20% to test its performance.

The problem with partner selection is trivial if there is only one partner to choose from. To not oversimplify the problem, we had to ensure that the test sample includes all possible quasi-partners for each person in the sample. To achieve this goal, we divided the quasi-couples’ graph G into connected components and sampled among those. An example of a connected component is depicted on Fig. 1. Most of the 11,334 components were of size 2 (61%) or 3 (22%), the largest components consisting of 13 people. Randomly selecting 80% of the components led to the inclusion of 79.9% of quasi-couples and 80.1% of people.

Figure 1.

An example of a connected component of quasi-couples’ graph. Each node represents a person, ages are given in the labels. Edge width represents simple sum of SOPs: number of positive SOPs minus number of negative SOPs.

All SOPs from Tables 1 and 3 are included as covariates in the logistic regression model. Time since last event is modelled geometrically, to assign more weight to recent events. If Y denotes the number of years since last positive (or negative) event, then 0.9Y is used in the model. Age disparity is handled as in Tiit et al. [6]. Let Δ= age𝑚𝑎𝑛 – age𝑤𝑜𝑚𝑎𝑛. We define two variables to account for unlikely age difference: let D𝑚𝑎𝑛= 1–0.95Δ-20 for Δ> 20 and D𝑚𝑎𝑛= 0, Δ⩽ 20; analogously let D𝑤𝑜𝑚𝑎𝑛= 1–0.97-Δ-15 for Δ<-15 and D𝑤𝑜𝑚𝑎𝑛= 0, Δ⩾-15.

Regression coefficients (log odds) are given in Table 4. In general, the coefficients are similar to Tiit et al. [6]. Marriage, half-marriage and co-owning a property strongly increase the odds of partnership. However, if there are more than two owners, the effect is much weaker. Declaration of income, housing loan, number of children and place of residence are also positive signs, as expected, and divorce, half-divorce and child maintenance lower the probability of partnership. Subsistence benefit and time since last negative event are not statistically significant in the model. Other time-related covariates are strong predictors – a recent positive event increases, while unlikely age difference reduces the odds of partnership.

Partnership index is the prediction of the logistic model. To classify quasi-couples into partners and non-partners, we define a threshold that is set on a level that gives same proportion of couples as in original data. If a quasi-couple’s index is higher than the threshold, it is considered a couple.

One shortcoming of this approach is that it does not guarantee each person appearing in one couple at most. By using Gale-Shapley algorithm, we create matching that guarantees determining stable marriages using index values for preferences. In the events of a person having high index values with multiple quasi-partners, the algorithm ensures that only one partner is assigned to him or her. We only include algorithm-generated couples if their index is high enough. A new – lower – threshold is selected to get the same proportion of couples as in original data.

Classification results on test data (3790 quasi-couples) are presented in Table 5. Logistic regression gives correct answers in 85.6% quasi-couples and 82.7% quasi-couples of lone parents. When using stable marriage approach to resolve conflicts, we get 86.1% correct results and 82.9% in lone parents.

There is no major improvement in model accuracy with adding stable marriage approach. This is expected, as there are not many people who would appear in multiple couples anyway – only 0.3% of persons in the test data had an index value exceeding the threshold with more than one quasi-partner. However, some selection method is vital for the census as each person should be assigned to one household only.

4.Further work

The first results of the partnership index with 86% accuracy are promising and it has been stated that the current version could already be used to reconsider the status of lone parents [6].

For wider use in census, the model accuracy must be improved. With current data, the options are limited. Comparison with survey data revealed that 10% of actual couples did not have any SOPs and some non-partners had many.

Adding more SOPs to the dataset should reduce errors of not finding actual couples. Currently, we plan to include data on shared vehicles from the Estonian Traffic Register, fathers using parental benefits, single parent’s child benefit, and paternity leaves from the Estonian National Pension Insurance Register. On the other hand, due to changes in the Income Tax Act, spouses cannot submit joint income tax return as of 2018, thus eliminating the option to use it as a SOP [16].

The model included elapsed times since events as covariates. Viewing erroneously classified records showed that time since last event may sometimes be more important than the quantity of SOPs. For example, after ending a relationship and starting a new one, a person may share more SOPs with ex-partner than his or her actual spouse. It is worth investigating whether giving priority to date information would enhance model behavior.

This example is a reflection of a wider issue. SOPs accumulate over time, therefore new couples may have none. Also, when couples separate, it may not appear immediately in the registers. Thus, there may be a risk of systematic bias in estimating partnership towards older age groups. To provide external validation of partnership index and other new census methodologies, Statistics Estonia has scheduled a control survey in 2018.

5.Conclusions

Household formation in register-based census in Estonia is complicated because place of residence data is inaccurate. Statistics Estonia is developing an index-based measure to estimate the probability of partnership. Partnership index is a function of signs of partnership (e.g. marriage, housing loan) in registers. Index based on logistic regression was accurate on 86% of potential couples. Considering partnership data as a bipartite graph and solving stable marriage problem ensures that each person is used once at most. The partnership index is under active development, with several updates planned in upcoming months to be applied in the next pilot census in 2019.