A methodological assessment of privacy preserving record linkage using survey and administrative data

1.Background

The National Center for Health Statistics (NCHS) serves as the nation’s principal federal health statistics agency, whose mission is to provide statistical information that can be used to guide actions and policies to improve the health of the American people. NCHS conducts several population-based and establishment health surveys designed to collect important information about the health of the U.S. population. Through the NCHS Data Linkage Program, data from these surveys are linked to mortality data from the National Death Index (NDI), health care utilization data from Medicare and Medicaid administrative records, and federal housing program participation records from the Department of Housing and Urban Development [1]. These data linkages are based on both deterministic and probabilistic linkage algorithms, which rely on the exchange and comparison of personally identifiable information (PII) between data sources. The linked data expand the scientific utility of surveys and enable richer analysis than would be possible with each data source alone. The NCHS linked data resources have supported over 1,000 PubMed-indexed scientific publications [2].

Privacy concerns are one of the greatest barriers to data linkage. Often the personal identifiers used for data linkage are protected by laws and regulations [3, 4]. For example, the Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule prohibits the release of identifiable health information except in certain circumstances [5]. However, the HIPAA Privacy Rule does allow for the release of protected health information that has been de-identified, and the Rule provides specific standards for de-identification [5]. Privacy preserving record linkage or “PPRL” is a method that can be used to link de-identified data [6]. One PPRL technique that meets HIPAA standards is called “hashing” [7]. Hashing converts names, addresses, and dates of birth into unique encrypted codes that protect the original values. Because record linkage often involves different combinations of PII, hashes are often combined to form multiple “tokens” for one individual [8]. One example of a token could include sex, date of birth, Social Security Number (SSN) [9] and another token could include sex, address, name and SSN.

Previous research has shown the feasibility of PPRL and assessed its accuracy [10, 11]. A PPRL study that used laboratory and clinical data included multiple records per individual and multiple reporting sites. The records in the multiple sources contained complete name, date-of-birth, ZIP code and a unique health insurance identification number (Medicare ID number). The results from the PPRL produced sensitivity and specificity that were as high as 100% [10]. In a larger dataset, with up to 20% of records missing Medicare ID number, sensitivity estimates reached 95% and specificity estimates were as high as 99% [10]. A study conducted using hospital admission records from Western Australia reported no difference in linkage quality when comparing PPRL to linkage with unencrypted identifiers. The same study reported slightly lower accuracy using PPRL to link records from a different part of Australia which had a higher percentage of missing PII values [11]. These studies demonstrate the potential of PPRL to accurately link healthcare records. They also highlight the importance of data quality and the potential for lower-quality linkages when personal identifiers are missing or incorrect.

PPRL techniques could expand the sources of data that could be used in linkages. However, before implementing these methods it is important to understand the impact of using a new linkage approach on data quality and secondary data analysis. Therefore, this analysis should be treated as a case study where an assessment of data quality is being performed prior to implementing PPRL methods with new sources however, future research is still needed.

2.Materials and methods

2.4Tokens

Rather than separately encrypting the individual identification fields, such as first name, last name, and DOB, these fields are concatenated into tokens to add privacy protections. For example, a single token is used to represent the concatenation of last name, first name, sex, and DOB. An example of a token developed using John Smith, sex = male, born March 27, 1968, could look like “iGy3RRqnKjO7cLUMF1z + er8SuR9F3WAmpqc8vsqCONQ =”. Of note, for some tokens, the Soundex function is also used with names. Soundex is a phonetic algorithm for indexing names by sound, as pronounced in English [17]. Datavant has developed a number of tokens for users to select from, including several tokens classified as high precision due to the inclusion of SSN as a unique identification number. Additional variables such as DOB are also included in these high-precision tokens to provide additional verification that in fact the two linked records belong to the same person. Although over 40 tokens were available, for this analysis only tokens for which the full complement of PII in the survey data were available were utilized. Figure 1 above illustrates the tokens used in this analysis which were all HIPAA certified by expert determination and deemed to have minimal risk of patient reidentification [8]. In addition, Datavant’s security has been tested and certified by third parties [18].

Figure 1.

Selected Datavant tokens used in this analysis, all tokens are HIPAA certified.

The actual process of comparing two files, based on the encryption key, is performed outside of the Datavant software package. For this evaluation, the token files were read into SAS statistical software (version 9.4) [19] so that tokens could be compared and all pairs that shared at least one common token between the two files were output to a results file. Note that if any of the identifiers within a given token were missing, that token was not created. File sequence numbers were carried through the process to enable evaluation of link validity and comparisons to a gold standard.

Once all possible token comparisons were made, a de-duplication process was implemented to create a linked file at the patient level. For each NHCS-NDI record pair identified by PPRL analysis as having at least one token in common, the number of linked high-precision tokens was counted. For each NHCS patient record, the paired NDI record having the highest number of high-precision tokens was selected as the link. If there were more than one paired NDI record with the same number of high-precision tokens, then the record was assessed to see if other non-high-precision tokens matched and the record with the highest number of additional tokens was selected as the link for that patient record. The remainder of analysis was conducted on a patient-level based on the links selected in the de-duplication process.

3.Results

The NHCS linkage submission file included 5,386,469 records of which 1,205,063 were alternate records (i.e., records that were generated to include formal name equivalents of known nicknames and different versions of name parsing). The NDI file included 6,373,038 records, of which 762,101 were alternate records. The gold standard linkage resulted in 212,922 unique links between patient record and the NDI. The de-duplicated (e.g., one record per patient as described above) PPRL resulted in 223,929 unique NHCS-NDI links – a 5.2% increase over the gold standard linkage. A patient that linked to the NDI is described as assumed deceased and those that did not link are assumed to be alive.

Largely the determinations of vital status have high concordance across these two linkages. The overall concordance was 99.6% for linkage-eligible patient records. The kappa statistic was 0.96, suggesting almost perfect agreement. The precision of the initial approach was 93.8% and the recall was 98.7%.

An examination of the links from the gold standard that were not returned by PPRL shows that the majority of these agreed on some part of the name (first or last), date of birth, state of residence and sex in the gold standard (data not shown). Differences between the two approaches may be due to how name fields were compared. In the gold standard, text strings were compared by modification to the Jaro-Winkler string comparator function [23]. By contrast, in the Datavant PPRL, since text strings are encrypted string comparator functions could not be used to compare names and the PPRL algorithm relied on encrypted tokens based on the exact name or indexed strings based on Soundex.

We next looked at the token combinations to assess which ones produced both FPs and TPs. A false positive rate was also calculated. Table 1 illustrates the 29 unique token combinations and the number of TPs and FPs associated with each combination. In the initial PPRL assessment, close to 60% of the TP links (n= 125,182) resulted from two high precision tokens that relied on SSN (i.e., Tokens 5 and 16). Close to 80% (n= 12,566) of the FP links were identified from links found by Token 1 or Token 2 only, which are based on agreement of first or last name, sex, and date of birth (see Fig. 1 for token definitions).

We note that links identified through Token 1, Token 2, the combination of Tokens 1 and 7, and the combination of Tokens 1 and 2 all have false positive rates > 50%. To assess the impact of removing the token combinations with high FP rates, we removed all links identified through these specific token combinations and assessed the new result set referred to as the refined PPRL links.

Table 2 summarizes the results for the initial and refined PPRL approaches.

Largely the determinations of vital status remained having a high concordance across these two linkages. The overall concordance was 99.8% for linkage-eligible patient records. The kappa statistic was slightly higher than the initial assessment at 0.98. The precision for the refined approach was 98.9% and the recall was 97.8%.

There were more FNs in the refined approach and less FPs compared to the initial PPRL assessment. An examination of the links from the gold standard that were not returned by refined PPRL shows similar results to the full PPRL result set. Most FNs had agreed on some part of the name (first or last), date of birth, state of residence and sex in the gold standard (data not shown).

3.1Impact on secondary data analysis

A final assessment looked at the impact of initial and refined PPRL links on secondary data analysis. The match rates by age and sex are presented below for the three approaches: gold standard, initial PPRL and refined PPRL (Table 3). In addition, the rate of SSN agreement is presented for the gold standard.

Figure 2.

Patient death rates by linkage approach and follow-up period and percent difference from gold standard.

The age and sex distribution of eligible linked survey participants appears similar across the three linkage approaches, although a higher number of links occur for those records with missing sex in both PPRL approaches (6.6% and 6.0% compared to 4.3% in the gold standard). On the other hand, there were a lower number of links found for records with missing age in PPRL (Table 3). Linkage rates are highest among adults aged 65 and over, with approximately 20% of adults aged 65 and over matching to the NDI for all three approaches. It is also important to note that this group (aged 65 and over) had the lowest percentage of missing SSN, 70.8% of eligible patients 65 and over had a non-missing SSN.

In addition, we performed a secondary analysis of death rates based on the results of the three linkage approaches: gold standard, initial PPRL, and refined PPRL links. Figure 2 shows a comparison of overall death rates during several follow-up periods: 30, 60, and 90 days from the last known discharge date for each patient.

Figure 2 shows that the initial PPRL approach leads to death rates being overestimated by 4.3% in the 90 day follow up period (and somewhat higher for shorter follow-up periods). By contrast, the refined PPRL links lead to a slight under-estimation of mortality (by less than 1%) for all follow up periods.

4.Discussion

PPRL is a technique that allows organizations to conduct linkages without sharing direct PII. As a federal statistical agency, it was important to assess data quality prior to implementing PPRL techniques to create data for use in official statistics. It should be noted that while we performed an assessment of data quality, we did not conduct a separate privacy analysis of the encoded databases. Prior to a full implementation of PPRL at NCHS, a privacy risk-reidentification analysis would need to be conducted as well [24, 25, 26]. This would enable us to identify any potential known risks and assess if there are any mitigation strategies that could be implemented to maintain the protection of privacy of the NCHS survey participants as required under Title III of Foundations for Evidence-Based Policymaking Act of 2018, Public Law 115–435 (the Confidential Information Protection and Statistical Efficiency Act or CIPSEA).

Nevertheless, the data quality results of this case study are encouraging. We examined the efficacy of PPRL compared to a gold standard by re-linking previously linked NHCS and NDI data using privacy protected, encrypted PII values (hashes/tokens) created in Datavant software rather than the actual data values. The standard NCHS linkage methodologies are based on deterministic and probabilistic techniques, using clear text matching. This research demonstrates that PPRL approaches, particularly the refined PPRL approach, produce results similar to a gold standard linkage (kappa statistics suggesting almost perfect concordance). The refined approach highlights the importance of the selection of tokens. When we removed the tokens that relied solely on first or last name, sex, and date of birth the concordance with the gold standard increased.

In terms of secondary data analysis of the match rates by age and sex, all three approaches produced similar results. Having a unique identifier such as SSN increases the likelihood that the records will match in all three approaches as noted in Table 3. In addition, in this analysis the percent with non-missing SSN is highest in the 65 and over age group which is also the age group most likely to link to the NDI.

This analysis quantifies how data linkage quality may impact inferences in secondary data analysis, which underscores the role of continued methodologic work to improve data linkage quality. The aggregated death rates computed from PPRL are less than 5% different than those from the gold standard and after refinement based on false positives, this falls to less than 1%. Next steps could include comparing the three approaches to assess death rates for different subpopulations and distributions by cause of death.

In addition, while this initial work laid the much-needed groundwork for assessing the accuracy of PPRL-created linked data when compared to traditional linkage methods, additional evaluation is needed to assess the quality of PPRL- based linkages when using data with lower quality PII. This initial analysis used two sources with high quality PII. Given the varying quality of PII within the wide range of data needed to conduct health related research it is important to expand the assessment of PPRL tools to include data sources with varying PII quality and completeness. While work to evaluate PPRL techniques is being conducted in the extramural research community, NCHS is in a unique position as the Department of Health and Human Services (HHS) federal statistical agency to assist with evaluating different PPRL strategies to potentially broaden linkage capabilities within HHS. It is of utmost importance to assess the quality of linkage processes and results when integrating data sources, using PPRL, so that researchers can determine whether the resulting data sets are suitable for inference and generalization to other populations.

In real-life scenarios researchers would not necessarily have a gold standard to be used for comparison or another way to determine linkage quality, therefore, these results could be used as a guide for the selection of tokens to reduce Type I errors. The generalizability of these findings for survey-based self-reported identification data compared to hospital patient records is still unclear and additional evaluations assessing PPRL techniques, including other PPRL methods, that rely on tokenization or hashing with lower quality PII is forthcoming. Future work will build on this analysis by testing a variety of scenarios, including PII that is non-standardized, incomplete (e.g., missing unique identification numbers such as Social Security Number (SSN)), and of varying levels of quality (e.g., informal names provided in survey collections) using previously linked NCHS data files as the gold standard to benchmark against PPRL-based results.

5.Conclusion

This research demonstrates that PPRL can be an effective record linkage technique that produces results similar to a gold standard. However, it is important to note that these results were obtained using sources that had a high rate of complete identifiers, including SSN.

6.Significance

This novel evaluation highlights the first time NCHS, a federal statistical agency, has utilized PPRL and compared it to an already released linkage that relied on clear text matching. The assessment builds on many of the guiding principles outlined in the Federal Committee on Statistical Methodology report “A Framework for Data Quality” [27]. The approach focused on assessments of threats, like accuracy and coherence, when utilizing new software to perform linkage and illustrates a way (e.g., the selection of refined tokens) to maintain scientific integrity and credibility while adhering to the protection of privacy. This is an important first step toward advancing the potential use of PPRL to integrate data without sharing direct identifiers.