Methodology for the generation of normative data for the U.S. adult Spanish-speaking population: A Bayesian approach

1Introduction

For many years, Hispanics have been recognized as the largest and fastest growing ethnic minority group in the U.S. (Pew Research Center, 2017). Projections indicate that Hispanics will reach almost 111 million people, making up 28% of the total U.S. population, by 2060 (U.S. Census Bureau, 2018). Clinical neuropsychologists base their diagnoses in part on the interpretation of neuropsychological test results, so accurate and reliable neuropsychological assessments are crucial. In the U.S., a growing number of clinicians are able to provide neuropsychological services in Spanish, but nonetheless face significant challenges when working with Spanish-speaking patients. One of the challenges is considering the influence of diverse sociocultural influences (e.g., country of origin, bilingualism, acculturation level, socioeconomic status) on patient performance as these factors are known to impact neuropsychological testing (Bialystok et al., 2012; Boone et al., 2007; Flores et al., 2017). The second challenge is related to the limited availability of normative data for this population. According to Gasquoine et al. (2021), neuropsychologists in the U.S. often attempt to interpret results for Spanish speakers using norms generated in the continental U.S. When not available, they resort to norms from Latin American countries or Spain (Rivera et al., 2019, 2021).

The majority of available norms for Spanish speakers living in the U.S. were established before 2010 and focus on only a few neuropsychological measures (mostly Verbal Fluency, Mini Mental State Examination, Clock Drawing Test), or are applicable for middle-aged and older individuals (>40 years old) residing along the Mexico/U.S. border (California, Arizona, New Mexico, Florida, and Texas) and Northern Manhattan (see Morlett et al., 2021 for a review). Previous norms also vary concerning administration language (exclusively Spanish or both Spanish and English).

More recently, Cherner et al. (2020) finished “The Neuropsychological Norms for the U.S.-Mexico Border Region in Spanish (NP-NUMBRS)” project, which provided norms for 15 neuropsychological measures. This project focused on native Spanish speakers from the U.S. (California/Arizona)-Mexico border region aged 19 to 60, with a total sample of 254 healthy adults. Although not taken into consideration in their normative sample, they recognized the relevance and potential impact of sociocultural variables on cognitive performance and accordingly encouraged future studies to incorporate variables such as level of bilingualism, acculturation, or age of language acquisition.

Despite significant progress in providing norms for U.S. neuropsychologists offering services in Spanish, there remains a considerable gap. For example, the norms mentioned above should be applied with caution in Spanish-speaking populations living in unexamined regions, such as the Northern U.S. Moreover, very few studies have considered sociocultural factors known to impact cognitive performance. Finally, most available norms were generated before 2010. For example, data for some of the most recently published norms in the NP-NUMBRS project were collected in two cohorts (1998– 2000 and 2006– 2009).

There are several approaches to generate normative data for neuropsychological tests. The two most common approaches are traditional and regression-based (delCacho-Tena et al., 2024; Innocenti et al., 2023; van Breukelen et al., 2005), both of which have limitations. The traditional approach involves dividing the sample into subgroups based on relevant demographic variables such as age (in ranges), level of education, and sex. Within each subgroup, the mean (x¯) and standard deviation (s) are calculated. Using these values, the z-score (z_i) of a participant can be estimated using the equation: zi=xi-x¯/s , where x_i represents the raw score. However, this approach has certain drawbacks. One major issue is the loss of information when dividing the sample into subgroups. For instance, splitting the sample by sex halves the sample size, and using five age groups further reduces the size of each subgroup to 10% of the total sample. Additionally, this method does not provide an adequate way to select cut-off points for creating age and education groups, making it impossible to detect gradual changes across continuous variables like age and education. As a result, it assumes no change in scores within each subgroup, which reduces the precision of the normative data.

The regression-based approach uses linear regression parameters β_k to adjust for demographic influences and create normative data. Examples of this approach can be found in the Mayo Clinic’s older Americans normative studies (MOANS; Ivnik et al., 1992), older African Americans (MOAANS; Lucas et al., 2005), the NEURONORMA project (Peña-Casanova et al., 2009) and NEURONORMA youth (Peña-Casanova et al., 2012). The procedure involves: a) creating age groups for generating normative data and evaluating the effect of age, sex, or education on neuropsychological scores using correlation coefficients (r) and determination (r²); b) creating age-adjusted normative tables (SS_A). For each age-range a cumulative frequency distribution of raw scores is generated, and these raw scores are assigned percentile ranks. Then, percentile ranks are converted to scaled scores (SS, scale of 2 to 18), aiming for a distribution x¯=10 and s = 3 to perform linear regressions; c) adjusting for education by modeling it using the following equation: SS_A = β₀+ β₁ · Education ; and d) using resulting equations to calculate age- and education-adjusted scaled scores SS_A&E = SS_A - β₁ · Education. The main issue with this methodology is the use of the age groups for generating normative data, the assumption of linearity in the relationship between covariates and the studied score, as well as the lack of evaluation of the regression model’s quality in terms of fit and predictive capability.

Another more recent regression-based approach uses multiple regression models and residual standard deviation. Examples of this approach are normative studies conducted for Spanish-speaking countries in both adults and children/adolescents (Rivera et al., 2019, 2021; Rodrıguez-Lorenzana et al., 2020), as well as in other countries (van der Elst, et al., 2012; Vicente et al., 2022). This approach allows an evaluation of the influence of covariates in the presence of other covariates within the same model rather than separately. Additionally, age and education can be included as continuous variables, and their quadratic effects are evaluated to identify gradual changes in scores across age and education. This eliminates the problem of abrupt changes seen in the traditional method. Using this multiple linear regression method, normative data are generated through the cumulative distribution of standardized residuals.

In this method, the multiple linear regression model assumes that Y_i = X_iβ + ɛ_i where Y_i is the vector of responses, X_i is the design matrix (including covariates such as age, age², education, sex, and interactions), β is the vector of regression parameters, and ɛ_i is the vector of residual components assumed to be normally distributed with mean 0 and variance σɛ2 . One advantage of this method is that for all multiple regression models, the statistical assumptions (multicollinearity, homoscedasticity, normality, and the presence of influential values) can be evaluated.

Using the final regression model that is obtained at the end of the stepwise procedure, normative data that are adjusted for demographic variables are established by means of a four-step procedure (Rivera et al., 2019; van der Elst et al., 2012; van Breukelen & Vlaeyen, 2005): a) the expected test score (Yˆi) is computed based on the fixed-effect parameter estimates of the established final regression model: Yˆi=Bˆ0+Bˆ1X1i+Bˆ2X2i+…+BˆkXki ; b) to obtain the residual value (e_i), the model-predicted value ( Yˆi ) is subtracted from the raw score on the neuropsychological test (Y_i), i.e., ei=Yi-Yˆi ; c) using the residual standard deviation (SD_e) value provided by the regression model, the residual is standardized: z_i = e_i/SD_e; and d) using the standard normal cumulative distribution function, the exact percentile value corresponding to the z_i-score previously calculated is obtained (if the model assumption of normality of the residuals was met in the normative sample) or via the empirical cumulative distribution function of the standardized residuals (if the standardized residuals were not normally distributed in the normative sample).

A recent review highlighted other regression-based methodologies for generating normative data (delCacho-Tena et al., 2024), such as fractional polynomial equations, which consider both linear and nonlinear effects of demographic factors. This method was primarily used in the NP-NUMBRS project (Cherner et al., 2021; Marquine et al., 2021). Its limited use may be due to the lack of information regarding variable selection, final model selection, and the transformations needed to convert raw scores to other scales.

Finally, all the previously mentioned studies (using both traditional and regression-based approaches) rely on classical (frequentist) statistical inference. There are few studies that employ Bayesian inference (delCacho-Tena et al., 2024), with notable exceptions like the work of Crawford et al. (2009), which highlight the unavoidable uncertainty over percentile ranks. Bayesian inference offers several advantages over the aforementioned discussed methods, including the ability to incorporate prior information regarding a parameter, even indicating that no information is available (Berger, 2006), provide more intuitive probability statements, and offer more flexible tools to model complex situations better accounting for uncertainty and variability in the data (Clayton et al., 2021).

Given this neuropsychology landscape for Spanish speakers, it is evident that updated normative data for Spanish speakers in the U.S. are essential considering that norms are most precise in the year of their creation (Mitrushina et al., 2005). Therefore, the purpose of this study was to present the methodology to generate normative data for healthy Spanish-speaking adults (18– 80 years old) living in the U.S. using Bayesian inference as an improved approach to normative data estimation for neuropsychological tests.

2Methods

2.3Statistical analyses

2.3.1Bayesian approach

For the present study, Bayesian inference was used. Bayesian inference offers an alternative to the classical approach by treating parameters as random variables with probability distributions, rather than fixed but unknown quantities. The goal is to compute the posterior distribution of the parameters given the observed data [P (θ|D)], using Bayes’ theorem:

P(θ|D)=P(D|θ)P(θ)P(D).

Where, P (D|θ) is the likelihood of the data given the parameters, P (θ) is the prior distribution for the unknown parameters, and P (D) is the marginal likelihood for the data. The marginal likelihood is often difficult to compute, so Bayes’ theorem is usually simplified to:

P(θ|D)∝P(D|θ)P(θ).

This means the posterior distribution can be determined by adjusting the product of the likelihood and the prior and using numerical or simulation strategies (Gómez-Rubio, 2021; Kruschke, 2015).

2.3.2Variable selection

A Bayesian approach was also adopted to determine which variables should be included as predictors. In this scenario, the different regression models are considered as the unknown parameters. A prior probability is then given to each of the models and combined with the information from the data. This combination is summarized in a posterior probability for each model. In this variable selection procedure, the number of models, or, equivalently, the number of possible combinations of variables, is 2^p where p refers to the number of covariates. After the posterior probabilities for each model are obtained, researchers can develop a better idea of which models are more strongly related to the output (Li & Clyde, 2018). As an example, imagine we have 2 variables (x₁, x₂) to include in a linear predictor η. With these two variables, we can form a total of 2² = 4 different models with linear predictors:

• M₁: η = β₀

• M₂: η = β₀ + β₁x₁

• M₃: η = β₀ + β₂x₂

• M₄: η = β₀ + β₁x₁ + β₂x₂

Starting from a prior probability for each of these models P (M_i) for i = 1, 2, 3, 4, and using the Bayes theorem, we can give a posterior probability P (M_i|D) for each model given data D . In the current study, there were 29 variables (including interactions), meaning the comparison of 2²⁹ models.

However, the posterior probability of a single model is not always the best summary, especially when the number of models is large, as the probabilities may be very low. Instead, researchers can use posterior inclusion probabilities (PIPs) for each variable. These PIPs are obtained by summing the probabilities of all the models that contain a given variable. For instance, in the example above, the PIP for x₁ is obtained as the sum of the posterior probability of models M₂ and M₃, such that:

P(x1∈MT|D)=P(M2|D)+P(M3|D).

For this study, once the PIPs for each variable (x_i) were obtained, an elbow plot (x-axis=each covariate, y-axis=PIP values) was created to examine substantial changes in PIPs. Variables with PIPs greater than.5 were selected for the regression model for each neuropsychological score. For a better understanding on how to perform Bayesian variable selection we recommend reading Bayarri et al. (2012).

2.3.3The effects of demographic variables

The effects of demographic variables on scores were evaluated by means of linear (LM) or generalized linear models (GLM). For each score, separate regression models were fitted to establish score-specific normative data. The full regression models included as predictors: age, age², education (log transformed), sex, time living in U.S., BAS, BDS, and all two-way interactions between these variables. Age was centered (age in years – X¯Age ) before computing quadratic age to reduce multicollinearity (Kutner et al., 2005). The natural logarithm (ln) of education was examined assuming that scores do not increase linearly as a function of education; instead, each one-point increase in education resulted in a slower rate of increase in neuropsychological scores.

2.3.4Probabilistic distributions

To implement the LM or GLM regression, different probability distributions were used depending on the type of neuropsychology test scores being studied. Therefore, for this study, regressions were implemented based on the following probabilistic distributions:

Binomial distribution. It is a discrete probability distribution that describes the number of successes in a fixed number of independent Bernoulli trials, where each trial can be classified as success or failure. This is Y_i ∼ Bin (m_i, π_i), where m_i is the total number of trials and π_i is the probability of success on each trial for each individual i. BNT and BTA scores were modeled using a Binomial regression, and the probability can be expressed as follows:

logit(πi)=log(πi1-πi)=β0+β1Xi(1)+…+βpXi(p).

Poisson distribution. The distribution most often used for modeling counts is the Poisson distribution Y_i ∼ Po (μ_i), where μ_i > 0 are the expected counts for individual i and it can be expressed as follows:

log(μi)=β0+β1Xi(1)+…+βpXi(p),

or, equivalently

μi=eβ0·eβ1Xi(1)·…·eβpXi(p).

Verbal fluency, Stroop test and SDMT scores were modeled using Poisson regression.

Beta distribution. This is a continuous probability distribution defined on the interval [0, 1], and it is commonly used to model random variables representing proportions or probabilities. A Beta distribution was assumed Y_i ∼ B (μ_i, φ_i) for ROCFT immediate recall, delayed recall and WHO-UCLA AVLT test scores. These scores were scaled by dividing the participant’s score by the maximum possible score for each test, resulting in proportional scores, and it can be expressed as follows:

logit(μi)=β0+β1Xi(1)+…+βpXi(p).

Gamma distribution. This is a continuous probability distribution when the variable of interest follows a skewed, positive distribution (i.e., asymmetric time). In this study, the TMT time scores were assumed following Y_i ∼ Ga (α_i, β_i), where the expected value is μi=αiβi and can be expressed as follows:

log(μi)=β0+β1Xi(1)+…+βpXi(p).

A particular case of the Gamma distribution is the exponential distribution, where α_i = 1 for all observations. This is a continuous probability distribution that has a memoryless property, meaning that the probability of an event occurring in the next interval is independent of how much time has already elapsed. It is characterized by a single parameter, Y_i ∼ Ex (λ_i), where λ_i represents the rate at which events occur and the expected time to an event is μ_i = 1/λ_i. ROCFT copy, the free-drawing and copy conditions of CDT, as well as Bells test scores were scaled using the calculation: [(maximum test score - participant score)/ maximum test score], and were modeled using Exponential regression.

2.3.5Priors distributions

Once the probability distribution for each test score was identified, Bayesian regression models were conducted, where prior distributions for each of the unknown parameters (β₀, β₁, … β_p) followed a normal distribution centered at 0, with large variance (i.e., σ² = 10⁴ and τ = 10^–3) for Beta, Binomial, Gamma, and Poisson models. For the Exponential model, a Gamma distribution was used centered at 10^–3 with large variance (σ² = 10⁴). Vague or non-informative prior distributions were assigned to the models. The Bayesian inference procedure was performed using Markov Chain Monte Carlo methods through the software JAGS (Just Another Gibbs Sampler) and its R interface rjags (Plummer, 2022). For the convergence of three chains for each parameter, a burning period was used. A total of 3,000 samples of each posterior distribution was left.

R 4.4.0 for Linux (R Development Core Team, 2024) was used to perform the analyses, and the mice package (Buuren & Groothuis-Oudshoorn, 2011) was used to conduct the MICE analysis. See supplemental material for illustration of the script of implementing the Bayesian generalized linear model (BGLM) for the generation of normative data in neuropsychological tests.

2.3.6Normative procedure

The norms (i.e., a percentile score) for the neuropsychological test scores were established using a four-step procedure: a) The expected test score (μˆi) was computed based on the parameter estimates from the established regression model (βˆp) according to the probability distribution. b) The probability based on the Beta, Binomial, Gamma, Exponential and Poisson Cumulative Distribution Function were estimated for each of the 3,000 sample parameters. c) A mean probability was calculated. Finally, d) this probability was multiplied by 100 to interpret it as a percentile.

Fig. 1

Flowchart of the methodology.

2.3.7User-friendly normative data

To facilitate the understanding of the procedure to obtain the percentile associated with a given score on a neuropsychological test, an example will be given. Suppose you need to find the probability for a woman, who is 50 years old and has 15 years of education. She obtained a BDS score of 10 and a score of 9 on the /r/ phoneme.

Since the method explained above is complex and can be prone to human error due to the number of required computations, an online calculator based on https://www.rstudio.com/products/shiny/ was created. This will facilitate probability calculation as clinicians only need to enter basic patient information into the calculator (i.e., raw score for the specific test, age, education, and so on). This tool is available for all users at https://github.com/diegoriveraps/calculators. Using the calculator and introducing the information requested for the example above, this woman would obtain a mean probability score of.213, that is, at the 21.3^th percentile (see Supplementary Material - point 4).

3Discussion

The purpose of this article was to describe the methodology and procedures utilized to generate normative data for 12 neuropsychological tests for healthy Spanish-speaking adults (18– 80 years old) living in the U.S. using Bayesian GLMs as a novel approach. The final sample size was comprised of 245 participants from eight U.S. states (California, Connecticut, Florida, Indiana, New Jersey, Oregon, Virginia, and Wisconsin) who immigrated or had ancestry from 17 different Latin American countries of origin (Argentina, Bolivia, Chile, Colombia, Cuba, Dominican Republic, Ecuador, El Salvador, Guatemala, Honduras, Mexico, Nicaragua, Panama, Peru, Puerto Rico, Uruguay, and Venezuela).

This study fills numerous gaps in the research to date, generating normative neuropsychological data for Spanish speakers living in the U.S. This study considers the impact of sociocultural factors including level of acculturation to Hispanic culture, bilingual language dominance, and time living in the U.S., in addition to the traditional sociodemographic variables such as age, education level, and sex (delCacho et al., 2024). Additionally, for the first time in neuropsychological normative data estimation, this study used probabilistic distributions other than the limited normal distribution (Poisson, Binomial, Exponential and Gamma), as well as Bayesian inference, to generate the most accurate neuropsychological norms possible.

Bayes’ theorem offers a robust methodology for statistical inference, enabling the incorporation of prior information and combining it with data to control uncertainty. Even without prior information, non-informative priors can be used to make neutral inferences (Berger, 2006). Results are presented as posterior probability distributions, overcoming the difficulties in interpreting p-values and confidence intervals (Trafimow & Marks, 2015). A key advantage of the Bayesian approach is the intuitive interpretation of results through direct probabilities. For example, it allows statements like, “a 95% probability that a parameter lies within a specific range,” which is more intuitive than frequentist confidence intervals (Kruschke, 2010).

Additionally, Bayesian methods provide flexibility and ensure convergence even as model complexity increases, where frequentist methods may fail. This is particularly useful in hierarchical modeling, capturing complex dependencies and variations at different levels (Ntzoufras, 2008; Sacchi & Swallow, 2021). The main criticism of using priors in Bayesian analysis is their subjectivity and influence on results. However, this can be mitigated by using non-informative priors, sensitivity analysis, and standardized priors (Berger, 2006; Congdon, 2006; Gelman et al., 2013). A good review of the advantages and performance of this methodology can be found in Steel (2020).

One key difference of this study compared to other normative data studies is the approach to selection of variables. The goal of variable selection is to identify a single ‘best’ model (Forte et al., 2018; James et al., 2021). The most common statistical methods used in neuropsychological normative data for this propose are linear regressions (49.5%), correlation and/or covariance coefficients (30.1%), analysis of variance (ANOVA), multivariate analysis of variance (MANOVA), analysis of covariance (ANCOVA; 22.1%), Student’s t and/or Mann– Whitney U (13.3%), and Chi-square (2.2%) (delCacho et al., 2024). However, while bivariate methods (i.e., Pearson’s, Spearman’s, bivariate correlation) are simple and useful for initial data exploration, they are inadequate for variable selection in a multivariate analysis. This is due to their inability to capture complex interactions, as they analyze each variable in isolation concerning the outcome (y). A variable might not show a strong correlation with the outcome on its own, but can be very informative when considered alongside other variables. In other words, bivariate methods do not evaluate how a variable contributes to the model when all other variables are considered, where the importance of a variable may change with the inclusion or exclusion of other variables in the model (James et al., 2021).

In terms of linear regression models, methods such as forward selection, backward elimination, and stepwise selection are commonly used. These methods are useful because they offer a balance between thoroughness and computational efficiency, but their sequential nature can lead to suboptimal solutions that may not consider all possible interactions. Therefore, it is important to combine them with cross-validation and complement them with more robust methods like regularization techniques (e.g., LASSO, Ridge; James et al., 2021; Miller, 2002).

Although it has a higher computational cost, this study used the 2^pvariable selection methodology, which is an exhaustive approach that considers all possible combinations of p variables to determine the best-fitting set of variables for a model. This methodology compares the performance of all generated models and selects the one that optimizes the chosen evaluation criterion. In this study, a primary model consisting of 29 covariates (including two-level interactions) was proposed, resulting in a total of 2^p models, or 536,870,912 models for each neuropsychological test score studied, to select the variables that should be considered in the creation of normative data. Unlike forward and backward selection, the 2^p methodology evaluates all possible combinations of variables, ensuring that no potentially relevant interactions are omitted. This can be particularly important in cases where there are complex interactions between variables that are not easily detected with sequential methods, as is the case when there are varied demographic and cultural factors that may impact test performance.

Finally, according to delCacho et al. (2024), normative data studies in neuropsychology tend to rely on regression-based linear models, regardless of the nature of the outcome variable, leading to failure to meet required statistical assumptions. A review of GLM methodology by Innonceti et al. (2023) highlights that 52% of models do not satisfy normality or it is not reported, and 66% do not satisfy homoscedasticity or it is not reported. This failure can be due to various reasons, but the most common is that the probabilistic model used in the regression is not appropriate, for example, using models with an inappropriate normal distribution.

Currently, GLMs offer significant advantages over traditional linear models for discrete (count) and time variables. Their flexibility to handle different error distributions, model non-constant variance, and provide interpretations consistent with the nature of the data make them a tool increasingly used in applied statistics. Additionally, as in the current study, Bayesian GLMs offer further advantages over traditional GLMs (Dey et al., 2000). These advantages include the ability to incorporate prior information, better estimation of uncertainty, greater flexibility in modeling, and more robust methods for model evaluation and comparison. These features make Bayesian GLM a versatile tool for data analysis in a wide range of scientific fields.

4Conclusion

Prior to the current study, tools available to neuropsychologists evaluating the cognitive performance predominantly Spanish-speaking adults covered a limited scope of cognitive abilities when compared to neuropsychological measures available for assessing English-speaking adults. This study, however, generated new normative neuropsychological data for Spanish-speaking adults living in the U.S. for many of the most commonly employed neuropsychological tests and therefore has the potential to improve the practice of neuropsychological assessment in the U.S. for this group. Further, for the first time in the known neuropsychological research literature, this study deployed modern statistical methods including the use of alternative score distributions and Bayesian prior probabilities to develop the most comprehensive and accurate prediction models. As a result, this study represents an advance not only in clinical but research methodology that future normative studies may consider emulating.

The current study has the potential to help reduce healthcare disparities in U.S. Hispanic populations via improvement in the accuracy of test results which has direct implications for diagnosis and treatment of neurological conditions. For all these reasons, the current study helps fill a large the gap in the research literature and in the standard of care for the neuropsychological assessment of Spanish-Speakers living in the U.S.