Hopkins Verbal Learning Test– Revised: Normative data for the Latin American Spanish speaking adult population

2.1Participants

The sample consisted of 3,977 healthy individuals who were recruited from Argentina, Bolivia, Chile, Cuba, El Salvador, Guatemala, Honduras, Mexico, Paraguay, Peru, and, Puerto Rico. The participants were selected according to the following criteria: a) were between 18 to 95 years of age, b) were born and currently lived in the country where the protocol was conducted, c) spoke Spanish as their native language, d) had completed at least one year of formal education, e) were able to read and write at the time of evaluation, f) scored ≥23 on the Mini-Mental State Examination (MMSE, Folstein, Folstein, & McHugh, 1975), g) scored ≤4 on the Patient Health Questionnaire– 9 (PHQ-9, Kroenke, Spitzer, & Williams, 2001), and h) scored ≥90 on the Barthel Index (Mahoney & Barthel, 1965).

Participants with self-reported neurologic or psychiatric disorders were excluded due to a potential effect on cognitive performance. Participants were volunteers from the community and signed an informed consent. Five participants were excluded from the analyses, with a final sample of 3,972 participants. Socio-demographic and participant characteristics for each of the countries’ samples have been reported elsewhere (Guàrdia-Olmos, Peró-Cebollero, Rivera, & Arango-Lasprilla, 2015). The multi-center study was approved by the Ethics Committee of the coordinating site, the University of Deusto, Spain.

2.2Instrument administration

The list applied in the study was from HVLT-R form 5, which contains a list of 12 semantically related words in three categories (professions, sports, and vegetables). Three trials of successive learning are presented where a list is read to the participant and the correct answers of each learning trial performed are recorded. Total Recall is the sum of words recalled correctly in the three trials. After 20– 25 minutes, the Delayed Recall phase occurs where the subject is asked to recall all the words from the initial list that they can remember (Benedict et al., 1998; Brandt, 1991).

2.3Statistical analyses

The detailed statistical analyses used to generate the normative data for this test are described in Guàrdia-Olmos et al. (2015). In summary, the data manipulation process for each country-specific dataset involved five-steps: a) t – tests for independent samples and effect sizes (r) were conducted to determine gender effects. If the effect size was larger than 0.3, gender was included in the model with gender dummy coded and female as the reference group (male = 1 and female = 0). b) A multivariable regression model was used to specify the predictive model including gender (if effect size was larger than 0.3), age as a continuous variable, and education as a dummy coded variable with 1 if the participant had >12 years of education and 0 if the participants had 1– 12 years of education. If gender, age and/or education was not statistically significant in this multivariate model with an alpha of 0.05, the non-significant variables were removed and the model was re-run. Then a final regression model was conducted that included age (if statistically significant in the multivariate model), dichotomized education (if statistically significant in the multivariate model), and/or gender (if effect size was greater than 0.3) [yˆi=β0+(βAge·Agei)+(βEduc·Educi)+(βGender·Genderi)]; c) residual scores were calculated based on this final model (ei=yi-yˆi); d) using the SD _e (residual) value provided by the regression model, residuals were standardized: z = e _i /SD _e , with SD _e (residual) = the standard deviation of the residuals in the normative sample; and e) standardized residuals were converted to percentile values (Strauss et al., 2006). Using each country’s dataset, these steps were applied to HVLT-R total recall and delayed recall scores.

3.1HVLT-R total recall

Regarding the effect of gender on HVLT-R total recall scores, the t-tests showed significant differences between men and women from Guatemala, however, its effect size was less than 0.3. Table 1 shows the results of the gender analyses by country on HVLT-R total recall scores. As shown in Table 1, the effect sizes for all countries were less than 0.3, and therefore gender was not taken into account to generate HVLT-R total recall normative data for any of the countries in the study.

The final eleven HVLT-R total recall multivariate linear regression models for each country are shown in Table 2. In all countries, the HVLT-R total recall score increased for those with more than 12 years of education (see Table 2) and decreased in a linear fashion as a function of age. The amount of variance explained in HVLT-R total recall scores ranged from 17% (in Guatemala) to 40% (in Paraguay).

3.2HVLT-R delayed recall

Regarding the effect of gender on HVLT-R delayed recall scores, the t-tests showed significant differences between men and women in Guatemala and Puerto Rico. Table 3 shows the results of the gender analysis by country on HVLT-R delayed recall scores. As shown in Table 3, the effect sizes for all countries were less than 0.3, and therefore gender was not taken into account to generate HVLT-R delayed recall normative data for any of the countries in the study.

The final eleven HVLT-R delayed recall multivariate linear regression models for each country are shown in Table 4. In all countries, the HVLT-R delayed recall score increased for those with more than 12 years of education in all countries except Bolivia (see Table 4), and decreased in a linear fashion as a function of age. The amount of variance explained in HVLT-R delayed recall scores ranged from 13% (in Guatemala) to 45% (in Paraguay).

3.3Normative procedure

Norms (e.g., a percentile score) for the HVLT-R total and delayed recall score test were established using the five-step procedure described above. To facilitate the understanding of the procedure to obtain the percentile associated with a score on this test, an example will be given. Suppose you need to find the percentile score for a Puerto Rican woman, who is 50 years old and has 8 years of education. She has a score of 15 on the HVLT-R total recall test. The steps to obtain the percentile for this score are: a) Check Table 1 to determine if the effect size of gender in the country of interest (Puerto Rico) on this test and time point (HVLT-R total recall) is greater than 0.3 by country. The column labelled r in Table 1 indicates the effect size and the superscript notation b next to the number indicates that the number is larger than 0.3. In this example, the effect size is 0.091, which is not greater than 0.3. For Puerto Ricans on this test, gender does not influence scores to a sufficient degree to take it into account gender when determining the percentile. b) Find Puerto Rico in Table 2, which provides the final regression models by country for HVLT-R total recall. Use the B weights to create an equation that will allow you to obtain the predicted HVLT-R total recall score. The corresponding B weights are multiplied by the actual age and dichotomized education scores and added to a constant in order to calculate the predicted value. In this case, the predicted HVLT-R total recall score would be calculated using the equation [yˆi=29.183+(-0.152·Agei)+(1.261·DichotomizedEducationalLeveli)] (the values have been rounded for presentation in the formula). The subscript notation i indicate the person of interest. The person’s age is 50, but the education variable is not continuous in the model. Years of education is split into either 1 to 12 years (and assigned a 0) or more than 12 years (and assigned a 1) in the model. Since our hypothetical person in the example has 8 years of education, her educational level value is 0. Thus the predicted value is yˆi=29.183+(-0.152·50)+(1.261·0)=29.183+(-7.6)+0=21.57). c) In order to calculate the residual value (indicated with an e in the equation), we subtract the actual value from the predicted value we just calculated (ei=yi-yˆi). In this case, it would be e _i  = 15 - 21.57 = -6.57. d) Next, consult the SD _e column in Table 2 to obtain the country-specific SD _e (residual) value. For Puerto Rico it is 4.060. Using this value, we can transform the residual value to a standardized z score using the equation (e _i /SD _e ). In this case, we have (-6.57)/4.060 = -1.618. This is the standardized z score for a Puerto Rican woman aged 50 and 8 years of education and a score of 15 on the HVLT-R total recall test. e) The last step is to use look-up the tables in the statistical reference books (e.g. Strauss et al., 2006) or use a trusted online calculator like the one available at http://www.measuringu.com/pcalcz.php. In the online calculator, you would enter the z score and choose a one-sided test and note the percent of area after hitting the submit button. In this case, the probability of -1.618 corresponds to the 5th percentile. Please remember to use the appropriate tables that correspond to each test (total recall vs. delayed recall) when performing these calculations. If the percentile for the HVLT-R delayed recall score is desired, Tables 3 and 4 must be used.

3.4User-friendly normative data tables

The five-step normative procedures explained above can provide more individualized norms. However, this method can be prone to human error due to the number of required computations. To enhance user-friendliness, the authors have completed these steps for a range of raw scores based on small age range groupings (see Guàrdia-Olmos et al., 2015) and created tables that clinicians can more easily use to obtain a percentile range associated with a given raw score on this test. These tables are available by country and type of test (HVLT-R total recall vs. HVLT-R delayed recall) in the Appendix. In order to obtain an approximate percentile for the above example (converting a raw score of 15 for a Puerto Rican woman who is 50 years old and has 8 years of education) using the simplified normative tables provided, the following steps are recommended. (1) First, identify the appropriate table ensuring the specific country and HVLT-R total recall scores. In this case, the table for HVLT-R total recall scores for Puerto Rico can be found in Table A11. (2) Note if the title of the table indicates that it is only to be used for one specific gender. In this case, gender is not specified in the table. Thus Table A11 is used for both males and females. (3) Next, the table is divided based on educational level (1 to 12 vs. more than 12 years of education). Since this woman has 8 years of education, she falls into the 1 to 12 years of education category. These data can be found in the top section of the table. (4) Determine the age range most appropriate for the individual. In this case, 50 fall into the column 48– 52 years of age. (5) Read down the age range column to find the approximate location of the raw score the person obtained on the test. Reading down the 48– 52 column, the score of 15 obtained by this Puerto Rican woman corresponds to an approximate percentile of 5.

4.1Limitations and future directions

This study has several limitations and directions for future research. First, clinicians should be careful in attempting to use the HVLT-R norms from this study for individuals in countries other than those from which data were collected. Future studies need to establish HVLT-R norms in other Latin American countries such as Ecuador, Uruguay, Venezuela, and Panama. Despite the severe limitations of using the current norms in other Spanish-speaking countries, the current norms may nonetheless be some of the most accurate to date for other regions in Latin America, as previous Spanish norms from the U.S.-Mexico border region have been shown to result in high rates of misclassification, especially in individuals with lower levels of education (Cherner et al., 2007). As a result, the current HVLT-R norms may be more accurate than others currently in use in routine neuropsychological assessment, but this generalizability is a critical area for futureresearch.

Second, several sampling limitations should be noted. The HVLT-R is a common neuropsychological measure in Latin America, but many other common assessments are left to be normed in the same manner, a ripe area for future research. Research should also investigate the ecological validity of various neuropsychological assessments in Latin America. If low, there would be a strong need to create instruments in those cultures with stronger ecological validity. Indeed, the HVLT-R has established construct, concurrent, and discriminant validity (Benedict et al., 1998; Brandt, 1991; Rasmussan et al., 1995), but this has not been established in Latin America. Future research needs to create assessments within local cultures which would add to their cultural sensitivity, not simply translate and norm tests from other cultures.

Third, because participants spoke Spanish as their primary language, HVLT-R performance could differ for individuals who are bilingual and speak English or other local dialects. As a result, an area for future research would be the influence of bilingualism on HVLT-R performance, which was beyond the scope of this study. Also, the data collection in this study was specific to several regions within the various countries, and the norms and their national representativeness is unknown. Future studies should collect data form more nationalized samples. But nonetheless, the current study is the largest neuropsychological normative study for the HVLT-R in any global region conducted to date, and it is a critical first step toward larger studies with nationally representative samples. In the current sample, a sizeable group had fewer than 12 years of education, but illiterate individuals were excluded. Therefore the current norms may not generalize well to illiterate adults, and future research should be conducted with illiterate samples, those with neurological conditions, as well as children, all groups that were not included.

Although these limitations are present, only one other study to date has produced HVLT-R norms for Spanish-speaking individuals, which was conducted in the U.S.-Mexico border region (Cherner et al., 2007). As a result, this study was the first to create HVLT-R norms across 11 countries in Latin America with nearly 4,000 participants. It was therefore the largest and most comprehensive HVLT-R normative study conducted in any global region, and its norms have the strong potential to affect the standard of neuropsychological assessment with the HVLT-R in Latin America unlike any study before it.