Modified Wisconsin Card Sorting Test (M-WCST): Normative data for Spanish-speaking pediatric population

2.1Participants

The sample consisted of 4,373 healthy children who were recruited from Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, Puerto Rico, and Spain. Participants were selected according to the following criteria: a) were between 6 and 17 years of age, b) were born and currently lived in a country where the study was conducted, c) spoke Spanish as their mother tongue, d) an IQ≥80 on the Test Of Non-verbal Intelligence (TONI-2, Brown, Sherbenou, & Johnsen, 2009), and e) scored <19 on the Children’s Depression Inventory (CDI, Kovacs, 1992).

Children with history of neurologic or psychiatric disorders as reported by the participant’s parent(s) were excluded due to its effects on cognitive performance. Participants in the study were from public and private schools, and signed an informed consent. Socio-demographic and participant characteristics for each of the countries’ samples have been reported elsewhere (Rivera & Arango-Lasprilla, 2017). Ethics Committee approval was obtained for the study in each country.

2.2Instrument administration

The M-WCST consists of four stimulus cards and 48 response cards. Each card varies in shape (cross, circle, triangle, or star), color (red, blue, yellow, or green), and number (one to four). The objective is to classify correctly the stimulus card according to certain rule until completion of a category. The test continues until all six categories are classified or until the whole volume has been used (Schretlen, 2010). The test allows for calculation of the number of categories, perseverations, and total errors.

2.3Statistical analyses

Detailed statistical analyses used to generate the normative data for the M-WCST scores are described in Rivera and Arango-Lasprilla (2017). In summary, the scores were standardized using multiple linear regression analyses by means of a four-step procedure: 1) First, the number of categories, perseverations, and total errors scores were computed separately by means of the final multiple regression models. The full regression models included as predictors: age, age², sex, and mean level of parental education (MLPE). Age was centered (= calendar age – mean age in the sample by country) before computing the quadratic age term to avoid multicollinearity (Aiken & West, 1991). Sex was coded as male = 1 and female = 0. The MLPE variable was coded as 1 if the participant’s parent(s) had >12 years of education or 0 if participant’s parent(s) had ≤12 years of education. If predicted variables were not statistically significant in the multivariate model with an alpha of 0.05, the non-significant variables were removed and the model was run again. A final regression model was conducted yˆi=B0+B1·(Age-x¯Age by country)i+B2·(Age-x¯Age by country)i2+B3·Sexi+B4·MLPEi . 2) Residual scores were calculated based on the final model ( ei=yi-yˆi ). 3) Residuals were standardized using the residual Standard Deviation (SD_e) value provided by the regression model: z_i = e_i/SD_e. 4) Standardized residuals were converted to percentile values using the standard normal cumulative distribution function. This four-step process was applied for number of categories, perseverations, and total errors scores separately for each country.

For all multiple linear regression models, the following assumptions were evaluated: a) multicollinearity by the values of the Variance Inflation Factor (VIF), which must not exceed 10, and the collinearity tolerance values, which must not exceed the value of 1 (Kutner, Nachtsheim, Neter, & Li, 2005), and b) the existence of influential values by calculating the Cook’s distance. The maximum Cook’s distance value was related to a F (p,  n - p) distribution. Influential values are considered when percentile value is equal or higher than 50 (Cook, 1977; Kutner et al., 2005). All analyzes were performed using SPSS version 23 (IBM Corp., Armonk, NY).

3.1M-WCST number of categories correct

The final multivariate linear regression models for the ten countries-specific number of categories correct were significant (see Table 1). In all countries, the number of categories correct increased linearly as a function of age. The number of categories correct for Chile, Cuba, Ecuador, and Spain were affected by a quadratic age effect. Children from Ecuador, Mexico, and Spain whose parent(s) had a MLPE >12 years obtained higher number of categories correct than children whose parent(s) had a MLPE ≤12 years. The child’s sex did not affect the numbers of categories correct for any country. The amount of variance these predictors explained in the number of categories correct ranged from 3.0% (in Puerto Rico) to 41.1% (in Cuba).

3.2M-WCST number of perseverative errors

The final multivariate linear regression models for the ten country-specific numbers of perseverative errors were significant (see Table 2). In all countries, the number of perseverative errors decreased linearly as a function of age. The number of perseverative errors for Chile, Cuba, Mexico, and Spain were affected by a quadratic age effect. Children from Mexico and Paraguay whose parent(s) had a MLPE >12 years made lower number of perseverative errors than children whose parent(s) had a MLPE ≤12 years. The child’s sex did not affect the number of perseverative errors for any country. The amount of variance these predictors explained in the number of perseverative errors ranged from 4.0% (in Paraguay) to 24.4% (in Cuba).

3.3M-WCST number of total errors

The final multivariate linear regression models for the ten country-specific number of total errors were significant (see Table 3). In all countries, the number of total errors decreased linearly as a function of age. The number of total errors for Chile, Cuba, Peru, and Spain were affected by a quadratic age effect. Children from Cuba, Ecuador, Mexico, Paraguay, and Spain whose parent(s) had a MLPE >12 years made lower number of total errors than children whose parent(s) had a MLPE ≤12 years. Child’s sex affected the number of total errors for Ecuador such that boys make fewer errors than girls. The amount of variance these predictors explained in the number of total errors ranged from 3.6% (in Puerto Rico) to 30.5% (in Cuba).

The assumptions of multiple linear regression analysis were met for all final models. There was not multicollinearity (the VIF values were below 10; VIF ≤1.048; collinearity tolerance values did not exceed the value of 1) or influential cases (the maximum Cook’s distance value was 0.242 in a F _(3,384)  distribution which correspond to percentile 13).

3.4Normative procedure

Norms (e.g., a percentile score) for the different M-WCST scores by country were established using the four-step procedure described in the statistical analysis section. An example will be provided to facilitate an improved understanding of the procedure used to obtain the percentile associated with a score on this test. Thus, let us assume we need to find the percentile score for a 16-year-old Spanish girl who scored a 9 on the M-WCST number of total errors and whose parent(s) have a mean of 8 years of education (MLPE). The steps to obtain the percentile for this score are: 1) Find Spain in Table 3, which provides the final regression models by country for the M-WCST number of total errors. Use the B weights to create an equation that will allow you to obtain the predicted M-WCST number of total errors for this girl using the coding provided in the statistical analysis section. The corresponding B weights are multiplied by the centered age (= calendar age – mean age in the Spanish sample which is equal to 11.3 years), centered age² (= calendar age – mean age in the Spanish sample which is equal to 11.3 years)², and MLPE which was coded based on the 12 years of education threshold. Sex was not a significant predictor, thus, it is not included in the model. See Rivera and Arango-Lasprilla (2017) to determine the mean age of each country’s sample. Then the result is added to the constant generated by the model in order to calculate the predicted value.

In the case of the Spanish girl, the M-WCST predicted number of total errors would be calculated using the following equation: yˆi=10.201+[-1.238·(Agei-11.3)]+[0.141·(Agei-11.3)i2]+(-1.936·MLPEi) . The girl’s age is 16. The MLPE (8 years) is split into either ≤12 years (and assigned a 0) or more than 12 years (and assigned a 1) in the model. Since the parent(s) of the hypothetical child in the example have 8 years of education, the MLPE value is 0. Thus, the predicted value equation is: yˆi=10.201+[-1.238·(16-11.3)]+[0.141·(16-11.3)2]+ (-1.936·0)=10.201+(-5.819)+3.115=7.497 .

2) In order to calculate the residual value (indicated with an e_i in the equation), we subtract the actual M-WCST number of total errors (she scored 9) from the predicted value we just calculated ( ei=yi-yˆi ). In this case, it would be e_i = 9 -7.497 = 1.503. 3) Next, consult the SD_e column in Table 3 to obtain the country-specific SD_e (residual) value. For Spain, it is 8.397. Using this value, we can transform the residual value to a standardized z score using the equation z_i = e_i/SD_e. In this case, we have 1.506/8.397 = 0.179. This is the standardized z score for a 16-year-old Spanish girl who scored a 9 on the M-WCST number of total errors who has parent(s) with 8 years of education (MLPE). 4) The last step is to use the tables available in most statistical reference books (e.g., Strauss, Sherman, & Spreen, 2006) or use a trusted online calculator like the one available at http://www.measuringu.com/pcalcz/. In the online calculator, you would enter the z score and choose a one sided test and note the 100-percent of area (in case of perseverations and total errors) or the percent of area (in case of categories) after hitting the submit button to convert z scores to percentiles. In this example, the z score (probability) of 0.179 corresponds to the 43 percentile. It is important to remember to use the appropriate tables that correspond to each score (number of categories, perseverations, and total errors scores) when performing these calculations.

3.5User-friendly normative data

The four-step normative procedures explained above offers the clinician the ability to determine an exact percentile for a child who has a specific score on the M-WCST number of total errors. However, this method can be prone to human error due to the number of required computations by hand. To enhance user-friendliness, the authors have completed these steps for a range of raw scores based on age, sex, and MLPE and created tables for clinicians to more easily obtain a percentile range/estimate associated with a given raw score on this test. These tables are available by country and type of test in the Appendix. In order to obtain an approximate percentile for the above example (converting a raw score of 9 on the M-WCST number of total errors for a Spanish girl who is 16 years old and whose parent(s) have 8 years of education) using the simplified normative tables provided in the Appendix, the following steps must be followed. (1) First, identify the appropriate table ensuring the appropriate country and score (number of categories, perseverations, and total errors scores). In this case, the table for M-WCST number of total errors for Spain can be found in Table A31. (2) Next, the table is divided based on MLPE (≤12 vs. more than 12 years of education). Since the parent(s) had 8 years of education, we will use the bottom section of the table for ≤12 years of MLPE. (3) Find the appropriate age of the child, in this case, 16 years old. (4) Next, look in the 16 years’ age column to find the approximate location of the raw score obtained on the test. Within the 16 years’ column, the score of 9 obtained by this Spanish girl corresponds to an approximate percentile of 40.

The percentile obtained using this user-friendly table sometimes could be slightly different than the hand-calculated, more accurate method (43 vs. 40) because the user-friendly table is based on a limited number of percentile values. Individual percentiles cannot be presented in these tables due to space limitations. If the exact score is not listed in the column, you must estimate the percentile value from the list of raw scores available.

4.1Implications

The results of this study are of great clinical relevance, since having validated and standardized tests to reliably measure problems of executive functioning in children and adolescents is extremely important for clinicians and researchers. Changes in executive functions are common in many developmental disorders such as autism (Robinson, Goddard, Dritschel, Wisley, & Howlin, 2009), learning disorders (Toll, Van der Ven, Kroesbergen, & Van Luit, 2011), attention-deficit hyperactivity disorders (Barkley, 1997), cerebral palsy (Bodimeade, Whittingham, Lloyd, & Boyd, 2013), and many others, such that the above data provide a highly useful tool to professionals working with these populations. Similarly, executive functions have been found to be one of the most sensitive functions to acquired brain injury in the pediatric population (Levin & Hanten, 2005) and appear to have a mediating effect on children’s adaptive behavior (Shultz et al., 2016).

4.2Limitations

Despite the wide-ranging implications of this study, as it is the largest in the world on validation and standardization of the M-WCST with Spanish speaking populations, results should be interpreted in the light of the following limitations: This study presents normative data of the M-WCST for nine countries in Latin America and Spain. For this reason, it is not advisable to use these norms in the pediatric population of those Spanish-speaking countries where the study was not performed. Future studies should be conducted to standardize this test in other Spanish-speaking countries.

Although the norms of the present study could be used by neuropsychologists in other countries to evaluate Spanish-speaking immigrant children from the countries where the sample was collected for this study, we advise caution since other variables such as level of acculturation, bilingualism, the number of years living in the country, and so on, could influence performance. In addition, the quality of education of both the children and parent(s) is another aspect that may influence the cognitive performance of children.

It is very important to keep in mind that no clinical diagnosis should be made based solely on the M-WCST scores. This test should be integrated as part of a larger battery that evaluates executive functions in more detailed. Because there are a limited number of tests and norms in Latin America and Spain to evaluate this cognitive function, more efforts should be made in the future to have other similar tools.

Although the size of the sample was adequate in each of the countries where the study was conducted, it is very important to note that only the sample in Chile, Mexico, Paraguay, Puerto Rico, and Spain was obtained from several regions of the country, while in the remaining countries were collected from only one geographic area. Future studies should expand the sample in other geographical areas of these countries to enhance representativeness of the sample.

The children who participated part in the present study speak Spanish as their first language. Although Spanish is the first language of the majority of the population in Latin America and Spain, it is important to keep in mind there is a great cultural and linguistic richness in this region, to the point that in many of these countries, sometimes the first language of many children may be different from Spanish (e.g., Portuguese, Euskera, Catalan, Guaraní, Maya, Quechua). For this reason, caution should be used when using these norms in children whose first language is not Spanish.

Finally, it is important to keep in mind that the present study was performed with a normal healthy population. Therefore, future studies should be performed with clinical populations to establish the sensitivity and specificity of this M-WCST among children and adolescents.