You are viewing a javascript disabled version of the site. Please enable Javascript for this site to function properly.
Go to headerGo to navigationGo to searchGo to contentsGo to footer
In content section. Select this link to jump to navigation

Symbol Digit Modalities Test: Normative data for Spanish-speaking pediatric population

Abstract

OBJECTIVE:

To generate normative data for the Symbol Digit Modalities Test (SDMT) in Spanish-speaking pediatric populations.

METHOD:

The sample consisted of 4,373 healthy children from nine countries in Latin America (Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, and Puerto Rico) and Spain. Each participant was administered the SDMT as part of a larger neuropsychological battery. SDMT scores were normed using multiple linear regressions and standard deviations of residual values. Age, age2, sex, and mean level of parental education (MLPE) were included as predictors in the analyses.

RESULTS:

The final multiple linear regression models showed main effects for age in all countries, such that score increased linearly as a function of age. In addition, age2 had a significant effect in all countries, except in Honduras and Puerto Rico. Models indicated that children whose parent(s) had a MLPE >12 years of education obtained higher score compared to children whose parent(s) had a MLPE ≤12 years for Chile, Guatemala, Mexico, and Spain. Sex affected SDMT score for Paraguay and Spain.

CONCLUSIONS:

This is the largest Spanish-speaking pediatric normative study in the world, and it will allow neuropsychologists from these countries to have a more accurate interpretation of the SDMT with pediatric populations.

1Introduction

The Symbol Digit Modalities Test (SDMT) is constructed as a substitution task that assesses neurocognitive functioning in the domains of attention, motor speed, and visual scanning (Smith, 1968; Smith, 1982; Smith, 2013; Straus, Sherman, & Spreen, 2006). The test is efficient to administer and both the oral and written formats are often completed in succession to produce data on both verbal and visuomotor responses, however they can be administered individually as needed (Berrigan et al., 2014; Straus et al., 2006). Correlations between the oral and written formats tend to range above r = 0.78, indicating a strong relation between the two versions (Lezak, Howieson, & Loring, 2004; Straus et al., 2006). The SDMT is used for identifying a number of neurocognitive conditions; however, it has particular sensitivity to cognitive changes related to multiple sclerosis (Benedict et al., 2008; Drake et al., 2009; Langdon et al., 2012).

The SDMT is considered appropriate for pediatric use, namely among children eight years and older (Smith, 2013; Straus et al., 2006). Similar to its use with adults, the instrument can be administered in both written and oral formats, or individually as needed (Straus et al., 2006). Correlations between the two versions among children have demonstrated some variation in the literature, but are considered acceptable to be used as needed for the purposes of neuropsychological testing. Oral performance has been found to be higher between the ages of 8– 13, but weakens as compared to written scores between ages 14– 17. This is likely associated with normative development in writing skills as children age.

Administration of the SDMT begins by presenting the test taker with a key comprised of paired shapes and numbers (Smith, 2013; Straus et al., 2006). The test taker has 90 seconds to scan an array of symbols and substitute a number associated with each symbol. The written version is administered first, and the test taker writes the number below the corresponding symbol. In the oral administration, the test proctor records the numbers spoken by the test taker. In both administrations, the proctor constructs a proportion of correctly answered items to total items answered. For example, 40/45 would indicate the participant responded with 40 correct items and 5 incorrect items, together summing to 45 total items attempted. A variation on the testing procedure includes a recall task, in which the test taker is provided a sheet with blanks to fill in the according numbers from memory (Uchiyama et al., 1994). This approach occurs between the written and oral test.

The SDMT is valuable for assessing neurocognitive symptoms related to a range of pediatric conditions including: traumatic brain injury (Babikian et al., 2011; Newman, Reesman, Vaughan, & Gioia, 2013), epilepsy (Lai et al., 2015), and brain cancer (Sands et al., 2012). Due to the role of motoric slowing in multiple sclerosis, the oral version of the SDMT has particular value when diagnosing this condition (Charvet, Beekman, Amadiume, Belman, & Krupp, 2014; Portaccio et al., 2009).

Pediatric normative data for the SDMT are sparse. The norms are based on a sample of 3,680 normal children from the Nebraska public school system (Smith, 1982). Although this offers foundational normative data, it is likely not appropriate to generalize these data due to cultural constraints of those studied in Nebraska and changes in responses to the test over time. A recent study developed norms for the oral SDMT for children between the ages of 6– 17 (Smerbeck et al., 2011). Although it was a relatively small sample, this study offers valuable normative data for children between 6– 17 years of age. Further, the scores were adjusted to the demographics of the sample and comparisons were made between controls and with children experiencing acute encephalomyelitis and those diagnosed with multiple sclerosis. Although normative data has been collected on cross-cultural groups of adults in the United States (González et al., 2007) and European countries (Amodio et al., 2002; Vogel, Stokholm, & Jørgensen, 2013), limited information is available on cross-cultural pediatric groups. To the authors’ knowledge, there is no normative data on the SDMT for pediatric groups across Central and South America. Nor is there literature as to the demographic variables that influence performance on the test in pediatric populations outside of the United States.

To date, only one major research effort has provided normative data for the SDMT for adults across a range of countries in Latin America (Arango-lasprilla et al., 2015). It is to the authors’ knowledge that no studies have gathered norms for pediatric use of the instrument across this diverse region. Although much work has been done in other regions of the world, to utilize norms from drastically different cultures would serve to promote testing errors that could misrepresent the cognitive profiles of individuals from Latin America. Considering the gap in the literature, the present study gathered SDMT data in nine countries in Latin America, and Spain to better inform clinical practice and research with culturally appropriate norms.

2Method

2.1Participants

The sample consisted of 4,373 healthy children who were recruited from Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, Puerto Rico, and Spain. Participants were selected according to the following criteria: a) were between 6 and 17 years of age, b) were born and currently lived in a country where the study was conducted, c) spoke Spanish as their mother tongue, d) an IQ ≥80 on the Test Of Non-verbal Intelligence (TONI-2, Brown, Sherbenou, & Johnsen, 2009), and e) scored <19 on the Children’s Depression Inventory (CDI, Kovacs, 1992).

Children with history of neurologic or psychiatric disorders, as reported by the participant’s parent(s), were excluded due to its effects on cognitive performance. Participants in the study were from public and private schools, and signed an informed consent. Socio-demographic and participant characteristics for each of the countries’ samples have been reported elsewhere (Rivera & Arango-Lasprilla, 2017). Ethics Committee approval was obtained for the study in each country.

2.2Instrument administration

The SDMT is a neuropsychological test especially useful for exploring processing speed and visual-motor coordination, as well as divided attention and memory (Lezak et al., 2004). It consists of a key of 9 geometric elements, each associated with a number from 1 to 9, with 10 training items and 110 test items. The subject’s score is the number of correct substitutions made at an interval of 90 seconds. The final score corresponds to the number of correct substitutions achieved (Smith, 2002). This study used normative data for the written form.

2.3Statistical analyses

Table 1

Final multiple linear regression models for SDMT score

CountryBStd. ErrortSig.R2SDe (residual)
Chile
  Constant40.4030.87945.976<0.0010.57110.302
  Age3.4410.15522.219<0.001
  Age2– 0.1450.050– 2.8800.004
  MLPE2.3771.1192.1230.034
Cuba
  Constant46.7290.89252.391<0.0010.60011.492
  Age3.9770.17123.261<0.001
  Age2– 0.2690.056– 4.788<0.001
Ecuador
  Constant40.5430.82549.160<0.0010.6029.410
  Age3.3540.15821.222<0.001
  Age2– 0.1610.052– 3.0880.002
Guatemala
  Constant34.3800.73346.933<0.0010.5198.008
  Age3.3350.23714.091<0.001
  Age2– 0.1520.075– 2.0270.044
  MLPE3.3891.4312.3690.019
Honduras
  Constant37.4620.50873.719<0.0010.6218.727
  Age3.4590.15821.933<0.001
Mexico
  Constant38.8620.58965.979<0.0010.6419.314
  Age3.4670.08839.300<0.001
  Age2– 0.2090.029– 7.162<0.001
  MLPE4.7030.6207.586<0.001
Paraguay
  Constant41.2561.29431.876<0.0010.40212.478
  Age2.8510.20813.701<0.001
  Age2– 0.2000.071– 2.8410.005
  Sex– 4.3161.456– 2.9640.003
Peru
  Constant42.8870.97244.130<0.0010.42611.919
  Age2.9130.19914.652<0.001
  Age2– 0.1660.065– 2.5380.012
Puerto Rico
  Constant40.5810.73555.227<0.0010.51210.623
  Age3.0570.20714.759<0.001
Spain
  Constant44.8110.64369.655<0.0010.6508.938
  Age3.6020.08641.911<0.001
  Age2– 0.2410.028– 8.688<0.001
  MLPE2.0130.5983.3670.001
  Sex– 1.1830.574– 2.0600.040

Note. MLPE: Mean level of parental education.

Detailed statistical analyses used to generate the normative data for the SDMT score are described in Rivera and Arango-Lasprilla (2017). In summary, the scores were standardized using multiple linear regression analyses by means of a four-step procedure. First, the SDMT scores were computed by means of the final multiple regression models. The full regression models included as predictors: age, age2, sex, and mean level of parental education (MLPE). Age was centered (= calendar age – mean age in the sample by country) before computing the quadratic age term to avoid multicollinearity (Aiken & West, 1991). Sex was coded as male = 1 and female = 0. The MLPE variable was coded as 1 if the participant’s parent(s) had >12 years of education or 0 if participant’s parent(s) had ≤12 years of education. If predicted variables were not statistically significant in the multivariate model with an alpha of 0.05, the non-significant variables were removed and the model was run again. A final regression model was conducted yˆi=B0+B1·(Age-x¯Agebycountry)i+B2·(Age-x¯Agebycountry)i2+B3·Sexi+B4·MLPEi . 2) Residual scores were calculated based on the final model ( ei=yi-yˆi ). 3) Residuals were standardized using the residual Standard Deviation (SDe) value provided by the regression model: zi = ei/SDe. 4) Standardized residuals were converted to percentile values using the standard normal cumulative distribution function. This four-step process was applied for SDMT scores separately for each country.

For all multiple linear regression models, the following assumptions were evaluated: a) multicollinearity by the values of the Variance Inflation Factor (VIF), which must not exceed 10, and the collinearity tolerance values, which must not exceed the value of 1 (Belsley, Kuh, & Welsch, 1980; Luque-Martínez, 2000), and b) the existence of influential values by calculating the Cook’s distance. The maximum Cook’s distance value was related to a F (p, n - p) distribution. Influential values are considered when percentile value is equal or higher than 50 (Cook, 1977; Kutner, Nachtsheim, Neter, & Li, 2005). All analyzes were performed using SPSS version 23 (IBM Corp., Armonk, NY).

3Results

The final multivariate linear regression models for the ten country-specific SDMT scores were significant (see Table 1). In all countries, the SDMT scores increased linearly as a function of age. The SDMT scores for all countries except for Honduras and Puerto Rico were affected by a quadratic age effect. Children from Chile, Guatemala, Mexico, and Spain whose parent(s) had a MLPE >12 years obtained higher SDMT scores than children whose parent(s) had a MLPE ≤12 years. The child’s sex affected the SDMT scores for Paraguay and Spain, such that girls achieved higher scores than boys. The amount of variance these predictors explained in this test ranged from 40.2% (in Paraguay) to 65.0% (in Spain).

The assumptions of multiple linear regression analysis were met for all final models. There was not multicollinearity (the VIF values were below 10; VIF ≤1.311; collinearity tolerance values did not exceed the value of 1) or influential cases (the maximum Cook’s distance value was 0.222 in a F(3, 299) distribution which correspond to percentile 6).

3.1Normative procedure

Norms (e.g., a percentile score) for the SDMT score by country were established using the four-step procedure described in the statistical analysis section. An example will be provided to facilitate an improved understanding of the procedure used to obtain the percentile associated with a score on this test. Let’s assume we need to find the percentile score for a 10-year-old Paraguayan boy who scored a 42 on the SDMT and whose parent(s) have a MLPE of 13 years. The steps to obtain the percentile for this score are: 1) Find Paraguay in Table 1, which provides the final regression models by country for the SDMT. Use the B weights to create an equation that will allow you to obtain the predicted SDMT score for this child using the coding provided in the statistical analysis section. The corresponding B weights are multiplied by the centered age (= calendar age – mean age in the Paraguayan sample which is equal to 11.6 years), centered age2 (= calendar age – mean age in the Paraguayan sample which is equal to 11.6 years)2, and sex which was coded as male = 1 and female = 0. MLPE was not a significant predictor, and therefore is not included in this model. See Rivera & Arango-Lasprilla (2017) to figure out the mean age of each country’s sample. Then the result is added to the constant generated by the model in order to calculate the predicted value.

In the case of the Paraguayan boy, the predicted SDMT score would be calculated using the following equation: yˆi=41.256+[2.851·(Agei-11.6)]+[-0.200·(Agei-11.6)2]+(-4.316·Sex) . The boy’s age is 10. Sex was coded as male = 1 and female = 0, so in this case as the child is a male, the sex value is 1. Thus, the predicted value equation is: yˆi=41.256+[2.851·(10-11.6)]+[-0.200·(10-11.6)2]+(-4.316·1)=41.256+(-4.562)+(-0.512)+(-4.316)=31.865 . 2). In order to calculate the residual value (indicated with an ei in the equation), we subtract the actual SDMT score (he scored 42) from the predicted value we just calculated ( ei=yi-yˆi ). In this case, it would be ei = 42 - 31.865 = 10.135. 3) Next, consult the SDe column in Table 1 to obtain the country-specific SDe (residual) value. For Paraguay, it is 12.478. Using this value, we can transform the residual value to a standardized z score using the equation zi = ei/SDe. In this case, we have 10.135/12.478 = 0.812. This is the standardized z score for a 10-year-old Paraguayan boy who scored a 42 on the SDMT who has parent(s) with 13 years of education (MLPE). 4) The last step is to use the tables available in most statistical reference books (e.g., Straus et al., 2006) to convert z scores to percentiles. In this example, the z score (probability) of 0.812 corresponds to the 79th percentile.

3.2User-friendly normative data

The four-step normative procedures explained above offer the clinician the ability to determine a percentile for a child who has a specific score on the SDMT. However, this method can be prone to human error due to the number of required computations by hand. To enhance user-friendliness, the authors have completed these steps for a range of raw scores based on age, sex, and MLPE and created tables for clinicians to more easily obtain a percentile range/estimate associated with a given raw score on this test. These tables are available by country in the Appendix. In order to obtain an approximate percentile for the above example (converting a raw score of 42 on the SDMT for a Paraguayan boy who is 10 years old and whose parent(s) have 13 years of education) using the simplified normative tables provided in the Appendix, the following steps must be followed. First, identify the appropriate table ensuring the appropriate country. In this case, the table for the SDMT for boys from Paraguay can be found in Table A7. Next, look in the 10 years’ age column to find the approximate location of the raw score obtained on the test. Within the 10 years’ column, the score of 42 obtained by this Paraguayan boy corresponds to an approximate percentile of 80.

The percentile obtained using this user-friendly table sometimes could be slightly different than the hand-calculated, more accurate method (79th vs. 80th) because the user-friendly table is based on a limited number of percentile values. Individual percentiles cannot be presented in these tables due to space limitations. If the exact score is not listed in the column, you must estimate the percentile value from the list of raw scores available.

4Discussion

The SDMT is one of the most widely used tests for the evaluation of divided attention, motor and perceptual processing speed, and visual scanning. Currently, this test is one of the most used by neuropsychologists in Latin America (Arango-Lasprilla, Stevens, Morlett Paredes, Ardila, & Rivera, 2016) and Spain (Olabarrieta-Landa et al., 2016). However, to date, there are only norms for the adult population in 12 countries from Latin American (Arango-Lasprilla, et al., 2015; Utria Rodríguez et al., 2015) and Spain (Peña Casanova et al., 2009; Tamayo et al., 2012). Therefore, the purpose of the present study was to obtain normative data for the SDMT in children and adolescents from nine Latin American countries (Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, and Puerto Rico) and Spain. The results of the study showed that there are different types of variables associated with the performance of this test such as age, quadratic age, sex and MLPE. In general it was found that final regression models explained between 40.2% and 65.0% of the variance for the SDMT total score.

In this study, it was observed that age was significantly related to the total SDMT score, with the score linearly increasing until the age of 17. However, the majority of studies using the SDMT has been with adult populations (≥18 years) and has found that from this age the total score of the test decreases proportionally as age advances (Arango-Lasprilla et al., 2015; Bowler, Sudia, Mergler, Harrison, & Cone, 1992; Feinstein, Brown, & Ron, 1994; Gilmore, Royer, & Gruhn, 1983; Richardson & Marottoli, 1996; Selnes et al., 1991).

Similarly, a curvilinear relationship between age and the total score of the SDMT was observed in all countries, except for Puerto Rico and Honduras. In general, a relative increase in the total test score between the ages 6 and 14 was observed, and from the age of 15 a less evident increase. For example, in the case of Chile, the SDMT scores of a 7-year-old child was approximately 25.2% higher than the test score of a 6-year-old child, while the test score of a 17-year-old child was only 3.6% higher than the score of a 16-year-old child. In the case of Puerto Rico and Honduras, the SDMT score only increased linearly as a function of age at a rate of approximately 14.1% to 5.9% and 17.8% to 6.4% units per year respectively, from the ages of 6 to 17 respectively.

In this study, the parents’ MLPE was associated with the total score of the SDMT in countries such as Chile, Guatemala, Mexico, and Spain, where children whose parents had >12 years of education had a higher total score in the SDMT than children whose parents had <12 years of education. To date, none of the normative studies in children have studied the influence of parental education on SDMT performance.

Smith (1991) reported that gender affects performance, with girls outperforming boys. This has also been demonstrated in non-Western children (Jinabhai et al., 2004). In general, both boys and girls show consistently higher oral than written scores for ages 8 to 13 years. However, the gender-based differences between written and oral scores diminish as age increases, particularly from ages 14 to 17 years. In the present study sex was not associated with test performance in any of the countries, except for Paraguay and Spain where girls presented a significantly higher score than boys.

5Implications

The SDMT is one of the most sensitive tests to brain injury in both adults and children (Strauss et al., 2006). For this reason neuropsychologists often include it in neuropsychological batteries to assess attention problems, speed of motor and perceptual processing in children with learning disabilities (Collins et al., 1999), traumatic brain injury (Kinsella et al., 1995), attention and hyperactivity disorder (Katz, Brown, Roth, & Beers, 2011), among others.

Despite its wide use, normative data for Spanish-speakers are focused on adult populations, with a lack of norms for children and adolescent populations. This is a great diagnostic limitation for professionals. Therefore, the norms presented in this study for ten Spanish-speaking countries will contribute to the improvement of neuropsychological clinical practice in these countries. From now on, professionals will be able to evaluate and diagnose their patients between the ages 6 and 17 years using norms adapted to the characteristics of children in their country.

6Limitations

Although this is the largest normative study in the world that has developed evidence for validation and standardization of the SDMT in Spanish speaking populations, a number of limitations exist. First, this study presents normative data for the SDMT for nine countries from Latin America and Spain. For this reason, it is not advisable to use these norms in the pediatric population of those Spanish-speaking countries where the study was not performed. Future studies should be conducted to standardize this test in other Spanish-speaking countries.

Although the norms of the present study could be used by neuropsychologists in other countries to evaluate Spanish-speaking immigrant children from the countries where the sample was collected for this study, they should be used with caution since other variables such as level of acculturation, bilingualism, the number of years living in the country, and so on, could influence performance. In addition, the quality of education of both the child and the parent(s) is another aspect that may influence the cognitive performance of children.

It is very important to keep in mind that no clinical diagnosis should be made based solely on the score of this test. This test should be integrated as part of a much larger battery that evaluates these processes in more detail. Because there are a limited number of tests and norms in Latin America and Spain to evaluate these processes, more efforts should be made in the future to have other similar tools.

Although the size of the sample was adequate in each of the countries where the study was conducted, it is very important to note that only the samples in Chile, Mexico, Paraguay, Puerto Rico, and Spain were obtained from several regions of the country, while in the remaining countries were collected from only one geographic area. Future studies should expand the sample in other geographical areas of these countries with the objective to be able to have a greater representativeness of the sample.

The children who participated in the present study had Spanish as their primary language. Although Spanish is the first language of the majority of the population in Latin America and Spain, it is important to note that the first language of many children may be different from Spanish (e.g., Euskera, Catalan, Guaraní, Maya, Quechua) due to cultural and linguistic diversity within the country. For this reason, caution should be used when using these norms in children whose first language is not Spanish.

Finally, it is important to keep in mind that the present study was performed with normal healthy population. Therefore, future studies should be performed with clinical population to establish the sensitivity and specificity of this test.

7Conclusions

The SDMT is one of the most widely used instruments in the world to measure divided attention, motor speed and perceptual processing, and visual scanning. Its high sensitivity to brain injury in both adults and children has made it an essential tool in any assessment protocol. Despite its wide use, normative data for Spanish-speaking children are limited. To date, this is the first study to develop norms for the SDMT in a pediatric population from nine Latin-Americans countries and Spain that include appropriate age, sex and MLPE-adjustments. This allows clinicians to more accurately interpret their performance and enhance diagnosis of their pediatric patients.

Conflict of interest

None to report.

Supplementary material

[1] The Appendix tables are available in the electronic version of this article: http://dx.doi.org/10.3233/NRE-172243.

References

1 

Aiken L. S. , & West S. G. ((1991) ), Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage.

2 

Amodio P. , Wenin H. , Del Piccolo F. , Mapelli D. , Montagnese S. , Pellegrini A. , & Umiltá C. ((2002) ). Variability of trail making test, symbol digit test and line trait test in normal people. A normative study taking into account age-dependent decline and sociobiological variables. Aging clinical and experimental research, 14: (2), 117–131.

3 

Arango-Lasprilla J. C. , Rivera D. , Rodríguez G. , Garza M. T. , Galarza-del-Angel J. , Rodriguez W. , & Longoni M. ((2015) ). Symbol Digit Modalities Test: Normative data for the Latin American Spanish speaking adult population. NeuroRehabilitation, 37: (4), 625–638.

4 

Arango-Lasprilla J. C. , Stevens L. , Morlett Paredes A. , Ardila A. , & Rivera D. ((2016) ). Profession of neuropsychology in Latin America. Applied Neuropsychology: Adult, 24: (4), 318–330. doi: 10.1080/23279095.2016.1185423.

5 

Babikian T. , Satz P. , Zaucha K. , Light R. , Lewis R. S. , & Asarnow R. F. ((2011) ). The UCLA longitudinal study of neurocognitive outcomes following mild pediatric traumatic brain injury. Journal of the International Neuropsychological Society, 17: (05), 886–895.

6 

Benedict R. H. B. , Duquin J. A. , Jurgensen S. , Rudick R. A. , Feitcher J. , Munschauer F. E. , & Weinstock-Guttman B. ((2008) ). Repeated assessment of neuropsychological deficits in multiple sclerosis using the Symbol Digit Modalities Test and the MS Neuropsychological Screening Questionnaire. Multiple sclerosis, 14: (7), 940–946.

7 

Berrigan L. I. , Fisk J. D. , Walker L. A. , Wojtowicz M. , Rees L. M. , Freedman M. S. , & Marrie R. A. ((2014) ). Reliability of regression-based normative data for the oral symbol digit modalities test: An evaluation of demographic influences, construct validity, and impairment classification rates in multiple sclerosis samples. The Clinical Neuropsychologist, 28: (2), 281–299.

8 

Bowler R. , Sudia S. , Mergler D. , Harrison R. , & Cone J. ((1992) ). Comparison of digit symbol and symbol digit modalities tests for assessing neurotoxic exposure. The Clinical Neuropsychologist, 6: (1), 103–104.

9 

Brown L. , Sherbenou R. J. , & Johnsen S. K. ((2009) ), Test de inteligencia no verbal TONI-2. Madrid: TEA ediciones.

10 

Charvet L. E. , Beekman R. , Amadiume N. , Belman A. L. , & Krupp L. B. ((2014) ). The Symbol Digit Modalities Test is an effective cognitive screen in pediatric onset multiple sclerosis (MS. Journal of the neurological sciences, 341: (1), 79–84.

11 

Collins M. W. , Grindel S. H. , Lovell M. R. , Dede D. E. , Moser D. J. , Phalin B. R. , & Sears S. F. ((1999) ). Relationship between concussion and neuropsychological performance in college football players. Jama, 282: (10), 964–970.

12 

Cook R. D. ((1977) ). Detection of influential observation in linear regression. Technometrics, 19: (1), 15–18. doi: 10.2307/1268249.

13 

Drake A. S. , Weinstock-Guttman B. , Morrow S. A. , Hojnacki D. , Munschauer F. N. , & Benedict R. H. B. ((2009) ). Psychometrics and normative data for the Multiple Sclerosis Functional Composite: Replacing the PASAT with the Symbol Digit Modalities Test. Multiple Sclerosis Journal, 16: (2), 228–237.

14 

Feinstein A. , Brown R. , & Ron M. ((1994) ). Effects of practice of serial tests of attention in healthy subjects. Journal of Clinical and Experimental Neuropsychology, 16: (3), 436–447.

15 

Gilmore G. C. , Royer F. L. , & Gruhn J. J. ((1983) ). Age differences in symbol-digit substitution task performance. Journal of clinical psychology, 39: (1), 114–124.

16 

González H. M. , Whitfield K. E. , West B. T. , Williams D. R. , Lichtenberg P. A. , & Jackson J. S. ((2007) ). Modified-symbol digit modalities test for African Americans, Caribbean Black Americans, and non-Latino Whites: Nationally representative normative data from the National Survey of American Life. Archives of Clinical Neuropsychology, 22: (5), 605–613.

17 

Katz L. J. , Brown F. C. , Roth R. M. , & Beers S. R. ((2011) ). Processing speed and working memory performance in those with both ADHD and a reading disorder compared with those with ADHD alone. Archives of Clinical Neuropsychology, acr026.

18 

Kinsella G. , Prior M. , Sawyer M. , Murtagh D. , Eisenmajer R. , Anderson V. , & Klug G. ((1995) ). Neuropsychological deficit and academic performance in children and adolescents following traumatic brain injury. Journal of Pediatric Psychology, 20: (6), 753–767.

19 

Kovacs M. ((1992) ), Children’s depression inventory. North Tonawanda, NY: Multi-Health System.

20 

Kutner M. H. , Nachtsheim C. J. , Neter J. , & Li W. ((2005) ), Applied linear statistical models (5th ed.). New York: McGraw Hill.

21 

Lai J. S. , Nowinski C. J. , Zelko F. , Wortman K. , Burns J. , Nordli D. R. , & Cella D. ((2015) ). Validation of the Neuro-QoL measurement system in children with epilepsy. Epilepsy & Behavior, 46: , 209–214.

22 

Langdon D. W. , Amato M. P. , Boringa J. , Brochet B. , Foley F. , Fredrikson S. , & Reder A. T. ((2012) ). Recommendations for a brief international cognitive assessment for multiple sclerosis (BICAMS. Multiple Sclerosis Journal, 18: (6), 891–898.

23 

Lezak M. , Howieson D. , & Loring D. ((2004) ), Neuropsychological Assessment (4th ed.). New York: Oxford University Press.

24 

Newman J. B. , Reesman J. H. , Vaughan C. G. , & Gioia G. A. ((2013) ). Assessment of processing speed in children with mild TBI: A “first look” at the validity of pediatric ImPACT. The Clinical Neuropsychologist, 27: (5), 779–793.

25 

Olabarrieta-Landa L. , Caracuel A. , Pérez-García M. , Panyavin I. , Morlett-Paredes A. , & Arango-Lasprilla J. C. ((2016) ). The profession of neuropsychology in Spain: Results of a national survey. Clinical Neuropsychology, 30: (8), 1335–1355.

26 

Peña-Casanova J. , Quiñones-Úbeda S. , Quintana-Aparicio M. , Aguilar M. , Badenes D. , Molinuevo J. L. , & Antúnez C. ((2009) ). Spanish Multicenter Normative Studies (NEURONORMA Project): Norms for verbal span, visuospatial span, letter and number sequencing, trail making test, and symbol digit modalities test. Archives of Clinical Neuropsychology, 24: (4), 321–341.

27 

Portaccio E. , Goretti B. , Lori S. , Zipoli V. , Centorrino S. , Ghezzi A. , & Amato M. P. ((2009) ). The brief neuropsychological battery for children: A screening tool for cognitive impairment in childhood and juvenile multiple sclerosis. Multiple Sclerosis, 15: , 620–626.

28 

Richardson E. D. , & Marottoli R. A. ((1996) ). Education-Specific normative data on common neuropsychological indices for individuals older than 75 Years. The Clinical Neuropsychologist, 10: (4), 375–381.

29 

Rivera D. , & Arango-Lasprilla J. C. ((2017) ). Methodology for the development of normative data for Spanish Speaking pediatric population. NeuroRehabilitation, 41: (3), 581–592.

30 

Sands S. A. , Zhou T. , O’Neil S. H. , Patel S. K. , Allen J. , Cullen P. M. , & Finlay J. L. ((2012) ). Long-term follow-up of children treated for high-grade gliomas: Children’s oncology group L991 final study report. Journal of Clinical Oncology, 30: (9), 943–949.

31 

Selnes O. A. , Jacobson L. , Machado A. M. , Becker J. T. , Wesch J. , Miller E. N. , & McArthur J. C. ((1991) ). Normative data for a brief neuropsychological screening battery. Perceptual and Motor Skills, 73: (2), 539–550.

32 

Smerbeck A. M. , Parrish J. , Yeh E. A. , Hoogs M. , Krupp L. B. , Weinstock-Guttman B. , & Benedict R. H. B. ((2011) ). Regression-based pediatric norms for the brief visuospatial memory test– Revised and the symbol digit modalities test. The Clinical Neuropsychologist, 25: (3), 402–412.

33 

Smith A. ((1968) ). The symbol-digit modalities test: A neuropsychologic test of learning and other cerebral disorders. Learning Disorders. Edited by: Helmuth J.

34 

Smith A. ((1982) ). Symbol digit modalities test (SDMT) manual (revised). Western Psychological Services. Los Angeles.

35 

Smith A. ((2002) ). Manual de test de s&mbolos y dígitos SDMT. Publicaciones de psicología aplicada. TEA ediciones .

36 

Smith A. ((2013) ), Symbol digit modalities test: Manual.

37 

Strauss E. , Sherman E. M. , & Spreen O. ((2006) ), A compendium of neuropsychological tests: Administration, norms, and commentary. American Chemical Society.

38 

Tamayo F. , Casals-Coll M. , Sánchez-Benavides G. , Quintana M. , Manero R. M. , Rognoni T. , & Peña-Casanova J. ((2012) ). Estudios normativos españoles en población adulta joven (Proyecto NEURONORMA jóvenes): Normas para las pruebas span verbal, span visuoespacial, Letter-Number Sequencing, Trail Making Test y Symbol Digit Modalities Test. Neurologia, 27: (6), 319–329.

39 

Uchiyama C. L. , D’elia L. F. , Dellinger A. M. , Seines O. A. , Becker J. T. , Wesch J. E. , & Miller E. N. ((1994) ). Longitudinal comparison of alternate versions of the Symbol Digit Modalities Test: Issues of form comparability and moderating demographic variables. The Clinical Neuropsychologist, 8: (2), 209–218.

40 

Utria Rodríguez O. E. , Olivera Plaza S. L. , Rivera D. , Quijano M. C. , Chacón Peralta H. , Rodríguez Días M. A. , Medina Salcedo J. M. , Olabarrieta Landa L. , Perrin P. B. , & Arango-Lasprilla J. C. ((2015) ). Datos Normativos del Test de Símbolos y Dígitos para población Colombiana [Normative data for Symbol Digit Test for Colombian population]. In Arango-Lasprilla, J. C., & Rivera, D. (Eds.), Neuropsicología en Colombia: Datos normativos, estado actual y retos a futuro Neuropsychology in Colombia: Normative data, current status and future challenges(pg. 223–238). Manizales, Colombia: Editorial Universidad Autónoma de Manizales.

41 

Vogel A. , Stokholm J. , & Jørgensen K. ((2013) ). Performances on Symbol Digit Modalities Test, Color Trails Test, and modified Stroop test in a healthy, elderly Danish sample. Aging, Neuropsychology, and Cognition, 20: (3), 370–382.