You are viewing a javascript disabled version of the site. Please enable Javascript for this site to function properly.
Go to headerGo to navigationGo to searchGo to contentsGo to footer
In content section. Select this link to jump to navigation

Verbal fluency tests: Normative data for Spanish-speaking pediatric population

Abstract

OBJECTIVE:

To generate normative data for the phonological and semantic verbal fluency tests (VFT) in Spanish-speaking pediatric populations.

METHOD:

The sample consisted of 4,373 healthy children from nine countries in Latin America (Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, and Puerto Rico) and Spain. Each participant was administered the VFT as part of a larger neuropsychological battery. Scores for letters F, A, S, and animals and fruit categories were normed using multiple linear regressions and standard deviations of residual values. Age, age2, sex, and mean level of parental education (MLPE) were included as predictors in the analyses.

RESULTS:

The final multiple linear regression models showed main effects for age on all scores, such that scores increased linearly as a function of age. Age2 had a significant effect in Chile (animals), Cuba (A letter, fruits), Ecuador (animals, fruits), Honduras (F letter), Mexico (animals, fruits), Peru (fruits), and Spain (S letters, animals, fruits). Models showed an effect for MLPE in Chile (A letters, animals, fruits), Ecuador (S letter, animals, fruits), Guatelama (F, S letter, animals), Honduras (animals), Mexico (F, A, S letters, animals, fruits), Puerto Rico (A, letters, animals), and Spain (all scores). Sex scores were found significant in Chile (animals), Ecuador (A letter, fruits), Mexico (F letter, fruits), Paraguay (F, A, S letters, fruits), Puerto Rico (F letter, animals, fruits), and Spain (F letter, fruits).

CONCLUSIONS:

This is the largest multi-national Spanish speaking-pediatric normative study in the world, and as such it will allow neuropsychologists from these countries to have a more accurate way to interpret the phonological and semantic VFT in pediatric populations.

1Introduction

Language is one of the most essential tools of humanity. Without the capacity to communicate with language, people would lack mutual understanding. The ability to use language allows for learning, societal integration, and the communication of thoughts, emotions, and needs. Past studies have demonstrated that language problems during childhood can lead to lasting repercussions later in life. For example, adolescents with a history of language problems throughout their childhood educational experiences have been more likely to have current attentional problems and social difficulties (Snowling, Bishop, Stothard, Chipchase, & Kaplan, 2006).

Several studies have estimated the prevalence of communication disorders in children to be between 2% and 25% (Law, Boyle, Harris, Harkness, & Nye, 2000; McLeod & Harrison, 2009; Tomblin et al., 1997). Based on these prevalence rates alone, it is clear that communication disorders are a relatively common problem of childhood that must be addressed. To treat these disorders, first it is essential to develop high-quality cognitive assessments. One example of these assessments includes verbal fluency tests (VFT), which have been used to measure individuals’ capacity for complex cognitive functioning, such as language and executive functioning (Strauss, Sherman, & Spreen, 2006). To perform well on these tests, an individual must be able to flexibly organize information and inhibit incorrect responses. Inability to perform in these tests has been linked to frontal lobe impairment. Due to the sensitivity of VFT to rapidly and accurately detect brain dysfunction, as well as its ease of administration has made VFT one of the most widely used tests in both clinical work and research (Azuma, 2004). VFT have been used to detect cognitive impairment in a wide range of populations such as children and adolescents with traumatic brain injury (Goldstrohm & Arffa, 2005); autism spectrum disorder (Begeer et al., 2013); attention deficit hyperactivity disorder (Rubiales, Baker, & Russo, 2013); schizophrenia (Landrø & Ueland, 2008); and children with communication difficulties (Cohen, Morgan, Vaughn, Riccio, & Hall, 1999; Henry, Messer, & Nash, 2015), among others.

The Thurstone Word Fluency Test (TWFT; Thurstone & Thurstone, 1962) was the first version of VFT. Due to its disadvantage of having to write the word, Benton developed an oral version called the Controlled Verbal Fluency Task (CVFT), which utilized the letters F, A, and S (Borkowski, Benton, & Spreen, 1967). Subsequently, the CVFT was revised and renamed as Controlled Oral Word Association Test (COWAT; Benton & Hamsher, 1989). It uses letters that were chosen according to an analysis of the difficulty of English letters (Mitrushina, Boone, Razani, & D’Elia, 2005). Currently, many different tests exist to measure verbal fluency; although, the most utilized among children are phonological and semantic VFT. Participants completing VFT are asked to generate as many words as they can that begin with a given letter (phonological VFT) or belonging to a particular category (semantic VFT).

Several variables have been associated with performance on VFT, but in the case of children and adolescents, the most significant variable is age (Beltrán Dulcey & Solís-Uribe, 2012; García et al., 2012; Lozano & Ostrosky-Solís, 2006; Malloy-Diniz et al., 2007; Martins, Mograbi, Andrade Gabrig, & Charchat-Fichman, 2016; Nieto, Galtier, Barroso, & Espinosa, 2008; Prigatano, Gray, & Lomay, 2008; Ruffieux et al., 2009; Tallberg, Carlsson, & Lieberman, 2011; Van der Elst, Hurks, Wassenberg, Meijs, & Jolles, 2011). Past research has consistently found that older children perform better on VFT. Other studies have found that the level of education of children’s parents also exerts an influence on their performance (Van der Elst et al., 2011). Taken together, both age and parents’ education level are extremely important variables to consider when assessing verbal fluency among children.

In pediatric neuropsychological assessment, it is also vital to consider the development of cognitive processes underlying neuropsychological tests. A healthy adult can produce around 12 and 16 words per minute in phonological and semantic VFT, respectively (Spreen & Strauss, 1998). As mentioned previously performance on both phonological and semantic tasks increases with age among young children (García et al., 2012; Matute, Rosselli, Ardila, & Morales, 2004; Riva, Nichelli, & Devoti, 2000; Van der Elst et al., 2011). Children between the ages of 6 and 8 produce between 3 and 12 words in phonological VFT, whereas children between 6 and 12 years can generate as many as 10 to 15 animals in a semantic VFT (Matute et al., 2004).

Phonological VFT are more difficult than semantic VFT for children, as well as among adults. In general, phonological VFT require more elaborate strategy and organization than semantic VFT (Riva et al., 2000), contributing to these differences in difficulty. However, neurodevelopmental reasons also explain differences in difficulty in performance of phonological VFT for children. Difficulty has been attributed to delayed maturational development of the frontal lobes, which are areas of the brain that control the executive functioning necessary for performing well on phonological VFT (Klenberg, Korkman, & Lahti-Nuuttila, 2001; Korkman, Kemp, & Kirk, 2001). Children between 14 and 15 years of age tend to perform as well as adults on semantic VFT; however, performance on phonological VFT do not match adults until children are older (Matute et al., 2004). Because of the difficulty children experience when completing phonological VFT, most research with children and adolescents only use semantic VFT.

Because VFT are widely used and highly useful, researchers have strived to provide normative data for children and adolescents between the ages of 6 and 20 years in countries such as Brazil (Malloy-Diniz et al., 2007; Martins et al., 2016), United States of America (Prigatano et al. 2008), Cameroon (Ruffieux et al., 2009), Sweden (Tallberg et al., 2011), and Holland (Van der Elst et al., 2011). Among Spanish-speaking countries, normative data for children has been collected in Spain (García et al., 2012; Nieto et al., 2008), Colombia (Beltrán Dulcey & Solís-Uribe, 2012), Argentina (Butman, Allegri, Harris, & Drake, 2000; Marino & Alderete, 2010), and Mexico (Lozano & Ostrosky-Solís, 2006).

Most of normative data studies focus solely on the semantic VFT as described above, using categories such as animals (Butman et al., 2000; García et al., 2012; Lozano & Ostrosky-Solís, 2006; Malloy-Diniz et al., 2007; Marino & Alderete, 2010; Martins et al., 2016; Nieto et al., 2008; Prigatano et al., 2008; Ruffieux et al., 2009; Tallberg et al., 2011; Van der Elst et al., 2011), fruits (Martins et al., 2016), clothing (Marino & Alderete, 2010; Martins et al., 2016), tools (Marino & Alderete, 2010), parts of the body (Malloy-Diniz et al., 2007), and food (Malloy-Diniz et al., 2007), all of which are appropriate for children. Among studies that have employed the phonological VFT, most opted for the classic letters (i.e., F, A, and S), although in some cases, especially in Spanish-speaking samples, letters such as P and M have been used (Butman et al., 2000; García et al., 2012; Marino & Alderete, 2010; Martins et al., 2016; Nieto et al., 2008). Similarly, some researchers have made other minor modifications. For example, Beltrán Dulcey and Solís-Uribe (2012) used the phonemes /f/, /a/, and /s/ instead of their respective letters. Furthermore, Nieto and colleagues (2008) decided to avoid using the letter S due to the “seseo” phenomenon present in some parts of Spain and Latin America that would complicate the task for children since the letters S, Z, and C share the same phoneme, /S/. However, Tallberg and colleagues (2011) hypothesized that children younger than 6 years old would be unable to complete the phonological VFT, because they generally do not know how to read, and thus would not be able to write the letters. Results indicated that the children could complete the task because they were guided by the sound of the letter (Tallberg et al., 2011).

In conclusion, few studies offer normative data for child and adolescent in Spanish-speaking populations. Moreover, the aforementioned studies have several limitations: 1) small sample sizes; 2) reduced age range among participants, notably failing to include school-age children for whom language problems may be detected easier; 3) applying normative data based on averages and standard deviations; and 4) local samples that are not representative of the country at large. On the other hand, there are still many countries for which normative data does not exist, including Chile, Cuba, Ecuador, Guatemala, Honduras, Paraguay, Peru, and Puerto Rico. To fill this gap in the literature, the objective of the present study is to obtain and present normative data for a population of children and adolescents from nine Latin American countries (Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, and Puerto Rico) and Spain.

2Method

2.1Participants

The sample consisted of 4,373 healthy children who were recruited from Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, Puerto Rico, and Spain. Participants were selected according to the following criteria: a) were between 6 and 17 years of age, b) were born and currently lived in the country where the study was conducted, c) Spanish as primary language, d) an IQ ≥80 on the Test of Non-Verbal Intelligence (TONI-2, Brown, Sherbenou, & Johnsen, 2009), and e) a score <19 on the Children’s Depression Inventory (CDI, Kovacs, 1992).

Children with history of neurologic or psychiatric disorders as reported by the participant’s parent(s), were excluded due to its effects on cognitive performance. Participants in the study were from public or private schools, and they signed an informed consent to participate. Socio-demographic and participant characteristics for each of the countries’ samples have been reported elsewhere (Rivera & Arango-Lasprilla, 2017). Ethics Committee approval was obtained for the study in each country.

2.2Instrument administration

VFT are commonly used in clinical practice and research for their sensitivity to brain damage. There are different modalities of VFT, but the two more commontly used are the phonological and semantic VFT. In phonological VFT, the participant is required to produce in 60 seconds as many words as s/he can beginning with certain letters (in this study F, A, and S). However, in semantic VFT, the participant is required to produce in 60 seconds as many words as s/he can belonging to a particular category (in this study, animals and fruits). In this process, cognitive flexibility, information organization, and inhibitory capacity are activated, so poor performance is indicative of dysfunction of the frontal lobe, specifically the left frontal cortex (Gouveia, Brucki, Malheiros, & Bueno, 2007). For administration and scoring, Olabarrieta-Landa, Landa-Torre, López-Mugartza, Bialystok, and Arango-Lasprilla (2017) guidelines were followed.

Table 1

Final multiple linear regression models for Letter F

CountryBStd. ErrortSig.R2SDe (residual)
Chile
  Constant6.8270.14547.015<0.0010.4172.853
  Age0.6980.04216.607<0.001
Cuba
  Constant8.5030.16651.153<0.0010.3103.240
  Age0.6280.04813.045<0.001
Ecuador
  Constant7.2430.14848.864<0.0010.3742.572
  Age0.5760.04313.388<0.001
Guatemala
  Constant6.7960.23229.246<0.0010.2602.836
  Age0.6220.0817.719<0.001
  MLPE1.0230.4622.2130.028
Honduras
  Constant6.3880.22428.579<0.0010.4222.646
  Age0.6750.04913.906<0.001
  Age20.0340.0152.1950.029
Mexico
  Constant6.9480.16242.992<0.0010.4202.843
  Age0.6850.02725.432<0.001
  MLPE0.8240.1884.376<0.001
  Sex– 0.4040.188– 2.1480.032
Paraguay
  Constant8.2300.24932.994<0.0010.3663.152
  Age0.6630.05212.636<0.001
  Sex– 1.5620.367– 4.256<0.001
Peru
  Constant8.4530.18346.105<0.0010.3843.381
  Age0.8020.05514.546<0.001
Puerto Rico
  Constant7.1510.26427.089<0.0010.2702.814
  Age0.4750.0568.533<0.001
  Sex– 0.8590.397– 2.1660.031
Spain
  Constant6.6930.17937.293<0.0010.3922.871
  Age0.6730.02824.465<0.001
  MLPE0.9630.1925.015<0.001
  Sex– 0.6200.184– 3.3650.001

Note. MLPE: Mean level of parental education.

2.3Statistical analyses

Detailed statistical analyses used to generate the normative data for the VFT scores are described in Rivera & Arango-Lasprilla (2017). In summary, the scores were standardized using multiple linear regression analyses by means of a four-step procedure.

1) The letters F, A, S and the categories of animals and fruits scores were computed separately by means of the final multiple regression models. The full regression models included as predictors: age, age2, sex, and mean level of parental education (MLPE). Age was centered (= calendar age – mean age in the sample by country) before computing the quadratic age term to avoid multicollinearity (Aiken & West, 1991). Sex was coded as male = 1 and female = 0. The MLPE variable was coded as 1 if the participant’s parent(s) had >12 years of education or 0 if participant’s parent(s) had ≤12 years of education. If predicted variables were not statistically significant in the multivariate model with an alpha of 0.05, the non-significant variables were removed and the model was run again. A final regression model was conducted: yˆi=B0+B1·(Age-x¯Agebycountry)i+B2·(Age-x¯Agebycountry)i2+B3·Sexi+B4·MLPEi . 2) Residual scores were calculated based on the final model ( ei=yi-yˆi ). 3) Residuals were standardized using the residual Standard Deviation (SDe) value provided by the regression model: zi = ei/SDe. 4) Standardized residuals were converted to percentile values using the standard normal cumulative distribution function. 4) Standardized residuals were converted to percentile values using the standard normal cumulative distribution function. This four-step process was applied to letters F, A, S and the categories of animals and fruits scores separately for each country.

For all multiple linear regression models, the following assumptions were evaluated: a) multicollinearity by the values of the Variance Inflation Factor (VIF), which must not exceed 10, and the collinearity tolerance values, which must not exceed the value of 1 (Kutner, Nachtsheim, Neter, & Li, 2005), and b) the existence of influential values by calculating the Cook’s distance. The maximum Cook’s distance value was related to a F (p, n - p) distribution. Influential values are considered when percentile value is equal or higher than 50 (Cook, 1977; Kutner et al., 2005). All analyzes were performed using SPSS version 23 (IBM Corp., Armonk, NY).

Table 2

Final multiple linear regression models for Letter A

BStd. ErrorTSig.R2SDe (residual)
Chile
  Constant7.6530.16646.072<0.0010.4932.604
  Age0.7490.03919.168<0.001
  MLPE0.5670.2822.0100.045
Cuba
  Constant7.6740.21635.504<0.0010.4132.785
  Age0.6610.04115.944<0.001
  Age20.0490.0143.627<0.001
Ecuador
  Constant9.1550.21542.556<0.0010.4492.836
  Age0.7280.04815.303<0.001
  Sex– 0.8080.332– 2.4350.015
Guatemala
  Constant7.8170.20637.955<0.0010.2302.927
  Age0.6370.0827.749<0.001
Honduras
  Constant7.6520.15948.064<0.0010.4182.730
  Age0.7160.04914.499<0.001
Mexico
  Constant7.6310.14154.124<0.0010.4102.976
  Age0.7030.02824.966<0.001
  MLPE0.9610.1974.878<0.001
Paraguay
  Constant9.3550.26535.261<0.0010.3303.353
  Age0.6670.05611.954<0.001
  Sex– 1.0340.390– 2.6480.009
Peru
  Constant9.2940.18250.941<0.0010.3833.364
  Age0.7940.05514.509<0.001
Puerto Rico
  Constant6.7940.37118.301<0.0010.4202.658
  Age0.6180.05411.369<0.001
  MLPE1.6090.4353.697<0.001
Spain
  Constant7.5210.16047.002<0.0010.3923.001
  Age0.7140.02924.862<0.001
  MLPE0.9150.2004.568<0.001

Note. MLPE: Mean level of parental education.

3Results

3.1Phonological fluency – letter F

The final multivariate linear regression models for the ten country-specific letter F score were significant (Table 1). In all countries, the letter F score increased linearly as a function of age. The letter F score for Honduras was also affected by a quadratic age effect. Children from Guatemala, Mexico and Spain who had parent(s) with a MLPE >12 years obtained higher letter F score than children whose parent(s) had a MLPE ≤12 years. The child’s sex affected letter F score for Mexico, Paraguay, Puerto Rico, and Spain, such that girls scored higher than boys. The amount of variance these predictors explained in the letter F score ranged from 26.0% (Guatemala) to 42.2% (Honduras).

3.2Phonological fluency – letter A

The final multivariate linear regression models for the ten country-specific letter A score were significant (Table 2). In all countries, the letter A score increased linearly as a function of age. The letter A score for Cuba was also affected by a quadratic age effect. Children from Chile, Mexico, Puerto Rico, and Spain who had parent(s) with a MLPE >12 years obtained higher letter A score than children who had parent(s) with a MLPE ≤12 years. The child’s sex affected letter A score for Ecuador and Paraguay such that girls scored higher than boys. The amount of variance these predictors explained in the letter A score ranged from 23.0% (Guatemala) to 49.3% (Chile).

3.3Phonological fluency – letter S

The final multivariate linear regression models for the ten country-specific letter S score were significant (Table 3). In all countries, the letter S score increased linearly as a function of age. The S letter score for Spain was also affected by a quadratic age effect. Children from Ecuador, Guatemala, Mexico, and Spain who had parent(s) with a MLPE >12 years obtained higher letter S score than children who had parent(s) with a MLPE ≤12 years. The child’s sex affected letter S score for Paraguay, such that girls scored higher than boys. The amount of variance these predictors explained in the letter S score ranged from 22.9% (Guatemala) to 44.4% (Honduras).

Table 3

Final multiple linear regression models for Letter S

BStd. ErrortSig.R2SDe (residual)
Chile
  Constant7.7180.15151.068<0.0010.4042.969
  Age0.7080.04416.166<0.001
Cuba
  Constant7.9880.15850.606<0.0010.3423.077
  Age0.6420.04614.048<0.001
Ecuador
  Constant7.3610.32822.414<0.0010.4232.871
  Age0.7070.04914.544<0.001
  MLPE0.9380.3822.4590.015
Guatemala
  Constant7.3480.24629.817<0.0010.2293.007
  Age0.5630.0856.595<0.001
  MLPE1.5230.4903.1080.002
Honduras
  Constant7.3350.16045.940<0.0010.4442.732
  Age0.7550.04915.261<0.001
Mexico
  Constant7.5810.14950.897<0.0010.3703.144
  Age0.6810.03022.888<0.001
  MLPE1.0340.2084.970<0.001
Paraguay
  Constant9.0760.27133.484<0.0010.3043.425
  Age0.6380.05711.191<0.001
  Sex– 1.1420.399– 2.8630.004
Peru
  Constant8.6600.17749.061<0.0010.3503.254
  Age0.7150.05313.501<0.001
Puerto Rico
  Constant7.4600.21135.353<0.0010.2763.029
  Age0.5270.0608.850<0.001
Spain
  Constant7.9600.19940.032<0.0010.4113.110
  Age0.7630.03025.551<0.001
  Age2– 0.0250.010– 2.5690.010
  MLPE1.3330.2086.416<0.001

Note. MLPE: Mean level of parental education.

3.4Semantic fluency – animals

The final multivariate linear regression models for the ten country-specific animals’ category score were significant (Table 4). In all countries, the animal’s category score increased linearly as a function of age. The animals’ category score for Chile, Ecuador, Mexico, and Spain was also affected by a quadratic age effect. Children from Chile, Ecuador, Guatelama, Honduras, Mexico, Puerto Rico, and Spain who had parent(s) with a MLPE >12 years obtained higher animals’ category score than children who had parent(s) with a MLPE ≤12 years. The child’s sex affected animals’ category score for Chile and Puerto Rico. In Chile, boys achieved higher scores than girls; however, in Puerto Rico, females achieved higher scores than males. The amount of variance these predictors explained in the animals’ category score ranged from 24.3% (Paraguay) to 46.6% (Cuba).

Table 4

Final multiple linear regression models for Animals’ category

BStd. ErrortSig.R2SDe (residual)
Chile
  Constant13.7340.32841.902<0.0010.4383.368
  Age0.8490.05116.746<0.001
  Age2– 0.0370.016– 2.2520.025
  MLPE1.1230.3703.0360.003
  Sex0.7510.3502.1430.033
Cuba
  Constant12.7810.19266.497<0.0010.4663.747
  Age1.0110.05618.169<0.001
Ecuador
  Constant14.8800.49729.968<0.0010.4183.821
  Age0.9080.06513.992<0.001
  Age2-0.0680.022– 3.1540.002
  MLPE1.7910.5093.5150.001
Guatemala
  Constant14.4030.29249.348<0.0010.3403.561
  Age0.8950.1018.852<0.001
  MLPE2.1840.5813.762<0.001
Honduras
  Constant14.0770.30745.915<0.0010.3713.586
  Age0.8250.06512.650<0.001
  MLPE1.1000.4222.6060.010
Mexico
  Constant15.6180.25062.397<0.0010.3893.951
  Age0.8420.03722.486<0.001
  Age2– 0.0830.012– 6.729<0.001
  MLPE1.6210.2636.161<0.001
Paraguay
  Constant14.5510.21966.326<0.0010.2433.793
  Age0.6150.0639.772<0.001
Peru
  Constant16.6100.22175.204<0.0010.2544.084
  Age0.7120.06610.782<0.001
Puerto Rico
  Constant13.2940.59522.353<0.0010.4293.917
  Age0.9400.08111.638<0.001
  MLPE1.9940.6443.0960.002
  Sex– 1.4120.577– 2.4460.015
Spain
  Constant15.9870.26460.551<0.0010.3914.129
  Age0.9600.04024.208<0.001
  Age2– 0.0640.013– 4.971<0.001
  MLPE1.8270.2766.622<0.001

Note. MLPE: Mean level of parental education.

3.5Semantic fluency – fruits

The final multivariate linear regression models for the ten country-specific fruit’s category score were significant (Table 5). In all countries, the fruit’s category score increased linearly as a function of age. The scores for Cuba, Ecuador, Mexico, Peru, and Spain were affected by a quadratic age effect. Children from Chile, Ecuador, Mexico, and Spain who had parent(s) with a MLPE >12 years obtained higher fruits’ category score than children who had parent(s) with a MLPE ≤12 years. The child’s sex affected fruits’ category score for Ecuador, Mexico, Paraguay, Puerto Rico, and Spain, such that girls achieved higher scores than boys. The amount of variance these predictors explained in the fruits’ category score ranged from 17.2% (Guatemala) to 46.2% (Cuba).

Table 5

Final multiple linear regression models for Fruits’ category

BStd. ErrortSig.R2SDe (residual)
Chile
  Constant9.6170.17455.400<0.0010.3922.684
  Age0.6280.04015.589<0.001
  MLPE0.6150.2912.1140.035
Cuba
  Constant9.1530.20544.747<0.0010.4622.635
  Age0.7020.03917.895<0.001
  Age20.0300.0132.3690.018
Ecuador
  Constant11.5950.39729.197<0.0010.3972.884
  Age0.6460.04913.154<0.001
  Age2– 0.0490.016– 3.0000.003
  MLPE1.1660.3853.0270.003
  Sex– 0.7280.343– 2.1230.035
Guatemala
  Constant10.7100.18458.088<0.0010.1722.620
  Age0.4760.0746.465<0.001
Honduras
  Constant10.5150.15667.554<0.0010.3572.668
  Age0.6160.04812.754<0.001
Mexico
  Constant11.1940.19956.320<0.0010.3432.807
  Age0.5520.02720.742<0.001
  Age2– 0.0400.009– 4.497<0.001
  MLPE0.6890.1873.682<0.001
  Sex– 0.7600.186– 4.092<0.001
Paraguay
  Constant11.6010.25545.561<0.0010.2523.218
  Age0.5170.0549.651<0.001
  Sex– 1.2340.375– 3.2940.001
Peru
  Constant12.5980.25549.473<0.0010.3513.109
  Age0.6140.05211.845<0.001
  Age2– 0.0650.017– 3.815<0.001
Puerto Rico
  Constant10.4840.28237.213<0.0010.3483.003
  Age0.6030.05910.155<0.001
  Sex– 1.3040.423– 3.0820.002
Spain
  Constant10.6480.19853.686<0.0010.3482.755
  Age0.5710.02621.582<0.001
  Age2– 0.0240.009– 2.7540.006
  MLPE0.9830.1845.336<0.001
  Sex– 0.9130.177– 5.163<0.001

Note. MLPE: Mean level of parental education.

The assumptions of multiple linear regression analysis were met for all final models. There was not multicollinearity (the VIF values were below 10; VIF ≤1.050; collinearity tolerance values did not exceed the value of 1) or influential cases (the maximum Cook’s distance value was 0.117 in a F (2,298)  distribution which correspond to percentile 11).

3.6Normative procedure

Norms (e.g., a percentile score) for the different VFT scores by country were established using the four-step procedure described in the statistical analysis section. An example is provided next to facilitate the understanding of the procedure used to obtain the percentile associated with a score on this test. To find the percentile score for an 8-year-old Mexican girl who scored a 15 on the animal’s category and whose parent(s) have a MLPE of 14, the steps are as follows:

1) Find Mexico in Table 4, which provides the final regression models by country for the animals’ category scores. Use the B weights to create an equation to obtain the predicted animals’ category score for this child using the coding provided in the statistical analysis section. The corresponding B weights are multiplied by the centered age (= calendar age – mean age in the Mexican sample which is equal to 11.4 years), centered age2 (= calendar age – mean age in the Mexican sample which is equal to 11.4 years)2, and MLPE code based on the 12 years of education threshold. See Rivera and Arango-Lasprilla (2017) to find the mean age of each country’s sample. Then the result is added to the constant generated by the model in order to calculate the predicted value. Child’s sex was not a significant predictor, and therefore is not included in this model.

In the case of our example of a Mexican girl, the predicted animals’ category score would be calculated using the following equation: yˆi=15.618+[0.842·(Agei-11.4)]+[(-0.083·(Agei-11.4)2]+(1.621·MLPE) . The girl’s age is 8. The MLPE (14 years) is split into either ≤12 years (and assigned a 0) or more than 12 years (and assigned a 1) in the model. Since the parent(s) of the hypothetical child in the example have 14 years of education, the MLPE value is 1. Thus, the predicted value equation is: yˆi=15.618+[0.842·(8-11.4)]+[-0.083·(8-11.4)2]+(1.621·1)=15.618+(-2.863)+(-0.959)+1.621=13.417.

2) To calculate the residual value (indicated with an ei in the equation), subtract the actual animals’ category score (she scored 15) from the predicted value we just calculated (ei=yi-yˆi) . In this case, the residual would be ei = 15 - 13.417 = 1.583. 3) Next, consult the SDe column in Table 4 to obtain the country-specific SDe (residual) value. For Mexico, it is 3.951. Using this value, the residual value can be transformed to a standardized z score using the equation zi = ei/SDe. In this case, the zi is 1.582/3.951 = 0.400. This is the standardized z score for an 8-year-old Mexican girl who scored a 15 on the animal’s category who has parent(s) with a MLPE of 14.

4) The last step is to use the tables available in most statistical reference books (e.g., Strauss et al., 2006) to convert z scores to percentiles. In this example, the z score (probability) of 0.400 corresponds to the 66th percentile. It is important to remember to use the appropriate tables that correspond to each test (letters F, A, S and the categories of animals and fruits) when performing these calculations.

3.7User-friendly normative data

The four-step normative procedures explained above offers the clinician the ability to determine an exact percentile for a child who has a specific score on the VFT. However, this method can be prone to human error due to the number of required computations by hand. To enhance user-friendliness, the authors have completed these steps for a range of raw scores based on age, sex, and MLPE and created tables for clinicians to more easily obtain a percentile range/estimate associated with a given raw score on this test. These tables are available by country and type of test in the Appendix. To obtain an approximate percentile for the above example (i.e., converting a raw score of 15 on the animals’ category test for a Mexican girl who is 8 years old whose parent(s) have a MLPE of 14) using the simplified normative tables provided in the Appendix, the following steps must be followed. (1) First, identify the appropriate table ensuring the appropriate country and test (letters F, A, S and the categories of animals and fruits). In this case, the table for animals’ category score for Mexico can be found in Table A44. (2) Next, the table is divided based on MLPE (≤12 vs. more than 12 years of education). Since the parent(s) had 14 years of education, the upper section of the table for >12 years of MLPE will be used. (3) Find the appropriate age of the child, in this case, 8 years old. (4) Next, look in the 8 years’ age column to find the approximate location of the raw score obtained on the test. Within the 8 years’ column, the score of 15 obtained by this Mexican girl corresponds to an approximate percentile of 70.

The percentile obtained using this user-friendly table is slightly different than the hand-calculated, more accurate method (i.e., 66th vs. 70th) because the user-friendly table is based on a limited number of percentile values. Individual percentiles cannot be presented in these tables due to space limitations. If the exact score is not listed in the column, the percentile value must be estimated from the list of raw scores available.

4Discussion

The purpose of this study was to obtain and present normative data of VFT for children and adolescents from nine Latin American countries (Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, and Puerto Rico) and Spain. The final regression models for the phonological VFT explained between 26.0% and 42.2% of the variance for the letter F, between 23.0% and 49.3% of the variance for the letter A, and between 22.9% and 44.4% of the variance for the letter S. For the semantic VFT, the final regression models explained between 24.3% and 46.6% of the variance for the animals’ category, and between 17.2% and 46.2% of the variance for the fruits’ category.

Age was significantly related to all VFT, such that scores increase linearly as children become older. Similarly, this pattern has been widely reported in other normative data studies of the same tests among children and adolescents (e.g., Beltrán Dulcey & Solís-Uribe, 2012; García et al., 2012; Lozano & Ostrosky-Solís, 2006; Malloy-Diniz et al., 2007; Martins et al., 2016; Nieto et al., 2008; Prigatano et al., 2008; Ruffieux et al., 2009; Tallberg et al., 2011; Van der Elst et al., 2011).

In contrast to most normative data studies in verbal fluency, the present study also determined the influence of a quadratic function of age (e.g., Honduras in the letter F or Chile, Ecuador, Mexico and Spain in the animals’ category). Van der Elst et al. (2011) also determined the influence of the linear and quadratic function of age, showing that the animals’ category displayed a linear increase based on the children’s age, but the design fluency showed a curvilinear relationship with age, with the highest performance increase among younger children compared to older children (Van der Elst et al., 2011).

Parents’ level of education influences children’s future cognitive development (Meador et al., 2011; Schady, 2011), level of education (Dubow, Boxer, & Huesmann, 2009; Ermisch & Pronzato, 2010) and occupation (Dubow et al., 2009). For this reason, MLPE was added in the regression models. Van der Elst and colleagues (2011) reported that children whose parent(s) had a higher educational level performed better on VFT compared to children whose parent(s) had a lower educational level. Congruent with Van der Elst and colleagues (2011), the results of the present study showed the same results: higher parental educational level was related to higher children’s performance in VFT in many countries (e.g., Mexico, Spain).

Gender was also associated with VFT. Girls generally scored better on these tests compared to boys, with the exception of the animal’s category in Chile where boys performed better than girls. Studies in adult samples indicate differences in gender depending on the category used during the semantic VFT (Acevedo et al., 2000, Egeland, Landrø, Tjemsland, & Walbækken, 2006; Peña-Casanova et al., 2009; Prigatano et al., 2008; Van Der Elst, Van Boxtel, Van Breukelen, & Jolles, 2006); however, most studies with children have not reported gender differences in phonological or semantic verbal fluency (Lozano & Ostrosky-Solís, 2006; Malloy-Diniz et al., 2007; Nieto et al., 2008; Riva et al., 2000; Ruffieux et al., 2009; Tallberg et al., 2011; Van der Elst, et al., 2011).

Some authors suggest that the differences in VFT may be due to gender stereotypes (Cadinu, Maass, Rosabianca, & Kiesner, 2005). For example, Hirnstein, Andrews, and Hausmann (2014) reported gender stereotypes effects on verbal fluency task in adults. Others have indicated that gender differences are due to sociocultural influences (Peña-Casanova et al., 2009). Despite these inconsistencies in the literature, gender differences are very small when compared to age and education effects. This is consistent with a meta-analysis study conducted mainly with children, which concluded that the gender differences in verbal abilities were very small and therefore non-existent (Hyde & Linn, 1988). However, only Canadian and North American samples were analyzed in this study. Since gender roles and patterns vary from culture to culture, these results cannot be generalized to the Latin American or Spanish population.

On the other hand, for the phonological VFT, the sounds of the letters F, A, and S originally used in the CVFT (Borkowski et al., 1967) were selected. The selection of letters during this task is of vital importance because it determines the difficulty of the task (Strauss et al., 2006), which is why authors such as Artiola, Hermosillo, Heaton, and Pardee (1999) suggested the use of the letters M, R, and P when evaluating Spanish-speakers. Even so, for this study we decided to use the original letters since professionals in Latin America frequently use them during their clinical assessments.

Unlike most verbal fluency studies, children in the present study were asked for words that began with the / f /, / a / and / s / phonemes, instead of the letters. With this approach, the test can be applied to children who have not consolidated their spelling knowledge and to adults with no education (Olabarrieta-Landa et al., 2017). Other authors such as Beltrán Dulcey and Solís Uribe (2012) also used the / f /, / a /, / s / phonemes when applying the test to a number of Colombian adolescents. Similarly, Nieto et al. (2008) did not use the letter S of the triad F, A, and S because of the “seseo” phenomenon presented in some parts of Spain, as this could complicate the task for children by confusing the letters S, Z, and C due to its same / s / phoneme. That is why some authors prefer to differentiate the tests and denominate them as phonological (when the phoneme is required) and letter (when the letter is required) verbal fluency test.

Finally, if the beta values of the constants are taken into account, for children, the phonological VFT was more complicated than the semantic. This difference may be due to the late maturation of the frontal lobes and executive functions that continue to develop beyond the age of 12 (Klenberg et al., 2001; Korkman et al., 2001). But this difference in performance also occurs in adults, so it seems that phonological verbal fluency requires a more elaborate strategy and organizational capacity than semantics (Riva et al., 2000), and therefore, these differences are not due exclusively to neurodevelopment characteristics.

4.1Limitations and future directions

The study has some limitations. First, only children whose primary language was Spanish were recruited. Therefore, the results of this study cannot be generalized to children whose primary language is not Spanish. This is especially important because many countries in Latin America and Spain are multilingual, with Spanish being one of the most spoken languages among its inhabitants. For example, in Mexico there are 62 indigenous languages (Chamoreau, 2014), and in Spain there are other official languages apart from Spanish such as Catalan, Basque or Galician (Garrido Medina, 2007).

Second, the results of this study cannot be generalized to other Latin American countries such as Argentina, Bolivia, Panama, or Venezuela, among others. Future studies should obtain normative data for VFT in these countries. Third, even though in Chile, Mexico, Paraguay, Puerto Rico, and Spain the sample was collected from several geographic areas of the country, for the rest of the countries of the study only children from a specific geographic area were recruited. It would have been ideal to recruit children from different areas of these countries as well. Future studies should expand the data from this study with samples from different geographic regions. Likewise, most of the children were recruited from urban areas, thus future studies should include more children from rural areas.

Fourth, the letters selected for the study were F, A, and S. These letters are commonly used by professionals in Latin America. However, other letters have been proposed to be used with Spanish-speaking populations, so future studies should also evaluate whether performance changes depending on letters selected.

Lastly, normative data were generated using healthy children and adolescents. Future studies should include clinical samples (e.g., children and adolescents with brain damage, epilepsy, etc.) to obtain proper cutoff points and calculate the sensitivity and specificity of VFT.

4.2Implications and conclusions

Despite the limitations, this is the first study to offer normative data for the phonological and semantic VFT in nine countries in Latin America and Spain among 4,373 children and adolescents. Most normative data studies have focused on adult populations, so to date this represents the largest normative data study of neuropsychological tests for children and adolescents in the world. In addition, for the creation of the normative data, more up-to-date approaches have been used to allow a more accurate calculation of the norms, such as the use of multiple regressions and residual values. Therefore, these norms are expected to be useful for those who work in the neuropsychological evaluation of children and adolescents in these 10 countries.

Conflict of interest

None to report.

Supplementary material

References

1 

Acevedo A. , Loewenstein D. A. , Barker W. W. , Harwood D. G. , Luis C. , Bravo M. , … & Duara R. (2000). Category fluency test: Normative data for English-and Spanish-speaking elderly. Journal of the International Neuropsychological Society, 6(07), 760–769. doi: 10.1017/S1355617700677032

2 

Aiken L. S. , & West S. G. (1991), Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage.

3 

Artiola L. , Hermosillo D. , Heaton R. , & Pardee R. E. (1999). Manual de normas y procedimientos para la baterίa neuropsicológica en español. Tucson: AZ.

4 

Azuma T. (2004). Working memory and perseveration in verbal fluency. Neuropsychology, 18(1), 69–77. doi:10.1037/0894-4105.18.1.69

5 

Begeer S. , Wierda M. , Scheeren A. M. , Teunisse J. P. , Koot H. M. , & Geurts H. M. (2013). Verbal fluency in children with autism disorders: Clustering and switching strategies. Autism, 18(8), 1014–1018. doi:10.1177/1362361313500381

6 

Beltrán Dulcey C. , & Solís-Uribe G. (2012). Evaluación Neuropsicológica en adolescentes: Normas para población de Bucaramanga.a, Neuropsiquiatría y Neurociencias. Revista Neuropsicologίa, 12(2), 77–93.

7 

Benton A. , & Hamsher K. S. (1989), Multilingual aphasia examination. Iowa City, IA: University of Iowa.

8 

Borkowski J. , Benton A. , & Spreen O. (1967). Word fluency and brain damage. Neuropsychologia, 5(2), 135–140. doi:10.1016/0028-3932(67)90015-2

9 

Brown L. , Sherbenou R. J. , & Johnsen S. K. (2009), Test de inteligencia no verbal TONI-2. Madrid: TEA ediciones.

10 

Butman J. , Allegri R. F. , Harris P. , & Drake M. (2000). Fluencia verbal en español. Datos normativos en Argentina. Medicina, 60(5/1), 561–564.

11 

Cadinu M. , Maass A. , Rosabianca A. , & Kiesner J. (2005). Why do women underperform under stereotype threat? Evidence for the role of negative thinking. Psychological science, 16(7), 572–578. doi:10.1111/j.0956-7976.2005.01577.x

12 

Chamoreau C. (2014). Diversidad lingüística en México. Amerindia - Langues du Mexique, 37(1), 3–20.

13 

Cohen M. J. , Morgan A. M. , Vaughn M. , Riccio C. A. , & Hall J. (1999). Verbal fluency in children: Developmental issues and differential validity in distinguishing children with attention-deficit hyperactivity disorder and two subtypes of dyslexia. Archives of Clinical Neuropsychology, 14(5), 433–443. doi:10.1016/s0887-6177(98)00038-9

14 

Cook R. D. (1977). Detection of influential observation in linear regression. Technometrics, 19(1), 15–18. doi:10.2307/1268249

15 

Dubow E. F. , Boxer P. , & Huesmann L. R. (2009). Long-term effects of parents’ education on children’s educational and occupational success: Mediation by family interactions, child aggression, and teenage aspirations. Merrill-Palmer quarterly (Wayne State University. Press), 55(3), 224–249.

16 

Egeland J. , Landrø N. I. , Tjemsland E. , & Walbækken K. (2006). Norwegian norms and factor-structure of phonemic and semantic word list generation. The Clinical Neuropsychologist, 20(4), 716–728.

17 

Ermisch J. , & Pronzato C. (2010). Causal effects of parents’ education on children’s education (No. 2010-16). ISER Working Paper Series.

18 

García E. , Rodríguez C. , Martín R. , Jiménez J. E. , Hernández S. , & Díaz A. (2012). Test de fluidez verbal: Datos normativos y desarrollo evolutivo en el alumnado de primaria. European Journal of Education and Psychology, 5(1), 53–64.

19 

Garrido Medina J. C. (2007). Multilingüismo y lengua externa e interna en la política lingüística en España.gicos. Anuario de estudios filolόgicos, 30, 131–149.

20 

Goldstrohm S. L. , & Arffa S. (2005). Preschool children with mild to moderate traumatic brain injury: An exploration of immediate and post-acute morbidity. Archives of Clinical Neuropsychology, 20(6), 675–695. doi:10.1016/j.acn.2005.02.005

21 

Gouveia P. A. R. , Brucki S. M. D. , Malheiros S. M. F. , & Bueno O. F. A. (2007). Disorders in planning and strategy application in frontal lobe lesion patients. Brain and cognition, 63(3), 240–246.

22 

Henry L. , Messer D. J. , & Nash G. (2015). Executive functioning and verbal fluency in children with language difficulties. Learning and Instruction, 39, 137–147. doi: 10.1016/j.learninstruc.2015.06.001

23 

Hirnstein M. , Andrews L. C. , & Hausmann M. (2014). Gender-stereotyping and cognitive sex differences in mixed-and same-sex groups. Archives of sexual behavior, 43(8), 1663–1673.

24 

Hyde J. S. , & Linn M. C. (1988). Gender differences in verbal ability: A meta-analysis. Psychological Bulletin, 104(1), 53–69.

25 

Klenberg L. , Korkman M. , & Lahti-Nuuttila P. (2001). Differential development of attention and executive functions in 3- to 12-year-old Finnish children. Developmental Neuropsychology, 20(1), 407–428. doi:10.1207/s15326942dn2001_6

26 

Korkman M. , Kemp S. L. , & Kirk U. (2001). Effects of age on neurocognitive measures of children ages 5 to 12: A cross-sectional study of 800 children from the United States. Developmental Neuropsychology, 20(1), 331–354. doi:10.1207/s15326942dn2001_2

27 

Kovacs M. (1992). Children Depression Inventory CDI (Manual), Toronto: Multihealth systems .

28 

Kutner M. H. , Nachtsheim C.J. , Neter J. , & Li W. (2005), Applied linear statistical models –(5th ed). New York: McGraw Hill.

29 

Landrø N.I. , & Ueland T. (2008). Verbal memory and verbal fluency in adolescents with schizophrenia spectrum disorders. Psychiatry and Clinical Neurosciences, 62(6), 653–661. doi:10.1111/j.1440-1819.2008.01864.x

30 

Law J. , Boyle J. , Harris F. , Harkness A. , & Nye C. (2000). Prevalence and natural history of primary speech and language delay: Findings from a systematic review of the literature. International Journal of Language & Communication Disorders, 35(2), 165–188. doi: 10.1080/136828200247133

31 

Lozano A. , & Ostrosky-Solís F. (2006). Efecto de la edad y la escolaridad en la fluidez verbal semántica: Datos normativos en población hispanohablante. Revista Mexicana de Psicologίa, 23(1), 37–44.

32 

Malloy-Diniz L. F. , Bentes R. C. , Figueiredo P. M. , Brandão-Bretas D. , da Costa-Abrantes S. , Parizzi A. M. ,… & Salgado J. V. (2007). Normalización de una batería de tests para evaluar las habilidades de comprensión del lenguaje, fluidez verbal y denominación en niños brasileños de 7 a 10 años: Resultados preliminares. Revista de Neurologίa, 44(5), 275–280.

33 

Marino J. , & Alderete A. M. (2010). Valores normativos de pruebas de fluidez verbal, categoriales, fonológicas, gramaticales y combinadas y análisis comparativo de la capacidad de iniciación. Revista Neuropsicología, Neuropsiquiatría y Neurociencias, 10(1), 79–93.

34 

Martins R. , Mograbi D. C. , Andrade Gabrig I. , & Charchat-Fichman H. (2016). Normative data and evidence of validity for the Rey Auditory Verbal Learning Test, Verbal Fluency Test, and Stroop Test with Brazilian children. Psychology & Neuroscience, 9(1), 54–67. doi: 10.1037/pne0000041

35 

Matute E. , Rosselli M. , Ardila A. , & Morales G. (2004). Verbal and nonverbal fluency in Spanish-speaking children. Developmental Neuropsychology, 26(2), 647–660. doi:10.1207/s15326942dn2602_7

36 

McLeod S. , & Harrison L. J. (2009). Epidemiology of speech and language impairment in a nationally representative sample of 4-to 5-year-old children. Journal of Speech, Language, and Hearing Research, 52(5), 1213–1229. doi: 10.1044/1092-4388(2009/08-0085)

37 

Meador K. J. , Baker G. A. , Browning N. , Clayton-Smith J. , Cohen M. J. , Kalayjian L. A. , … & Loring D. W. (2011). Relationship of child IQ to parental IQ and education in children with fetal antiepileptic drug exposure. Epilepsy & Behavior, 21(2), 147–152.

38 

Mitrushina M. , Boone K. B. , Razani J. , & D’Elia L. F. (2005), Handbook of normative data for neuropsychological assessment. Oxford University Press.

39 

Nieto A. , Galtier I. , Barroso J. , & Espinosa G. (2008). Fluencia verbal en niños españoles en edad escolar: Estudio normativo piloto y análisis de las estrategias organizativas. Revista de Neurología, 46(1), 2–6.

40 

Olabarrieta Landa L. , Landa Torre E. , López-Mugartza J. C. , Bialystok E. , & Arango-Lasprilla J. C. (2017). Verbal fluency tests: Developing a new model of administration and scoring for Spanish language. NeuroRehabilitation, 41(2), 539–565.

41 

Peña-Casanova J. , Quiñones-Úbeda S. , Gramunt-Fombuena N. , Quintana-Aparicio M. , Aguilar M. , Badenes D. , … & NEURONORMA Study Team. (2009). Spanish Multicenter Normative Studies (NEURONORMA Project): Norms for verbal fluency tests. Archives of Clinical Neuropsychology, 24(4), 395–411. doi:10.1093/arclin/acp042. Epub 2009 Aug 1

42 

Prigatano G. P. , Gray J. A. , & Lomay, V. T. (2008). Verbal (animal) fluency scores in age/grade appropriate minority children from low socioeconomic backgrounds. Journal of the International Neuropsychological Society, 14(1), 143–147. doi: 10.10170S1355617708080089

43 

Riva D. , Nichelli F. , & Devoti M. (2000). Developmental aspects of verbal fluency and confrontation naming in children. Brain and Language, 71(2), 267–284. doi: 10.1006/brln.1999.2166

44 

Rivera D. , & Arango-Lasprilla, J. C. (2017). Methodology for the development of normative data for Spanish Speaking pediatric population. Neuro Rehabilitation, 41(3), 581–592.

45 

Rubiales J. , Bakker L. , & Russo D. (2013). Fluidez verbal fonológica y semántica en niños con trastorno por déficit de atención e hiperactividad. Neuropsicologia Latinoamericana, 5(3), 7–15.

46 

Ruffieux N. , Njamnshi A. K. , Mayer E. , Sztajzel R. , Eta S. C. , & Doh R. F. … Hauert C. A. (2009). Neuropsychology in Cameroon: First normative data for cognitive tests among school-aged children, child neuropsychology. A Journal on Normal and Abnormal Development in Childhood and Adolescence, 16(1), 1–19. doi: 10.1080/09297040902802932

47 

Schady N. (2011). Parents’ education, mothers’ vocabulary, and cognitive development in early childhood: Longitudinal evidence from Ecuador. American Journal of Public Health, 101(12), 2299–2307.

48 

Snowling M. J. , Bishop D. V. M. , Stothard S. E. , Chipchase B. , & Kaplan C. (2006). Psychosocial outcomes at 15 years of children with a preschool history of speech-Language impairment. Journal of Child Psychology and Psychiatry, 47(8), 759–765.doi: 10.1111/j.1469-7610.2006.01631.x

49 

Spreen O. , & Strauss E. (1998), A compendium of neuropsychological tests (2nd Ed.). New York, NY: Oxford University Press.

50 

Strauss E. , Sherman E. M. , & Spreen O. (2006), A compendium of neuropsychological tests: Administration, norms, and commentary. American Chemical Society.

51 

Tallberg I. M. , Carlsson S. , & Lieberman M. (2011). Children’s word fluency strategies. Scandinavian Journal of Psychology, 52(1), 35–42.doi: 10.1111/j.1467-9450.2010.00842.x

52 

Thurstone L. L. , & Thurstone T. G. (1962). Primary mental abilities (Rev.). Chicago: Science Research Associates.

53 

Tomblin J. B. , Records N. L. , Buckwalter P. , Zhang X. , Smith E. , & O’Brien M. (1997). Prevalence of specific language impairment in kindergarten children. Journal of Speech, Language, & Hearing Research, 40(6), 1245–1260.

54 

Van der Elst W. , Hurks P. , Wassenberg R. , Meijs C. , & Jolles J. (2011). Animal Verbal Fluency and Design Fluency in school-aged children: Effects of age, sex, and mean level of parental education, and regression-based normative data. Journal of Clinical & Experimental Neuropsychology, 33(9), 1005–1015.doi: 10.1080/13803395.2011.589509

55 

Van Der Elst W. , Van Boxtel M. P. , Van Breukelen G. J. , & Jolles J. (2006). Normative data for the Animal, Profession and Letter M Naming verbal fluency tests for Dutch speaking participants and the effects of age, education, and sex. Journal of the International Neuropsychological Society, 12(01), 80–89. doi:10.1017/S1355617706060115