Culture-Fair Cognitive Screening Tools for Assessment of Cognitive Impairment: A Systematic Review

INTRODUCTION

Dementia affects approximately 500,000 people in the UK with profound socioeconomic implications; the care of people living with dementia is estimated to cost up to £14.93 billion annually, exceeding the cost of stroke, heart disease, and cancer combined [1]. While the number of people living with dementia in the UK is expected to double by 2050, prevalence among ethnic minority communities is expected to increase sevenfold [2]. Commonly used cognitive assessment tools include the Addenbrooke’s Cognitive Examination (ACE III), Mini-Mental State Examination (MMSE), and the Montreal Cognitive Assessment (MoCA), which have been shown to be helpful in aiding diagnosis of dementia [3]. However, these tools were created for “Western” populations and a multitude of studies have demonstrated that performance is significantly influenced by age, educational status, ethnicity, and language [3–5], thus resulting in poor test accuracy for non-Caucasian patients [6]. Comprehensive and accurate cognitive assessment is very important in the workup for dementia. For example, it can inform referral to other members of the multi-disciplinary team such as speech and language, or occupational therapy, monitor longitudinally cognitive changes and guide pharmacological and social interventions. Under-diagnosis of dementia can have negative implications including inadequate support and delayed use of anti-dementia drugs, which may reduce the efficacy even during the earliest stages of Alzheimer’s disease [7]. There are also negative implications of over-diagnosis, which can lead to unnecessary anxiety, and implications for driving, work, and insurance. Moreover, the accurate identification of mild cognitive impairment is essential to the advancement of preventive and therapeutic research.

Attempts to translate or adapt commonly used cognitive tests have been explored, however these still pose issues [8]. For example, repetition of the English idiom “no ifs, ands, or buts” used in the MMSE does not maintain the same level of linguistic difficulty, test of fluency or cultural relevance upon translation. Ethnic disparities in performance during neuropsychological testing have long been recognized, regardless of socioeconomic status or level of education [9]. Such findings are attributed to an over-reliance of these tools on cultural norms and language [10]. A variety of cognitive assessments have been developed for specific populations. However, at present there is no universal gold standard tool for the diagnosis of cognitive impairment in ethnic minority groups. A culture-free test are test which is devoid of all culture-related content, which can be very difficult to achieve. In contrast, a culture fair test is a test designed to reduce the influence of cultural elements and therefore provide an accurate assessment without favoring any one culture [11].

Thus far, there has not been an up-to-date review comparing the variety of available culture-fair cognitive screening tools. This systematic review aims to determine the commonly used culture-fair cognitive screening tools and evaluate their test accuracy, strengths, and limitations.

MATERIALS AND METHODS

Search

This review was reported in accordance with the PRISMA guidance for reporting systematic reviews and a PRISMA checklist is included in the Supplementary Material. Medline, Embase, PsychINFO, and CINAHL databases were searched from inception using the search strategy presented in the Supplementary Material. The search strategy was developed in conjunction with a librarian at the Leicestershire Partnership Trust (LH). The search was completed in November 2021 and updated in July 2023. No additional studies were found in the second search in July 2023. Included articles were published in any year and there was no restriction on time. Reference lists of included articles and citation indices were screened for additional relevant material. Only adult human studies where patient data could be successfully extracted were considered. Included studies had to use a cognitive screening tool, however, were not limited to a specific cognitive test and included neuropsychological test batteries. Abstracts, conference papers, posters, translated or adapted tools, adapted for specific populations, or studies reporting on less than five subjects were excluded. The review protocol was registered on PROSPERO prior to commencement of the review (CRD42021288776). The PRISMA flow diagram is presented in Fig. 1, to outline the papers identified through the database searches.

Fig. 1

Prisma flow diagram.

RESULTS

The search strategy identified 5,865 articles, with 4,040 remaining after duplicates were removed (Fig. 1). An additional four studies were found through manual search of the references. A total of 92 full texts were identified based on initial screening of abstracts and 28 studies were included in the review. As described in Table 1, studies took place in multiple countries including Singapore, Malaysia, Indonesia, Philippines, Netherlands, Tunisia, Germany, Belgium, Denmark, Sweden, Norway, Greece, India, Taiwan, Israel, Japan, USA, Nigeria, Jamaica, Canada, Latin America, Chine, Africa, Australia, Spain, and Tanzania. There were 18 test accuracy studies, seven cross sectional studies, one normative data study and one longitudinal study. Six studies were multicenter. The studies ranged between local community, general practice, outpatient memory clinic, geriatric rehabilitation, and inpatient settings. Mean age of participants ranged between 65–80 years old.

Eleven studies assessed RUDAS as the primary cognitive tool, of which nine studies used MMSE or MoCA as reference tests. RUDAS was found to be as accurate as the MMSE [14–16], with a similar sensitivity (range: 52–94%) and better specificity (range: 70% –98%) [17]. Area under the curve (AUC) values reported ranged between 0.62 [18] to 0.93 [19]. The majority of studies showed that RUDAS scores were found to be uninfluenced by education [16], language [16, 20], and sex [16]; however, three studies identified that RUDAS scores were influenced by education, as highlighted in Table 2 [14, 17, 19]. Nielsen et al. (2013) established it to be superior to MMSE, for use in an older multicultural population [20].

The remaining seventeen studies analyzed an array of other cognitive tests; three studies used the Community Screening Instrument for Dementia (CSI-D) [21–23], three studies used the Identification of Dementia in Elderly Africans (IDEA) [24–26], two studies for the Kimberley Indigenous Cognitive Assessment tool (KICA) [27, 28], two studies used the Visual Cognitive Assessment Test (VCAT) [6, 29], and there was one study each for the Cross-Cultural Dementia Screening (CCD) [30], Dementia screening battery (DSB-100) [31], Multicultural Cognitive Examination (MCE) [20], Neuropsychological Evaluation Screening tool (NEST) [32], Modified Mini Mental Test (MMT) [33], Iowa screening test [34], Cross-cultural cognitive examination (CCCE) [35], and the Elderly Cognitive Assessment Questionnaire (ECAQ) [36]. Sensitivity testing demonstrated NEST had better sensitivity compared to the Hindi Mental State Examination (HMSE) at all educational levels (94.8%) [32]. CSI-D had good sensitivity across different countries and the brief version of the full CSI-D shared favorable culture-fair screening properties of the full assessment [21]. The Iowa test, ECAQ and CCCE showed similar sensitivity of 85%, 85.3% and 88% respectively [34–36]. In terms of specificity of dementia identification, Iowa test had 66%, CCCE showed 97.4% and ECAQ 91.5% [34–36]. VCAT had good construct validity and good internal consistency [6] and was not affected by age, years of education, race, or employment, whereas MoCA was [29]. Receiver operating characteristic (ROC) analysis found VCAT to be equivalent or more effective to the MMSE and MoCA (AUC 0.84–0.91) [6, 29] and CCD to have high predictive validity for dementia (AUC 0.85–0.95, sensitivity 85%, specificity 89%) [30]. CCD showed no differences between the ethnic groups after adjusting for age and education [30]. MCE had significantly improved test accuracy (AUC 0.99) compared with using the RUDAS alone (AUC 0.92) [19]. The KICA showed good test accuracy based upon AUC scores of 0.93 and 0.95 [27, 28] with a sensitivity of 90.6% and specificity 92.6% [28]. The IDEA had an AUC of 0.78–0.87 [24, 26] which was improved in combination with the IDEA-IADL to 0.94 [26]. The MMT was useful in specifically testing for vascular dementia, however, was affected by educational level [30] and DSB-100 was valid in a Tunisian elderly population [31]. As detailed in Table 2, the DSB-100 was found to be affected by education level. The CSI-D, brief-CSI-D, CCD, and VCAT were found to have little effect of educational level. The RUDAS, VCAT, and Iowa screening test were all found to be acceptable to patients. The Iowa screening test, NEST, and ECAQ were all quick to administer (under 15 min). Many studies assessed the accuracy of tools at different cut off points [6, 14–16, 18, 20, 23, 24, 27, 28, 33]. Cut off for IDEA was quoted as < 7 out of 12 for the 5-item instrument, or < 6 out of 9 for the 7-item instrument [24]. Optimal cut off to detect cognitive impairment was≤24. Optimal cut off for the KICA was quoted as 31/32 (KICA) and 34 [27]. RUDAS cut offs varied from 21 [14–16, 18, 19, 27] to 24 [19]. Cut off for CSI-D was five or less to favor specificity or six or less to favor sensitivity [23] and for MMT was 33/55 [33]. Optimal cut off for NEST was three or more errors [32].

DISCUSSION

Our systematic review highlights that there have been multiple attempts across various countries to create a culture-fair cognitive assessment tool. The need for such a tool has been widely recognized and there have been numerous studies highlighting the shortcomings of the currently used cognitive assessment tools in screening for cognitive impairment in multicultural populations [3–5]. We propose the universal use of cognitive assessment tools free of cultural bias as a practical approach for memory services in response to the ever-growing diversity of the UK population. Our review examined 28 studies identifying eleven different culture-fair cognitive assessment tools, all of which show promising results in studies in their country of development. The RUDAS, VCAT, KICA, and CCD were found to have preferable specificity and sensitivity to the MMSE when used in ethnic minorities.

A strength of the review is the inclusion of studies assessing patients from multiple different countries, and both native and migrant patients in those countries. Including studies from a variety of countries and multi-country studies increases the likelihood of generalizability of the results across the world. However, there are multiple countries which have not been studied, including the UK, which brings scope for further research.

The RUDAS and KICA have been recognized as promising options to reduce cultural bias in ethnic minorities [13]. Both have been compared to the MMSE and shown superior or comparable diagnostic accuracy, and less effect of both education and culture. One study compared the modified KICA and RUDAS together, and showed the modified KICA had greater diagnostic accuracy (94%) compared to the RUDAS (89%) [27]. The diagnostic accuracy of the RUDAS is increasingly well documented as a cross-cultural cognitive test [40]. One meta-analysis that included 26 studies found the RUDAS to have a comparable diagnostic accuracy across a multitude of sociocultural settings, with limited impact from educational, linguistic, or cultural biases [13]. However, despite being deemed culture-fair, both have cultural adaptations available. For example, for the RUDAS, in both the Peruvian and Spanish validation study [14, 41], the word tea in the supermarket list in the memory item was changed to coffee, and in the Thai validation study, animal fluency was substituted with fruit fluency to reduce the impact of formal education on the language item [40]. Although these may appear to be minor changes, the fact that cultural adaptations are required suggests that the tool is not entirely culture-fair, as without the adaptations the results may be skewed. KICA has been incorporated into Australia’s dementia services following recognition of its’ superior sensitivity and specificity for the diagnosis of cognitive impairment in Indigenous communities where illiteracy, language, or cultural barriers may impede assessment [27]. The KICA has been adapted for Canadian, Persian, and Brazilian populations, as discussed in the review by Mukaetova-Ladinska et al. [42]. Neither the KICA nor RUDAS have been validated for use in the UK memory services as of yet. Furthermore, it is important to note that our review included participants aged 65 and over. In a study of 70 UK participants, in the younger persons memory service the RUDAS was found to have modest accuracy (AUC = 0.70 (0.57–0.82), p = 0.002, sensitivity 71.7%, specificity 58%) and in the younger adults was found to be influenced by educational and ethnic background, which is in direct contrast to results in the older people as stated above [43]. Further research should be done in validating culture-fair tools for diagnosing dementia in the younger population.

The review has highlighted the challenges in separating the effects of culture and education in cognitive tools, and many patients may be affected by both cultural and educational biases. The RUDAS, MMT, and DSB-100 were found to be affected by educational level to varying degrees and may require adjustment for educational qualifications. However, regarding the RUDAS, the effect of education was still much less than the effect on the MMSE [17, 44] and some studies did not find it to be affected by educational level at all. Educational level was found to have little impact on the CSI-D, brief-CSI-D, CCD and VCAT [16, 22, 29, 30].

The VCAT provides an option which is neutral to both language and education [29]. It is a 30-item tool and includes testing of executive functioning which is very useful, and something not all cognitive tools cover, including the widely used MMSE. The items are visual based and therefore may be an extremely useful tool across cultures, removing the barriers of language [29].

An interesting finding from the review was the study testing the IDEA-IADL, which combines the cognitive assessment with an assessment of instrumental activities of daily living [26]. These are complex activities which are generally affected earlier in cognitive impairment, and therefore can aid in the holistic diagnosis of dementia. It is well known that diagnosis of dementia does not rely on cognitive screens alone, and an assessment of functioning is required. Activity of Daily Living (ADL) questionnaires can be a useful tool, however the authors highlighted that the commonly used ADL tools, such as the Lawton’s ADL tool, were developed in high income countries and therefore may also be subject to cultural biases [24, 25]. As seen above, AUC curve was 0.896 for diagnosing dementia when using the IDEA-IADL alone and 0.94 when used with in combination with the IDEA [24, 25]. This could indicate areas for further research, to create tools which can be easily administered, avoid cultural biases, and can be used to assess both cognitive abilities and instrumental activities of daily living.

A major hurdle identified through this review for many of the cognitive assessment tools is the lack of validation worldwide and in varying populations. Many countries, including the UK now have a very multicultural elderly population, and therefore having a tool suitable for one ethnic population is of minimal use. Therefore, many of the studies will need to be validated across more countries.