You are viewing a javascript disabled version of the site. Please enable Javascript for this site to function properly.
Go to headerGo to navigationGo to searchGo to contentsGo to footer
In content section. Select this link to jump to navigation

Digital Neuropsychological Assessment: New Technology for Measuring Subtle Neuropsychological Behavior


Technology has transformed the science and practice of medicine. In this special mini-forum, data using digital neuropsychological technology are reported. All of these papers demonstrate how coupling digital technology with standard paper and pencil neuropsychological tests are able to extract behavior not otherwise obtainable. As digital assessment methods mature, early identification of persons with emergent neurodegenerative and other neurological illness may be possible.

Necessity is the mother of invention. The COVID-19 pandemic has significantly altered the delivery of health care including neuropsychological evaluations. Telehealth procedures previously deemed either not possible or impracticable may become a standard feature in health care after the COVID-19 crisis has resolved. Perhaps a serendipitous effect of the COVID-19 crisis is the determination that standard paper and pencil clinical neuropsychological evaluations can be administered electronically using a digital medium. Using this technology, patients who either cannot come to the office, perhaps because of medical reasons or location, can benefit from these much-needed services.

An equally exciting benefit of digital assessment technology is the granularity of the data that can be obtained. This is relevant for studies examining the effect of disease modifying medication to treat neurodegenerative illness such as mild cognitive impairment (MCI) and dementia such as Alzheimer’s disease (AD). One reason that clinical drug trials designed to treat MCI and AD have not been successful is that by the time patients are recruited, the illness may be too advanced. Neuropsychological assessment is key for both diagnosis and as outcome measures to assess treatment efficacy. To meet this challenge there is an urgent need to develop neuropsychological assessment paradigms that can identify emergent pre-dementia syndromes as early as possible.

Coupling digital technology with traditional paper and pencil neuropsychological tests may be a way to flag or identify subtle neuropsychological alterations before patients actually meet current diagnostic criteria for either MCI or dementia, such as AD or vascular dementia. However, to realize the full potential of digital technology to identify emergent pre-dementia and dementia syndromes at least two requirements should be met. First, digitally administered tests should be able to uncover and measure behavior not otherwise obtainable using standard paper and pencil tests. Second, this behavior should be used to operationally define neurocognitive constructs known to underlie pre-dementia and dementia syndromes. The five papers that comprise this special topic project fulfill both requirements.

Emrani and colleagues [1] used an iPad to administer and score the Backward Digit Span Test (BDST) [2, 3]. The origins of digit span as a psychological construct date from the work of Gottfried Leibniz (1646–1716). Leibniz suggested that individuals have a finite capacity to prospectively process or hold in mind information from the environment. He termed this capacity the Span of Apperception [4]. In the nineteenth century, Ebbinghaus (1850–1909; 1885/1964) [4] was the first cognitive scientist to show how span could be used as an experimental paradigm to investigate memory and learning. In America,1 Oliver Wendell Homes Pater (1809–1894; 1871) [4] made cogent observations about digit span as a method to assess span of apperception, i.e., “in uttering distinctly a series of unconnected numbers or letters before a succession of careful listeners, I have been surprised to find how generally they break down, in trying to repeat them, between seven and ten figures or letters” (Holmes, 1871) [4]. Emrani and colleagues [1] administered the BDST to memory clinic patients classified using actuarial criteria [5, 6] with MCI versus a group of memory clinic patients not meeting criteria for MCI (i.e., non-MCI). Analyses were undertaken consistent with constructs drawn from the theory of executive attention [7] and were confined to only trials where performance was 100%correct. By analyzing the latency necessary to generate all responses, Emrani and colleagues [1] were able to differentiate MCI from non-MCI patients.

The Digit Symbol subtest is drawn from the corpus of intelligence tests authored by David Wechsler. Since Wechsler’s publication of the original Bellevue Intelligence Scale [8, 9], the basic format of this test has changed very little. The patient is asked to use a key to transpose geometric symbols with their corresponding Arabic numbers. The Digit Symbol subtest is extremely sensitive to the presence of neurocognitive impairment [10] because of the multiple cognitive skills necessary for optimum performance. For example, at a minimum, the examinee must be able to produce written output (graphomotor skills); visually track back and forth from the test key to the test form (visual scanning); and locate the test items (visuospatial ability). Performance improves to the extent to which the examinee “learns” the symbol/ number pairs without referring to the test key for each test item (incidental memory). The unique contribution of all three neurocognitive operations has been explored by Joy and colleagues [11–13].

Andersen and colleagues [14] administered the Digit Symbol subtest using a digital pen and ‘smart’ paper. In this research a standard version of the Wechsler Adult Intelligence Scale Digit Symbol subtest was administered to participants from the Long-Life Family Study [15, 16]. Statistical analyses found distinct trajectories for time spent ‘drawing’ versus ‘non-drawing’. Analyses revealed four ‘non-drawing’ time trajectories. Selected participants in these clusters obtained lower episodic memory scores. Analyses of ‘drawing’ time also revealed four performance trajectories. However, some of these participants presented with weaker grip strength and slower gait speed, but no differences in cognitive test scores.

Parjane and colleagues [17] recorded speech using a picture description task from dementia patients diagnosed with corticobasal syndrome and primary progressive non-fluent aphasia. Both acoustic and linguistic parameters were calculated to provide a means of measuring overall speech prosody and meaningful between-group differences were reported. The Clock Drawing Test (CDT) is a widely used and popular neuropsychological test [18, 19]. In the command condition, patients are presented with a blank sheet of paper and are asked to draw the face of a clock showing the numbers and the two hands set for ‘10 after 11’. In the copy condition, a pre-drawn model of a clock with numbers and hands set for ‘10 after 11’ is presented, and patients are asked to copy it. However, analog clock drawing systems can be very time consuming and difficult to score [20, 21]. Recently, a digital clock drawing test (dCDT) has been introduced [22, 23]. Davoudi and colleagues [24] administered the dCDT to patients diagnosed with AD, vascular dementia, and healthy elderly controls. Using machine learning statistical analyses, these researchers were able to extract motor, kinematic and visuospatial features not otherwise obtainable that classified patients and participants into their respective groups. In a second paper, Davoudi and colleagues [25] asked the question - what exactly is a normal clock? Using digital technology, elements such as the size and shape of the clock face, and digit and hand placement are easily measured, providing much needed quantification of patients’ drawings.

Digit Span Backward, the Digit Symbol subtest, analysis of speech, and the Clock Drawing Test are among the oldest and most thoroughly researched neuropsychological tests and paradigms. Writing during a time before digital assessment technology was available, Edith Kaplan [18, 19] championed the analysis of errors, and the process by which tests are completed as a means of understanding brain and cognition. All these papers provide excellent examples of Kaplan’s Process Approach [26] and show how digital technology can extract behavior not otherwise obtainable, i.e., how to identify and operationally define the process by which tests are completed. Collectively, the data presented in these papers show how digital technology can calculate potentially sensitive clinical neurocognitive biomarkers that may flag early emergent neurodegenerative illness.



Emrani S , Lamar M , Price C , Baliga S , Wasserman V , Matusz EF , Saunders J , Gietka V , Strate J , Swenson R , Baliga G , Libon DJ ((2021) ) Neurocognitive constructs underlying executive control in statistically-determined mild cognitive impairment. J Alzheimers Dis 82: , 5–16.


Lamar M , Catani M , Price CC , Heilman KM , Libon DJ ((2008) ) The impact of region-specific leukoaraiosis on working memory deficits in dementia. Neuropsychologia 46: , 2597–2601.


Lamar M , Price CC , Libon DJ , Penney DL , Kaplan E , Grossman M , Heilman KM ((2007) ) Alterations in working memory as a function of leukoaraiosis in dementia. Neuropsychologia 45: , 245–254.


Richardson JT ((2007) ) Measures of short-term memory: A historical review. Cortex 43: , 635–650.


Bondi MW , Edmonds EC , Jak AJ , Clark LR , Delano-Wood L , McDonald CR , Nation DA , Libon DJ , Au R , Galasko D , Salmon DP ((2014) ) Neuropsychological criteria for mild cognitive impairment improves diagnostic precision, biomarker associations, and progression rates. J Alzheimers Dis 42: , 275–289.


Jak AJ , Bondi MW , Delano-Wood L , Wierenga C , Corey-Bloom J , Salmon DP , Delis DC ((2009) ) Quantification of five neuropsychological approaches to defining mild cognitive impairment. Am J Geriatr Psychiatry 17: , 368–375.


Fuster JM ((2015) ) The Prefrontal Cortex (5th ed). Lippincott-Raven, Philadelphia, PA.


Wechsler D ((1939) ) The measurement and appraisal of adult intelligence (1st ed). Williams and Wilkins Corporation, Baltimore, MD.


Wechsler D ((1944) ) The measurement and appraisal of adult intelligence (3rd ed). Williams and Wilkins Corporation, Baltimore, MD.


Lezak M , Howieson DB , Loring DW ((2004) ), Neuropsychological assessment (4th ed.). Oxford University Press, New York.


Joy S , Fein D , Kaplan E ((2003) ) Decoding digit symbol: Speed, memory, and visual scanning. Assessment 10: , 56–65.


Joy S , Fein D , Kaplan E , Freedman M ((2000) ) Speed and memory in WAIS-R-NI Digit Symbol performance among healthy older adults. J Int Neuropsychol Soc 6: , 770–780.


Joy S , Kaplan E , Fein D ((2004) ) Speed and memory in the WAIS-III Digit Symbol–Coding subtest across the adult lifespan. Arch Clin Neuropsychol 19: , 759–767.


Andersen SL , Sweigart B , Glynn NW , Wojczynski MK , Thyagarajan B , Mengel-From J , Thielke S , Perls TT , Libon DJ , Au R , Cosentino S , Sebastiani P ; Long Life Family Study ((2021) ) Digital technology differentiates graphomotor and information processing speed patterns of behavior. J Alzheimers Dis 82: , 17–32.


Newman AB , Glynn NW , Taylor CA , Sebastiani P , Perls TT , Mayeux R , Christensen K , Zmuda JM , Barral S , Lee JH , Simonsick EM , Walston JD , Yashin AI , Hadley E ((2011) ) Health and function of participants in the Long Life Family Study: A comparison with other cohorts. Aging (Albany NY) 3: , 63–76.


Sebastiani P , Hadley EC , Province M , Christensen K , Rossi W , Perls TT , Ash AS ((2009) ) A family longevity selection score: Ranking sibships by their longevity, size, and availability for study. Am J Epidemiol 170: , 1555–1562.


Parjane N , Cho S , Ash S , Cousins KAQ , Shellikeri S , Liberman M , Shaw LM , Irwin DJ , Grossman M , Nevler N ((2021) ) Digital speech analysis in progressive supranuclear palsy and corticobasal syndromes. J Alzheimers Dis 82: , 33–45.


Kaplan E ((1988) ) A process approach to neuropsychological assessment. In Clinical neuropsychology and brain function: Research, measurement, and practice, Boll T, Bryant BK, eds. American Psychological Association, Washington, DC.


Kaplan E ((1990) ) The process approach to neuropsychological assessment of psychiatric patients. J Neuropsychiatry Clin Neurosci 2: , 72–87.


Libon DJ , Malamut BL , Swenson R , Sands LP , Cloud BS ((1996) ) Further analyses of clock drawings among demented and nondemented older subjects. Arch Clin Neuropsychol 11: , 193–205.


Rouleau I , Salmon DP , Butters N , Kennedy C , McGuire K ((1992) ) Quantitative and qualitative analyses of clock drawings in Alzheimer’s and Huntington’s disease. Brain Cogn 18: , 70–87.


Libon DJ , Penney DL , Davis R , Tabby DS , Eppig J , Nieves C , Wicas G , Lamar M , Price CC , Au R , Swenson R , Garrett KD , on behalf of the Clock Sketch Consortium ((2014) ) Deficits in processing speed and decision making in relapsing-remitting Multiple Sclerosis: The Digit Clock Drawing Test (dCDT). J Mult Scler 1: , 113.


Davis R , Libon DJ , Au R , Pitman D , Penney DL ((2014) ) THink: Inferring cognitive status from subtle behaviors. Proc Conf AAAI Artif Intell 2014: , 2898–2905.


Davoudi A , Dion C , Amini S , Tighe PJ , Price CC , Libon DJ , Rashidi P ((2021) ) Classifying non-dementia and Alzheimer’s disease/vascular dementia patients using kinematic, time-based, and visuospatial parameters: The Digital Clock Drawing Test. J Alzheimers Dis 82: , 47–57.


Davoudi A , Dion C , Formanski E , Frank BE , Amini S , Matusz EF , Wasserman V , Penney DL , Davis R , Rashidi P , Tighe P , Heilman KM , Au R , Libon DJ , Price CC ((2021) ) Normative references for graphomotor and latency digital clock drawing metrics for adults age 55 and older: Operationalizing the production of a normal appearing clock. J Alzheimers Dis 82: , 59–70.


Libon DJ , Swenson R , Ashendorf L , Bauer RM , Bowers D ((2013) ) Edith Kaplan and the Boston Process Approach. Clin Neuropsychol 27: , 1223–1233.


1 Oliver Wendell Holmes, Sr., MD, a resident of Cambridge, Massachusetts, was a very influential nineteenth century physician. In addition to his achievements as Dean of Harvard Medical School, Holmes was a member of an illustrious group of Boston literati and was a founder of the Atlantic Monthly magazine. In addition to his other achievements Dr. Holmes is credited for naming the city of Boston The Hub of the Universe.