Automated Video Analysis of Audio-Visual Approaches to Predict and Detect Mild Cognitive Impairment and Dementia in Older Adults

Article type: Research Article

Authors: Chu, Che-Sheng^{a; b; c; d} | Wang, Di-Yuan^e | Liang, Chih-Kuang^{b; f; g; h} | Chou, Ming-Yueh^{b; f; g} | Hsu, Ying-Hsin^{b; h; i} | Wang, Yu-Chun^b | Liao, Mei-Chen^b | Chu, Wei-Ta^{e; 1; *} | Lin, Yu-Te^{b; g; h; j; 1; *}

Affiliations: [a] Department of Psychiatry, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan | [b] Center for Geriatrics and Gerontology, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan | [c] Non-Invasive Neuromodulation Consortium for Mental Disorders, Society of Psychophysiology, Taipei, Taiwan | [d] Graduate Institute of Medicine, College of Medicine, Kaohsiung Medical University, Kaohsiung, Taiwan | [e] Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan | [f] Center for Healthy Longevity and Aging Sciences, National Yang Ming Chiao Tung University, Taipei, Taiwan | [g] Department of Geriatric Medicine, School of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan | [h] Division of Neurology, Department of Internal Medicine, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan | [i] Chia Nan University, Tainan, Taiwan | [j] School of Medicine, College of Medicine, National Sun Yat-sen University, Kaohsiung, Taiwan

Correspondence: [*] Correspondence to: Wei-Ta Chu, PhD, Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan. Tel.: +886 6 2757575; [email protected]; ORCID: 0000-0001-5722-7239 and Yu-Te Lin, MD, PhD, Center for Geriatrics and Gerontology, Kaohsiung Veterans General Hospital, No.386, Dazhong 1st Rd., Zuoying Dist., Kaohsiung City 813414, Taiwan. Tel.: +886 7 3422526; E-mail: [email protected].

Note: [1] These authors contributed equally to this work.

Abstract: Background:Early identification of different stages of cognitive impairment is important to provide available intervention and timely care for the elderly. Objective:This study aimed to examine the ability of the artificial intelligence (AI) technology to distinguish participants with mild cognitive impairment (MCI) from those with mild to moderate dementia based on automated video analysis. Methods:A total of 95 participants were recruited (MCI, 41; mild to moderate dementia, 54). The videos were captured during the Short Portable Mental Status Questionnaire process; the visual and aural features were extracted using these videos. Deep learning models were subsequently constructed for the binary differentiation of MCI and mild to moderate dementia. Correlation analysis of the predicted Mini-Mental State Examination, Cognitive Abilities Screening Instrument scores, and ground truth was also performed. Results:Deep learning models combining both the visual and aural features discriminated MCI from mild to moderate dementia with an area under the curve (AUC) of 77.0% and accuracy of 76.0%. The AUC and accuracy increased to 93.0% and 88.0%, respectively, when depression and anxiety were excluded. Significant moderate correlations were observed between the predicted cognitive function and ground truth, and the correlation was strong excluding depression and anxiety. Interestingly, female, but not male, exhibited a correlation. Conclusion:The study showed that video-based deep learning models can differentiate participants with MCI from those with mild to moderate dementia and can predict cognitive function. This approach may offer a cost-effective and easily applicable method for early detection of cognitive impairment.

Keywords: Artificial intelligence, dementia, machine learning, mild cognitive impairment, video analysis

DOI: 10.3233/JAD-220999

Journal: Journal of Alzheimer's Disease, vol. 92, no. 3, pp. 875-886, 2023