Virtual Reality Functional Capacity Assessment Tool (VRFCAT-SL) in Parkinson’s Disease

Background: Cognitive impairment is common in Parkinson’s disease (PD) and highly associated with loss of independence, caregiver burden, and assisted living placement. The need for cognitive functional capacity tools validated for use in PD clinical and research applications has thus been emphasized in the literature. The Virtual Reality Functional Capacity Assessment Tool (VRFCAT-SL) is a tablet-based instrument that assesses proficiency for performing real world tasks in a highly realistic environment. Objective: The present study explored application of the VRFCAT-SL in clinical assessments of patients with PD. Specifically, we examined associations between VRFCAT-SL performance and measures of cognition, motor severity, and self-reported cognitive functioning. Methods: The VRFCAT-SL was completed by a sample of 29 PD patients seen in clinic for a comprehensive neuropsychological evaluation. Fifteen patients met Movement Disorders Society Task Force criteria for mild cognitive impairment (PD-MCI); no patients were diagnosed with dementia. Non-parametric correlations between VRFCAT-SL performance and standardized neuropsychological tests and clinical measures were examined. Results: VRFCAT-SL performance was moderately associated with global rank on neuropsychological testing and discriminated PD-MCI. Follow-up analyses found completion time was associated with visual memory, sustained attention, and set-switching, while errors were associated with psychomotor inhibition. No clinical or motor measures were associated with VRFCAT-SL performance. Self-report was not associated with VRFCAT-SL or neuropsychological test performance. Conclusion: The VRFCAT-SL appears to provide a useful measure of cognitive functional capacity that is not confounded by PD motor symptoms. Future studies will examine utility in PD dementia.


INTRODUCTION
Parkinson's disease (PD) is a progressive neurodegenerative disorder involving both motor and non-motor symptoms. Cognitive deficits are common in PD, with an approximate 40% meeting criteria for mild cognitive impairment (PD-MCI) and 80% or more progressing to Parkinson's disease dementia (PDD) over the course of illness [1,2]. Impaired cognition is highly associated with loss of independence and caregiver burden, and the strongest predictor of need for placement in assisted living facilities [3,4].
In movement disorder clinics, neuropsychological evaluations are useful for determining whether subjective reporting of decline in memory and cognition by the patient is supported by objective findings of deficits on standardized tests [5]. Test performances determine the magnitude of impairment with cognitive domains, and the overall profile is interpreted with respect to differential diagnosis of changes typical for PD versus other etiologies. Neuropsychological tests also serve as a proxy for impairment in everyday life. For instance, if an individual performs poorly on standardized tests of attention, they may have difficulty performing tasks such as balancing a checkbook or counting pills [6]. Impairment on a test of verbal fluency may relate to word-finding difficulties in conversation [7]. However, such associations are certainly not perfect, and the need for ecologically valid measures of functional ability have been emphasized in the literature [8][9][10]. Such measures have also become increasingly emphasized by the Food and Drug Administration (FDA) as outcome measures in clinical trials [11].
Several informant-based questionnaires have been validated for use in assessing cognitive functional capacity in PD. However, knowledgeable informants are not always present during clinical evaluations, and these ratings can be biased by caregiver mood, burden, and other factors [12,13]. Anosognosia of cognitive deficits [14,15] and motor impairment [16] in PD also present limitations on the reliability of patient self-report.
Several standardized tests of real-world functioning have been developed and explored in PD populations. The Everyday cognition battery assesses medication use, financial planning, and food preparation and nutrition skills through a series of tasks that involve reading, episodic memory, working memory, and inductive reasoning [17]. It has been shown to be valid in assessing older adults with subjective cognitive decline and demonstrated sensitivity to treatment with rivastigmine in a clinical trial with PD [18]. However, administration procedures are rather complex and completion time of one hour limits application in both clinical evaluations and research. The University of California Performancebased Skills Assessment (UPSA) assesses abilities on tasks related to finances, communication, planning/organization, travel, and household chores and was developed to be used as a functional outcome measure in schizophrenia trials [19]. It takes about 30 minutes to administer and was recently validated in a PD population against a battery of standardized neuropsychological tests. Performance on the UPSA was found to be significantly related to standardized neuropsychological tests even after controlling for demographic factors and PD motor symptoms and discriminated PD with normal cognition from PD-MCI and PDD [20]. The Complex Task Performance Assessment (CTPA) requires examinees to execute the role of librarian in a structured format [21]. This includes managing inventory, listening and responding to telephone messages, and planning. It was developed for use in older adults and also explored in a recent study of PD patients without cognitive impairment. The investigators found that completion time discriminated PD patients from healthy controls, but error rates were similar [22]. Moreover, PD patients on average took about 40 minutes to complete the CTPA, which is near the published cutoff time.
The Virtual Reality Functional Capacity Assessment Tool (VRFCAT) is a tablet-based task that requires examinees to complete a series of real-world tasks in a virtual-reality environment [23]. The task was developed with grant support from the National Institute of Health (NIH) and the FDA to serve as a measure of functional improvement in clinical trials. Like the UPSA, it was originally standardized for use in schizophrenia, with subsequent studies demonstrating validity for assessment of functional capacity in older adults with cognitive impairment [24,25]. The VRFCAT was developed to measure four different functional abilities: meal preparation, using transportation, shopping, and managing currency. These scenarios were developed using immersive "first-person" gaming technology. The realistic, interactive environment challenges the examinee to a series of tasks related to making a food dish. First, they explore a kitchen to see what items they have for the recipe. Then, they use a bus schedule to find a bus that will take them to a grocery store. They find and purchase the necessary items at the store and use the schedule once again to find the bus that will take them home. Exact change is needed for both bus trips and to purchase the food. Patients sequentially complete the scenarios through a progressive storyboard design. Across the 4 scenarios, there is total of 12 different tasks or "objectives". For each objective, participants who are unable to complete the objective within a pre-specified time period are "pushed" to the next objective (referred to as "Forced Progression"). The assessment takes approximately 30 minutes to complete. The primary VRFCAT end point is Total Time. Secondary endpoints are Total Errors and Total Forced Progressions.
An abbreviated version of the test, the VRFCAT-SL was recently developed that reduces completion time to less than 15 minutes in healthy older adults. Aside from technical assistance that might be needed from the examiner, the task is self-administered and scoring is automated. Normative data with gender, age, and education correction are available from the publisher. These features, along with elimination of additional test materials and manipulatives, make the VRFCAT-SL particularly attractive for assessing functional abilities within the context of a comprehensive neuropsychological evaluation. There are also multiple versions available to reduce practice effect with serial testing.
We recently began administering the VRFCAT-SL to PD patients referred for clinical neuropsychological evaluations to assess functional capacity. The present study is a retrospective chart analysis to explore utility of the VRFCAT-SL as measure of functional capacity in clinical neuropsychological evaluations of subjective cognitive decline in PD. Specifically, we sought to determine whether VRFCAT-SL performance measures were associated with scores on standardized neuropsychological tests and sensitive to cognitive impairment in PD. We also wanted to assess the potential impact of motor disturbances on ability to complete the VRFCAT-SL. Finally, we examined whether the use of a standardized self-report scale for impact of cognition changes on instrumental ADLs, the Parkinson's disease Cognitive Function Rating Scale (PD-CFRS) [26], would provide similar information regarding real-world functional ability.

METHODS
This retrospective chart review was approved by the Human Subjects Protection Program at the Medical University of South Carolina, Pro00098834.
Participants were 30 consecutive patients followed in a tertiary care Movement Disorders clinic for idiopathic PD at an academic medical center who were referred for neuropsychological evaluation. PD diagnosis was made according to UK Brain Bank criteria by a fellowship-trained movement disorder neurologist. All patients were referred for neuropsychological evaluation for concerns of cognitive deficits as noted by the patient, informant, or referring neurologist. The VRFCAT-SL was administered to all patients seen during this time period (i.e., none were excluded from analysis). Administration was paused after these 30 assessments to perform these analyses related to quality assurance. One participant was unable to complete the VRFCAT-SL due to visual impairment (advanced age-related macular degeneration); this individual was also not able to complete other tests with small visual stimuli, the PD-CFRS, and other self-report inventories.
The first author (THT) performed all neuropsychological evaluations and administered the VRFCAT-SL. A core battery of tests was administered that included: Mattis Dementia Rating Scale (DRS), Brief Visuospatial Memory Test [27] (BVMT-R, Total Learning and Delayed Recall), Hopkins Verbal Learning Test [28] (HVLT-R, Total Learning and Delayed Recall), Judgment of Line Orientation (JOLO) [29], Neuropsychological Assessment Battery (NAB) [ [32]. Other tests were administered as indicated. The Geriatric Depression Scale (GDS) [33] and Geriatric Anxiety Inventory (GAI) [34] were also administered. Raw scores were corrected for demographics according to test manuals (BVMT, HVLT, and Hayling corrected for age; NAB corrected for age and education; COWAT and Trails for age, gender, education, and ethnicity) and converted to scaled scores. Diagnosis of cognitive status was based on clinical history and performance on standardized neuropsychological testing in keeping with Movement Disorders Society Task Force Level II criteria for PD-MCI [35]. Specifically, the presence of two or more scores within one domain falling more than 1.5 SD below normative expectations were interpreted to reflect MCI.
All standardized testing was completed first, followed by the PD-CFRS, and then the VRFCAT-SL. Performance on the VRFCAT-SL was interpreted qualitatively with respect to implications for functional independence but not was not considered for diagnosis. For all analyses, VRFCAT-SL performances were corrected for age, gender, and education  Figure 1). Notably, only 4 patients had any forced progressions, and just one patient had 2 forced progressions. Accordingly, non-parametric tests were used for all analyses (i.e., Spearman-Brown for correlations and Mann-Whitney U-tests for group comparisons).
To examine association between VRFCAT-SL performance and overall cognition, a global cognitive index for each participant was derived based on the average rank (higher = better) for all standardized neuropsychological tests excluding the DRS-2. Similarly, a global score on the VRFCAT-SL was derived based on the average rank for its primary outcome measures (time, errors, and forced progressions). In exploring associations between VRFCAT-SL measures and neuropsychological test performances, the Bonferroni method was used to control for increased Type-I error rate associated with multiple comparisons (i.e., 17 measures, p < 0.002). SPSS version 25 ® was used for all statistical analyses.

RESULTS
Measures of central tendency for demographic, clinical characteristics, and test performances are provided for the entire sample, and separately by cognitive status group in Table 1. Boxplots illustrating

Association with standardized neuropsychological tests
Spearman-brown correlation analysis assessed relationship between overall VRFCAT performance measure (i.e., corrected Time, Errors, and Forced Progressions) and standardized neuropsychological testing. As shown in Fig. 1, there was a modest association between overall ranks, rho = 0.404, p = 0.037. The relationship was not driven by outliers. Overall standardized neuropsychological test performance was associated VRFCAT-SL time, rho = 0.532, p = 0.004, but not Errors or Forced Progressions.
Relationships between VRFCAT-SL outcome measures and primary neuropsychological outcome measures were also explored. As shown in Table 2, T-score for VRFCAT-SL Time was positively asso-  correlations were significant at the p < 0.05 level, and none of the associations exceeded Bonferroni correction for multiple comparison (i.e., 17 measures, p < 0.002). Raw total score from the DRS-2 was not significantly associated with any VRFCAT-SL measure.

Sensitivity to cognitive status
About one-half of the sample (n = 15) met MDS Task Force Level II criteria for PD-MCI. Independent sample Mann-Whitney U-tests compared performance on VRFCAT-SL outcomes between cognitive status groups. Statistically significant group differences were found for Time, p = 0.001, and Errors, p = 0.005. Statistically significant group differences were not observed for Forced Progressions; however, all 4 patients with a Forced Progression were in the PD-MCI group. Notably, group differences based on cognitive status were also not observed on the DRS-2, p = 0.331.

Association with motor symptoms
Spearman-Brown correlation analyses explored impact of PD motor disturbance on VRFCAT-SL performance. No statistically significant correlations were observed between any VRFCAT-SL measure and duration of illness (years), levodopa equivalent daily dose, or MDS-UPDRS Part III scores. When analyzed separately by cognitive status, a trend was seen for the relationship between Errors and durations of illness, rho = 0.500, p = 0.069 in the cognitively normal group (n = 14). In the PD-MCI group (n = 15), a statistically significant correlation was observed between Time and LEDD, rho = -0.588, p = 0.021, and a trend was observed between Errors and LEDD, rho = -0.477, p = 0.072.

Association with self-reported impact of cognition on instrumental ADLs
The composite rank for VRFCAT-SL performance was not associated with self-report on the PD-CFRS, rho = 0.067, p = 0.774. Follow-up analyses did not reveal any statistically significant associations with VRFCAT-SL Time, Errors, or Forced Progressions. Self-report on the PD-CFRS was also not associated with composite rank on the standardized neuropsychological tests, rho = -0.289, p = 0.170. Though falling short of a trend, the direction of this association suggests greater subjective impairment in those with worse performance on neuropsychological testing. Mann-Whitney U-tests indicated that scores from the CFRS also did not differ between cognitive status groups, p = 0.531. PD-CFRS scores were also not associated with motor disturbance (MDS-UPDRS Part III scores), rho = 0.199, p = 0.428. However, clinically significant associations were observed between PD-CFRS ratings and self-report depression (GDS), rho = 0.621, p = 0.001, and anxiety (GAI), rho = 0.412, p = 0.046.

DISCUSSION
Current results support use of the VRFCAT-SL for providing an efficient global assessment of functional cognition in PD patients with subjective cognitive decline. A modest correlation was observed between composite ranks for overall VRFCAT-SL and standardized neuropsychological measures. VRFCAT-SL performance measures also discriminated patients meeting MDS Task Force Level II criteria for PD-MCI from PD patients with normal cognition. Importantly, there were no indications that motor impairment significantly impacts task performance. Taken together, our findings using the VRFCAT-SL are consistent with those previously reported using other functional capacity tools. As with the CTPA, the VRFCAT-SL identified mild cognitive impairment in PD [22]. This study also replicated findings using the UPSA, which not only showed that performances differed between PD normal and PD-MCI, but those performances were associated with cognition independent of motor symptoms [21]. However, the average completion time of about 13 minutes for the VRFCAT-SL was found to be much lower than reported times for the UPSA (30 minutes), CTPA (40 minutes), and ECB (60 minutes) [17]. Thus, in addition to tablet-based administration and automated scoring, reduced time demands associated with the VRFCAT-SL facilitates inclusion in more comprehensive clinical and research batteries.
Findings from this study did not support the use of patient's self-report on the PD-CFRS for assessing impact of cognitive deficits on instrumental ADLs. While Kulisevsky et al. suggest use of the PD-CFRS for self-report [26], in our sample PD-CFRS ratings were not associated with VRFCAT-SL performance or standardized neuropsychological measures and did not differ between cognitive diagnosis groups. Rather than reflecting cognition and motor function, PD-CFRS ratings in this sample appeared to capture anxiety and depressive mood symptomatology. This finding is in keeping with a number of prior studies describing anosognosia for both cognitive and motor disturbances in PD [14][15][16].
One limitation of the VRFCAT-SL with respect to providing a comprehensive functional capacity index in PD is the lack of a medication management task. Most patients with progressed PD follow a complex regimen involving variable doses of one or more dopamine replacement therapies throughout the day, as well as medications that might be taken for non-motor symptoms and other health conditions. The ability to reliably take medications as prescribed has presents a significant cognitive challenge and obvious relevance to overall functioning. Lack of a medication condition is also a weakness of the UPSA and CTPA. The ECB includes several items that require participants to remember information contained on a pill bottle and answer multiple choice questions but does not measure ability to actually follow a medication schedule. As such, this would appear to be more of a test of attention, learning, and memory with medication-themed stimuli, rather than a medication adherence task, per se. The Medication Management Abilities Assessment (MMAA) [37] was specifically designed to assess to evaluate this domain. The MMAA requires examinees to learn instructions for taking 4 different medications, retain this information over a one-hour delay, and properly distribute the correct number of pills along with instructions (e.g., with or without food) to the rater. In a study by Pirogovsky et al., PD patients meeting criteria for mild cognitive impairment based on standardized neuropsychological evaluation performed worse than those with normal cognition and a healthy control group [10]. The ecological validity of the VRFCAT-SL in PD might therefore be improved if medication adherence were assessed, perhaps by requiring examinees to observe the passing of time and take virtual pills as directed within a certain time window on several occasions throughout the task.
A methodological limitation of this study is that the provider (THT) performed neuropsychological testing, administered the VRFCAT-SL, and interpreted test performances. While cognitive status was based on clinical history and standardized neuropsychological testing, the possibility of diagnostic bias from observing VRFCAT-SL performance must be considered with respect to sensitivity and specificity estimates for discriminating PD-MCI. This study also included a relatively small sample of PD patients, and only a subset of participants had MDS-UDPRS Part III scores thus limiting generalizability. Future studies with neuropsychological evaluation independent of VRFCAT-SL administration are therefore recommended.
In the course of this study, no patients were found to meet MDS Task Force criteria for dementia in PD. This is not particularly surprising given that referring Movement Disorder neurologists generally request neuropsychological testing to identify mild cognitive impairment, while diagnosis of dementia is based on more obvious deficits and loss of independence. Future studies are thus needed to determine whether this can be used effectively in PD dementia populations.
Taken together, the results from this retrospective chart review provide provisional support for the VRFCAT-SL as an efficient tool for obtaining a global estimate of cognitive functioning in moderately advanced PD. Compared to other standardized functional capacity tools that have been used in PD, the tablet-based VRFCAT-SL eliminates the need for manipulatives, reduces administration and scoring complexities, and appears to be less burdensome with respect to time. Further investigation is required to evaluate test-retest reliability and determine whether it provides an objective measure of functional cognitive capacity across the full spectrum of disease progression.

CONFLICT OF INTEREST
Travis H. Turner is a consultant for VeraSci, Inc., Alexandra Atkins is an employee of VeraSci, Inc., and Richard S. Keefe is an owner of VeraSci, Inc.
VeraSci is a for-profit company that provides services to over 100 business entities, most of which are pharmaceutical companies. https://verasci.com.

SUPPLEMENTARY MATERIAL
The supplementary material is available in the electronic version of this article: https://dx.doi.org/ 10.3233/JPD-212688.