A Consensus Set of Outcomes for Parkinson’s Disease from the International Consortium for Health Outcomes Measurement

Working group

The formation of the Working Group was based on the principles of previous ICHOM working groups [11]. The PD Working Group consisted of 12 members from eight countries (USA, Canada, UK, Spain, Italy, Germany, Netherlands, and Sweden) and included expert neurologists (n = 9), a psychiatrist, and a nurse specializing in PD, as well as an experienced patient advocate (Table 1). Working Group members were identified by reviewing authors of leading papers on PD care quality, and by identifying members of international patient advocacy groups, leading international PD scientific organizations, and leading physicians in existing national and international quality measurement efforts.

Process

Following the process used in earlier ICHOM work [10, 11], a modified Delphi technique was employed to define the outcomes and case-mix variables. Case mix variables are defined as those variables that capture the state of the patient independent of the medical condition for which they are being treated. This includes demographic factors, health status (e.g. co-morbidities) and treatments. The process is a structured, consensus-driven approach, with teleconferences and post-teleconference surveys to reach decisions. Proposals for each teleconference were generated in advance by a core ICHOM project team (RU, TAK, PdR). These were based on a literature review of existing guidelines and standards, as well as individual interviews with each Working Group member.

The Working Group was officially announced in December 2013 and launched with an in-person meeting at the conference of the International Association of Parkinsonism and Related Disorders (IAPRD). This was followed by five 75-minute teleconferences, which took place every month between January and May 2014. All of these teleconferences were followed by a survey of the Working Group members to make decisions on key discussion areas. A 2/3 majority was required, being a commonly used threshold for Delphi and modified Delphi processes, on each survey question to reach consensus. Shifting the threshold a bit did not have an impact on the selection process. When a 2/3 majority was not reached, the topic was brought up for re-discussion at the following teleconference. The standard set of outcomes was then launched at the International Parkinson and Movement Disorder Society (MDS) Conference in June 2014.

The process began with defining the scope of the Working Group by deciding which causes of parkinsonism to include in the set. Subsequently, key outcome domains that are meaningful to patients were identified based on relevant literature and outcome measurement programmes [6–11]. These were then reviewed with each Working Group member individually to determine if additional domains, not identified by the search, should be considered. The resultant list of outcome domains was then organized based on four criteria. Each criterion was rated on a Likert scale of 1–4, where one was the lowest and four was the highest score given: (1) Frequency of the outcome domain in the patient population – an important consideration for a set that aims to be parsimonious; (2) Impact of the outcome domain on the patient – an essential consideration for a set that aims to reflect what is most meaningful to patients; (3) Preventability/treatability of the outcome domain – a necessary consideration for a set that aims to be used in the clinic to generate meaningful data on which clinicians can act to modify their practice; and (4) Feasibility to capture the outcome domain in clinical practice – this is essential as the set is designed to be used in routine clinical practice. This formed the basis for the first teleconference discussion.

Once the outcome domains were decided, the tools for data collection were determined. Relevant scales or items were identified and prioritized using specific criteria. Again, each criterion was rated on a Likert scale of 1–4, where one was the lowest and four was the highest score given. The criteria were as follows: (1) Domain coverage – this set aims to be of minimal burden and complexity. Thus, tools that cover many domains were preferable; (2) Psychometric properties – the data collected must be accurate, and thus patient-reported tools were prioritized based on psychometric properties; (3) Feasibility to implement – the tool must be practical for day-to-day use in the clinic; and (4) Clinical interpretability – clinical teams must be able to understand the results. This formed the basis for the second teleconference discussion. Finally, we sought to reach agreement on the frequency of data collection, balancing comprehensiveness, practicalities for clinics, and what would be best for patients.

This was followed by identification of the baseline case-mix variables, which are necessary to make meaningful comparisons between patients. Case-mix variables to measure were prioritized based on three criteria. Each criterion was rated on a Likert scale of 1–4, where one was the lowest score and four was the highest score given. The criteria were as follows: (1) Relevance (strength of association between the case-mix variable and the outcome) – we aimed to identify case-mix variables that could strongly affect the outcome; (2) Case-mix variable independency – given the aim to collect a minimum set of case-mix variables, the aim was to identify variables that would independently affect the outcome; (3) Feasibility to collect – the set must be practical for use in the clinic. This formed the basis for the third teleconference discussion.

The fourth teleconference focused on reaching agreement around internationally acceptable ways to measure case mix adjustment variables. The fifth teleconference focused on reviewing the set prior to its launch to the international community.

Literature search strategy

The following PubMed MeSH terms and Boolean logic were used to perform a search to identify outcomes that matter to PD patients, as well as scales to collect those outcomes: (“Parkinson’s disease” OR “Parkinson disease” OR “Parkinsonism”) AND (“critique” OR “recommendation” OR “review”) AND (“scale” OR “scales” OR “instrument” OR “instruments” OR “questionnaire” OR “questionnaires”). Limitations were applied, which included the need to be review articles, written in the English language, and published in the 10 years preceding January 2015.

From this search, article titles and abstracts were reviewed to identify those that had a clear focus on scales used in clinical practice. From these results, references to scales were extracted and through targeted searches, original validation studies and use of the respective instruments were identified.

Scope

The set was designed to cover all cases of adult (>18 years of age) idiopathic PD. Atypical parkinsonism was excluded, as the consensus was that this would require different outcome measures. We recommend that atypical causes of parkinsonism be considered in future outcome sets. This set is intended to be relevant to PD patients receiving all common treatment options for motor and non-motor symptoms, including pharmacotherapy (including infusion or injection-based delivery), deep brain stimulation, and rehabilitation-based therapy (including allied health interventions, nursing, and behavioral therapy).

Outcomes

A series of motor, non-motor and other outcomes were agreed upon by the Working Group as essential to collect.

Non-motor symptoms

Non-motor outcomes impact the ability of patients with PD to carry out normal day-to-day activities [12] and are key determinants of their perceived health [13, 14]. Based on the current literature, non-motor symptoms that are most important for PD patients were listed [6, 7, 11, 15, 16]. As described in Methods, the project team then prioritized this list and suggested the following outcome domains for inclusion in the standard set: depression, anxiety, cognitive function, urinary function, gastrointestinal function, pain, sleep, sexual function, treatment complications (hemorrhage and behavior change). These were deemed frequent, of high impact on patients, treatable and feasible to capture in clinical practice. During the teleconference, the group agreed with their inclusion but additionally felt that fatigue, hallucinations and sweating should also be included, due to their impact on patients. In the survey following the teleconference, the voting confirmed inclusion of the aforementioned outcomes with the exception of treatment complications – specifically, hemorrhage, as it is very uncommon, and behavior change, as this is captured under the cognitive and psychiatric domains. Additionally, the survey revealed that psychosis, apathy, impulse control disorder and dizziness/syncope were further domains deemed necessary to be part of the standard set, again due to their impact on patients. These were reviewed at the next teleconference and agreed by all WG members to be included in the Set. (See Table 2 for the full list of outcome domains and suggested scales).

A range of tools for data collection were identified. These included the Scale for Outcomes of Parkinson’s disease (SCOPA-AUT) [17], the Non-Motor Symptom Questionnaire (NMSQuest) [18], the Non-Motor Symptoms Scale (NMSS) [19], the Movement Disorder Society – Unified Parkinson’s Disease Rating Scale (MDS-UPDRS) [20], as well as specific scales relating to depression [21, 22], anxiety [23], apathy [24], psychosis [25], fatigue [26], sleep [27] and cognition [28, 29].

It was felt that it would be simpler and less burdensome for patients and health systems to have a single instrument rather than many individual patient-reported outcome measurements. A number of scales were considered, including NMSS [19], NMSQuest [19, 30], SCOPA-AUT [17] and MDS-UPDRS Part 1 [31, 32]. Ultimately, the MDS-UPDRS part 1 was chosen, as it has the highest test-retest reliability and internal consistency (as measured by Cronbach’s alpha), in comparison to the other tools, as well as having acceptable construct validity. Additionally, it poses minimal burden on the health system, with the clinician-recorded component taking <10 minutes to complete and the rest being patient reported [32]. Additionally, the MDS-UPDRS Part 2 (see below) is recommended for collection of the motor outcomes and thus it was felt simpler for clinics to use the MDS-UPDRS for both motor and non-motor assessment.

Two of the selected domains (sweating and sexual function) are not covered in the MDS-UPDRS part 1 survey, so it was decided to use the questions addressing these issues that are in the NMSQuest [21]. While not a perfect solution, the Working Group prioritized the selection of two simple, easy to administer, patient-reported questions. The Working Group encourages the MDS to consider including questions relating to sweating and sexual dysfunction in future iterations of the MDS-UPDRS.

We initially considered using the MDS-UPDRS part 1 as a screening tool for anxiety, depression and cognitive symptoms, and to use domain specific scales such as Beck Depression Inventory (depression) [33], State Trait Anxiety Inventory (anxiety) [34] and Montreal Cognitive Assessment (cognition) [35] to investigate these non-motor symptoms in more detail. However, it was decided that this would miss a key principle underpinning the work (i.e., to produce a practical, minimum set of outcomes that is of minimal burden to patients and staff). Therefore, only the MDS-UPDRS part 1 was included as part of the set.

Motor symptoms

Motor symptoms are an important problem in PD and their presence is relied upon to make a clinical diagnosis of PD. Motor features that were considered to be most important to the PD patient were identified and listed [6, 7, 11]. The outcome domains that the project team suggested including in the standard set (following the process set out under the methods) included: mobility – ability to walk; activities of daily living – living independently, handwriting and keyboard capabilities; ability to self-care; tremor; speech; swallowing; treatment complications (dyskinesia and dystonia).

During the teleconference (and confirmed by the post-teleconference survey) it was agreed to include these proposed outcome domains, and it was suggested and agreed upon in the post-call survey to include additional ones. The additional outcomes included: leisure activities, saliva and drooling, and ability to move in bed at night. These were agreed upon as they are domains that can have a significant impact on the patient’s quality of life. Ultimately, the only outcome domains from the initial list not to be included in the standard set were treatment complications – specifically, dyskinesia and dystonia – as it was felt that we should focus on motor function, not specific symptoms or side effects.

A wide variety of rating instruments were identified for different motor symptoms, including the Hoehn and Yahr staging [36, 37], the Schwab and England ADL scale [38], PD-related health status questionnaires [39] such as PDQ39 [40], the MDS-UPDRS, and scales which can be used to report motor complications, such as “wearing off” [41], risk of falling (including the Berg Balance Scale [42] and others [43, 44]) and mobility (Timed Get Up and Go Test) [45]. During the teleconference discussions it was agreed that many domain-specific scales would be needed and that this would be too burdensome and complicated for patients and clinical teams. Therefore, the MDS-UPDRS and the PDQ-39 were ultimately identified as the potential tools for data collection. The PDQ-39 is available in multiple languages and is free to use, but only covers 6/10 motor domains that we identified as being important. In contrast, the MDS-UPDRS part 2 questionnaire is also available in multiple languages and is free to use clinically but covers 10/10 domains. MDS-UPDRS part 2 has excellent psychometric properties [20]. Therefore, the MDS-UPDRS part 2 was decided as the motor tool of choice by the Working Group.

Additional health outcomes

We identified four additional domains as important for patients with PD: ability to work, hospital admissions, overall PD-related health status, and falls. These were selected by the group, particularly the patient representative, as important outcomes to assess. To assess ability to work, hospital admissions, and falls, the questions currently used in the recently developed Dutch National Parkinson’s Disease Registry (www.ParkinsonInzicht.nl), which cover these domains, were selected for use in the ICHOM set. The Dutch registry uses the PDQ-39 to assess PD-related health status. The PDQ-8 and PDQ-39 are comparable as health status indices, but the PDQ-8 is significantly less burdensome to complete [46–48]. We recognize the value of having a single PD-related health status score and decided to include the PDQ-8.

Finally, there was also a discussion around the assessment of cost of accessing care for the patient. While we agreed that cost is vitally important, it was best included not as an outcome but rather the denominator of the value equation. Reporting cost was therefore seen as out of the scope of this work.

Case-mix variables

Patients with PD have a broad range of characteristics both related and unrelated to their neurodegenerative disease that may influence their outcomes. A parsimonious set of case-mix variables (Table 2) that were felt to strongly impact outcomes, based on existing literature [49, 50] and informal discussions, was proposed. For demographic variables: age, gender, level of education, and living status (i.e. whether the patient was living alone) were proposed. Age and gender are associated with anxiety, cognitive function, urinary function, GI function, pain, sexual function and fatigue. Gender is associated with depression [51]. Level of education, gender and living status are associated with cognitive function [49, 52]. For baseline health status: early age at onset of PD, depression earlier in life, PD motor subtype, non-PD related cognitive dysfunction, non-PD related co-morbidities, and non-PD related medication affecting sleep, sexual function, and dizziness were proposed. During the teleconference it was suggested and agreed upon to include marital status as an additional demographic variable, as not being married is known to be associated with the risk for cognitive decline in the elderly general population [53]. Other constructs such as loneliness and social networks in late life also include marital status and are known to be correlated to cognitive function [50]. There was unanimous agreement to remove PD motor subtype and all medication side effects due to the difficulty of recording this information accurately. There was agreement to change early age at diagnosis of PD to age at diagnosis of PD, as there are conflicting views on the definition of “early”, while age would provide a more specific time point assuring less ambiguity in the data collected. For baseline health status, the age of PD onset and diagnosis, the diagnosis of depression, anxiety or rapid eye movement (REM) sleep behavior disorder (RBD) before PD diagnosis [53], and comorbidities were included. We agreed on definitions for each of the case-mix variables. For marital status and living status we decided to use the widely accepted definitions developed by the European Social Survey [54]. For level of education, the United Nations Educational, Scientific and Cultural Organization (UNESCO) definitions of education levels, which allow for international and cross-cultural comparisons, were selected [55]. We decided to change the term “tertiary” to “University or equivalent” as it was felt that this wording would be easier for patients and care providers to understand. For the case-mix variables, depression and anxiety, we developed two new yes/no questions. We agreed to include a single baseline patient-reported question used to assess previous REM sleep behavior disorder [53]. A validated patient-reported Charlson Comorbidity Index currently in use by the United Kingdom National Health Service [56] was chosen to reduce data collection burden on physicians.

Data collection

In order to be able to easily compare between providers, centers and countries, the use of established instruments with multiple translations was prioritized and data collection methods that can be applied across different countries and settings were proposed. We aimed to reduce the reporting burden on clinicians and as such the vast majority of outcomes in the set are patient-reported, with the exception of the cognitive and mental health outcomes. We recommend all outcomes to be recorded annually.