A Canadian Adult Spinal Muscular Atrophy Outcome Measures Toolkit: Results of a National Consensus using a Modified Delphi Method

INTRODUCTION

Spinal Muscular Atrophy (SMA) is a recessive neuromuscular genetic disorder which affects 1 in 11 000 live births and has a prevalence of approximately 1-2 per 100,000 persons [1, 2]. There are at least 33 causative genes identified for conditions with SMA manifestations, with the most common causative gene being the SMN1 gene located on the long arm of chromosome 5, which produces a variety of SMA known as 5q SMA [3]. 5q SMA encompasses 92%of all SMA, of which 95%have a homozygous deletion of SMN1 resulting in a significant decrease in the amount of functional SMN protein [2, 4]. 5q SMA disease severity is modulated by the number of copies of SMN2, a low-functioning homologue of SMN1 which produces a lesser amount of functional SMN protein [2, 4]. The number of copies of SMN2 naturally varies within the general population and is inversely related to disease severity of 5q SMA [2–4].

Recently, there have been significant advancements in SMA treatments which utilize therapeutic mechanisms such as SMN1 gene replacement via viral vectors [5], synthetic olignoucleotides and small molecules which encourage production of SMN1 protein via altered SMN2 splicing [6, 7], neuroprotection, and enhancement of nerve and/or muscle function [4, 8]. These therapeutic advances have widened the spectrum of clinical presentation and will lead to a higher disease prevalence due to longer life expectancies [6, 9]. Despite progress in treatments, there is a paucity of data on the natural history, prevalence, and progression patterns of SMA, particularly in adults [10].

Given the changing therapeutic environment of SMA and the crucial need for high quality real-world data, routine use of a standardized toolkit of nationally agreed upon outcome measures (OM) will advance our ability to elucidate important patterns of disease trajectory and response to therapy, enabling a deeper understanding of both the disease and the effectiveness of emerging treatments [11]. Furthermore, the creation of a standard consensus-derived toolkit will support the development of best practices. Due to the heterogenous nature of disease progression and multi-system involvement, this toolkit shall include a diverse set of OM which capture, but are not limited to, patient reported outcome measures (PROM), and domains of gross motor function, fine motor function, quality of life, and respiratory function. McGraw et al. and Sansone et al. both advocate for the inclusion of OM, which assess a wide range of activities and capture meaningful clinical changes [12, 13]. To date, few efforts have been made to derive consensus on which OM should be used to assess adult SMA. The purpose of this study was to derive a national consensus on a set of recommended OM and frequency of assessment for adult SMA.

MATERIALS AND METHODS

A modified delphi method was chosen as the process to achieve a national consensus on OM. The delphi method is an anonymized serial survey methodology to elicit expert opinion in an iterative voting process to distill judgements into a consensus [14, 15]. The delphi method has been widely used in healthcare for making decisions with conflicting evidence, poor availability of evidence, or when judgemental information is necessary [15–18]. The study participants (the study authors with exception to JS, VH, and JL) were clinical leads or directors from clinics treating adults with SMA in Canada, and members of the Canadian Neuromuscular Disease Network. Participants were encouraged to discuss the OM project with their teams throughout the process. The delphi process began in June 2019 and was completed in April 2020. This study modified the delphi method by including a final virtual conference following the survey-based voting rounds, during which all participants could discuss and interpret the results of the voting rounds (Fig. 1) and achieve a final consensus.

Fig. 1

Flowchart of modified delphi methodology.

In order to effectively vote on appropriateness of outcome measures, an initial table was developed to provide information and background on validity and reliability of currently available OM. Firstly, a literature review of OM in SMA was conducted, and a questionnaire sent to the participants requesting input on OM for consideration. Following the literature review and participant feedback, a comprehensive table of all potential OM was generated. The OM table included available evidence of validity, reliability, clinical application and domain (e.g. adult/paediatric, gross motor/fine motor), and any additional information (such as time to completion, resources required, and references) for each of the OM (Supplementary Table 1). The voting surveys were developed and delivered through both google forms (first voting round) [19] and SurveyMonkey (second voting round) [20], and were distributed through the Canadian Neuromuscular Disease Registry (CNDR) Network [11].

The first voting round asked the participant to assess the quality and the clinical utility of each of the OM. Participants were provided with and instructed to refer to the comprehensive table of OM, including the embedded links to available publications and OM instruction manuals. This round served to integrate all of the clinical expertise of the participants into a global rating. The survey used a 5-point likert scale from ‘very weak’ (1) to ‘very strong’ (5). The first voting round was available for 8 weeks, after which the results were analyzed. The OM were grouped based on cut point values of the mean and lower bounds of the standard deviation. A cut point of a mean score ≥3 (neither weak nor strong) was determined for inclusion in the next round of voting. A report was produced (Supplementary Report 1) outlining the results of the voting, which was then distributed to the participants prior to the next voting round.

The second round of voting rated the remaining OM across stages of disease, and their frequency of use. Participants were instructed to consider the quality and utility of the measures, and not necessarily the OM currently used in their practice. The OM were assessed for each of 3 functional populations of patients (non-sitters, sitters, and walkers). The frequency of use ratings identified which OM were used in practice, and thus could more easily be implemented if included in the toolkit. The response options were also expanded to allow respondents who were unfamiliar with a particular outcome measure to opt out of that question. This modification allowed for an informed ranking of OM by limiting the responses to participants familiar with the OM in question. The second round of voting was available for 10 weeks, after which results were analyzed and a second report was compiled and distributed to the participants (Supplementary Report 2). Once all participants had reviewed the report, a virtual conference was held with the participants to discuss the results of both rounds and to generate a final consensus-derived toolkit of recommended OM. This last step allowed for participants to discuss the merits and/or barriers of the selected measures, and included agreement to re-visit recommendations in 1-2 years, as new evidence becomes available.

As a secondary initiative that evolved from the virtual conference, the participants requested a post-hoc survey to determine the optimal frequency of clinical assessment of OM, in part due to a range of provincial regulatory criteria tied to reimbursement and drug approvals. The post-hoc survey aimed to describe clinically appropriate assessment timing and frequency of each of the recommended OMs while considering both treated (i.e –on a disease-modifying medication) and untreated (i.e –natural history) populations. Additionally, the post-hoc survey aimed to gather baseline information on interprovincial variance of assessment frequency practice.

RESULTS

A total of 34 OM were identified through literature review and questionnaire of Canadian SMA adult clinicians. From these 34 OM, 44 peer-reviewed articles and supporting literature were identified which provided supplementary information for the OM included in the voting process (Supplementary Table 1).

Health care providers from ten clinics treating adults with SMA (neurologists, physiatrists, physiotherapists) across Canada participated in each of the first two rounds of online voting. After the first round, the cut point (mean score ≥3) narrowed the number of OM from 34 to 21 (Table 1; Fig. 2). The results from the second round of voting for each of the functional populations are outlined in Table 1. The most frequently used OM were Forced Vital Capacity (FVC), Peak Cough Flow (PCF), Maximum inspiratory and expiratory pressures (MIP and MEP), Hammersmith Functional Motor Scale –Expanded (HFMSE), Revised Upper Limb Module (RULM), 6 Minute Walk Test (6MWT), and Timed Up and Go Test (TUG). Other assessed measures in the second round were used only sporadically.

Table 1

Round 1 and 2 results of OM which were included in 2^nd voting round. The mean is scored on a scale of 1-5 with 5 (very strong), 3 (neither weak nor strong), and 1 (very weak)

*Included for future considerations; ∼Combined count of Always and Often responses; Bolded OM are those included in the final consensus; Shading indicates type of OM. CHOP-ATEND: Childrens Hospital of Philadelphia – Adult Test of Neuromuscular Disorders.

Fig. 2

Mean ratings of outcome measures from round 1. Grey shading indicates outcome measures below cut point of 3 “Neither weak nor strong”. Error bars indicate standard deviation.

During the final virtual conference, consensus was achieved to recommend the routine use of 8 OM across each of the functional populations (Table 2). The final 8 OM have administration time estimates between 3 minutes (TUG), and 30–40 minutes (HFMSE) with a total estimated time required to administer the OM being between 70 and 85 minutes for each of the three populations. Although the SF-36, Motor Function Measure (MFM), and MIP/MEP were not included in the final consensus group, they were recommended as optional as the participants determined them to be highly important and with suggestion to re-consider within a 2 year period as evidence and clinical needs evolve.

The post-hoc frequency survey was completed by 9 clinics across Canada (Ontario, Alberta, New Brunswick, Quebec, Saskatchewan and British Columbia). The survey identified that adult SMA patients should be evaluated on a yearly basis, while patients receiving disease-modifying therapy should be assessed every 6 to 12 months. All OM included in the final consensus are best assessed on a yearly basis, with exceptions to the FVC and PCF which both had a mixed response of a 6 to 12 month frequency (Table 3). Additionally, the post-hoc frequency survey identified two major barriers for clinicians in conducting outcome assessments: 1. Availability of and access to allied health professionals and 2. lack of available clinical time.

DISCUSSION

In rare diseases such as SMA, the use of several different OM nationally and internationally creates challenges in monitoring disease progression, ability to pool data, or assessing therapeutic response and resource utilization. This study provides a consensus-derived toolkit developed via a modified delphi method with national experts who treat adults with SMA. The national OM toolkit can lead to improved monitoring and larger, robust datasets which are of great importance in the adult SMA population [1, 2]. Operationalizing a national toolkit with a patient registry such as the Canadian Neuromuscular Disease Registry (CNDR) [11] provides a valuable data resource for clinicians and researchers to facilitate further insight into SMA [13].

The final OM are specific to each of the three functional populations and were chosen with careful consideration to their practicality, feasibility, supporting evidence, domain of function, and clinical importance. Among the 8 OM included in the final consensus, there are 5 functional measures (3 gross and/or fine motor, 2 ambulation), 2 respiratory measures, and 1 PROM. Each of the included OM take less than 30 minutes to conduct, with a total estimated administration time of less than two hours.

The SF-36 was included in the optional group of OM and recommended for future consideration, as there is a limited pool of OM which assess quality of life in adult SMA population. Additional quality of life measures for SMA should be developed to better capture these needs [12, 21]. The MFM was felt by the participants to be comparable to the HFMSE but was excluded from the final consensus due to a lack of familiarity or experience with the measure, but participants recognize the evolving field. The MFM is currently widely used in Europe [22], and has been the primary motor measure selected for some clinical trial programs [23]. Over time, additional evidence for the MFM will be generated and continued monitoring of the MFM is important, with future reconsideration in the toolkit. The MIP and MEP were included as optional OM as they were considered by the participants to have high intra-individual variability due to reliance on patient effort [24]. The intra-individual variability and the availability of more reliable alternatives such as the FVC and PCF, led the participants to select FVC and PCF as the recommended respiratory measures.

The results of the present study are fundamentally in accordance with recent research, which have largely included the same OM as those in the present consensus. Sansone et al. recently recommended OM in adult SMA based upon expert opinion of international neuromuscular experts [13] and their results align with the findings of the present study, with differences only in the selection of the SMAFRS and the TUG. Similarly, Hagenacker et al., Walter et al. and Maggi et al. studied the effects of nusinersen (an SMN2-targeted anti-sense oligonucleotide [6]) in an adult SMA population and all utilized the HFMSE, RULM, and 6MWT [25–27] in evaluation of therapy effectiveness and safety. In 2016, Mercuri et al. selected the HFMSE as the primary outcome measure when reporting upon the natural progression of SMA in children and adults [28]. In a natural history study by Chabanon et al. the MFM, Pulmonary Function Tests (including FVC, PCF, MIP and MEP), and the 6MWT were included among additional OM [22]. A 2019 report by the Institut National d’Excellence en Santé et en Services Sociaux (INESSS) in Québec, recommended the CHOP-ATEND, HFMSE, RULM, 6MWT and the SMAFRS as adult SMA OM [29]. The SF-36 was chosen as an optional measure over the PedsQL 4.0 in the present study, although neither OM is particularly well suited to the adult SMA population.

The final consensus-derived toolkit of recommended OM is not fully comprehensive, and there remain recognized gaps in the assessment of adult SMA. Consistent with the literature, the participants in our study indicated poor availability of OM that assess important domains of quality of life, bulbar function (speech and swallowing), fine motor skills, and endurance (particularly in non-sitter and sitter populations) [4, 10, 12, 25, 30]. Capturing additional domains is important to further increase sensitivity to detect changes and to better assess other aspects of life [12]. For example, Sansone et al. described the importance of finger and hand strength, while Kizina et al. and Binz et al. examined the presence of fatigue [13, 31, 32]. Bulbar function was previously identified by Rouault et al. as a priority for stabilization for persons with SMA II and III [30]. Additional domains such as bone health, metabolic effects and participation reflect larger aspects of a persons health. In addition to standardized OM, PROM are integral in capturing individual needs and goals and can provide unique person-focused data [13, 33]. Development of additional standardized OM and PROM is needed to capture the breadth of changes and systemic nature of the disease [12].

Within Canada, at the time of publication, provincial drug reimbursement requirements ranged from assessments every 4 to 12 months, or after every injected drug dose, with varying OM province to province [34]. The post-hoc frequency survey identified an evaluation frequency of one year for adults with SMA, with those receiving disease-modifying therapy being assessed every 6 to 12 months. This recommendation is in keeping with the consensus findings of Hodgkinson et al. which suggested visits should occur at 6 months, and annually thereafter [34]. Advocating for a unified assessement recommendation that is based on clinical expertise and outcome measure sensitivity is a needed step towards providing evidence-based care.

There are a number of barriers to routine use of OM in the clinic setting; some OM require specific training or specialized equipment, certain measures are time consuming and resource heavy to administer (patients may need to be transferred with a lift), others may be burdensome or fatiguing for the patient to complete. Such factors can lead to variable time commitment for OM completion, as well as poor compliance with OM administration. An efficient system for performing OM depends on having the necessary support for required resources: equipment, time, space and health professionals. Further, poor sensitivity to change coupled with floor and ceiling effects may limit the utility of any particular outcome measure. Developing OM which are easily administered, sensitive and reliable can greatly facilitate their uptake [13, 35]. Alternatively, introducing additional means to provide access to allied health professionals and increased clinical time is also important. The finding of limited access to appropriate health professionals and clinic time underscores the importance of interdisciplinary and interspecialty care in the treatment of adult SMA, and need for valid and reliable PROM and OM which do not require intensive use of clinical resources. Another consideration is the timing of outcome collection. Due to the slow progression of adult SMA, too frequent evaluations may be burdensome and yield little change; data collection schedules informed by long-term real-world studies could aid in optimizing data quality.

Due to the evolving nature of the field, it is possible that some standardized OM may have been missed. While this study did not formally correct for the potential bias of multiple respondants within a province, all CNDR adult SMA sites were engaged throughout the process, representing a majority of clinical adult SMA expertise in Canada. It is possible that factors such as trial participation experience, resource availability, and regulatory criteria could have influenced individual OM ratings despite instructions to rate the OM based on clinical experience, review of the literature, and the evidence available, rather than strictly current practice or regulations. Lastly, this study did not separate OM selections into patients with 5q SMA, and those with non-5q SMA; despite this limitation, 5q SMA represents the vast majority of all SMA patients (96%) [2], and clinicians with non-5q SMA patients may require additional or different OM.

The implementation of this toolkit as standard practice in SMA clinics will help capture standard real-world evidence and allow a deeper understanding of both the natural history of adult SMA and the effectiveness of emerging treatments. While this Canadian toolkit is largely in agreement with existing studies (mostly from Europe), an international consensus study could greatly benefit alignment of patient assessments globally, facilitating data pooling through partners such as the TREAT-NMD Neuromuscular Network. Furthermore, an international consensus would support advocacy efforts for improved availability of clinical services.

This work represents a important first step in achieving an accessible, standardized assessment for adults living with SMA. As a living document, this toolkit will be revisited and refined, with incorporation of SMA-specific measures as they evolve, and much needed measures to capture what is meaningful for persons with lived experience.