Neuromodulation for Apathy in Alzheimer’s Disease: A Double-Blind, Randomized, Sham-Controlled Pilot Study
Abstract
Background:
Apathy, a profound loss of motivation, initiation, and goal directed cognition, is a common comorbidity of Alzheimer’s disease (AD). The presence of apathy is associated with rapid progression of AD, long-term impairment, disability, and higher mortality. Pharmacological treatments of apathy are limited.
Objective:
The primary objective was to evaluate the efficacy of repetitive transcranial magnetic stimulation (rTMS) for apathy in AD.
Methods:
A randomized, double-blind, parallel-arm, sham-controlled pilot study was conducted in subjects with AD and apathy (N = 20). Subjects were randomized to rTMS or sham treatment (5 days/week) for four weeks. Primary outcome, apathy evaluation scale-clinician version (AES-C), and secondary outcome measures, modified-Mini Mental State Examination (3MS), instrumental activities of daily living (IADL), and clinical global impression (CGI), were assessed at baseline and four weeks. Follow-up visits were conducted at 8 and 12 weeks to test the durability of effects of intervention.
Results:
Mean age was 77.3 (±7.2) years, 80% were Caucasians and 10% were females. After adjusting for baseline, there was a significantly greater improvement in the AES-C with rTMS compared to sham treatment (–10.1 (–15.9 to –4.3); t (16) = –3.69; p = 0.002) at 4 weeks. There was also significantly greater improvement in 3MS (6.9 (1.7 to 12.0); t (15) = 2.85; p = 0.012), IADL (3.4 (1.0 to 5.9); χ21 = 7.72; p = 0.006), CGI-S (1.4 (0.5 to 2.3), t (16) = 3.29; p = 0.005), and CGI-I (–2.56 (–3.5 to –1.6), t (17) = –5.72; p < 0.001) for rTMS compared to the sham at 4 weeks. The effects of rTMS were durable at 12 weeks.
Conclusion:
rTMS may be safely used in subjects with AD and may improve apathy, function, and some aspects of cognition.
INTRODUCTION
Alzheimer’s disease (AD) impacts over 30 million people and is one of the largest public health problems. AD plus the accompanying neuropsychiatric symptoms, functional impairment, caregiver burden, and morbidity and mortality add up to over $100 billion in annual US healthcare costs [1]. The neuropsychiatric symptoms of AD, such as apathy, contribute to over a third of the associated disability. Apathy is a disorder of behavioral initiation that manifests as slowed but not necessarily impaired cognition, retarded emotions but not depression, and failure to initiate activities of daily living (ADLs). Although there is overlap with depressive symptoms, mounting evidence shows apathy as a distinct entity that lacks dysphoria, suicidal ideation, self-criticism, and hopelessness [2, 3]. This distinction is validated by the anatomical and functional areas of the brain involved in apathy [4, 5] and its worsening with use of antidepressants [6, 7].
Apathy and other behavioral problems have a greater impact on daily function than cognition in AD patients [8, 9]. Apathy is associated with decline in ADLs and instrumental activities of daily living (IADLs) even after controlling for age, education, and depression [10]. Those with apathy are 2–3 times more likely to have deficits in ADLs [11, 12]. Apathetic AD patients require more management and support which increases caregiver burden and service utilization. Furthermore, apathy is associated with three-times higher mortality [12]. Unfortunately, treatment options are limited. Antidepressants may worsen apathy and dementia medications have shown mixed results [14–16]. Short-term trials indicate that the stimulant methylphenidate improves apathy in patients with AD [17–19], but many individuals cannot tolerate stimulants due to cardiovascular concerns [20]. Thus, non-pharmacological treatment options for apathy are urgently needed [21].
Repetitive transcranial magnetic stimulation (rTMS) is a non-invasive tool used for treatment of several neuropsychiatric conditions including treatment-resistant depression [22]. Our team has shown the feasibility and efficacy of rTMS for apathy and certain cognitive measures in patients with mild cognitive impairment (MCI) [23]. Left dorsolateral prefrontal cortex stimulation at 10 Hz for ten treatments significantly improved apathy, global cognition, and executive function with the rTMS treatment compared to the sham treatment [23]. Due to the potential neuroprotective effects of rTMS, it is being studied to enhance cognition in those with MCI [24]. The physiological effects and the potential benefits of rTMS warrant its testing for apathy in AD. In fact, because the primary deficit in apathy is lack of initiation, we hypothesized that rTMS treatment can improve cognition, emotion, and function through improved initiation. The primary objective was to evaluate the efficacy of rTMS for apathy in patients with AD. The secondary objectives were to examine the effects of rTMS on cognition, daily function, safety, and tolerability. To our knowledge, this is the first study to examine the effects of rTMS for apathy in AD.
METHODS
Study design and participation
This pilot feasibility study was a single site, prospective, double-blind, randomized, parallel-arm, sham-controlled study of rTMS treatment for apathy in older adults with AD. The study was conducted at the Central Arkansas Veterans Healthcare System as approved by its Institutional Review Board. Subjects were recruited via advertisements in clinical areas and provider referrals. Subjects were pre-screened by medical records review, and those appearing eligible were invited for the baseline visit. At the baseline visit, capacity to consent was assessed with UCSD Brief Assessment of Capacity to Consent (UBACC) [25]. Subjects scoring 15 or higher were deemed capable and provided written informed consent. If the UBACC score was lower than 15, caregivers provided the consent. Additionally, all caregivers provided written consent for their participation. The inclusion criteria were age 55 years or older, diagnosed with AD, scored 30 or higher on The Apathy Evaluation Scale-clinician version (AES-C) [26], scored 18 or higher on the Mini-Mental State Examination (MMSE) [27], cleared the TMS adult safety scale (TASS) [28], and on a stable dose of antidepressants (if applicable) for at least two months. A caregiver who spent at least several hours per week with the patient was required to participate. The exclusion criteria were use of medications that increase risk of seizures or ototoxicity, history of bipolar disorder, seizure disorder, seizure disorder in first degree relatives, implanted device, stroke, aneurysm, cranial neurosurgery, alcohol-related problems, or current episode of major depressive disorder. Subjects were randomized using computer-generated, double-blind assignments based on a random permuted block design. Subjects/caregivers, study personnel, and the statistician were masked to treatment assignment until analyses were completed.
Intervention
The NeuroStar® TMS Therapy and XPLOR System consisting of an XPLOR standard treatment coil, a blinded rTMS-coil, a blinded sham-coil, and acoustic blinding hardware were used (Neuronetics, Inc., Malvern, PA). The XPLOR blinded rTMS-coil was identical in appearance and function to the NeuroStar TMS Therapy System treatment rTMS-coil except for a “coil type” label, “X” and “Y”. During use, the sham-coil produced an equivalent sound to the rTMS-coil but did not produce a therapeutic magnetic field. The acoustic blinding hardware disguised the acoustic tones of the blinded XPLOR coils. Subjects used foam earplugs and were not allowed to sleep during treatments.
Motor threshold (MT) determination and treatment
Single pulse TMS was used to find the scalp position of lowest resting MT for the right first dorsal interosseous or abductor pollicis brevis muscle using the NeuroStar device algorithm. The stimulation site was the left dorsolateral prefrontal cortex (DLPFC) defined as 5.5 cm anterior to the MT location. For twenty consecutive weekdays, the subject received 3000 pulses at 10 Hz, 4 s train duration, and 26 s inter-train interval at 120% MT using the assigned coil though protocols were in place to lower the MT if necessary. These parameters are consistent with safety guidelines and depression treatment [29, 30]. Furthermore, a large metanalysis of negative symptoms in schizophrenia (N = 827), concluded that left DLPFC stimulation at 10 Hz and greater than 100% motor threshold was more effective than other protocols hence we chose to stimulate left DLPFC in the current study [31]. Certified technicians, not raters, delivered the treatments. Adverse events were assessed at each visit by questionnaire and/or patient and caregiver self-report. Primary and secondary outcomes were assessed at 4, 8, and 12 weeks.
Primary outcome measure
Apathy was assessed with the AES-C. AES-C measures behavioral, cognitive, and emotional domains of apathy for the previous four-weeks. Scores range from 18 to 72, with higher scores reflecting more severe apathy. A score of ≥30 is considered clinically significant apathy. AES-C has good internal consistency (Cronbach’s α> 0.86), and test-retest reliability (Pearson’s r > 0.76) [26]. Information was collected from the patient and the caregiver through a semi-structured interview. We chose AES-C over other available measures of apathy and other versions of AES as it was the most validated measure for apathy at the time of this study and best suited version of AES for cognitively impaired subjects [32]. More recently, the Dementia Apathy Interview and Rating (DAIR) [33] and the apathy domain of the neuropsychiatric inventory [34] have been found to have a high degree of overlap with the diagnostic criteria for apathy [35, 36].
Secondary outcome measures
Cognition was measured with modified Mini-Mental State Exam (3MS) and MMSE. The 3MS is a global screen for cognition [37]. Scores range from 30 to100 to provide finer discrimination, superior to MMSE, as a community screen for dementia [37]. MMSE scores were derived from 3MS. Executive function was assessed with trail making tests- A and B (TMT-A and TMT-B), and executive interview (Exit-25). TMT-B test differentiates apathetic from non-apathetic AD patients [38, 39]. Time taken to complete the tests in seconds along with errors made were recorded. EXIT-25 has high inter-rater reliability (Pearson’s r = 0.90) [40]. Scores range from 0 to 50 with higher scores reflecting poor executive function. Functional status was assessed with IADL and ADL. ADL scores range from 0 to 24 while IADL scores range from 0 to 23 with higher scores reflecting better function [41]. Clinical Global Impression (CGI) is an observational scale of global evaluation, which assesses the change in degree of illness in relation to the original assessment [42]. Two components are used to assess overall clinical severity (CGI-S) and improvement (CGI-I), each with a seven-point scale. At baseline, only severity was rated. In subsequent visits, severity and improvement were rated. Caregiver burden was assessed with Zarit Burden Scale (ZBS). It is a 22-item scale that assesses caregiver’s burden related to patient relationship, physical and mental health, finances, and social life. Scores range from 0 to 88 with higher scores reflecting higher caregiver burden. It has excellent reliability [43].
Statistical analysis
Descriptive statistics for demographics and baseline cognitive measures were compared between groups using the two-sample t-test for continuous data or Fisher’s exact test for categorical data. Changes in apathy score from baseline at week 4 (primary endpoint) were analyzed using an analysis of covariance with a main effect for treatment (rTMS or sham). Baseline measurements of the outcome were included as a covariate. A similar approach was used to analyze other cognitive measures, and robust regression techniques (Huber M estimation with a bisquare weight function) were used when normality was rejected. CGI-I, which measures improvement, did not have a baseline measure by definition so those results did not adjust for baseline. Additional sensitivity analyses also investigated the impact of controlling for age and depression on efficacy results. Durability of the treatment effect was assessed in the rTMS arm at 8 and 12 weeks by analyzing change from baseline using the repeated measures mixed model analysis of covariance with main effects for arm and time and the interaction between arm and time with the baseline measurements of the outcome included as a covariate. A random subject effect accounted for the correlation arising from repeated measurements on the same subject. Two-sided p-values less than 0.05 indicated significance; p-values were not adjusted for multiple comparisons in this pilot study. Data were analyzed using SAS Enterprise Guide v5.1 (SAS, Cary, SC). This pilot study was designed to have 80% power to detect a difference of at least 7.3 points on AES-C between the two groups assuming a standard deviation of 5.5 using a two-sided test with alpha = 0.05. The difference of 7.3 was chosen based on a study of methylphenidate for treatment of apathy in AD [44].
RESULTS
A total of 53 recruits were screened to randomize 20 subjects (N = 9 rTMS and n = 11 Sham). Screening, enrollment, and participation are depicted in Fig. 1. Mean (SD) age of the subjects was 77.3 (7.2) years, 80% were Caucasians, 10% African Americans, 10% Hispanic, and 10% were females. Of the subjects, 40% were on cholinesterase inhibitors and 10% were on memantine. Nineteen subjects completed the study. At baseline, there were no significant differences between the two arms with regards to the demographics, concomitant medications, and comorbidities (Table 1). There were no significant differences between the arms with regards to any primary or secondary endpoints except Exit-25, which was significantly higher in the sham group (t (18) = –3.25; p = 0.005) (Table 2).
Fig. 1
Screening, Enrollment, and Participation aReason for exclusion: 1-substance abuse, 3-travel, 6-not interested, 4-time, 4-bipolar disorder, 4-did not meet criteria for AD, 3-h/o head trauma, 1-contraindicated medication (bupropion), 1-implants, 1-another study, 2-younger age, 2-h/o stroke, 1-deteriorating caregiver health.
Table 1
Descriptive characteristics according to the randomized groups
| All subjects | rTMS | Sham | Statistic | Pa | |
| (N = 20) | (N = 9) | (N = 11) | |||
| Continuous variables | mean (SD) | mean (SD) | mean (SD) | ||
| Age in years | 77.3 (7.2) | 74.3 (5.7) | 79.6 (7.7) | t18 = –1.7 | 0.105 |
| Anthropometry | |||||
| Height (inches) | 70.0 (3.4) | 70.3 (4.1) | 69.8 (2.9) | t18 = 0.3 | 0.751 |
| Weight (lbs.) | 194.4 (28.6) | 205.9 (35.1) | 185.0 (18.8) | t18 = 1.7 | 0.105 |
| BMI (kg/m2) | 28.2 (6.1) | 29.9 (8.0) | 26.9 (4.0) | t11 = 1.0 | 0.332 |
| Categorical variables | n (%) | n (%) | n (%) | ||
| Male | 18 (90) | 8 (89) | 10 (91) | FET | > 0.999 |
| Race/Ethnicity | FET | 0.728 | |||
| Non-Hispanic Caucasian | 16 (80) | 8 (89) | 8 (73) | ||
| Non-Hispanic African-American | 2 (10) | 0 | 2 (18) | ||
| Hispanic | 2 (10) | 1 (11) | 1 (9) | ||
| Education Category | FET | 0.153 | |||
| Less than High School | 1 (5) | 0 (0) | 1 (9) | ||
| High School diploma | 9 (45) | 2 (22) | 7 (64) | ||
| Some college degree | 4 (20) | 2 (22) | 2 (18) | ||
| Bachelor’s degree | 4 (20) | 3 (33) | 1 (9) | ||
| Professional/Graduate degree | 2 (10) | 2 (22) | 0 | ||
| Concomitant medications | |||||
| Anti-Depressants | 9 (45) | 4 (44) | 5 (45) | FET | > 0.999 |
| Acetylcholinesterase Inhibitors | 8 (40) | 2 (22) | 6 (55) | FET | 0.197 |
| Memantine | 2 (10) | 0 | 2 (18) | FET | 0.479 |
| Comorbidities | |||||
| Hypertension | 13 (65) | 5 (56) | 8 (73) | FET | 0.642 |
| Diabetes | 9 (45) | 4 (44) | 5 (45) | FET | > 0.999 |
| Depression | 3 (15) | 1 (11) | 2 (18) | FET | > 0.999 |
| Coronary Artery Disease | 6 (30) | 4 (44) | 2 (18) | FET | 0.336 |
| Hypothyroidism | 0 | 0 | 0 | FET | > 0.999 |
| Hyperlipidemia | 11 (55) | 6 (67) | 5 (45) | FET | 0.406 |
| Degenerative Joint Disease | 1 (5) | 0 | 1 (9) | FET | > 0.999 |
| Hearing Loss | 1 (5) | 1 (11) | 0 | FET | 0.450 |
ap-values calculated using the two-sample t-test or Fisher’s exact test (FET). IQR, interquartile range.
Table 2
Baseline measures according to the randomized groups
| All subjects (N = 20) | rTMS (N = 9) | Sham (N = 11) | Statistic | Pa | |
| mean (SD) | mean (SD) | mean (SD) | |||
| Primary endpoint | |||||
| Apathy Evaluation Scale | 49.1 (7.7) | 47.7 (5.8) | 50.2 (9.1) | t18 = –0.72 | 0.482 |
| Secondary endpoints | |||||
| Activities of Daily Living | 21.4 (4.3) | 22.9 (1.8) | 20.1 (5.4) | t13 = 1.62 | 0.130 |
| Instrumental Activities of Daily Living | 12.8 (6.0) | 13.9 (6.7) | 10.8 (6.1) | t18 = 1.07 | 0.298 |
| Modified Mini-Mental State Examb | 72.6 (13.7) | 75.3 (14.3) | 70.6 (13.7) | t17 = 0.71 | 0.486 |
| Mini Mental State Exam | 22.1 (3.4) | 22.9 (3.4) | 21.4 (3.3) | t18 = 1.01 | 0.324 |
| Trails Making Test A | 114.2 (84.9) | 107.8 (81.5) | 119.4 (91.2) | t18 = –0.30 | 0.771 |
| Trails Making Test B | 189.1 (64.6) | 169.8 (64.0) | 204.9 (63.7) | t18 = –1.23 | 0.236 |
| Exit-25 | 16.4 (6.6) | 12.0 (5.5) | 19.9 (5.3) | t18 = –3.25 | 0.005 |
| Cognitive Global Impression - Severity | 5.3 (0.9) | 5.1 (0.8) | 5.5 (0.9) | t18 = –0.88 | 0.391 |
| Zarit Burden Scale | 32.7 (20.5) | 29.4 (24.6) | 35.3 (17.2) | t18 = –0.62 | 0.541 |
ap-values calculated using the two-sample t-test; unequal variance t-test used for ADL. b3MS was missing at baseline for one participant in the rTMS group.
Primary outcome
There was a significant between-group difference [average difference (95% CI)] in the change in AES-C scores for rTMS treatment compared with the sham treatment [–10.1 (–15.9 to –4.3); t (16) = –3.69; p = 0.002] (Table 3). Within-group analysis showed significant improvement in AES with rTMS treatment exhibiting a mean reduction of 11.0 points from baseline (t (16) = –5.52; p < 0.001) and no improvement with the sham treatment (t (16) = –0.45; p = 0.662). The mean improvement of AES-C score in the rTMS treatment group was 10.1 points which is clinically significant.
Table 3
Changes from baseline in outcomes with rTMS and sham treatments and the differences between the two treatments
| Variables | Change with rTMS treatment (n = 9) Mean (95% CI) | Change with Sham treatment (n = 10) Mean (95% CI) | Differencea Mean (95% CI) | Statistic | pb |
| Primary endpoint | |||||
| AES | –11.0 (–15.2 to –6.7) | –0.8 (–4.8 to 3.2) | –10.1 (–15.9 to –4.3) | t16 = –3.69 | 0.002 |
| Secondary endpoints | |||||
| ADL | 0.1 (–0.7 to 1.0) | 0.3 (–0.6 to 1.1) | –0.1 (–1.4 to 1.1) | t16 = –0.22 | 0.829 |
| IADLc | 1.0 (–2.3 to 4.4) | –2.4 (–5.4 to 0.5) | 3.4 (1.0 to 5.9) | X21 = 7.72 | 0.006 |
| 3MS | 7.2 (3.4 to 11.0) | 0.3 (–3.1 to 3.8) | 6.9 (1.7 to 12.0) | t15 = 2.85 | 0.012 |
| MMSE | 1.0 (–1.1 to 3.0) | 0.1 (–1.8 to 2.0) | 0.8 (–2.0 to 3.6) | t16 = 0.62 | 0.542 |
| TMT-A | –4.8 (–23.2 to 13.5) | –1.9 (–19.3 to 15.5) | –2.9 (–28.2 to 22.4) | t16 = –0.24 | 0.812 |
| TMT-B | –10.8 (–28.3 to 6.6) | 11.0 (–5.6 to 27.5) | –21.8 (–46.1 to 2.5) | t16 = –1.90 | 0.075 |
| Exit-25 | 1.5 (–2.1 to 5.1) | –3.1 (–6.6 to 0.3) | 4.6 (–0.8 to 10.1) | t16 = 1.79 | 0.092 |
| CGI-S | 1.7(1.1 to 2.4) | 0.4 (–0.3 to 1.0) | 1.4 (0.5 to 2.3) | t16 = 3.29 | 0.005 |
| CGI-I | 1.8 (1.1 to 2.5) | 4.3 (3.7 to 4.9) | –2.5 (–3.5 to –1.6) | t17 = –5.72 | < 0.001 |
| ZBS | –3.2 (–11.9 to 5.6) | 0.7 (–7.6 to 9.1) | –3.9 (–16.0 to 8.2) | t16 = –0.68 | 0.506 |
aAll means are estimates from a regression model of 4-week change from baseline, except for CGI-I, which is assessed at 4-weeks. Difference reflects rTMS group change minus sham group change and is adjusted for corresponding baseline measure. bp-values comparing rTMS and sham treatment are model-based. cSince normality of model residuals were rejected by the Shapiro-Wilk test (p = 0.005), estimates were derived from using a robust modeling technique (Huber M estimation with a bisquare weight function). Normality was not rejected for all other endpoints. AES, Apathy Evaluation Scale; ADL, Activities of Daily Living; IADL, Instrumental Activities of Daily Living; 3MS, Modified Mini-Mental State Exam; MMSE, Mini-Mental State Exam; TMT-A, Trails Making Test A; TMT-B, Trails Making Test B; Exit-25, Executive function-25; CGI-I, Clinical Global Impression-Improvement; CGI-S, Clinical Global Impression-Severity; ZBS, Zarit Burden Scale.
Secondary outcomes
There was a significant between-group improvement in 3MS favoring the rTMS treatment [6.9 (1.7 to 12.0); t (15) = 2.85; p = 0.012] (Table 3). Within-group analysis showed significant improvement in 3MS with the rTMS treatment (t (15) = 4.02; p = 0.001) and minimal improvement with the sham treatment (t (15) = 0.21; p = 0.834). There was a significant between-group improvement in IADL favoring the rTMS treatment [3.4 (1.0 to 5.9); χ21 = 7.72; p = 0.006] (Table 3). There was a significant between-group improvement in CGI-S favoring the rTMS treatment [1.4 (0.5 to 2.3), t (16) = 3.29; p = 0.005]. Within-group analysis showed significant improvement in CGI-S with the rTMS treatment (t (16) = 5.7; p < 0.001) and no improvement with the sham treatment (t (16) = 1.2; p = 0.238). There was a significant between-group improvement in CGI-I favoring the rTMS treatment [–2.56 (–3.5 to –1.6), t (17) = –5.72; p < 0.001] (Table 3). Within-group analysis showed significant improvement in CGI-I in both groups, the rTMS treatment (t (17) = 5.6; p < 0.001) and the sham treatment (t (17) = 14.2; p < 0.001). There were no significant between-group or within-group differences for any of the other secondary outcomes. Efficacy findings were similar and significance unaffected when also controlling for age and depression for primary and secondary outcomes (results not shown).
Adverse events
There were 43 adverse events reported in 11 subjects. All adverse events occurred only during treatment sessions and resolved with completion of treatment. Application site pain was the most common adverse event, followed by headache, discomfort, and eye twitching. Details of the adverse events are listed in Table 4. One subject from the sham group withdrew due to tolerability issues. The proportion of subjects experiencing adverse events did not significantly differ between groups (78% rTMS versus 40% sham, p = 0.170) though the sample size was small (Table 4). There was an appreciable difference for application site pain (56% in rTMS group versus 10% in sham group, p = 0.057).
Table 4
Adverse events in the subjects receiving rTMS or Sham treatment
| Adverse Event | rTMS | Sham | Total | |||
| N = 9 | N = 10 | N = 19 | ||||
| Occ | Subj | Occ | Subj | Occ | Subj | |
| Application site pain | 8 | 5 (56%) | 1 | 1 (10%) | 9 | 6 (32%) |
| Headache | 7 | 3 (33%) | 2 | 2 (20%) | 9 | 5 (26%) |
| Discomfort | 3 | 2 (22%) | 2 | 1 (10%) | 5 | 3 (16%) |
| Eye twitching | 3 | 2 (22%) | 1 | 1 (10%) | 4 | 3 (16%) |
| Ringing in ears | 0 | 0 | 3 | 1 (10%) | 3 | 1 (5%) |
| Difficulty with correct alignment | 1 | 1 (11%) | 0 | 0 | 1 | 1 (5%) |
| Tooth Ache | 1 | 1 (11%) | 0 | 0 | 1 | 1 (5%) |
| Dizzy and lightheaded | 1 | 1 (11%) | 0 | 0 | 1 | 1 (5%) |
| Confusion | 0 | 0 | 1 | 1 (10%) | 1 | 1 (5%) |
| Buzzing in head | 0 | 0 | 1 | 1 (10%) | 1 | 1 (5%) |
| Diarrhea | 0 | 0 | 1 | 1 (10%) | 1 | 1 (5%) |
| More apathetic &argumentative | 0 | 0 | 1 | 1 (10%) | 1 | 1 (5%) |
| Slurring words | 0 | 0 | 1 | 1 (10%) | 1 | 1 (5%) |
| Trouble staying asleep | 0 | 0 | 1 | 1 (10%) | 1 | 1 (5%) |
| Other –unspecified* | 4 | 4 (44%) | 0 | 0 | 4 | 4 (21%) |
| Total | 28 | 7 (78%) | 15 | 4 (40%) | 43 | 11 (58%) |
*Note 2 comments were patient wanted % MT reduced, and head cushion removed to better align the LC. The other 2 AEs had no comments. Occ, occurrences; Subj, subjects.
Durability
The within-rTMS group significance seen at 4 weeks for AES-C was not maintained at 8 and 12 weeks. For AES-C, the average changes from baseline at 8 and 12 weeks were –3.5 (95% CI, –9.6 to 2.6) and –4.4 (95% CI, –10.6 to 1.8), respectively, which are clinically relevant changes from baseline for the rTMS group, albeit nonsignificant, and corresponded to maintaining 32% and 40% of the change seen at 4 weeks. 3MS maintained significance at 12 weeks but not at 8 weeks; the average changes for 3MS were 3.4 (95% CI, –0.9 to 7.7; p = 0.114) at 8 weeks and 7.0 (95% CI, 2.6 to 11.3; p = 0.003) at 12 weeks. For CGI-S, the changes were 1.0 (95% CI, 0.2 to 1.9; p = 0.019) and 1.1 (95% CI, 0.2 to 1.9; p = 0.016) at 8 and 12 weeks, and for CGI-I, the changes were 4.7 (95% CI, 3.8 to 5.6; p < 0.001) and 3.7 (95% CI, 2.6 to 4.8; p < 0.001).
DISCUSSION
The main objective of this study was to show that rTMS is a safe treatment for apathy in patients with AD. The hypothesis that patients receiving rTMS treatment would show significant improvement in apathy (AES-C) and cognition (3MS) versus the sham group was supported. A change of 3.3 points or higher on AES-C is clinically significant [45]. AES-C scores on average improved by 11 points in the rTMS group and all subjects achieved a clinically significant improvement. This within-group improvement in apathy was better than 6 weeks of methylphenidate treatment in the ADMET study and 4 weeks of methylphenidate treatment reported by Padala et al. [19, 44]. In the latter study, average AES-C scores continued to improve with 12 weeks of methylphenidate treatment [44]. Notably, the improvement in AES-C in our study rTMS group was greater than that observed with a 2-week rTMS study, giving preliminary evidence for duration of treatment [23]. Average 3MS scores improved by 7.2 points with rTMS which is greater than that reported with 12 weeks of methylphenidate treatment [44]. Although there was no statistically significant improvement in MMSE, perhaps due to a ceiling effect, the within-group change with rTMS of 1 point is similar to the change seen with 6 weeks of methylphenidate and close to being clinically significant [19, 46]. Furthermore, there was statistically significant between-group improvement in measures of daily function (IADL) and clinical global impression (CGI-I), which is important especially when the short duration of treatment is considered.
Although the mechanism of action for rTMS on apathy is unknown, application of rTMS in high frequencies (≥1 Hertz) over the left DLPFC enhances dopamine transmission, increases neuronal activity in the prefrontal cortex, and has neurotrophic and neuroprotective effects [47–51]. Our team and others have studied its efficacy on cognition and apathy in patients with MCI and post-stroke [23, 24, 52]. Increase in dopamine release in the DLPFC has been observed in primates [53] and rodents [54] following oral methylphenidate administration, as well as, increase in dopamine release in the caudate and prefrontal cortex in humans following rTMS stimulation [55]. This dopamine release in the prefrontal cortex may be the common mechanism underlying the benefits of rTMS and methylphenidate [17, 19, 44]. We speculate that the cascade of events in the rTMS therapeutic action would include increased dopamine release in the DLPFC, leading to an improvement in processing speed, apathy and ADLs.
The adverse events reported in this study were transient, mild, and occurred only during treatment as also occurred in rTMS studies of depression [29, 56]. While one subject withdrew due to tolerability issues, the person was in the sham group. Although there were no statistically significant between-group differences in application site pain, probably due to the small sample size, a larger percentage of patients reported discomfort with rTMS than sham.
The durability effects of rTMS were encouraging. Although there were no statistically significant within-group improvements in AES-C scores at 8 weeks and 12 weeks, the improvements in AES-C in the rTMS group remained clinically significant. There were statistically significant within-group improvements in 3MS and CGI-I in the rTMS group at 12 weeks. A case could be made for longer duration of treatment or adding maintenance treatments for apathy as done for depression [29, 56]. Pairing rTMS sessions with activities of premorbid interest could potentially enhance durability of the improvement in apathy. These questions should be investigated in future effectiveness studies.
The major strength of the study is its design as a prospective, double-blind, randomized, parallel-arm, sham-controlled trial. The use of a validated sham coil is a key strength as expectation of outcome is a major concern in intervention studies. Other researchers have tilted the coil 90° for the sham-effect though this approach could still induce voltage to the brain [57]. Our sham coil also controlled for the look and sound of stimulation. We have used sham coils successfully to blind subjects and raters in earlier protocols; however, we did not attempt to control for the feel of rTMS using electrical stimulation of the scalp as in previous studies [58]. Other strengths include the high adherence rate and use of blinded raters. The masking was only broken after the analyses were completed. Limited exclusionary criteria resulted in a study population that is broadly representative of community dwelling AD patients except for the predominantly male representation due to the Veteran population. Reporting durability of the effect of rTMS is another strength, as such information is seldom reported in neuropsychiatric symptom research.
Limitations of the study include the small sample size, short duration of treatment, inclusion of apathy only in the context of AD, and predominantly male subjects. Although patients with a current episode of major depressive disorder were excluded, and the dose of antidepressants (if applicable) was required to be stable, it is possible that some of the improvements could result from improvement in residual depression. Larger studies would be needed to systematically measure depression and control for change in depressive symptoms and antidepressant treatment in post-hoc analyses. Another limitation of the study was the use of a single measure for apathy. Since the conceptualization of this study, the diagnostic criteria for apathy have been operationalized and validated. Future studies may want to strongly consider the use of the revised criteria for apathy in AD [28]. We also acknowledge that our pilot study preselected subjects for apathy rather than random selection from a larger pool. As rTMS is explored further for the treatment of apathy, future studies need to explore other anatomically relevant regions of the brain such as the anterior cingulate gyrus.
Conclusion
Identifying non-pharmacological treatments for apathy in AD with minimal adverse effects is a high priority since such treatments could improve quality of life for AD patients and their caregivers. rTMS is a plausible non-pharmacological option for apathy in AD with potential to rapidly improve apathy and slow cognitive decline. The identified improvement in function could be explained by improved initiative and motivation reported in other rTMS studies. Future studies will need robust batteries of cognition to delineate the effects of rTMS on cognition, especially executive function. If replicated in larger studies, such findings could have a significant impact on the trajectory of neurodegeneration associated with AD. Studies of longer treatment duration and follow-up are needed to identify if rTMS alters AD progression. Future studies also need to increase the frequency of outcome measures to identify the earliest indicator of success and define the optimal rTMS dose.
ACKNOWLEDGMENTS
We would like to thank all the subjects and their caregivers. We would also like to thank Ms. Ashlyn Jendro for critical review of the paper.
This study (ClinicalTrials.gov Identifier: NCT02190084) was supported by SCVAHCN Network Research Grant Program by Department of Veterans Affairs (PRP), and Neuronetics Inc. Investigator Initiated Trial award (supplies only) (PI: Prasad Padala, MD). The sponsors had no role in the design, methods, subject recruitment, data collections, analysis or preparation of paper.
Oral presentation: Padala PR. Transcranial magnetic stimulation for apathy in Alzheimer’s disease: A double blind sham-controlled trial. Oral presentation at 2018 American Association for Geriatric Psychiatry annual meeting, Honolulu, HI.
Abstract published: Padala PR, Padala KP, Lensing SY, Hunter CR, Parkes CM, Bopp MM, Caceda R, Dennis RA, Mennemeier MS, Sullivan DH. Transcranial magnetic stimulation for apathy in Alzheimer’s disease: A double blind sham-controlled trial. Am J Geriatr Psychiatry. 2018 Mar;26(3): S56-57. doi: 10.1016/j.jagp.2018.01.190.
Authors’ disclosures available online (https://www.j-alz.com/manuscript-disclosures/20-0640r2).
REFERENCES
[1] | GBD 2016 Dementia Collaborators ((2019) ) Global, regional, and national burden of Alzheimer’s disease and other dementias, 1990–2016: A systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol 18: , 88–106. |
[2] | Ishii S , Weintraub N , Mervis JR ((2009) ) Apathy: A common psychiatric syndrome in the elderly. J Am Med Dir Assoc 10: , 381–393. |
[3] | Robert P , Onyike CU , Leentjens AF , Dujardin K , Aalten P , Starkstein S , Verhey FR , Yessavage J , Clement JP , Drapier D , Bayle F , Benoit M , Boyer P , Lorca PM , Thibaut F , Gauthier S , Grossberg G , Vellas B , Byrne J ((2009) ) Proposed diagnostic criteria for apathy in Alzheimer’s disease and other neuropsychiatric disorders. Eur Psychiatry 24: , 98–104. |
[4] | Holthoff VA , Beuthien-Baumann B , Kalbe E , Lüdecke S , Lenz O , Zündorf G , Spirling S , Schierz K , Winiecki P , Sorbi S , Herholz K ((2005) ) Regional cerebral metabolism in early Alzheimer’s disease with clinically significant apathy or depression. Biol Psychiatry 57: , 412–421. |
[5] | Starkstein SE , Mizrahi R , Capizzano AA , Acion L , Brockman S , Power BD ((2009) ) Neuroimaging correlates of apathy and depression in Alzheimer’s disease. J Neuropsychiatry Clin Neurosci 21: , 259–265. |
[6] | Wongpakaran N , van RR , Wongpakaran T , Clarke D ((2007) ) Selective serotonin reuptake inhibitor use associates with apathy among depressed elderly: A case-control study. Ann Gen Psychiatry 6: , 7. |
[7] | Barnhart WJ , Makela EH , Latocha MJ ((2004) ) SSRI-induced apathy syndrome: A clinical review. J Psychiatr Pract 10: , 196–199. |
[8] | Rog LA , Park LQ , Harvey DJ , Huang CJ , Mackin S , Farias ST ((2014) ) The independent contributions of cognitive impairment and neuropsychiatric symptoms to everyday function in older adults. Clin Neuropsychol 28: , 215–236. |
[9] | Onyike CU , Sheppard JM , Tschanz JT , Norton MC , Green RC , Steinberg M , Welsh-Bohmer KA , Breitner JC , Lyketsos CG ((2007) ) Epidemiology of apathy in older adults: The Cache County Study. Am J Geriatr Psychiatry 15: , 365–375. |
[10] | Clarke DE , Ko JY , Lyketsos C , Rebok GW , Eaton WW ((2010) ) Apathy and cognitive and functional decline in community-dwelling older adults: Results from the Baltimore ECA longitudinal study. Int Psychogeriatr 22: , 819–829. |
[11] | Freels S , Cohen D , Eisdorfer C , Paveza G , Gorelick P , Luchins DJ , Hirschman R , Ashford JW , Levy P , Semla T ((1992) ) Functional status and clinical findings in patients with Alzheimer’s disease. J Gerontol 47: , M177–M182. |
[12] | Okura T , Plassman BL , Steffens DC , Llewellyn DJ , Potter GG , Langa KM ((2010) ) Prevalence of neuropsychiatric symptoms and their association with functional limitations in older adults in the United States: The aging, demographics, and memory study. J Am Geriatr Soc 58: , 330–337. |
[13] | van der Linde RM , Matthews FE , Dening T , Brayne C ((2016) ) Patterns and persistence of behavioural and psychological symptoms in those with cognitive impairment: The importance of apathy. Int J Geriatr Psychiatry 32: , 306–315. |
[14] | Padala PR , Padala KP , Monga V , Ramirez DA , Sullivan DH ((2012) ) Reversal of SSRI-associated apathy syndrome by discontinuation of therapy. Ann Pharmacother 46: , e8. |
[15] | Zhang N , Wei C , Du H , Shi FD , Cheng Y ((2015) ) The effect of memantine on cognitive function and behavioral and psychological symptoms in mild-to-moderate Alzheimer’s disease patients. Dement Geriatr Cogn Disord 40: , 85–93. |
[16] | Cummings JL , Koumaras B , Chen M , Mirski D ((2005) ) Effects of rivastigmine treatment on the neuropsychiatric and behavioral disturbances of nursing home residents with moderate to severe probable Alzheimer’s disease: A 26-week, multicenter, open-label study. Am J Geriatr Pharmacother 3: , 137–148. |
[17] | Herrmann N , Rothenburg LS , Black SE , Ryan M , Liu BA , Busto UE , Lanctôt KL ((2008) ) Methylphenidate for the treatment of apathy in Alzheimer disease: Prediction of response using dextroamphetamine challenge. J Clin Psychopharmacol 28: , 296–301. |
[18] | Padala PR , Burke WJ , Shostrom VK , Bhatia SC , Wengel SP , Potter JF , Petty F ((2010) ) Methylphenidate for apathy and functional status in dementia of the Alzheimer type. Am J Geriatr Psychiatry 18: , 371–374. |
[19] | Rosenberg PB , Lanctôt KL , Drye LT , Herrmann N , Scherer RW , Bachman DL , Mintzer JE , ADMET Investigators ((2013) ) Safety and efficacy of methylphenidate for apathy in Alzheimer’s disease: A randomized, placebo-controlled trial. J Clin Psychiatry 74: , 810–816. |
[20] | Scherer RW , Drye L , Mintzer J , Lanctôt K , Rosenberg P , Herrmann N , Padala PR , Brawman-Mintzer O , Burke W , Craft S , Lerner AJ , Levey A , Porsteinsson A , van Dyck CH , ADMET 2 Research Group ((2018) ) The Apathy in Dementia Methylphenidate Trial 2 (ADMET 2): Study protocol for a randomized controlled trial. Trials 19: , 46. |
[21] | Swierkosz-Lenart K , Mall JF , von Gunten A ((2019) ) Interventional psychiatry in the management of behavioural and psychological symptoms of dementia: A qualitative review. Swiss Med Wkly 149: , w20140. |
[22] | George MS , Wassermann EM , Williams WA , Callahan A , Ketter TA , Basser P , Hallett M , Post RM ((1995) ) Daily repetitive transcranial magnetic stimulation (rTMS) improves mood in depression. Neuroreport 6: , 1853–1856. |
[23] | Padala PR , Padala KP , Lensing SY , Jackson AN , Hunter CR , Parkes CM , Dennis RA , Bopp MM , Caceda R , Mennemeier MS , Roberson PK , Sullivan DH ((2018) ) Repetitive transcranial magnetic stimulation for apathy in mild cognitive impairment: A double-blind, randomized, sham-controlled, cross-over pilot study. Psychiatry Res 261: , 312–318. |
[24] | Taylor JL , Hambro BC , Strossman ND , Bhatt P , Hernandez B , Ashford JW , Cheng JJ , Iv M , Adamson MM , Lazzeroni LC , McNerney MW ((2019) ) The effects of repetitive transcranial magnetic stimulation in older adults with mild cognitive impairment: A protocol for a randomized, controlled three-arm trial. BMC Neurol 19: , 326. |
[25] | Jeste DV , Palmer BW , Appelbaum PS , Golshan S , Glorioso D , Dunn LB , Kim K , Meeks T , Kraemer HC ((2007) ) A new brief instrument for assessing decisional capacity for clinical research. Arch Gen Psychiatry 64: , 966–974. |
[26] | Marin RS , Biedrzycki RC , Firinciogullari S ((1991) ) Reliability and validity of the Apathy Evaluation Scale. Psychiatry Res 38: , 143–162. |
[27] | Folstein MF , Folstein SE , McHugh PR ((1975) ) “Mini-mental state”: A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 12: , 189–198. |
[28] | Keel JC , Smith MJ , Wassermann EM ((2001) ) A safety screening questionnaire for transcranial magnetic stimulation. Clin Neurophysiol 112: , 720. |
[29] | George MS ((2010) ) Transcranial magnetic stimulation for the treatment of depression. Expert Rev Neurother 10: , 1761–1772. |
[30] | Rossi S , Hallett M , Rossini PM , Pascual-Leone A ((2009) ) Safety, ethical considerations, and application guidelines for the use of transcranial magnetic stimulation in clinical practice and research. Clin Neurophysiol 120: , 2008–2039. |
[31] | Aleman A , Enriquez-Geppert S , Knegtering H , Dlabac-de Lange JJ ((2018) ) Moderate effects of noninvasive brain stimulation of the frontal cortex for improving negative symptoms in schizophrenia: Meta-analysis of controlled trials. Neurosci Biobehav Rev 89: , 111–118. |
[32] | Guercio BJ , Donovan NJ , Munro CE , Aghjayan SL , Wigman SE , Locascio JJ , Amariglio RE , Rentz DM , Johnson KA , Sperling RA , Marshall GA ((2015) ) The Apathy Evaluation Scale: A comparison of subject, informant, and clinician report in cognitively normal elderly and mild cognitive impairment. J Alzheimers Dis 47: , 421–432. |
[33] | Strauss ME , Sperry SD ((2002) ) An informant-based assessment of apathy in Alzheimer disease. Neuropsychiatry Neuropsychol Behav Neurol 15: , 176–183. |
[34] | Cummings JL , Mega M , Gray K , Rosenberg-Thompson S , Carusi DA , Gornbein J ((1994) ) The Neuropsychiatric Inventory: Comprehensive assessment of psychopathology in dementia. Neurology 44: , 2308–2314. |
[35] | Robert P , Lanctôt KL , Agüera-Ortiz L , Aalten P , Bremond F , Defrancesco M , Hanon C , David R , Dubois B , Dujardin K , Husain M , König A , Levy R , Mantua V , Meulien D , Miller D , Moebius HJ , Rasmussen J , Robert G , Ruthirakuhan M , Stella F , Yesavage J , Zeghari R , Manera V ((2018) ) Is it time to revise the diagnostic criteria for apathy in brain disorders? The 2018 international consensus group. Eur Psychiatry 54: , 71–76. |
[36] | Lanctôt KL , Scherer RW , Li A , Vieira D , Coulibaly H , Rosenberg PB , Herrmann N , Lerner AJ , Padala PR , Brawman-Mintzer O , van Dyck CH , Porsteinsson AP , Craft S , Levey AI , Burke W , Mintzer JE ((2020) ) Measuring apathy in Alzheimer’s disease in the Apathy in Dementia Methylphenidate Trial 2 (ADMET 2): A comparison of instruments. Am J Ger Psychiatr 20: , 30349–3. |
[37] | McDowell I , Kristjansson B , Hill GB , Hebert R ((1997) ) Community screening for dementia: The Mini Mental State Exam (MMSE) and Modified Mini-Mental State Exam (3MS) compared. J Clin Epidemiol 50: , 377–383. |
[38] | Corrigan JD , Hinkeldey NS ((1987) ) Relationships between parts A and B of the Trail Making Test. J Clin Psychol 43: , 402–409. |
[39] | Starkstein SE , Sabe L , Vázquez S , Di Lorenzo G , Martínez A , Petracca G , Tesón A , Chemerinski E , Leiguarda R ((1997) ) Neuropsychological, psychiatric, and cerebral perfusion correlates of leukoaraiosis in Alzheimer’s disease. J Neurol Neurosurg Psychiatry 63: , 66–73. |
[40] | Royall DR , Mahurin RK , Gray KF ((1992) ) Bedside assessment of executive cognitive impairment: The executive interview. J Am Geriatr Soc 40: , 1221–1226. |
[41] | Lawton MP , Brody EM ((1969) ) Assessment of older people: Self-maintaining and instrumental activities of daily living. Gerontologist 9: , 179–186. |
[42] | Guy W ((2016) ) ECDEU Assessment Manual for Psychopharmacology-Revised. InRockville, MD, U.S. DHEW Publ No ADM 76-338. 70: , 218–222. |
[43] | Zarit SH , Reever KE , Bach-Peterson J ((1980) ) Relatives of the impaired elderly: Correlates of feelings of burden. Gerontologist 20: , 649–655. |
[44] | Padala PR , Padala KP , Lensing SY , Ramirez D , Monga V , Bopp MM , Roberson PK , Dennis RA , Petty F , Sullivan DH , Burke WJ ((2018) ) Methylphenidate for apathy in community-dwelling older veterans with mild Alzheimer’s disease: A double-blind, randomized, placebo-controlled trial. Am J Psychiatry 175: , 159–168. |
[45] | Lanctôt KL , Chau SA , Herrmann N , Drye LT , Rosenberg PB , Scherer RW , Black SE , Vaidya V , Bachman DL , Mintzer JE ((2014) ) Effect of methylphenidate on attention in apathetic AD patients in a randomized, placebo-controlled trial. Int Psychogeriatr 26: , 239–246. |
[46] | Howard R , Phillips P , Johnson T , O’Brien J , Sheehan B , Lindesay J , Bentham P , Burns A , Ballard C , Holmes C , McKeith I , Barber R , Dening T , Ritchie C , Jones R , Baldwin A , Passmore P , Findlay D , Hughes A , Macharouthu A , Banerjee S , Jones R , Knapp M , Brown RB , Jacoby R , Adams J , Griffin M , Gray R ((2011) ) Determining the minimum clinically important differences for outcomes in the DOMINO trial. Int J Geriatr Psychiatry 26: , 812–817. |
[47] | Cho SS , Strafella AP ((2009) ) rTMS of the left dorsolateral prefrontal cortex modulates dopamine release in the ipsilateral anterior cingulate cortex and orbitofrontal cortex. PLoS One 4: , e6725. |
[48] | Pogarell O , Koch W , Pöpperl G , Tatsch K , Jakob F , Zwanzger P , Mulert C , Rupprecht R , Möller HJ , Hegerl U , Padberg F ((2006) ) Striatal dopamine release after prefrontal repetitive transcranial magnetic stimulation in major depression: Preliminary results of a dynamic [123I] IBZM SPECT study. J Psychiatr Res 40: , 307–314. |
[49] | Pogarell O , Koch W , Pöpperl G , Tatsch K , Jakob F , Mulert C , Grossheinrich N , Rupprecht R , Möller HJ , Hegerl U , Padberg F ((2007) ) Acute prefrontal rTMS increases striatal dopamine to a similar degree as D-amphetamine. Psychiatry Res 156: , 251–255. |
[50] | Chervyakov AV , Chernyavsky AY , Sinitsyn DO , Piradov MA ((2015) ) Possible mechanisms underlying the therapeutic effects of transcranial magnetic stimulation. Front Hum Neurosci 9: , 303. |
[51] | Li X , Nahas Z , Kozel FA , Anderson B , Bohning DE , George MS ((2004) ) Acute left prefrontal transcranial magnetic stimulation in depressed patients is associated with immediately increased activity in prefrontal cortical as well as subcortical regions. Biol Psychiatry 55: , 882–890. |
[52] | Sasaki N , Hara T , Yamada N , Niimi M , Kakuda W , Abo M ((2017) ) The efficacy of high-frequency repetitive transcranial magnetic stimulation for improving apathy in chronic stroke patients. Eur Neurol 78: , 28–32. |
[53] | Kodama T , Kojima T , Honda Y , Hosokawa T , Tsutsui KI , Watanabe M ((2017) ) Oral administration of methylphenidate (Ritalin) affects dopamine release differentially between the prefrontal cortex and striatum: A microdialysis study in the monkey. J Neurosci 37: , 2387–2394. |
[54] | Arnsten AF , Dudley AG ((2005) ) Methylphenidate improves prefrontal cortical cognitive function through alpha2 adrenoceptor and dopamine D1 receptor actions: Relevance to therapeutic effects in attention deficit hyperactivity disorder. Behav Brain Funct 1: , 2. |
[55] | Takada M , Tokuno H , Nambu A , Inase M ((1998) ) Corticostriatal projections from the somatic motor areas of the frontal cortex in the macaque monkey: Segregation versus overlap of input zones from the primary motor cortex, the supplementary motor area, and the premotor cortex. Exp Brain Res 120: , 114–128. |
[56] | Janicak PG , O’Reardon JP , Sampson SM , Husain MM , Lisanby SH , Rado JT , Heart KL , Demitrack MA ((2008) ) Transcranial magnetic stimulation in the treatment of major depressive disorder: A comprehensive summary of safety experience from acute exposure, extended exposure, and during reintroduction treatment. J Clin Psychiatry 69: , 222–232. |
[57] | Lisanby SH , Gutman D , Luber B , Schroeder C , Sackeim HA ((2001) ) Sham TMS: Intracerebral measurement of the induced electrical field and the induction of motor-evoked potentials. Biol Psychiatry 49: , 460–463. |
[58] | Mennemeier M , Triggs W , Chelette K , Woods A , Kimbrell T , Dornhoffer J ((2009) ) Sham transcranial magnetic stimulation using electrical stimulation of the scalp. Brain Stimul 2: , 168–173. |
