Pilot Study Results Assessing the Accuracy of a Ballistic Sleep Monitor Relative to Polysomnography in Parkinson’s Disease
Sleep disturbance is the most prevalent non-motor symptom in Parkinson’s disease (PD), affecting over 90% of individuals [1], and associating with increased fall frequency, impulsive behavior, and cognitive dysfunction [2, 3]. While sleep disturbance typically worsens as the disease progresses [3], some conditions, such as REM sleep behavior disorder (RBD), may present decades before a clinical diagnosis [2]. Therefore, better understanding of sleep disturbances in PD may improve symptom management, as well as facilitate disease detection in the prodromal stage when disease modifying treatments may be most effective [2].
Polysomnography (PSG) is the gold-standard sleep assessment; however, the associated equipment may cause a significant departure from the usual night’s sleep. In addition, PD is characterized by fluctuations and variability so assessment of a single night may fail to capture the spectrum of sleep alterations [4], potentially leading to missed opportunities for intervention and an incomplete understanding of sleep disturbance in PD. Digital devices afford continuous home monitoring and the large-scale data may generate new signatures of health and disease [5]. Despite potential advantages, progress has been hampered by methodological variability and insufficient validation [6]. Of the few studies comparing digital devices to PSG in PD, most have used actigraphy and have reported moderate correlations with wide variability [7]. Therefore, the goal of the present pilot study was to compare the accuracy of a ballistic sleep monitor relative to PSG among adults with early-stage PD using standardized analytic procedures [8].
Participants were recruited from Department of Neurology at the University of Texas Health San Antonio. Inclusion criteria included 1) Age 18–88 years; 2) Clinical diagnosis of idiopathic early-stage PD (Hoehen & Yahr Stage 1-2); 3) A study partner capable of providing collateral sleep information; 4) In-home broadband internet and a smart phone. Exclusion criteria included 1) Active/current insomnia; 2) Use of a CPAP or BiPAP; 3) Untreated sleep apnea (self-report or STOP-BANG score >2 [9]); 4) Untreated restless legs syndrome; 5) Body mass index ≥40 kg/m2; 6) Current sedative-hypnotic medication use; 7) Current alcoholism or drug abuse; 8) Diagnosed dementia or Montreal Cognitive Assessment score <19 [10]. Twenty participants enrolled in the study. Concurrent data from the ballistic sleep monitor was unavailable on 12 participants due to issues with data downloads and the manual resetting of devices. The study was conducted in adherence with The Code of Ethics of the World Medical Association and the protocol was approved by the local institutional review board. Participants provided written informed consent prior to enrollment.
The first visit consisted of history and physical examination, cognitive assessments, and questionnaires. RBD symptoms were assessed with the Mayo Sleep Questionnaire [11]. An in-clinic overnight PSG was conducted and sleep stages were scored using the Academy of Sleep Medicine Manual for Scoring of Sleep Studies and Associated Events v2.6 by a registered sleep technologist [12]. During the PSG, a ballistic sleep monitor (Emfit QS, Emfit Corp., Kuopio, Finland) was placed under the mattress at thoracic level [13, 14]. An electromechanical film sensor detects pressure changes associated with respiration and heart rate, which are used to derive sleep stage estimation using algorithms previously validated against simultaneously collected electrocardiogram and respiratory inductive plethysmography [15].
Demographic and clinical characteristics were assessed using descriptive statistics. Sleep analyses were conducted using standardized guidelines and open-source R code by Menghini et al. [8]. Sleep staging and summary metrics derived from the ballistic sleep monitor and PSG were compared using paired sample’s t-tests or Wilcoxon signed rank tests. Light sleep was derived as the sum of N1 and N2. Biases of the device relative to PSG were assessed using Bland-Altman plots with 95% confidence intervals (CI). Epoch-by-epoch (EBE) analyses of thirty second periods were performed following temporal synchronization of the device with the PSG. Group-level accuracy, sensitivity, and specificity for sleep staging with 95% CIs were calculated. Statistical analyses were performed using R v4.2.2 and all tests were two-sided with statistical significance set at p < 0.05.
Eight participants (mean age 66±8 years, 25% female) were included in the study. The average PD diagnosis duration was 2.7±2.9 years. Five participants endorsed RBD (62.5%) and seven (87.5%) were prescribed levodopa. During the PSG, two participants did not achieve REM sleep and five did not achieve Stage 3 sleep. Relative to PSG, the device’s algorithm overestimated total sleep time (TST), sleep efficiency (SE), and deep and REM sleep duration, while underestimating light sleep duration (Table 1). Similarly, stratified analyses indicated the device overestimated TST, SE, and deep sleep in both RBD and non-RBD groups (all p < 0.05). EBE analyses indicated relatively high specificity, while sensitivity was consistently poor. The accuracy of the sleep stage discrimination ranged from 55 to 82 with the lowest levels observed for light sleep duration and the highest for deep sleep duration. Our findings are similar to a prior validation study using this device in a convenience sample of adults undergoing PSG, which reported accuracies ranging from 55 to 78 [13].
Table 1
Summary Metrics | ||||
PSG | Device | p, Effect size | Bias (Lower, | |
Mean±SD | Mean±SD | (95% CI) | Upper LOAs) | |
Total sleep time, min | 231±102 | 343±51 | p = 0.001*, 2.36/ | 112 (3, 220) |
(0.64, 4.07) | ||||
Sleep efficiency, % | 58±25 | 86±3 | p = 0.031*, 1.34 | 23 (–2, 49) |
(0.56, 2.11) | ||||
Light sleep duration, min | 337±140 | 196±27 | p < 0.001*, –3.67 | –150 (–265, –34) |
(–6.73, –0.62) | ||||
Deep sleep duration, min | 3±8 | 61±17 | p < 0.001*, 4.95 | 59 (19, 98) |
(0.27, 9.64) | ||||
REM sleep duration, min | 44±52 | 86±21 | p = 0.021*, 0.98 | 39 (–34, 111) |
(0.12, 1.83) | ||||
Mean heart rate | 62±10 | 60±8 | p = 0.057, 0.04 | 0.7 (–5, 7) |
(–0.11, 0.19) | ||||
EBE Analysis | ||||
PSG Stage | Sensitivity | Specificity | Accuracy | Kappa |
(95% CI) | (95% CI) | (95% CI) | value | |
Wake | 15 (12, 18) | 96 (94, 98) | 67 (60, 75) | 0.64 |
Light sleep | 57 (52, 62) | 53 (47, 57) | 55 (50, 59) | 0.12 |
Deep sleep | 28 (–11, 51) | 83 (80, 86) | 82 (80, 85) | 0.76 |
REM sleep | 31 (25, 38) | 78 (76, 80) | 75 (72, 78) | 0.61 |
*p < 0.05. Differences between PSG and sleep monitor device were assessed with paired samples t-test or Wilcox test. Bias (95% CIs) and upper and lower LOAs were analyzed using Bland-Altman plots. EBE analysis cells indicate the percentage of epochs that the device correctly or incorrectly classified relative to PSG. PSG, polysomnography; SD, standard deviation; 95% CI, 95% confidence interval; LOA, limits of agreement; min, minutes; REM, rapid eye movement.
In summary, our pilot study provides novel data demonstrating the accuracy of a ballistic sleep monitor relative to PSG among individuals with early-stage PD. The sensitivity and accuracy of the device were poor with overestimation of TST, SE, and deep and REM sleep duration, as well as underestimation of light sleep duration. Primary limitations of the study included the small sample and sizable data loss, which may contribute to bias. Future studies with larger samples and well-matched control groups are necessary to confirm the findings. Overall, the results highlight the importance of validating digital devices against gold-standard PSG within individuals with PD prior to implementation within clinical settings.
ACKNOWLEDGMENTS
Funding was provided by a UTHSA Stevens Parkinson’s Disease Center pilot award and the National Institute on Aging [R01AG077472, P30AG066546].
CONFLICT OF INTEREST
Dr. Gonzales has personal stock in Abbie. Ms. Pollet and her spouse are employed by Academy Diagnostics Sleep and EEG Center. Dr. Seshadri has consulted for Eisai and Biogen outside the current work. All other authors report no disclosures.
DATA AVAILABILITY
The dataset is available on reasonable request by qualified researchers by contacting the corresponding author.
REFERENCES
[1] | Zhang Y , Ren R , Sanford LD , Yang L , Zhou J , Tan L , Li T , Zhang J , Wing Y-K , Shi J , Lu L , Tang X ((2020) ) Sleep in Parkinson’s disease: A systematic review and meta-analysis of polysomnographic findings. Sleep Med Rev 51: , 101281. |
[2] | Albers JA , Chand P , Anch AM ((2017) ) Multifactorial sleep disturbance in Parkinson’s disease. Sleep Med 35: , 41–48. |
[3] | Jester DJ , Lee S , Molinari V , Volicer L ((2020) ) Cognitive deficits in Parkinson’s disease with excessive daytime sleepiness: A systematic review. Aging Mental Health 24: , 1769–1780. |
[4] | Zhang J , Lam SP , Ho CKW , Li AM , Tsoh J , Mok V , Wing YK ((2008) ) Diagnosis of REM sleep behavior disorder by video-polysomnographic study: Is one night enough? Sleep 31: , 1179–1185. |
[5] | Barnes K ((2021) ) Sleep monitoring in mental health goes digital. Commun Med (Lond) 1: , 50. |
[6] | Depner CM , Cheng PC , Devine JK , Khosla S , De Zambotti M , Robillard R , Vakulin A , Drummond SPA ((2020) ) Wearable technologies for developing sleep and circadian biomarkers: A summary of workshop discussions. Sleep 43: , zsz254. |
[7] | Zampogna A , Manoni A , Asci F , Liguori C , Irrera F , Suppa A ((2020) ) Shedding light on nocturnal movements in Parkinson’s disease: Evidence from wearable technologies. Sensors 20: , 5171. |
[8] | Menghini L , Cellini N , Goldstone A , Baker FC , de Zambotti M ((2021) ) A standardized framework for testing the performance of sleep-tracking technology: Step-by-step guidelines and open-source code. Sleep 44: , zsaas170. |
[9] | Chung F , Yegneswaran B , Liao P , Chung SA , Vairavanathan S , Islam S , Khajehdehi A , Shapiro CM ((2008) ) STOP questionnaire: A tool to screen patients for obstructive sleep apnea. Anesthesiology 108: , 812–821. |
[10] | Nasreddine ZS , Phillips NA , Bédirian V , Charbonneau S , Whitehead V , Collin I , Cummings JL , Chertkow H ((2005) ) The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. J Am Geriatr Soc 53: , 695–699. |
[11] | Boeve BF , Molano JR , Ferman TJ , Smith GE , Lin S-C , Bieniek K , Haidar W , Tippmann-Peikert M , Knopman DS , Graff-Radford NR ((2011) ) Validation of the Mayo Sleep Questionnaire to screen for REM sleep behavior disorder in an aging and dementia cohort. Sleep Med 12: , 445–453. |
[12] | Berry RB , Quan SF , Abreu AR , Bibbs ML , DelRosso L , Harding SM , Mao M-M , Plante DT , Pressman MR , Troester MM (2020) The AASM manual for the scoring of sleep and associated events: Rules, terminology and technical specifications, American Academy of Sleep Medicine. |
[13] | Kholghi M , Szollosi I , Hollamby M , Bradford D , Zhang Q ((2021) ) A validation study of a ballistocardiograph sleep tracker EMFIT QS against polysomnography. J Clin Sleep Med 18: , 1203–1210. |
[14] | Huysmans D , Borzée P , Testelmans D , Buyse B , Willemen T , Van Huffel S , Varon C ((2019) ) Evaluation of a commercial ballistocardiography sensor for sleep apnea screening and sleep monitoring. Sensors 19: , 2133. |
[15] | Ranta J , Aittokoski T , Tenhunen M , Alasaukko-Oja M ((2019) ) EMFIT QS heart rate and respiration rate validation. Biomed Phys Eng Express 5: , 025016. |