Validation of a leg movements count and periodic leg movements analysis in a custom polysomnography system

Background Periodic leg movements (PLM) during sleep (PLMS) are considered strongly related to restless legs syndrome (RLS), and are associated with polymorphisms in RLS risk genes. Various software for automatic analysis of PLMS are available, but only few of them have been validated. Aim of this study was to validate a leg movements count and analysis integrated in a commercially available polysomnography (PSG) system against manual scoring. Methods Twenty RLS patients with a PLMS index > 20/h and 20 controls with a PLMS index < 5/h were included. Manual and computerized scoring of leg movements (LM) and PLM was performed according to the standard American Academy of Sleep Medicine (AASM) criteria. LM and PLM indices during sleep and wakefulness, the rate of PLMS associated with respiratory events, intermovement interval and periodicity indices were manually and automatically scored. Results The correlation between manual and computerized scoring was high for all investigated parameters (Spearman correlation coefficients 0.751–0.996, p < 0.001; intraclass correlation coefficients 0.775–0.999, p < 0.001). Bland-Altman plots showed high agreement between manual and automatic analysis. Conclusions This study validated an automatic LM count and PLM analysis against the gold standard manual scoring according to AASM criteria. The data demonstrate that the software used in this study has an outstanding performance for computerized LM and PLM scoring, and LM and PLM indices generated with this software can be reliably integrated in the routine PSG report. This automatic analysis is also an excellent tool for research purposes. Electronic supplementary material The online version of this article (doi:10.1186/s12883-017-0821-6) contains supplementary material, which is available to authorized users.

Although several software programs for automatic detection and analysis of leg movements (LM) during sleep (LMS) have been developed and are commonly used in the clinical polysomnography (PSG) routine and research applications, only few of them have been validated and time-consuming visual detection and manual scoring of PLMS is still considered the gold standard for research purposes.
The aim of this study was to validate a LM detection and PLM analysis program integrated in a commercially available custom PSG system against manual scoring, which could be useful not only for clinical but also for research purposes.

Selection of participants
Routine PSG reports of the Sleep Disorders Unit, Department of Neurology, Innsbruck Medical University were screened to find 20 patients with RLS with an automatic-scored PLMS index higher than 20/h. RLS was diagnosed according to the International RLS Study Group (IRLSSG) criteria [10], based on an urge to move the legs, usually accompanied by unpleasant sensations, which begin or worsen during rest and is relieved by movement, worsen in the evening or night, and is not explained by other conditions. RLS mimics were excluded.
Control subjects were selected among patients without RLS who underwent PSG for other reasons, and had an automatic-scored PLMS index ≤ 5/h. For both groups, exclusion criteria were an apneahypopnea index (AHI) higher than 5/h or the use of a continuous positive airway pressure (CPAP) therapy. RLS treatment did not represent an exclusion criterion.
This study was approved by the local ethic committee of Innsbruck Medical University. All participants granted written informed consent prior to study participation.

Video-PSG
All subjects underwent at least one night of 8-h video-PSG according to the American Academy of Sleep Medicine (AASM) 2012 standards [11]. In case of more nights of PSG, the second night was used for this study, unless technical reasons prevented this.
Leg movements were recorded using surface electrodes placed longitudinally and symmetrically around the middle of the tibialis anterior muscle, 2-3 cm apart. For scoring of EMG activity, bipolar surface EMG was recorded with the low frequency filter at 50 Hz, the high frequency filter at 300 Hz, and a sampling rate of 1000 Hz. Amplification was set at 10 μV per mm. Impedance of surface EMG electrodes had to be lower than 10 kΩ.

Sleep and LM scoring criteria
Sleep was scored according to American Academy of Sleep Medicine (AASM) criteria [1].
According to the AASM criteria [11], the onset of a leg movement event is defined as the point at which there is an 8 μV increase in EMG voltage above resting EMG; the ending of a leg movement event is defined as the start of a period lasting at least 0.5 s during which the EMG does not exceed 2 μV above resting EMG. Bilateral leg movements separated by less than 5 s between movement onsets are counted as a single leg movement. Leg movements occurring during a period from 0.5 s preceding a respiratory event to 0.5 s following are not scored according to AASM criteria. We performed two separate analyses, one according to AASM and one including also respiratory-related leg movements.
Periodic limb movements (PLM) are defined as repetitive leg movements lasting from 0.5 to 10 s, separated by an intermovement interval (defined as the time between onsets of consecutive leg movements) ranging from 5 to 90 s, organized in series of at least 4 leg movements [11].
The periodicity index, which is an independent measure able to pick up the time structure of LM activity in patients with RLS and PLMS, was calculated as defined by Ferri et al. as the ratio of the number of periodic LM intervals (at least 3, 10-90 s) divided by the total number of intervals found [12].

Description of manual and computerized PLM quantification and analysis
PLMS indices in NREM, REM and total sleep, PLM during wakefulness (PLMW) index, the intermovement interval in NREM, REM and total sleep, and in wakefulness, index of PLMS associated with respiratory events and periodicity index were all scored manually and automatically as explained below, and inserted in a SPSS database for the statistical analysis. The manual scoring was performed first, without running the automatic analysis, blind for the automatic scoring.

Manual quantification of LM and PLM
LM and PLM during sleep and during wakefulness were manually analysed by a trained scorer (AS) according to AASM criteria [1]. All identified leg movements (LM) were tabulated in an excel spread-sheet, and associated to sleep stages manually. The duration of each LM and the interval between two consecutive LM were measured. Indices were calculated for PLMS during NREM, REM and total sleep, and for PLMW. Intermovement intervals for LMS and PLMS during NREM, REM and total sleep, as well as LM during wakefulness (LMW) and PLMW, were scored. The periodicity index according to Ferri et al. was also calculated [12].
PLMS associated with respiratory events were excluded from the analysis according to the AASM standard criteria [11]. All PLMS were however recorded in the excel spread-sheet, including informations about association or not with respiratory events, so that separate analysis including or excluding PLMS associated with respiratory events were possible. Indices of PLMS associated with respiratory events per hour of total sleep time were separately calculated.

Computerized scoring algorithm for PLM detection and analysis
The computerized software algorithm for detection and analysis of PLM is a feature of the Brain RT PSG system by OSG (2840 Rumst, Belgium; http://www.osg.be). The algorithm was developed by OSG and adjusted in several site visits and steps upon author's request, to automatically detect PLM according to AASM criteria.
For this study, LMS and PLMS indices in NREM, REM and total sleep, LMW and PLMW index, the intermovement interval in NREM, REM and total sleep, and in wakefulness were automatically calculated. In addition, the analysis was run again changing the settings in order to include also PLMS associated with respiratory events, and indices of PLMS associated with respiratory events per hour of total sleep time were automatically scored. No artefact correction was performed.
The analysis was then run changing in the settings the criterion for the interval between two consecutive PLMS from 5-90 s to 10-90 s, in order to be able to automatically calculate the periodicity index [12].
The settings of the software for detection and analysis of LM are shown in Additional file 1: Figure S1, and an example of the computerized detection of PLM is shown in Additional file 2: Figure S2.

Event per event analysis
An event per event analysis was performed for each LM. All LM detected by both manual and computerized analysis were counted as matched in case of overlap between a manually detected LM and at least half of the automatically detected LM. The LMs detected only manually, as well as those detected only by computerized analysis, were also counted. The sensitivity of the computerized detection was calculated as the percentage of manually scored leg movements also detected automatically. The false positive rate was calculated as the percentage of automatically detected LM that did not match manually detected LM.

Statistics
IBM SPSS 21 (SPSS, Inc., Chicago, IL) was used for all statistical analysis. Data were tested for normal distribution using the Shapiro-Wilk test. Descriptive statistics are given as numbers (percentages) as well as medians (range), as data were not normally distributed. Nonparametric statistics were applied. Correlations and agreement between manual and computerised quantification of LM and PLM were calculated by means of the Spearman correlation coefficients, the intraclass correlation coefficients and the Bland-Altman plots. P-values <0.05 were considered significant. In case of multiple comparisons, correction for Bonferroni was performed, and p-values were set accordingly (p < 0.01).

Results
Demographic, clinical and sleep characteristics of the RLS patients and the control group Twenty patients with RLS (14 men, 6 women) with a median age of 51.5 (37-73) years were included in this study.
The sleep parameters of the two groups are provided in Additional file 3: Table S1.
Tables 1 and 2 provide the LM and PLM measures obtained by manual and computerized detection and analysis according to AASM criteria [11], as well as the manual and computerized calculation of the periodicity index according to Ferri et al. [12] Of note, all values calculated by manual and computerized analysis were very similar. In line with this, all Spearman correlation coefficients were between 0.751 and 0.996, and all intraclass correlation coefficients between 0.775 and 0.999. Correlations between manual and computerized detection and analysis of LM and PLM are also shown in The event per event analysis showed a high agreement between the two methods, as shown by sensitivity percentages of the computerized LM and PLM analysis ranging from 95 to 100%, and false positive percentages between 0 and 11% (Additional file 4: Table S2).

Discussion
This study validated a software for the detection and analysis of LM integrated into a commercialy available PSG system, which quantifies different LM and PLM indices according to AASM criteria. The correlation between the computerized scoring of different LM and PLM indices and the standard visual detection and manual scoring was remarkable high. The validated integrated software allows the correlation of LM and PLM data with all other PSG data without needing to export data to another program or to run another analysis on the same platform, as the analysis can be done within the same single routine PSG report, and gives in addition visual informations. All LM and PLM parameters used not only in the routine but also for research purposes, including intermovement intervals and periodicity index, can be automatically scored with this software, and the scoring settings can be modified by the user.
Several software programs for the automatic detection and analysis of PLM are available. The first study evaluating an algorithm for automatic detection of PLM dates back to 1990 [13], and after that several algorithms for the automatic scoring of PLM were tested [14][15][16][17][18][19][20], but not adequately validated. To our knowledge only one [20] is included in a commercially available software for  Although in this study agreement between manual and automatic analysis was high for all investigated parameters, it was higher for PLM indices than for intermovement intervals. This is probably due to the fact that scoring or not scoring a few PLM makes little difference in the PLM index but can produce a sizeable difference in the intermovement interval, specifically when overall PLM indices are low. Those differences were more pronounced in REM sleep. A possible explanation is the more difficult quantification of PLM during REM sleep due to fragmentation of movements. The difference in intermovement intervals between manual and computerized scoring was more evident in the control group, which may be attributed to the reason explained above. In fact in such cases, including or not including in the analysis few PLM can produce a relevant change in intermovement intervals. Nevertheless, the software we validated showed high correlations and high agreement between both methods also in subjects with extreme PLM indices, which ranged from 0.2 to 194.6/h. The inclusion and exclusion criteria were specifically designed to validate this software in both subjects without relevant PLM and in subjects with high indices.
Of note, the periodicity indices in the current study were higher than previously reported in both RLS patients and control subjects [11]. In the original study proposing the periodicity index, Ferri and colleagues analyzed PLMS and not PLMW. Although periodicity index is based only on PLMS, including PLMW series occurring during wake after sleep onset (WASO) allows the inclusion of a higher number of PLMS that are part of mixed PLM series containing both PLMS and PLMW but constituted by <4 PLMS, that otherwise would not be counted as PLMS.
The software used for this study is fully implemented in a routine PSG system. This is on the one hand essential for the clinical routine; on the other hand, it is a timesaving system to obtain good data also for research purposes, as all the LM and PLM indices and variables can be easily calculated. No artifact correction was performed, so that the software can also be used by untrained scorers. However, the validation was done in a study with high recording quality, but manual artifact correction could be needed in recordings with worst signal quality. This software also gives the users the possibility to change several settings, with the result that it can detect and analyze LM also according different criteria.
A limitation of this study is the exclusion of patients with sleep-related breathing disorders, with or without nCPAP therapy. This should be done in further studies. Studies including different age groups, e.g. children, showing more spontaneous activity during the night, would also be of interest. Another aspect that remains to be explored is the importance of movement amplitude [21]. This was not done in this study, but amplitude measurement is possible with the software and this issue should be addressed in future studies.
As this study also validated LM count in addition to PLM analysis, these results will remain of interest also if PLM criteria will change in the future. The algorithm for LM count and PLM analysis is published, so results of the study can be replicated by other groups.