Validation of quantitative gait analysis systems for Parkinson’s disease for use in supervised and unsupervised environments

Background Gait impairments are among the most common and impactful symptoms of Parkinson’s disease (PD). Recent technological advances aim to quantify these impairments using low-cost wearable systems for use in either supervised clinical consultations or long-term unsupervised monitoring of gait in ecological environments. However, very few of these wearable systems have been validated comparatively to a criterion of established validity. Objective We developed two movement analysis solutions (3D full-body kinematics based on inertial sensors, and a smartphone application) in which validity was assessed versus the optoelectronic criterion in a population of PD patients. Methods Nineteen subjects with PD (7 female) participated in the study (age: 62 ± 12.27 years; disease duration: 6.39 ± 3.70 years; HY: 2 ± 0.23). Each participant underwent a gait analysis whilst barefoot, at a self-selected speed, for a distance of 3 times 10 m in a straight line, assessed simultaneously with all three systems. Results Our results show excellent agreement between either solution and the optoelectronic criterion. Both systems differentiate between PD patients and healthy controls, and between PD patients in ON or OFF medication states (normal difference distributions pooled from published research in PD patients in ON and OFF states that included an age-matched healthy control group). Fair to high waveform similarity and mean absolute errors below the mean relative orientation accuracy of the equipment were found when comparing the angular kinematics between the full-body inertial sensor-based system and the optoelectronic criterion. Conclusions We conclude that the presented solutions produce accurate results and can capture clinically relevant parameters using commodity wearable sensors or a simple smartphone. This validation will hopefully enable the adoption of these systems for supervised and unsupervised gait analysis in clinical practice and clinical trials.


Background
Parkinson's disease (PD) presents a multitude of motor and non-motor symptoms heterogeneous in expression and progression over time [1,2]. Alterations in full-body kinematics are seen since early stages of the disease and are characterized by reduced velocity, step length, arm swing and smoothness, increased inter-limb asymmetry, impairments in complex locomotor tasks (e.g., turning) and reduced range of motion at several joints (e.g., shoulder), leading in later stages to an increase in double-support time and cadence, shuffling steps, freezing of gait and festination [3]. Changes in lower and upper body kinematics have shown to be potential markers of disease progression in the early stage (number of steps, total duration, harmonic ratios and arm swing asymmetry) and middle stage (stride time variability and stride regularity) disease progression [4,5].
Motor symptoms of PD are commonly rated using the Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS), which has guided therapeutic decisions for several decades [1]. This approach is subjective and time-consuming, limited in assessment repetition and lacking in quantitative outcomes [6]. Fullbody kinematic analysis systems are needed to obtain a full and objective assessment of movement as a basis for individually tailored clinical decision making and prognostication.
For decades, optoelectronic tracking systems have been established as a criterion in quantitative movement analysis [7]. However, such equipment is expensive, restricted to a controlled and calibrated environment, requiring a complex setup phase [7][8][9]. Wearable inertial sensors (e.g., inertial measurement units -IMU) have been introduced into clinical practice due to their relatively low cost, lightweight and ease of use [7][8][9] and have been proven to be useful and feasible for kinematic analysis of PD gait [10,11]. However, very few studies applying inertial sensor systems quantitatively describe the changes in full-body kinematics in PD gait [10][11][12][13].
The previously described motor symptom assessment methods all require an in-person consultation with the patient. In between such assessments, motor symptoms and their fluctuations are reported using patient diaries [14,15]. The risk of poor adherence, limited time resolution and subjective nature of such information raises concerns regarding their accuracy and reliability [14,15]. It is necessary to expand beyond conventional clinical assessments and aim for movement analysis in unsupervised, ecologically valid and patient-relevant environments to achieve a more accurate kinematic characterization.
Quantitative unsupervised movement analysis could capture symptom fluctuations and rare events in PD while minimizing the effects of supervision [11,14]. This would require an affordable, user-friendly system with minimal impact on daily living [16,17]. Smartphones are owned by 81% of adults in the United States [18], contain one or several embedded inertial sensors and have been validated as a platform for ecological movement analysis [16,[19][20][21][22]. Benefits in terms of healthcare accessibility, patient engagement and reduction of clinician workloads are also envisioned or already proven to emerge from the use of mobile health technologies [23]. However, many studies addressing movement analysis using smartphone sensors have requirements that do not reflect typical usage, reducing ecological validity, and validation of applications for gait analysis of PD patients is limited [19].
Attempting to address current issues of quantitative motion analysis in terms of cost, time consumption and influence of supervision, two methods of objective and quantitative gait analysis were developed and are presented in this study: a 3D full-body kinematics analysis system based on inertial sensors for supervised objective evaluations, and a smartphone application for characterization of gait-related motor symptoms of PD in daily living scenarios beyond clinical assessments. To assess the validity of these methods for clinical practice and clinical trials, this study aims to simultaneously assess the estimation of the agreement between each of these two methods of objective and quantitative gait analysis and the optoelectronic criterion in patients with PD.

Recruitment and eligibility
The study was approved by Campus Neurológico Sénior (CNS) Ethics Committee (Ref. 04/2019) and all participants gave their written informed consent in accordance with the Declaration of Helsinki. Study participants were recruited between July and October 2019 from the inand outpatients of the CNS, a tertiary specialized movement disorders center. After providing written informed consent, subjects with a PD diagnosis were evaluated for eligibility for study enrolment. Eligibility criteria included the ability to understand the potential risks and benefits of the study, willingness and ability to provide written informed consent to participate in the study, PD diagnosis (according to MDS criteria [24]), ability to walk unassisted, perform the Timed Up and Go test in a normal pace and without assistance in less than 11,5 s (i.e. no fall risk) [25], in ON phase, and Hoehn & Yahr scale < III. Exclusion criteria were defined as cardiovascular, pulmonary or musculoskeletal conditions that could affect patients' ability to participate in the study, inability to correctly respond to the assessment protocol according to the clinician's judgment or lack of support from a caregiver for this purpose and permanent use of gait assistance. A certified rater performed the MDS-UPDRS on all subjects. A summary of the subjects' clinical outcomes is shown in Table 1.
A unique identification number was attributed to each participant enrolled in the study to maintain confidentiality.

Experimental setup
Participants underwent a gait analysis barefoot, at a selfselected speed, for a distance of 10 m along the laboratory's diagonal, going back and forth three times, never exceeding 1 min, and allowing the participant to rest whenever necessary. All participants received a careful explanation and were allowed a familiarization period in which they were given time to get accustomed to the equipment and space, and performed the task once whilst yet unrecorded. The participants' motion during the walking trials was assessed simultaneously with an optoelectronic (used as the criterion), an inertial and a smartphone-based capture system.
Time synchronization between motion capture systems was achieved by sending a trigger signal from the optoelectronic system to the inertial system to start data capture.
For the optoelectronic system, 48 retroreflective markers were placed on the participants' skin over specific anatomical landmarks (1st and 5th distal metatarsal head, Hallux's distal phalanx, heel, medial and lateral malleoli, medial and lateral femoral condyles, anterior and posterior superior iliac spines, xiphoid process, jugular notch, spinous process of 7th cervical and thoracic vertebrae, acromion, medial and lateral humeral epicondyles, head of the ulna, styloid process of the radius, and distal phalanx of the middle finger). Additionally, four rigid clusters, each with four markers, were attached to the lateral sides of the thighs and shanks. The markers were placed by an experienced physiotherapist, always prior to placement of any IMU and were held in position with double-sided adhesive tape. The 3D coordinates of each marker were collected at 120 Hz by a system of 10 infrared cameras (Oqus 300+, Qualisys AB, Gothenburg, Sweden). For the inertial system, fifteen IMU (Xsens, Enschede, Netherlands) with a sample rate set to 120 Hz, were placed and secured using elasticated velcro straps, in different body segments: head, thorax, scapulae, upperarms, forearms, hands, sacrum, thighs, shanks and feet. Care was taken not to interfere with the reflective markers.
Two Nokia 5.1TM smartphones (Nokia, Espoo, Finland) were placed on the participants' right and left front pockets, where Kinetikos CE-marked smartphone application (Kinetikos, Coimbra, Portugal) developed for Android (Google Inc., Mountain View, CA, USA) recorded linear and angular quantities captured by the device's built-in hardware sensors throughout the trial (accelerometer, gyroscope, magnetometer and orientation data) at 100 Hz.

Data analysis
Kinetikos CE-marked cloud-based platform (Kinetikos, Coimbra, Portugal) was used to reconstruct participants' full-body motion using a 3D kinematic computer model of the skeletal system that includes representation of the head, thorax, upper and lower extremities and respective joints, totaling 26 degrees of freedom (DoF). In this model, the hip was a ball and socket joint (3 DoF). Both the knee and ankle had a single DoF each set by a revolute joint (1 DoF + 1 DoF). Head and lumbar motion were each modelled as ball-and-socket joints (3 DoF + 3 DoF). Each upper extremity consisted of 5 DoF; the shoulder was modelled as a ball-and-socket joint (3 DoF), and the elbow and forearm rotation were each modelled with revolute joints (1 DoF). Each joint's coordinate system matches the International Society of Biomechanics (ISB) recommendations [26,27]. Rotations were represented by Cardan-Euler angles, following the sequences proposed by the ISB [26,27] for all joints except the shoulder, where the sequence XZY was chosen, which has shown superior results in terms of gimbal lock incidence and clinical coherence when compared to the ISB recommendation [28]. For this study, agreement between 11 DoF was considered: hip flexion/extension, hip adduction/abduction, hip internal/external rotation, knee flexion/extension, ankle dorsi/plantar flexion, arm flexion/extension, arm adduction/abduction, arm internal/ external rotation, elbow flexion/extension, forearm pronation/supination and wrist flexion/extension. Both virtual markers and virtual IMUs were placed on the 3D biomechanical model to match the locations of the experimental ones.
The markers' 3D trajectories and IMU orientations were identified in Qualisys Track Manager (Qualisys AB, Gothenburg, Sweden) and Xsens MT Manager (Xsens, Enschede, Netherlands), respectively. The optoelectronic system's markers' trajectories were then exported while for each inertial sensor a file with inertial, angular and orientation data was generated. The model's segments were proportionally scaled based on each participant's height and the identified trajectories and orientations were expressed in the model's coordinate system. While orientations from the inertial based system were centered around the pelvis by default, trajectories from the optoelectronic system were centered around the model's pelvis by defining a coordinate system using the left and right anterior and posterior superior iliac spine markers [27] and expressing all marker trajectories in this system. Inverse kinematics was used to reconstruct the optimal body pose in each frame based on a global optimization procedure aiming to minimize the weighted sum of squared distances between measured and modeldetermined marker positions [29,30]. This procedure was conducted, firstly, based on the optoelectronic data by minimizing the errors between experimental and model markers 3D coordinates, and secondly, on the IMUs data by minimizing the errors between the experimental IMU orientations and the model's IMU Frames. The previously described method allows the model's joint angles to be used to track the orientation of each body part. The results were two sets of coordinate values of the model's DoF for each time frame of the gait trial. Gait events (left and right heel strike and toe-off events) were detected based on four kinematic-based procedures [31][32][33]. These algorithms use the calcaneus and toe marker displacement and velocity time series for the identification of: horizontal inter-heel distance [31]; foot center-of-mass vertical velocity [32]; horizontal heel and toe position relative to the pelvis; horizontal heel and toe velocity [33]. Since these estimates were found to identify heel strike and toe-off with unequal consistency [34], the event indices were further combined in a custom algorithm yielding a single, more robust outcome. Events with no correspondence on the optoelectronic system (i.e., events outside the area captured by the infrared cameras) were not considered for analysis. Because the camera system did not capture the beginning and end of the laboratory's diagonal, gait events during turns were not captured by this system. This process is described in Fig. 1.
Sensor synchronization in the Kinetikos CE-marked smartphone application was achieved through linear interpolation and spherical linear interpolation (for orientation data). Sensor fusion was performed using Madgwick's gradient descent IMU orientation estimation, which combines attitude estimates by integration of gyroscope measurements and direction obtained by accelerometer measurementsto compensate for long term gyroscope integration driftto obtain the device's global orientation [35]. Gait events (heel strikes and toe offs ipsilateral to the side of the pocket the smartphone   was placed in) were detected using a custom peak detection algorithm on the global vertical acceleration time series after filtering with a 4th order Butterworth lowpass filter at 2 Hz, based on the assessment that the vertical acceleration time series contains alternating high and low amplitude peaks and that the nadirs immediately following such peaks are correlated with heel-strike and toe-off events, respectively [36]. Only events from the mobile application with a corresponding event on the criterion were considered for analysis. This sequence of events is described in Fig. 2. Several gait parameters were calculated for both systems: stride duration as the time difference between two ipsilateral heel-strikes; stance phase duration as the time between a heel-strike event and the following ipsilateral toe-off event; swing phase duration as the time between a toe-off event and the following ipsilateral heel-strike event; cadence as the number of strides per minute; stride length given by the difference in a foot's position in the sagittal plane between consecutive ipsilateral heelstrikes for the inertial-based system and by the linear relation of stride length, stride frequency and acceleration variance [37] for the smartphone application; stride speed as stride length divided by stride time. In the case of the inertial sensor-based system, double-support phase duration and step width were also assessed, as the time between a heel-strike and the following contralateral toe-off and as the distance in the coronal plane between the two feet at the instant of a heel-strike, respectively.

Statistical analysis
Bland-Altman analysis [38] was used to assess the agreement between the 3D full-body kinematics based on inertial sensors and the smartphone application, and the optoelectronic criterion. Differences between methods versus the average of their values (bias) and Limits of Agreement (LoA) were calculated along with 95% confidence intervals (CI). For each gait parameter, the assumption of normality of the differences between methods was checked with the D'Agostino-Pearson omnibus test. Bland-Altman analysis was performed with GraphPad Prism software (GraphPad Software, Inc., La Jolla, CA).
Welch's two-sample t-test was used to analyze the significance of the differences in each gait parameter between the Normal difference distribution from PD patients versus healthy controls and from PD patients during ON versus OFF states and the bias of the inertial-based 3D full-body and the smartphone-based kinematics. Normal difference distributions were pooled from studies conducted in PD patients in ON and OFF states, that included an age-matched healthy control group [39][40][41][42].
Agreement between angular kinematics waveforms obtained from marker-based and IMU-driven inverse kinematics was assessed as follows: (i) average reconstruction error was taken as the mean absolute error (MAE); and (ii) waveform similarity was evaluated through the Linear Fit Method (LFM) [43]. The LFM yields three coefficients: α 1 , the amplitude scaling factor between the curves being compared and the reference time series; α 0 , which predicts any offset; and R 2 , the strength of the linear relation between the two time series. LoA for α 0 and α 1 were calculated to assess whether these parameters are statistically different from the same parameters of the line of equality. Concurrent validity was considered very high if R 2 is above 0.75, fair-to-high if R 2 is between 0.4-0.75, and low if R 2 is below 0.4. The agreement between angular kinematics waveforms was performed in Python software (Python Language Reference, version 3.7, available at http://www.python.org).

Demographic
Nineteen subjects with PD (7 female) were enrolled in the study (age: 62 ± 12.27 years; disease duration: 6.39 ± 3.70 years; HY: 2 ± 0.23), recruited from CNS, Torres Vedras, Portugal. Six hundred eighty five gait cycles were identified from the criterion system and compared to the corresponding gait cycles selected from the inertial sensor-based system and the mobile application. Due to data integrity problems, two subjects were excluded from the former validation and one from the latter.

Agreement between methods
Agreement of spatiotemporal parameters between either presented solution and the optoelectronic system used as a criterion was assessed through Bland-Altman analysis [38] (Figs. 3 and 4), resulting in an excellent agreement for all metrics. Bias for all spatiotemporal metrics was at least one order of magnitude below the mean values (Tables 2 and 3).
(See figure on previous page.) Fig. 3 Bland-Altman analysis. Each gait parameter presents (for the left and ride side) a plot of differences between 3D full-body kinematics based on inertial sensors and the optoelectronic criterion versus the mean of the two measurements, and a distribution plot of differences between measurement by methods. Grey areas represent the 95% CI for bias and limits of agreement Fig. 4 Bland-Altman analysis. Each gait parameter presents (for the left and ride side) a plot of differences between smartphone-based kinematics and the optoelectronic criterion versus the mean of the two measurements, and a distribution plot of differences between measurement by methods. Grey areas represent the 95% CI for bias and limits of agreement

Resolution
To analyze the differences between the bias of either presented system and the Normal difference distribution from PD patients versus healthy controls and PD patients during ON and OFF medication states, Welch's two-sample t-test was used, with normal difference distributions pooled from published research in PD patients in ON and OFF states that included age-matched healthy control groups [39][40][41][42]. As shown in Table 4, both gait analysis methods presented enough resolution to capture the differences between PD patients in the OFF phase and age-matched healthy controls, and between PD patients in OFF and ON medication states in all gait parameters with the exceptions of swing duration and step width.

Reconstruction error and waveform similarity
Average reconstruction error was taken as the MAE. MAE of 3091 ± 0,136°was achieved at the five lower limb DoF, while errors of 2348 ± 0,172°were found at the six upper extremity DoF. Waveform similarity, evaluated through the LFM [43], yielded mean coefficients of determination (R 2 ) of 0,781 ± 0,050 for the lower DoF and 0,750 ± 0,022 for the upper DoF, indicating a very high waveform similarity. For the α 1 (scaling factor) and α 0 (scalar addition) parameters, respectively, mean values Table 2 Agreement between the gait parameters (mean, standard deviation and coefficient of variance -CV) measured by the inertial-based 3D full-body kinematics (IMU) and the optoelectronic criterion system (OS  These parameters indicate the validity of this method to support in-depth analysis of the full-body movement patterns of PD patients. More detailed results of the LFM are presented in Table 5. Figure 5 presents the mean and 95% confidence interval for joint angles throughout the normalized gait cycle, for the optoelectronic criterion and the inertial sensor-based system.

Discussion
Gait impairments in PD can have a great impact on functional ability and quality of life and serve as indicators of global health, disease progression, cognition, fall risk and mortality [4,5]. In this study, two methods for characterization of PD gait were developed and validated comparatively to the current optoelectronic criterion: a 3D full-body kinematic method using inertial sensors for use in clinical settings, and a mobile-based method to capture ecologically valid gait parameters.
Many relevant kinematics-related parameters of PD gait cannot be accurately captured through clinical observation in supervised in-office visits or subjective patient journals [6,14,15]. Limitations in conventional methods of motor symptom rating motivate the addition of quantitative and objective, supervised and unsupervised assessment strategies for effective and impactful clinical decisions, improving symptom ratings at inoffice consultations and long-term, ecologically valid monitoring.
Wearable sensors have proven to be feasible instruments for accurate, quantitative movement analysis of PD patients in supervised environments, with advantages over the current optoelectronic systems in terms of cost, ease of use and environment requirements, although validation of full-body kinematic outputs using inertial sensor systems in PD patients is still lacking [10][11][12][13].
Current smartphones, with embedded inertial sensors, have shown potential and feasibility for unsupervised kinematic analysis during activities of daily living (ADL), Table 3 Agreement between the gait parameters (mean, standard deviation and coefficient of variance -CV) measured by the smartphone-based kinematics (Mob) and the optoelectronic criterion system (OS). Bias and Limits of Agreement (LoA) from the Bland-Altman analysis with 95% Confident Intervals (CI  effectively capturing relevant parameters [16]. Nonetheless, validation of smartphones' use for movement analysis in PD patients is still limited [19][20][21][22].

Agreement between methods
Bland Altman analysis for both the 3D full-body kinematics based on inertial sensors and the smartphone-based analysis shows an excellent agreement between the presented methods and the criterion. Some assumptions need to be taken into consideration: the mean and standard deviation of the differences are constant throughout the range of measurements; and these are from an approximately normal distribution. Scatter plots and histograms of Figs Tables 2 and 3). Bias and LoA amplitude for all metrics was at least one order of magnitude below the mean values for each parameter, suggesting high agreement between the methods. The lack of observable trend in the differences across the measurements reinforces that there is a strong agreement between the systems across a range of measurements.

Resolution
Either of the presented gait analysis methods showed enough resolution to capture the differences between PD patients and age-matched healthy controls, and PD Table 4 Absolute normal difference distribution (mean ± standard deviation) from PD patients versus healthy controls (PDvsControl) and Parkinson's disease patients between ON and OFF states (ONvsOFF) for each gait parameter [39][40][41][42]. Bias (mean ± standard deviation) values for each gait parameter for the inertial-based 3D full-body kinematics (IMU Bias) and the smartphone-based kinematics (Mob Bias Step width (m) 0.008 ± 0,031 0.001 ± 0,038 0.008 ± 0,021 -> 0,9999 -0,2520 aindicates that statistical significant differences were reached in the original study from which the normal difference distribution was pooled. Significant results are displayed in bold patients' differences between ON and OFF medication states in all gait parameters with exception of swing duration and step widththese differences were found to be non-statistically different in the original studies [39][40][41][42] (Table 4). These non-statistically significant differences are on the order of a thousandth of a second (swing duration) and thousandth of a meter (step width) which may raise a question about their clinical relevance.

Reconstruction error and waveform similarity
Agreement between angular kinematics waveforms obtained from marker-based and IMU-driven inverse kinematics was assessed based on the MAE and the LFM. Average reconstruction error was estimated by the MAE, resulting in magnitudes inferior to the mean relative orientation accuracy of Xsens IMUs sensors of 5°f or trials up to 60 s [44]. Results from the LFM showed high mean coefficients of determination, indicating a high waveform similarity across the full gait cycle for both lower-and upper-body DoF. The LoA of the scaling factor and scalar addition parameters of the LFM include 1 and 0, respectively, leading to conclude there exists a significant relationship between results obtained through the two movement analysis methods [45], albeit whether the LoA are small enough to ensure agreement requires a clinical perspective. The low MAE and high waveform similarity between the 3D full-body IMUbased movement reconstruction and optoelectronic criterion provide the needed evidence to support an indepth analysis of the full-body movement patterns in PD patients using this method.
The previous points corroborate that the IMU-based gait analysis system is capable of producing results comparable to the criterion, providing a valid, objective fullbody kinematic analysis with advantages in terms of cost, setup complexity and time consumption [7][8][9]. Nonetheless, this analysis is still subject to the effects of supervision and requires an in-person consultation. Complementing this approach, the smartphone-based solution, while unable to produce angular results or full-body gait analysis, has shown good agreement with the criterion and enough resolution to capture clinically relevant kinematic parameters, providing a quantitative assessment of spatiotemporal gait parameters with potential for use in ecologically valid environments with minimal impact on daily living, going beyond traditional, subjective measures of disease severity between consultations [14,15]. The combination of these solutions allows for a more holistic, quantitative assessment of PD gait.
Further studies will be required to estimate the validity and reliability of these systems in a broader spectrum of the disease stage and respective motor symptoms. In addition, it is important to expand validation of the smartphone-based system beyond supervised environments and into patient-relevant scenarios to evaluate performance in unsupervised ADL involving more complex motions and providing more accurate insight into the patient's motor symptoms.

Conclusions
Our results show that the IMU-based full-body gait analysis provides accurate results using simple sensors with a reduced setup and environment complexity when compared to an optoelectronic system. The presented mobile application can be a user-friendly solution to collect ecologically valid gait parameters, translating into a greater clinical insight on previously unreported or subjectively reported motor symptoms. The resolution of either method is sufficient to capture relevant statistical differences in PD patients gait parameters when compared to age-matched controls, and between their ON and OFF states, making both presented systems clinically viable solutions with the potential to improve PD patients' quality of life by opening the door to more personalized care based on quantitative objective metrics obtained not only during the moments of contact between patient and physician but longitudinally during ADL. Both the IMUbased and smartphone-based solutions produce results comparable to the criterion using commodity wearable technology. This validation will hopefully enable more widespread adoption of such systems for clinical practice and clinical trials.