Quantitative home-based assessment of Parkinson’s symptoms: The SENSE-PARK feasibility and usability study
BMC Neurology volume 15, Article number: 89 (2015)
Currently, assessment of symptoms associated with Parkinson’s disease is mainly performed in the clinic. However, these assessments have limitations because they provide only a snapshot of the condition.
The feasibility and usability of an objective, continuous and relatively unobtrusive system (SENSE-PARK System), which consists of wearable sensors (three worn during the day and one worn at night), a smartphone-based App, a balance board and computer software, was tested 24/7 over 12 weeks in a study including 22 PD patients. During the first four weeks of the study, patients did not get feedback about their performance, during the last eight weeks they did. The study included seven clinical visits with standardized interviews, and regular phone contact. The primary outcome was the number of drop-outs during the study. As secondary outcomes, the Post-Study System Usability Questionnaire (PSSUQ), score and information obtained from the standardized interviews were used to evaluate the usability of the system.
All patients completed the study. The participants rated the usability of the SENSE-PARK System with a mean score of 2.67 (±0.49) on the PSSUQ. The interviews revealed that most participants liked using the system and appreciated that it signaled changes in their health condition.
This 12 week controlled study demonstrates that the acceptance level of PD patients using the SENSE-PARK System as a home-based 24/7 assessment is very good. Particular emphasis should be given to a user-friendly design. Motivation to wear such a system can be increased by providing direct feedback about the individual health condition.
At present, measurements of Parkinson’s disease symptoms are almost all performed in a clinical setting, which may not reflect daily life situations. The most widely used assessment scale for PD symptoms is the Unified Parkinson’s Disease Rating Scale (MDS - UPDRS) which includes interviews asking the patient for historical information referring to the previous week, and a clinical rating scale which includes semi-quantitative assessment of motor (dys) function . However, there is broad agreement in the scientific field that new assessment strategies are needed, in particular those which have high ecological validity, multiple time points of evaluation and are effective .
Quantitative assessments using wearable technology may allow for continuous, objective and ecologically valid data collection and can be applied frequently at short intervals outside of the physician’s office, allowing real-time monitoring of symptom changes. This approach may also improve patient-doctor interaction, influence therapeutic decisions and ultimately ameliorate patients’ global health status. In addition, such measures have the potential to be used as outcome parameters in clinical trials, allowing for frequent assessments (e.g., in the home setting). These wearable sensors are of particular interest as they can be worn unobtrusively, so they do not relevantly influence the person who wears the sensor during a test, or during daily life. In addition, they can measure movements, and can be attached to almost every part of the body where symptoms of interest can occur .
The first studies with such sensors were performed more than 10 years ago and focused on the assessment of tremor and dyskinesia. Feeding the information obtained by such sensors back to the individual user, so that he or she can learn more about the individual disease presentations and how to counteract disease-associated symptoms, may provide additional motivation for users.
Within the framework of an EU-funded project (www-sense-park.eu), a device consisting of four components: software, a smartphone app, a Wii balance board and a set of sensors, was developed . This paper focuses on the results of this study with regard to feasibility and usability of the SENSE-PARK System over a prolonged time frame in the home environment of the users.
A multi-centre, open-label, feasibility and usability study comprising 22 participants was conducted. Eleven PD patients were invited to use the SENSE-PARK System for 12 weeks and perform clinical assessments every other week. The other 11 patients only completed the clinical assessments.
The primary outcome was the frequency of drop-outs and the first secondary outcome was assessed through the IBM Post-Study Usability Questionnaire (PSSUQ - rating score from 1, best, to 7, worst).
The SENSE-PARK System consists of a set of wearable sensors (3 to be used during the day and one at night), a Wii Balance Board, software and a smartphone app (Fig. 1).
The sensors monitor movements of PD patients during daily activities, collecting motor-related raw data. Accelerometers and angular rate sensors measure the motion of the user, and map certain activities. When awake, the user wears a small sensor at the wrist and the leg of the more affected side, as well as at the lower back according to Fig. 1. When asleep, the user wears one sensor at the lower back only. This set of sensors, together with algorithms developed during the SENSE-PARK project phase, allows monitoring parameters associated with gait, hypokinesia, dyskinesia, tremor and sleep. Through the Wii Balance Board, data such as body weight and sway are also collected. The user also performs cognitive tests through specific software. These tests use virtual environments on a screen to evaluate specific cognitive domains, including alertness, divided attention, response control, visual exploration and working and topological memory.
A two day investigator meeting was conducted to train staff, including installation of equipment, and procedures for training participants in the use of the System. Investigators received training in test administration and scoring.
A sample of 22 PD patients was divided into two groups: SENSE-PARK System users and non-users. Inclusion criteria were (1) PD patients between 40 and 85 years of age, (2) stage 1 to 2.5 (ON) of the Modified Hoehn and Yahr (H&Y) scale  and (3) having experience or interest in technical equipment (computer, regular mouse and keyboard).
Exclusion criteria included (1) illiteracy, (2) ≤24 in the Mini Mental State Examination (MMSE) , (3) postural instability item MDS-UPDRS > 2, (4) inability to handle the device for some other reason.
There were no restrictions on prior and concomitant therapy. An approval was obtained from the local ethics committee at the three sites (University of Lisbon, Portugal; University of Tübingen, Germany; and University of Tromsö, Norway), and in compliance with national legislation and Declaration of Helsinki.
Study design and assessments
After signing the informed consent, participants were screened for eligibility. For the group of users, a home visit was scheduled to ensure users had the required home facilities for the use of the SENSE-PARK System and to install the full system (the mouse setting was defined according to the hand the user normally writes with). The users were also invited to perform a demo session, in order to get acquainted with the SENSE-PARK System. Only balance and cognitive testing had to be performed “actively”. On cognitive test occasions, patients were instructed to be seated in a chair in front of the computer and place the computer and mouse on a table. Balance tests had to be performed with a Wii Balance Board which was connected to the local computer and the SENSE-PARK System. The other domains’ data were collected with the sensors which were worn 24 hours per day: three to be used during the day and one at night.
After the installation, participants used the SENSE-PARK System at home for one week and then they were observed by the investigator who registered whether the participants made mistakes during the tasks and whether they needed assistance. When necessary, another week of training was allowed. The time needed for confident SENSE-PARK System operation, test taking, data storage and data transfer was recorded. As soon as the user was able to use the SENSE-PARK System adequately and it was working properly, the assessment phase of the study started.
During the assessment period, patients who used the SENSE-PARK System were asked to perform sway assessment and cognitive tests every other day at a similar time of day during ON stage. Patients selected, in advance, the days of the week they would perform their testing, which was then programmed into the SENSE-PARK System with a reminder coming up on the interface of the software. An ongoing check of data arrival allowed users who were near to missing their two day window to be contacted by the study staff. If any problems or questions developed, users had access to the study staff for backup support.
Every other week during the study, participants from both groups returned to a health professional expert in movement disorders or were visited to perform the control of concomitant medications, check data download, record the occurrence of any Adverse Event, confirm patient compliance and to perform the following rating scales: MDS-UPDRS , H&Y  , Montreal Cognitive Assessment (MoCA) , Mini–Mental State Examination (MMSE) , Parkinson's Disease Questionnaire-39 (PDQ-39) , EQ-5D , Epworth Sleepiness Scale (ESS) , Panic Disorder Severity Scale (PDSS) , Non-Motor Symptoms Scale (NMSS) , Unified Dyskinesia Rating Scale (UdysRS) , Clinical Global Impression - Severity Scale and Improvement Scale (CGI-S and CGI-I) .
Between investigator visits, semi-structured interviews were conducted by phone to gain insight into the experiences of the participants using the SENSE-PARK System. Topics discussed were: willingness to continue in the study, satisfaction with the SENSE-PARK System, changes in health status or medical condition, adverse events, feedback messages and doubts about the system. Those participants who were not using the SENSE-PARK System were asked about willingness to continue in the study, health status, medical condition and adverse events (usability evaluation).
After 4 weeks of active data collection with the SENSE-PARK System, the participants received a modified version of the software. This new version provided feedback to users (Fig. 2).
At the end of the study period (V8), participants were asked to fill in the IBM Post-Study Usability Questionnaire (PSSUQ)  which assesses user satisfaction with, and usability of, technical devices. It is a 19-item closed-ended ordinal questionnaire, based on 7-point graphic Likert scales. The items address five important components of user satisfaction with general computer systems usability: ease of use, ease of learning, simplicity, effectiveness, information, and the user interface .
Statistical analyses were performed in R, version 3.0.3. Subject demographic and clinical data were examined by descriptive summary. Usability was assessed for system users and was tested with the PSSUQ values obtained at the end of the study (see above). Feasibility was assessed for users and non-users. For non-users, feasibility was assessed through willingness to continue in the study and through completeness of the study, i.e. withdrawals from the study. For users, feasibility was assessed through the following indices: willingness to continue in the study and through completeness of the study, duration of time to train participants to use the SENSE-PARK System confidently, compliance in terms of the number of completed test sessions and their timeliness with every other day assessments, compliance/adherence to system measures, success of data transfer and decryption, ease of use and program satisfaction [16, 17].
Twenty two idiopathic PD patients (14 male, 8 female) were included in the study (Table 1). There were no significant differences regarding age and sex distribution as well as MMSE scores among the countries. Although not significantly different, study participants included in Norway were slightly younger.
All participants showed willingness to continue in the study from visit 1 to visit 8, and all of them completed the study. Most participants completed the tasks asked for during the study and according to the protocol. Number of doubts and difficulties decreased along the study (Fig. 3). Only one participant failed to transfer the data when asked. All the other SENSE-PARK System users were able to download the data along the study.
All participants showed willingness to continue in the study from visit 1 to visit 8, and all of them completed the study. Since these patients were not system users questions concerning data transfer, number of doubts and difficulties and completion of tasks were not assessed.
No adverse events were recorded.
Usability aspects were only assessed for system users. Overall analyses from the PSSUQ scale showed that the system was generally well accepted (Fig. 4). Analysis of the PSSUQ scale by country showed that study participants from Norway had a slightly different profile than those from the other countries, scoring less (i.e. better) with regard to system use in items such as ability to efficiently complete tasks, comfort with using the system, and ease of learning.
Problems that arise from using this device
There were four adverse events registered, three of them being classified as definitely related to the study and one being possibly related to the study (Table 2).
We found the SENSE-PARK System highly feasible in terms of patient compliance, satisfaction and ease of use. Patients maintained their involvement with the program over 16 weeks, and several requested a continuation of the program at study end. We were initially concerned that participants would find the testing process, of every other day participation and weekly visits, tedious, but the satisfaction ratings did not support this concern, and the computer-based technology guided the test-taking smoothly. However, we encountered more problems with transmission and sensor errors compared to other tasks, but by mid-study, technical correction was achieved.
Hardware related problems consisted mainly of a broken sensor. Battery replacements were not required.
The training period was short and the testing was self-administered without direct investigator involvement. Consistent with the concept of the SENSE-PARK System as a monitoring tool, after the first 4 weeks we allowed users to see their previous scores when they took their next test.
This pilot study is a “proof of concept” and establishes patient and technical feasibility of the SENSE-PARK system.
There is, as yet, no evidence that using this test array will improve treatment of patients during clinical practice or that a cost benefit analysis is favorable. Further studies are needed to test how the test array performs in a clinical setting with larger patient populations. One question to be addressed is whether the SENSE-PARK System can detect a need for treatment changes. Additionally, whether there are country-specific issues, creating different requirements for its introduction into clinical practice, needs to be evaluated.
Clinical global impression - Improvement scale
Clinical global impression - Severity scale
Epworth sleepiness scale
Hoehn and Yahr
(Movement disorder society) Unified Parkinson’s disease rating scale
Mini–mental state examination
Montreal cognitive assessment
Non-motor symptoms scale PDQ-39, Parkinson's disease questionnaire-39
Panic disorder severity scale
Post-study system usability questionnaire
Unified Dyskinesia rating scale
Goetz CG, Fahn S, Martinez-Martin P, Poewe W, Sampaio C, Stebbins GT, et al. Movement Disorder Society-sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS): Process, format, and clinimetric testing plan. Mov Disord. 2007;22:41–7.
Maetzler W, Domingos J, Srulijes K, Ferreira JJ, Bloem BR. Quantitative wearable sensors for objective assessment of Parkinson's disease. Mov Disord. 2013;28:1628–37.
Serrano JA, Thoms A, Weber P. Patients Initiated Timeline Marking of Events in Parkinson’s Disease: Visualization of Time Correlation between Patients Marked Events and Acquired Data from Sensors. In: Schmorrow D, Fidopiastis C, editors. Foundations of Augmented Cognition Advancing Human Performance and Decision-Making through Adaptive Systems. Lecture Notes in Computer Science. Springer International Publishing. 2014;8534:325-34.
Hoehn MM, Yahr MD. Parkinsonism: onset, progression and mortality. Neurology. 1967;17:427–42.
Pangman VC, Sloan J, Guse L. An examination of psychometric properties of the mini-mental state examination and the standardized mini-mental state examination: implications for clinical practice. ANR. 2000;13:209–13.
Gill DJ, Freshman A, Blender JA, Ravina B. The Montreal cognitive assessment as a screening tool for cognitive impairment in Parkinson's disease. Mov Disord. 2008;23:1043–6.
Jenkinson C, Peto V, Fitzpatrick R, Greenhall R, Hyman N. Self-reported functioning and well-being in patients with Parkinson's disease: comparison of the short-form health survey (SF-36) and the Parkinson's Disease Questionnaire (PDQ-39). Age Ageing. 1995;24:505–9.
Rabin R, de Charro F. EQ-5D: a measure of health status from the EuroQol Group. Ann Med. 2001;33:337–43.
Johns MW. A new method for measuring daytime sleepiness: the Epworth sleepiness scale. Sleep. 1991;14:540–5.
Shear MK, Brown TA, Barlow DH, Money R, Sholomskas DE, Woods SW, et al. Multicenter collaborative panic disorder severity scale. J Psychiatry. 1997;154:1571–5.
Chaudhuri KR, Martinez-Martin P, Brown RG, Sethi K, Stocchi F, Odin P, et al. The metric properties of a novel non-motor symptoms scale for Parkinson's disease: Results from an international pilot study. Mov Disord. 2007;22:1901–11.
Goetz CG, Nutt JG, Stebbins GT. The Unified Dyskinesia Rating Scale: presentation and clinimetric profile. Mov Disord. 2008;23:2398–403.
Guy., William. ECDEU assessment manual for psychopharmacology Rockville, Md.: U.S. Dept. of Health, Education, and Welfare, Public Health Service, Alcohol, Drug Abuse, and Mental Health Administration, National Institute of Mental Health, Psychopharmacology Research Branch, Division of Extramural Research Programs. Maryland, US: U.S. Dept. of Health 1976.
Fruhling A, Lee S. Assessing the Reliability, Validity and Adaptability of PSSUQ. AMCIS 2005 Proceedings 2005.
Lewis JR. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum-Comput Interact. 1995;7:57–78.
Parsons A, McCullough C, Wang J, Shih S. Validity of electronic health record-derived quality measurement for performance monitoring. Journal of the American Medical Informatics Association 2012;19(4):604–9.
Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, et al. How we design feasibility studies. Am J Prev Med. 2009;36:452–7.
This study was performed in the frame of the EU project SENSE-PARK, funded under the Seventh Framework Programme, Cooperation – ICT, Grant Agreement no. 288557.
Joaquim J. Ferreira has held consultancy functions with GlaxoSmithKline, Novartis, TEVA, Lundbeck, Solvay, Abbott, BIAL, Merck-Serono, Merz, Ipsen; has received grants from GlaxoSmithKline, Grunenthal, Fundação MSD (Portugal) and Teva; has been a member of the European Huntington Disease Network and has been employed by Centro Hospitalar Lisboa Norte, Faculdade de Medicina de Lisboa. Helen Matthews has been employed byThe Cure Parkinson’s Trust. Walter Maetzler has received speaker honoraria from UCB, GlaxoSmithKline and Rölke Pharma. None of the other authors have any competing interests to report.
JJF: conception and design, revising it critically for important intellectual content, has given final approval of the version to be published. ATS: conception and design, drafting the manuscript, has given final approval of the version to be published. CG: acquisition of data, drafting the manuscript, has given final approval of the version to be published. JD: conception and design, drafting the manuscript, has given final approval of the version to be published. DA: analysis and interpretation of data, drafting the manuscript, has given final approval of the version to be published. NG: analysis and interpretation of data, drafting the manuscript, has given final approval of the version to be published. MB: analysis and interpretation of data, drafting the manuscript, has given final approval of the version to be published. FL: conception and design, drafting the manuscript, has given final approval of the version to be published. IA: acquisition of data, drafting the manuscript, has given final approval of the version to be published. HW: acquisition of data, drafting the manuscript, has given final approval of the version to be published. AS: conception and design, drafting the manuscript, has given final approval of the version to be published. PW: conception and design, drafting the manuscript, has given final approval of the version to be published. AT: conception and design, drafting the manuscript, has given final approval of the version to be published. SM: conception and design, drafting the manuscript, has given final approval of the version to be published. SS: conception and design, drafting the manuscript, has given final approval of the version to be published. JU: acquisition of data, drafting the manuscript, has given final approval of the version to be published. MAH: acquisition of data, drafting the manuscript, has given final approval of the version to be published. KSM: acquisition of data, drafting the manuscript, has given final approval of the version to be published. HM: conception and design, drafting the manuscript, has given final approval of the version to be published. TI: conception and design, drafting the manuscript, has given final approval of the version to be published. JD: conception and design, drafting the manuscript, has given final approval of the version to be published. HG: conception and design, revising it critically for important intellectual content, has given final approval of the version to be published. WM: conception and design, revising it critically for important intellectual content, has given final approval of the version to be published. All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
About this article
Cite this article
Ferreira, J.J., Godinho, C., Santos, A.T. et al. Quantitative home-based assessment of Parkinson’s symptoms: The SENSE-PARK feasibility and usability study. BMC Neurol 15, 89 (2015). https://doi.org/10.1186/s12883-015-0343-z