Reliability and consistency of the Japanese version of the Primary Lateral Sclerosis Functional Rating Scale

Abstract

Background

Primary lateral sclerosis (PLS) is an extremely rare condition; therefore, to date no clinical studies have been conducted. The Primary Lateral Sclerosis Functional Rating Scale (PLSFRS) was developed in the United States of America. The PLSFRS is a crucial assessment scale for international collaborative research and future clinical trials for PLS. It is useful for evaluating medical conditions through face-to-face assessments and telephone interviews such as when a face-to-face assessment is not possible due to disasters or the burden of hospital visits. This study assessed the reliability and consistency of in-person and telephone interviews using the Japanese version of the PLSFRS.

Methods

We enrolled 19 Japanese patients who met the specific criteria for inclusion at the six collaborating institutions. The PLSFRS assessments were performed by two evaluators at defined time points and analyzed for intra-rater and inter-rater reliability and consistency between the in-person and telephone interviews.

Results

The Japanese version of the PLSFRS was developed by a specialized company and translator, and modified to consider the Japanese lifestyle through a consensus among motor neuron specialists. The quadratic-weighted kappa coefficients for the intra-rater and the inter-rater agreement were substantial (intra-rater: 0.691-1.000, inter-rater: 0.634-1.000). Moreover, the intraclass correlation coefficient for the PLSFRS total score was 0.997 (95% confidence interval, 0.992–0.999).

Conclusions

This study provides results regarding the Japanese version of the PLSFRS intra-rater and inter-rater reliability and consistency between in-person and telephone interviews.

Background

Primary lateral sclerosis (PLS) is a progressive neurodegenerative disease in which upper motor neurons are selectively damaged. PLS is rare, occurring in less than 5% of motor neuron diseases [1, 2]; therefore to date, no therapeutic agents or clinical trials have been evaluated. Since there have been no indicators to evaluate PLS, in 2020 a PLS Functional Rating Scale (PLSFRS) was developed in the United States of America, and the results were consistent with intra-rater, inter-rater, face-to-face visits, and telephone test-retest reliability during 1 year [3]. The PLSFRS is an essential tool for future international collaborative studies and clinical trials for PLS. Our study evaluated the reliability of the Japanese version of the PLSFRS and assessed the consistency between face-to-face evaluations and telephone interviews.

Methods

Institutional review board and volunteer participation

The study protocol was approved by the Ethics Committee of Toho University Faculty of Medicine, Tokyo, Japan (A21013, M20282), and other participating institutions (4058, rinfu 21 − 008, TS R03-017, ippan2021-318, 2021 − 0101). Written informed consent was obtained from all the patients. The PLSFRS was translated into Japanese by a specialized company and translator (2-14-10 Sotokanda Chiyoda-ku Tokyo 101 − 0021 Crimson Interactive Japan Co., ltd.), and it was further modified to account for the Japanese lifestyle based on consensus among the motor neuron specialists. Nineteen patients were enrolled in the study.

Protocol and inclusion and exclusion criteria

The study and evaluation protocols are presented in Table 1. Face-to-face examinations and telephone interviews were conducted at intervals of at least 24 h but no more than 7 days. All evaluators for this study were board-certified neurologists and were trained appropriately before the evaluation. The inclusion and exclusion criteria are listed in Table 2. The patients were encouraged to state their condition, and the caregivers were advised not to make their interpretations. Participants were excluded if they withdraw consent, died, or were deemed inappropriate to continue in the study.

Table 1 Study protocol

Table 2 Inclusion and exclusion criteria

Statistical analyses

We conducted statistical analyses of the PLSFRS to assess intra-rater and inter-rater reliability for each item of the PLSFRS and the consistency of the PLSFRS total score between a face-to-face examination and telephone interview using Microsoft Excel version 2210 (Microsoft Corp., Redmond, WA, USA) and IBM SPSS Statistics version 29.0 (IBM Corp., Armonk, NY, USA). Intra-rater reliability was assessed between the baseline and 8 weeks after baseline examination. Inter-rater reliability was assessed between the baseline and 4 weeks after baseline examination. The consistency between a face-to-face examination and telephone interview was assessed 4 weeks after the baseline examination.

Results

Demographics

Nineteen patients were enrolled at baseline. The average age of the participants at baseline was 68.2 (42–85) years. The sample included eight men (42.1%) and 11 women (57.9%). The median disease duration was 72 (12–264) months.

Dropouts

Four participants were excluded from the study. Two patients had aspiration pneumonia during the evaluation period; however this was not considered an adverse event associated with the evaluation. The evaluation could not be completed for the other two patients because they could not consult the doctor within the evaluation period.

Intra-rater and inter-rater reliability

The quadratic-weighted kappa coefficients for intra-rater and inter-rater agreement were substantial (intra-rater; 0.691-1.000, inter-rater; 0.634-1.000) (Table 3).

Table 3 Intra-rater and inter-rater consistency in this study

Consistency

The intraclass correlation coefficient for the PLSFRS total score between face-to-face examination and telephone interview was 0.997 (95% confidence interval, 0.992–0.999)

Discussion

PLS is a rare motor neuron disease, and to date, no clinical trials have been conducted. To develop therapeutic drugs in the future, it will be necessary to work globally, which includes collecting cases; therefore, creating a Japanese version of the PLSFRS is essential. Previously, the Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised (ALSFRS-R) alone evaluated the intra-rater and inter-rater reliabilities. The usefulness of administering the ALSFRS-R over the telephone was subsequently evaluated [4, 5]. Telephone interviews minimized the burden on participants and caregivers by reducing the number of visits required to the study sites. Four consultations, including three visits in 8 weeks, may have increased the burden; however, no adverse events were associated with the visits for this evaluation. The availability of quick and reliable measures that can be administered over the telephone is indispensable for conducting this type of research. Thus, in this study, the consistency of the telephone interviews was ensured from the outset. Repeated evaluator training is crucial for maintaining reliability of the scale. When creating this scale, we considered the Japanese lifestyle and, added instructions to prevent confusion during the evaluation. The Japanese version of the PLSFRS is provided as a supplement, and the original instructions were referenced and added to the table for the evaluators’ convenience.

The present study has several limitations. The current diagnostic criteria require the absence of clinical and electrophysiological lower motor neuron involvement for at least 4 years after symptom onset before diagnosing PLS [6]. Although the participants in this study were recruited according to the criteria, some patients were in the early stage of disease onset and may have been diagnosed with upper motor neuron-dominant amyotrophic lateral sclerosis. Moreover, we implemented less stringent electromyography criteria to enroll more participants and accommodate a wider spectrum of patients with PLS. This approach can be an ambivalent issue because the participants who developed pure upper motor disease for less than 4 years could not receive a definite diagnosis and may have been classified as having predominantly upper motor neuron dominant amyotrophic lateral sclerosis [7]. The concern with the PLSFRS was that, since there were two additional score tiers compared to the ALSFRS-R, except for orthopnea and respiratory insufficiency, the evaluator and the participant can get confused about the subtleties. We initially believed that evaluations by phone or in person would be difficult; however, the results showed no significant differences. Therefore, the Japanese version of the PLSFRS may be a valuable indicator for assessing medical conditions, even when face-to-face assessment is impossible due to disasters or hospital visit burdens.

Conclusions

This study provides the results for the Japanese version of the PLSFRS intra-rater and inter-rater reliability and an evaluation of the consistency between in-person and telephone interviews. Similar to the ALSFRS-R for amyotrophic lateral sclerosis, we believe that the Japanese version of the PLSFRS will be widely used not only by neurologists but also by internists and medical staff other than doctors and is a well-validated tool for PLS researchers to conduct future clinical trials.