Stability and relative validity of the Neuromuscular Disease Impact Profile (NMDIP)

Sample and procedure

A cross sectional study, using a postal survey, was administered to patients diagnosed with a NMD who were registered at the Department of Neurology of the University Medical Center Groningen, University of Groningen, the Netherlands. Inclusion criteria for this study were: diagnosis with a NMD and representing one of Rowland’s NMD classification groups: motor-neuron disorders, muscle disorders, junction disorders and peripheral nerve disorders [7]; being aged 18 or older; being able to read and write in Dutch; and being able to give informed consent. No exclusion criteria were formulated.

A total of 1003 eligible patients were selected from the hospital patient records system. To avoid inappropriately sending the questionnaires, we crosschecked for deceased patients using the national population register. Patients received information about the study and were invited to participate.

Respondents completed demographic and disease specific questions, the NMDIP, two criterion variables to measure the ‘Extent of Limitations’ and ‘Quality of Life’. Also, concurrent measures were completed: two generic multidimensional health impact measures (the Medical Outcome study Short Form Questionnaire (SF-36) [8], and the World Health Organization Quality Of Life-abbreviation version (WHOQOL-bref) [9], and two generic domain specific measures the Groningen Activity Restriction Scale (GARS) [10] and the Impact on Participation and Autonomy Questionnaire (IPAQ) [11]. To assess stability over time, the NMDIP was administered on two occasions to patients who agreed to fill in the questionnaire twice. We, arbitrary, selected a time frame from eight to 10 weeks to be sure that patients could not remember their answers on the first questionnaire, and the likelihood of changes in the health situation was minimal.

Measurement instruments

The NMDIP includes 36 items and consists of eight scales and four additional items. The 36 items were divided over the four ICF components. For the Body Functions component items and for the Participation component items scoring options ranged from 0 (no disability) to 4 (complete disability); for the Activities component items scoring options ranged from 0 (no disability) to 3 (complete disability); and for the Environmental Factors component items scoring options ranged from 0 (no support) to 2 (full support) [4]. Item scores were summed into a scale with higher scores indicating more disability. To evaluate the RV, we used the ‘Physical Functioning’ construct as represented by the ‘Activities of Moving around’ and ‘Self-care and Domestic Activities’ scales, the ‘Psychological Functioning’ construct as represented by the ‘Mental Functions and Pain’ scale, and the ‘Social Functioning’ construct as represented by the ‘Participation in Life Situations’ scale. These scales were selected because items in these scales are closely associated with the scales in the concurrent measures.

The SF-36 was selected as a well-known reliable and valid generic multidimensional health-impact measure used for NMD [12, 13]. The SF-36 [8] comprises 36 items with eight functional dimensions. Three scales were used to examine the RV: ‘Physical Functioning’, ‘Mental Health’ and ‘Social Functioning’. Item scores were coded, summed and transformed to a score of 0 (worst health) to 100 (best health) for each scale. The overall Cronbach’s alpha for these scales was 0.79 in a study of Amyotrophic Lateral Sclerosis patients [14]. In our previous study the Cronbach’s alpha for the selected scales ranged from 0.77 to 0.94 [4].

The WHOQOL-bref [9] was selected as a generic measurement instrument for a broad evaluation of quality of life. It consists of 28 items in four constructs and two separate questions. Three scales were used to examine the RV: ‘Physical Health and Autonomy’, ‘Psychological Health’, and ‘Social Relations’. Item scores from each scale were coded, summed and transformed to a score of 0 (worst health) to 20 (best health). The Cronbach’s alpha ranged from 0.63 to 0.81 in a study of Multiple Sclerosis patients [15]. In our previous study the Cronbach’s alpha for the selected scales ranged from 0.60 to 0.84 [4].

The GARS [10] is a domain specific generic measurement instrument for assessing disability in ‘Activities of daily living’ (ADL) and ‘Instrumental activities of daily living’ (IADL). It consists of eleven ADL items and seven IADL items. A four-category response format was used, and ranged from 1 (no problem in performing without help) to 4 (impossible to perform). The scores were summed for each subscale. The Cronbach’s alpha ranged from 0.95 to 0.97 in a study of Multiple Sclerosis patients [15]. In our previous study the Cronbach’s alpha ranged from 0.93 to 0.95 [4].

The IPAQ [11, 16] is a domain specific generic measurement instrument for assessing participation. It consists of fifteen items focusing on person-perceived participation and autonomy. The instrument assesses two aspects of participation: perceived participation and the perceived problems with participation. In this study the perceived participation aspect was used since this construct is closely associated with the ‘Participation in Life Situations’ construct in the NMDIP questionnaire. The sub-domains were ‘Autonomy Indoors’, ‘Family Role’, ‘Autonomy Outdoors’, and ‘Social Relations’. The response options ranged from 1 (very good) to 5 (very poor). Scores were summed for each domain. The Cronbach’s alpha ranged from 0.86 to 0.94 in a study of Multiple Sclerosis patients [15]. In our previous study the Cronbach’s alpha ranged from 0.84 to 0.94 [4].

Criterion variables

Two questions were selected as criterion variables: ‘Extent of limitations’ and ‘Quality of life’.

To evaluate the ‘Extent of Limitations’ respondents were asked to answer the question: ‘To what extent are you limited due to your NMD?’ Responses were on a ten-point scale ranging from 1 (not limited at all) to 10 (completely limited). Respondents were classified into one of four groups: Group A with a ‘very low extent of limitation’ (score 1–2), Group B with a ‘moderate extent of limitation’ (score 3–5), Group C with a ‘high extent of limitation’ (score 6–8) and, Group D with a ‘very high extent of limitation’ (score 9–10).

The second criterion variable for evaluation of quality of life was one of the two single items adapted from the WHOQOL-bref. Respondents were asked: ‘How would you rate your quality of life?’. Response options were: 1 = very poor, 2 = poor, 3 = neither poor nor good, 4 = good and 5 = very good. Respondents were classified into three groups: Group A– ‘very poor or poor quality of life’, Group B– ‘neither poor nor good’, and Group C– ‘good or very good quality of life’.

Analysis

Descriptive statistics were used to characterize the total sample and the test-retest sample. Differences between both samples were examined using the difference in proportions test, the two-sample t-test, and if data are not normally distributed a non-parametric test for independent samples were used.

Test-retest reliability or stability over time was examined using the Wilcoxon Signed Test and the one-way random intraclass correlation coefficients (ICCs) [17].

Relative Validity was examined in several steps. First, the Chi-square was computed for each scale by calculating the Kruskal-Wallis H-test. Second, the RV of each scale was computed by dividing each H-statistic by the H-statistic for the scale with the highest H-statistic, and multiplied by one hundred. The resulting RV-estimate indicates the extent to which a scale or construct is able to discriminate between two groups compared to the measure with the highest H-statistic [18, 19]. Finally, the clinical relevance of the differences between respondent subgroups, and the nonparametric effect size (coefficient r) for unrelated samples, was calculated for statistically significant group differences (α = 0.05) with post hoc tests (Bonferroni correction) [20]. Effect sizes where estimated through coefficient r, which was calculated by dividing the z-statistic (obtained from the Mann-Whitney U test) by the root of the sample size (n). To interpret this nonparametric effect sizes (coefficient r), Cohen suggested the following thresholds: an r of <0.10 indicates a trivial effect, an r of ≥0.10 to <0.24 a small effect, a r of ≥0.24 to <0.37 a moderate effect, and an r ≥ 0.37 a large effect. A r ≥ 0.10 reflects a clinically relevant difference between groups [20, 21].

IBM SPSS statistics version 22 was used.

Test-retest reliability

Criterion-related relative validity

Extent of limitations

About 16% (n = 110) of the respondents reported ‘low extent of limitations’ (Group A) due to NMD, while 36% (n = 250) reported a ‘moderate extent of limitation’ (Group B), and 39% (n = 270) reported a ‘high extent of limitation’ (Group C). About 8% (n = 58) of the respondents reported a ‘very high extent of limitations’ (Group D).

Comparisons of the RV coefficients, as summarized in Table 3, revealed that the NMDIP ‘Activities of Moving around’ scale and SF-36 ‘Physical Functioning’ scale were the most valid in discriminating between groups with an increasing extent of limitation.

We then examined the performance of the NMDIP-scales in indicating the differences between extreme groups (A-D) and subgroups (A-B, B-C, C-D) regarding the physical-, psychological- and social functioning constructs, as they relate to similar constructs in the concurrent measurement instruments. Regarding physical functioning, we found that both NMDIP activity scales turned out to be the most sensitive (followed by the ‘Muscle Functions’ scale) for measuring differences between extreme groups and subgroups. However, the performance of the concurrent SF-36 ‘Physical functioning’ scale and both GARS scales were almost identical. Regarding the psychological functioning construct we found that the NMDIP ‘Mental Functions and Pain’ scale was the best performing scale compared to the SF-36 ‘Mental Health’ scale and the WHOQOL-bref ‘Psychological Health’ scale, showing the highest extreme group and subgroup differences. Regarding the social functioning construct the NMDIP ‘Participation in Life Situations’ scale performed better than the SF-36 ‘Social Functioning’ and the WHOQOL-bref ‘Social Relations’ scales, and roughly as well as the same as the comparable constructs in the domain-specific IPAQ.

In summary, the NMDIP scales performed sufficient to good in discriminating between (sub) groups with an increasing extent of limitations compared to similar constructs in concurrent measures regarding physical functioning, psychological functioning and social functioning constructs.

Quality of life

Eight percent (n = 53) of the respondents reported poor or very poor quality of life (Group A), while 25% (n = 175) experienced their quality of life as neither poor nor good (Group B) and 67% (n = 474) reported a good or very good quality of life’ (Group C).

Comparisons of the RV-coefficients, as summarized in Table 4, revealed that the SF-36 ‘Psychological Health’ scale and IPAQ ‘Autonomy outdoors’ scales were the most valid in discriminating between groups with differences in quality of life. The ‘Mental Functions and Pain’ NMDIP scale was the third most valid scale.

When examining the performance of the NMDIP-scales in indicating the differences between extreme and subgroups for quality of life, we found about the same extreme group differences for the physical functioning scales for all concurrent constructs with moderate Effect Sizes (ESs). The same goes for the subgroup differences, although the NMDIP ‘Mental Functions and Pain’ scale, and the WHOQOL-bref ‘Psychological Health’ scale performed slightly better than the SF-36 ‘Mental Health’ scale. Finally, when examining the social functioning scales we found that the comparable NMDIP ‘Participation in Life Situations’ scale performed about as well as the SF-36 ‘Social Functioning’ scale and the IPAQ scales with a moderate to large ESs for extreme group differences. The NMDIP ‘Participation in Life Situations’ scale also performed better compared to the social functioning construct of the WHOQOL-bref, the ‘Social Relations’ scale. The same goes for the subgroup differences.

In summary, the NMDIP scales performed well in discriminating between subgroups with differences in quality of life compared to similar constructs in concurrent measures concerning the physical functioning, psychological functioning and social functioning constructs.