Skip to main content

Don’t be late! Timely identification of cognitive impairment in people with multiple sclerosis: a study protocol

A Study Protocol to this article was published on 15 January 2024

Abstract

Background

Cognitive impairment occurs in up to 65% of people with multiple sclerosis (PwMS), negatively affecting daily functioning and health-related quality of life. In general, neuropsychological testing is not part of standard MS-care due to insufficient time and trained personnel. Consequently, a baseline assessment of cognitive functioning is often lacking, hampering early identification of cognitive decline and change within a person over time. To assess cognitive functioning in PwMS in a time-efficient manner, a BICAMS-based self-explanatory digital screening tool called the Multiple Screener©, has recently been developed. The aim of the current study is to validate the Multiple Screener© in a representative sample of PwMS in the Netherlands. Additionally, we aim to investigate how cognitive functioning is related to psychological factors, and both work and societal participation.

Methods

In this cross-sectional multicentre study, 750 PwMS (aged 18–67 years) are included. To obtain a representative sample, PwMS are recruited via 12 hospitals across the Netherlands. They undergo assessment with the Minimal Assessment of Cognitive Functioning in MS (MACFIMS; reference-standard) and the Multiple Screener©. Sensitivity, specificity, and predictive values for identifying (mild) cognitive impairment are determined in a subset of 300 participants. In a second step, the identified cut-off values are tested in an independent subset of at least 150 PwMS. Moreover, test–retest reliability for the Multiple Screener© is determined in 30 PwMS. Information on psychological and work-related factors is assessed with questionnaires.

Discussion

Validating the Multiple Screener© in PwMS and investigating cognition and its determinants will further facilitate early identification and adequate monitoring of cognitive decline in PwMS.

Background

Multiple sclerosis (MS) is a chronic inflammatory disease of the central nervous system characterized by demyelination and neurodegeneration [1]. In addition to physical limitations, 43–65% of people with MS (PwMS) develop cognitive symptoms that may severely affect daily life functioning and consequently health-related quality of life [2,3,4]. The most commonly and earliest affected cognitive domains are information processing speed, verbal memory, and visuospatial memory [3, 5].

The impact of cognitive impairment on daily life functioning can be significant, especially since most PwMS are relatively young at disease onset [3]. As such, cognitive impairment is one of the main reasons for unemployment in MS [6, 7]. About 43% of PwMS become unemployed within three years of diagnosis due to fatigue and physical impairment, but also cognitive impairment [8,9,10]. This early unemployment has a large impact on PwMS, their families, and on society in general [11]. However, by the time PwMS with self-perceived cognitive problems approach health care professionals, their cognitive deficits are often already advanced and potentially more difficult to treat, suggesting that early intervention might be promising [12, 13].

The need for early intervention is emphasized by a recent study showing that successful response to cognitive rehabilitation depends on the status of the brain’s functional network before the intervention [14]. PwMS with a functional connectivity that is more like that of healthy controls were able to benefit from a cognitive rehabilitation program (i.e., these participants significantly improved on neuropsychological tests and had better self-perceived cognitive functioning). However, those PwMS with a less efficient brain network at baseline (indicative of more MS-related pathology) were non-responsive, suggesting the existence of a small window of opportunity for intervention early in the development of the disease [14]. Additionally, other studies show that less MS-related brain damage (e.g., higher grey matter volume) are linked to better cognitive rehabilitation outcomes [15,16,17]. Therefore, it is crucial to identify PwMS at the earliest stages of cognitive impairment to allow for intervention when it is most effective to improve cognitive functioning.

Recent international recommendations for measuring and monitoring cognitive functioning in PwMS propose a baseline cognitive screening and annual follow-up [18]. However, in contrast to these recommendations, neuropsychological testing is not part of standard MS-care in current clinical practice in the Netherlands (and several other countries) [19]. Consequently, a good reference assessment of baseline cognitive performance is often lacking, hampering the detection of the first subtle changes in cognition. Detecting these early changes is particularly difficult when PwMS already experience difficulties in daily life functioning but still perform (above) average on neuropsychological assessments [3]. The main reason for not following the international recommendations is the lack of time and specialized personnel to assess cognitive functioning [19]. As digital assessment tools may lower the threshold for systematic assessment of cognitive functioning in PwMS, we recently developed a self-explanatory, time-efficient digital screening tool, the Multiple Screener© [20].

The Multiple Screener© consists of an adjusted version of the validated and recommended BICAMS (Brief International Cognitive Assessment for MS) paper-and-pencil assessment [21] and takes 15 min to complete. It assesses the most frequently impaired cognitive domains in MS: information processing speed via the Symbol Digit Modalities Test (SDMT [22]), verbal learning and memory via the Dutch version of California Verbal Learning Test Second Edition (CVLT-II [23,24,25]), and visuospatial learning and memory via the Spatial Recall Test (SPART [26]) [20]. In addition, the Multiple Screener© also includes questionnaires on depression and anxiety [27], fatigue [28] and self-perceived cognitive symptoms [29], taking into account psychological factors when screening for cognitive deficits in MS. The main advantages of the Multiple Screener© are that it does not require specialized personnel for administration (i.e., PwMS can perform the tests on their own), has automated scoring, and is time-efficient. The Multiple Screener© has been tested in 236 healthy controls and normative data are available [20]. In healthy controls, the correlations between the Multiple Screener© and the paper-and-pencil versions of the neuropsychological tests have been shown to be good to excellent [20]. However, a next essential step before the Multiple Screener© can be used in clinical practice is to investigate its diagnostic accuracy especially in identifying PwMS with mild cognitive impairment according to a reference standard (the Minimal Assessment of Cognitive Function in Multiple Sclerosis, MACFIMS [30]), allowing for timely identification.

Objectives

As part of a larger research project (i.e., the Don’t be late! study, see Table 1) the primary objective of this study is to determine diagnostic accuracy of the Multiple Screener© in a representative Dutch sample of PwMS. Specifically, we aim to determine how well the Multiple Screener© can differentiate between PwMS with no cognitive impairment, mild cognitive impairment, and cognitive impairment according to the reference-standard (MACFIMS [30]). In a second step we aim to confirm the observed diagnostic accuracy of the Multiple Screener© in differentiating between PwMS with no cognitive impairment, mild cognitive impairment, and cognitive impairment in an independent subset of PwMS. When reporting on the diagnostic accuracy of the Multiple Screener©, the Standards for Reporting Diagnostic Accuracy guidelines from the Equator-Network (STARD 15, [31]) will be followed.

Table 1 Don’t be late study!

The study has the following secondary objectives:

  1. 1)

    To determine the test–retest reliability of the Multiple Screener©;

  2. 2)

    To determine how cognitive, psychological, work-related, and health-related quality of life outcomes are related.

Methods

Design and setting

The present study is a cross-sectional multicentre study in which a representative sample of 750 PwMS will be included. In the 12 participating Dutch hospitals, demographical and medical information will be collected, and cognitive functioning of PwMS will be assessed with both the reference standard (MACFIMS) and the Multiple Screener©. In line with international validation guidelines [32], the assessment of the Multiple Screener© will be repeated within 3 weeks after the hospital visit in a small subset of participants (N = 30) in order to determine test–retest reliability. Finally, all participants will fill in several online questionnaires at home.

Participants

Recruitment and consent

We aim to recruit a representative sample of PwMS in the Netherlands that visit the neurologist in light of their MS. We will include PwMS with a variety in MS types (relapsing remitting, secondary progressive (85% of the population) and primary progressive (15%)), disease duration and age. All participating hospitals are asked to provide a patient information letter for a set period of time to all PwMS that visit the outpatient clinic, independent of cognitive status, employment status, disease status and meeting the in- and exclusion criteria. Contact details of PwMS that give permission to be approached about participation are shared with the researchers from the Amsterdam UMC, Vrije Universiteit Amsterdam. After at least one week, they contact the potential participant to provide additional information if requested (or refer to an independent physician) and to ask whether they would like to participate in the study. When PwMS decide to participate, the researcher will screen the subjects for eligibility via telephone (see below for inclusion and exclusion criteria) such that an unnecessary hospital visit will be avoided when a subject is not eligible. In case of a positive screening outcome, a visit for the assessment will be scheduled at which written informed consent will be obtained.

Inclusion criteria

To be eligible to participate in this study, people must fulfil the following criteria: a confirmed MS diagnosis according to the McDonald 2017 criteria [33], age between 18 and 67 years, no changes in disease modifying therapy within the last 3 months, and no relapse or steroid treatment six weeks prior to the study visit.

Exclusion criteria

Participants will be excluded from participation in this study if they have other neurological or psychiatric comorbidities that can potentially influence cognitive functioning, a current or history of drug or alcohol abuse, have insufficient vison or hearing, or are unable to speak or read Dutch. The reasons for excluding participants from the current study will be documented.

Ethical approval

The study will be conducted according to the principles of the Declaration of Helsinki (2013) and in accordance with the Dutch Medical Research Involving Human Subjects Act (WMO). The Medical Ethical Committee (METC) of the Amsterdam UMC, Vrije Universiteit Amsterdam has approved this study (METC 2021.0707) on 4 May 2022.

Measures and procedures

Demographic and clinical characteristics

During the assessment at the participating hospitals, information on demographical and clinical characteristics will be collected from participants and their medical file. The following characteristics will be collected: age in years, sex, educational level (Dutch Verhage scale), work status, date of diagnosis, MS subtype, MS severity assessed with the telephone version of the Expanded Disability Status Scale (EDSS) [34], medication usage, medical history, and comorbidities.

Neuropsychological assessment

Participants will undergo an extensive neuropsychological assessment (120–150 min) at the location of the participating site. The assessment consists of the Multiple Screener©, the MACFIMS test battery [30], a social cognition test, performance validity tests and an assessment of awareness of cognitive functioning. Parallel versions will be used for the tests that are overlapping between the MACFIMS and Multiple Screener© and the order of administration will be counterbalanced to minimize learning effects and influence of fatigue.

Multiple Screener©

The Multiple Screener© is a digital tool aiming to assess cognitive functioning in PwMS. It is a digital, self-explanatory version of the validated and recommended BICAMS [21] and takes 15 min to complete. It includes the following three tests:

  • Digital version of the CVLT-II [23,24,25]: Verbal learning and memory. The ability to learn 16 auditory presented semantically related words is examined over five trials. After each trial participants are asked to type the remembered words (direct recall). The total number of the correctly remembered words is calculated.

  • Digital version of the SDMT [22]: Processing speed and working memory. Nine pairs of digits and symbols are visually presented. Participants are asked to type the numbers associated with the paired symbols as fast as possible. The total number of correct answers within 90 s is calculated.

  • Digital version of the SPART [26]: Visuospatial memory. A 6 × 6 grid with 10 black checkers is displayed three times for ten seconds. After each time, an empty grid is displayed with ten black checkers next to it. Participants must swipe the black checkers to the correct places in the empty grid to match what they observed. The total number of correctly placed checkers is calculated. See Fig. 1 for an illustration of the SDMT and the SPART.

Fig. 1
figure 1

The digital version of the Digit Modalities Test (SDMT) (left) and the Spatial Recall Test (SPART) (right) in the Multiple Screener© application. The Dutch version of the California Verbal Learning Test–second edition (CVLT-II) is not depicted as this test has an auditory format

The software of the Multiple Screener© is produced by the manufacturer Sherpa B.V. In accordance with the legislation of the Medical Device Directive, the software is qualified as a medical device, classified in risk class I (low risk), reported to FARMATEC-CIBG-VWS, and CE-certified by the manufacturer.

A subset of participants (n = 30) will be invited to return to the hospital within 3 weeks after the initial assessment to complete The Multiple Screener© for a second time to determine the test–retest reliability.

MACFIMS

The MACFIMS is an internationally renowned and well-validated, 90-min, paper-and-pencil test battery that is commonly used to determine cognitive impairment in MS. It consists of tests for verbal and visuospatial learning and memory and information processing speed (cf. the Multiple Screener©) and in addition tests for language and working memory, visuospatial orientation, and executive functioning [30]. The tests and corresponding cognitive domain(s) are summarized in Table 2.

Table 2 Minimal assessment of cognitive functioning in MS test battery

Performance validity

The Amsterdam Short Term Memory Test (ASTM) [39] will be used to assess performance validity in all participants. In case the ASTM indicates underperformance (cut-off of ≤ 84; [40]) the Rey 15-item Test (higher specificity compared to the ASTM) [41] will additionally be performed.

Social cognition

Social cognition and in particular affective theory of mind (i.e., the ability to recognise the thought or feelings of others) will be measured with the revised version of the Reading the Mind in the Eyes Test [42].

Awareness of cognitive functioning

Finally, to assess (online) awareness of global cognitive functioning, a subset of the participants (N = 200) will be asked to estimate their own performance immediately before and after completion of the MACFIMS battery. More specifically, they will be asked to estimate what percentile score they believe that they would receive for the overall test battery if compared with a randomly selected demographically matched peer group. A normal distribution including brief explanations of percentiles scores (inspired by Rothlind et al. [43]) will serve as a visual aid for participants.

Questionnaires

To reduce the burden on the day in the hospital, participants will fill out several online questionnaires at home (for an overview of the questionnaires see Table 3). Participants will be asked to complete the questionnaires within one week after the hospital visit to ensure that the collected data most closely resembles the status of the participant during the hospital visit. The researcher will send reminders if the questionnaires have not been returned.

Table 3 Questionnaires on MS-related and psychological factors, work and societal participation

Outcomes

Primary outcome

The primary outcome measures for the first study objective are sensitivity, specificity, negative and positive predictive value, and the receiver-operating characteristic of the Multiple Screener©.

Secondary outcomes

A secondary outcome measure is the test–retest reliability (i.e., intraclass correlation coefficients) of the Multiple Screener©. Additionally, secondary outcome measures include the relationships between cognitive functioning (as measured with the Multiple Screener© and the MACFIMS test battery [30]) with the following measures (the hypothesized directions of these relationships are summarized in Table 4):

  1. 1.

    Psychological measures: self-perceived cognitive functioning [29], awareness of cognitive functioning [43], physical and psychological impact of MS [45], mood [27], fatigue [28], personality traits [50, 52], stressful life events [53], resilience [48] and mastery [49]

  2. 2.

    Patient-reported health-related quality of life [44]

  3. 3.

    Work-related measures: MS-related work difficulties [55], work productivity and activity impairment [54], negative work events and work accommodations [57]

  4. 4.

    Health and lifestyle measures: Physical exercise, smoking, alcohol, diet, sleep, BMI, household composition, and social activities.

Table 4 Hypothesized direction of correlations between cognitive scores with health-related quality of life, psychological, and work-related, health and lifestyle measures

Power calculation

Because the Multiple Screener© is aimed at assessing cognitive decline, especially sensitivity to detect (mild) cognitive impairment should be high, while a relatively lower degree of specificity can be tolerated. Based on accuracy values from the paper–pencil version of the Multiple Screener© (BICAMS) in MS and comparable cognitive screening instruments frequently used in people with Parkinson’s disease, we aim for sensitivity values of at least 0.80 and specificity values of at least 0.70 [58, 59]. As reported by Amato et al. [60] we expect that approximately 50% of PwMS at the outpatient clinics will classify as having no cognitive impairment (i.e., cognitively preserved; CP), 30% will classify as having mild cognitive impairment (MCI), and 20% will classify as having overt cognitive impairment (CI). Based on these prevalence estimations, a minimum sample size of 100 will be required to detect sensitivity values of at least 0.80 with a power of 80% and alpha threshold of 0.05 [61]. In total 198 participants will be required to detect specificity values of at least 0.70 with identical power and significance level. However, a sample size of 300 is recommended to reliably evaluate accuracy values of screening tools [61]. As such, for our primary objective we aim to include 300 PwMS. Moreover, we aim to confirm the accuracy of the Multiple Screener© in an independent sample of at least 150 PwMS (i.e., another subset of our sample).

This study is part of a larger research project and a subset of participants from the current study (i.e., participants with mild cognitive impairment) will be selected for the intervention study of the second work package (see Table 1). Therefore, the overall required sample size (N = 750) is based on the power calculation for the intervention study. For additional information, the reader is referred to Aarts et al. [62].

Statistical analysis

Data will be analysed using R Studio software (at least version 4.2.1; [63]) and IBM SPSS Statistics (at least version 28 [64]). In case of non-normality, data will be presented as median and inter-quartile range and transformed for further analyses if appropriate or non-parametric tests will be applied. Participants with missing data and outliers will be excluded for that particular analysis. A p-value of 0.05 will be considered as statistically significant for all analyses.

Primary study parameters

For our primary objective we will determine sensitivity, specificity, positive and negative predictive values of the Multiple Screener© as compared to the MACFIMS. Based on previous definitions for cognitive impairment among PwMS [65], participants will be divided into three subgroups depending on their severity of cognitive impairment. Participants scoring at least 2 standard deviations (SDs) below the mean normative values on at least 2 out of 6 cognitive domains assessed with MACFIMS will be classified as having CI. Participants who score 1 to 1.99 SDs below the mean normative values on at least 1 cognitive domain and/or at least 2 SDs below the mean normative values on 1 cognitive domain (not fulfilling the CI criteria) will be classified as having MCI. The remaining participants will be defined as CP [65]. For the Multiple Screener, participants scoring at least 2 SDs below the mean normative values on at least 1 of the 3 tests will be classified as CI. Participants scoring 1 to 1.99 SDs below the mean normative values on at least 1 of the 3 tests will be classified as MCI. The remaining participants will be defined as CP. Overall, regression-based norms adjusted for age, sex, and education will be used for individual cognitive tests before determining cognitive status.

Participants' cognitive status will be determined via the MACFIMS and will be investigated in relation to the scores detected with Multiple Screener©. All accuracy values will be calculated separately for the detection of MCI and CI (one against all approach for multiclass classification) and will be presented as percentages. Additionally, receiver-operating characteristic (ROC) analyses will be performed to determine overall accuracy and optimal cut-off scores of the Multiple Screener© for detecting MCI and CI in people with MS. Once we have determined accuracy values and optimal cut-off scores in 300 participants, we will test these in another subset of at least 150 participants to confirm their correctness.

The Multiple Screener© will be considered a sufficiently adequate screening instrument for the detection of (M)CI if its overall sensitivity values are at least 0.80 and specificity values at least 0.70. However, if one of the individual tests does not meet these criteria, we will determine accuracy values of the two other tests over and above that of all three tests combined.

Secondary study parameters

Test–retest reliability

Test–retest reliability of the Multiple Screener© will be determined by calculating intraclass correlation coefficients (ICCs) for absolute agreement, using a two-way mixed model. Based on the 95% confidence interval of the ICC estimate, values will be considered to reflect poor reliability (< 0.5), moderate (0.5 -0.75), good (0.75–0.9), and excellent (> 0.90) [66]. The coefficients will be calculated separately for the SDMT, CVLT-II, and the SPART.

Relationships between cognition and psychological, work-related, and patient-reported health-related quality of life measures

Cross-sectional associations between cognition and psychological, work-related, and health-related quality of life measures will be analysed using Pearson’s or Point-Biserial correlations and linear regression analyses (including stepwise procedures) in both subsets and the overall sample. Correlations coefficients of less than 0.3, between 0.3 and 0.7, and greater than 0.7 will be considered weak, moderate, and strong, respectively [67]. An overview of the hypothesized correlations can be found in Table 4.

Additionally, logistic regression analyses will be used to identify the predictive value of demographical and disease characteristics (such as sex, MS subtype, medication, comorbidities) on cognitive functioning. Additionally, differences between groups (CP, MCI and CI) in demographic and clinical characteristics and other outcome measurements (e.g., psychological, work-related and health-related quality of life measures) will be analysed using independent samples t-tests, Mann–Whitney U tests and Pearson’s chi-square tests. For particular analyses, confounding variables (such as age, sex, education, EDSS score, disease duration, mood, fatigue etc.) will be inserted. Bonferroni corrections will be applied to correct for multiple comparisons within each objective.

Safety reporting

We will not collect information on (serious) adverse events due to the observational and non-interventional nature of this study.

Study status

The first participant was included on 19 July 2022. Currently 216 participants have been enrolled in the study (December 2023).

Discussion

Cognitive impairment is common in PwMS and can severely affect health-related quality of life. In order to intervene timely, a baseline assessment and frequent monitoring of cognitive functioning seems crucial. However, in the Netherlands, neuropsychological assessment is not (yet) integrated into standard care due to the time-consuming nature of cognitive testing and limited availability of trained personnel [19]. The current study will validate a digital screening tool with the primary objective to enable early identification of cognitive decline in PwMS. The validation of the Multiple Screener© within a representative sample of PwMS that visit a neurologist, will lay the foundation for implementing a cognitive screening tool for annual testing in clinical practice in the near future. This study will further help raise awareness among health care professionals about cognitive impairment in MS and its significance within the broader scheme of priorities in MS-care. The fact that 12 hospitals in the Netherlands are interested in participating in the study further emphasizes the need for such screening methods. In addition, this study will also contribute to the development of practical guidelines for Dutch professionals regarding the screening and subsequent monitoring of cognitive decline in MS.

Moreover, with the present study we are collecting one of the largest datasets on cognition and its determinants in PwMS which will provide us with a wealth of data that can be used to answer multiple relevant related research questions. Specifically, it will enhance our understanding of the relationship between cognition and relevant confounders, ranging from cognitive self-awareness to fatigue and mood problems.

To conclude, the validation of the Multiple Screener© will facilitate early identification of cognitive impairment in PwMS; ultimately enabling better management of cognitive symptoms in this population. Additionally, the study's comprehensive dataset will allow new insights into factors related to cognition in PwMS, thus informing future research and clinical practices. Finally, timely identification of cognitive impairment is a crucial step for initiating early interventions, an important aspect that will be explored in subsequent phases of the larger Don't be late! study.

Availability of data and materials

Not applicable.

Abbreviations

AIS:

Athens Insomnia Scale

ASMT:

Amsterdam Short Term Memory Test

BICAMS:

Brief International Cognitive Assessment for MS

BVMS NL version:

Buffalo Vocational Monitoring Survey

BVMT-R:

Brief visuospatial memory test-revised

CD-RISC 25:

Connor Davidson Resilience Scale

CI:

Cognitively impaired

COWAT:

Controlled oral word association test

CP:

Cognitively preserved

CVLT:

California verbal learning test

D-KEFS:

Delis-Kaplan Executive Function System sorting test

EDSS:

Expanded disability status scale

HADS:

Hospital anxiety and depression scale

ICCs:

Intraclass correlation coefficients

JLO:

Judgment of line orientation test

LTE:

List of Threatening Events Questionnaire

MACFIMS:

Minimal Assessment of Cognitive Function in Multiple Sclerosis

MCI:

Mild cognitive impairment

METC:

Medical Ethics Committee

MFIS:

Modified Fatigue Impact Scale

MS:

Multiple Sclerosis

MSIS-29:

Multiple Sclerosis Impact Scale

MSNQ:

Multiple Sclerosis Neuropsychological Questionnaire

MSWDQ-23:

Multiple Sclerosis Work Difficulties Questionnaire

NEO-FFI:

NEO Five-Factor Inventory

PMS:

Pearlin Mastery Scale

PwMS:

People with Multiple Sclerosis

ROC:

Receiver-operating characteristic

SDMT:

Symbol digit modalities test

SF-36:

36-Item Short Form

SPART:

Spatial Recall Test

WMO:

Medical Research Involving Human Subjects Act

WPAI:

Work Productivity and Activity Impairment Questionnaire

References

  1. Katz Sand I. Classification, diagnosis, and differential diagnosis of multiple sclerosis. Curr Opin Neurol. 2015;28(3):193–205.

    Article  PubMed  Google Scholar 

  2. Campbell J, Rashid W, Cercignani M, Langdon D. Cognitive impairment among patients with multiple sclerosis: associations with employment and quality of life. Postgrad Med J. 2017;93(1097):143–7.

    Article  CAS  PubMed  Google Scholar 

  3. Sumowski JF, et al. Cognition in multiple sclerosis: State of the field and priorities for the future. Neurology. 2018;90(6):278–88.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Eijlers AJC, et al. Predicting cognitive decline in multiple sclerosis: a 5-year follow-up study. Brain. 2018;141(9):2605–18.

    PubMed  Google Scholar 

  5. Wojcik C, et al. Staging and stratifying cognitive dysfunction in multiple sclerosis. Mult Scler. 2022;28(3):463–71.

    Article  PubMed  Google Scholar 

  6. Benedict RH, et al. Negative work events and accommodations in employed multiple sclerosis patients. Mult Scler. 2014;20(1):116–9.

    Article  PubMed  Google Scholar 

  7. Kordovski VM, et al. Identifying employed multiple sclerosis patients at-risk for job loss: when do negative work events pose a threat? Mult Scler Relat Disord. 2015;4(5):409–13.

    Article  PubMed  Google Scholar 

  8. van der Hiele K, et al. Work participation and executive abilities in patients with relapsing-remitting multiple sclerosis. PLoS ONE. 2015;10(6):e0129228.

    Article  PubMed  PubMed Central  Google Scholar 

  9. van Gorp DAM, et al. Cognitive functioning as a predictor of employment status in relapsing-remitting multiple sclerosis: a 2-year longitudinal study. Neurol Sci. 2019;40(12):2555–64.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Global MS Employment Report 2016. 2016, MS International Federation: msif.org.

  11. Uitdehaag B, et al. New insights into the burden and costs of multiple sclerosis in Europe: results for the Netherlands. Multiple Sclerosis J. 2017;23(2_suppl):117–29.

    Article  Google Scholar 

  12. Meide HVD, Gorp DV, Van Der Hiele K, Visser L. “Always looking for a new balance”: toward an understanding of what it takes to continue working while being diagnosed with relapsing-remitting multiple sclerosis. Disabil Rehabil. 2018;40(21):2545–52.

    Article  PubMed  Google Scholar 

  13. Pope C, Ziebland S, Mays N. Qualitative research in health care. Analysing qualitative data. BMJ. 2000;320(7227):114–6.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Prouskas SE, et al. A randomized trial predicting response to cognitive rehabilitation in multiple sclerosis: Is there a window of opportunity? Mult Scler. 2022:13524585221103134.

  15. Fuchs TA, et al. Response heterogeneity to home-based restorative cognitive rehabilitation in multiple sclerosis: an exploratory study. Mult Scler Relat Disord. 2019;34:103–11.

    Article  PubMed  Google Scholar 

  16. Fuchs TA, et al. Functional connectivity and structural disruption in the default-mode network predicts cognitive rehabilitation outcomes in multiple sclerosis. J Neuroimaging. 2020;30(4):523–30.

    Article  PubMed  Google Scholar 

  17. Taylor LA, et al. Understanding who benefits most from cognitive rehabilitation for multiple sclerosis: a secondary data analysis. Mult Scler. 2023;29(11–12):1482–92.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Kalb R, et al. Recommendations for cognitive screening and management in multiple sclerosis care. Mult Scler. 2018;24(13):1665–80.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Klein OA, das Nair R, Ablewhite J, Drummond A. Assessment and management of cognitive problems in people with multiple sclerosis: a National Survey of Clinical Practice. Int J Clin Pract. 2018:e13300.

  20. van Dongen L, et al. Introducing multiple screener: an unsupervised digital screening tool for cognitive deficits in MS. Mult Scler Relat Disord. 2019;38:101479.

    Article  PubMed  Google Scholar 

  21. Langdon DW, et al. Recommendations for a Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Mult Scler. 2012;18(6):891–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Smith A. Symbol digit modalities test. 1973: Western Psychological Services Los Angeles.

  23. Delis D, et al. Neuropsychological assessment of learning and memory. Cortex. 1989;10(3):308–17.

    Google Scholar 

  24. Woods SP, et al. The California Verbal Learning Test–second edition: Test-retest reliability, practice effects, and reliable change indices for the standard and alternate forms. Arch Clin Neuropsychol. 2006;21(5):413–20.

    Article  PubMed  Google Scholar 

  25. Mulder JL, Dekker R, Dekker PH. Verbale Leer en Geheugen Test. Lisse: Swets & Zeitlinger; 1996.

    Google Scholar 

  26. Rao SM. Cognitive Function Study Group, N. A Manual for the Brief Repeatable Battery of Neuropsychological Tests in Multiple Sclerosis., N.M.S. Society, Editor. 1990: New York.

  27. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67(6):361–70.

    Article  CAS  PubMed  Google Scholar 

  28. Kos D, et al. Assessing fatigue in multiple sclerosis: Dutch modified fatigue impact scale. Acta Neurol Belg. 2003;103(4):185–91.

    PubMed  Google Scholar 

  29. Benedict RH, et al. Screening for multiple sclerosis cognitive impairment using a self-administered 15-item questionnaire. Mult Scler. 2003;9(1):95–101.

    Article  CAS  PubMed  Google Scholar 

  30. Benedict RH, et al. Validity of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). J Int Neuropsychol Soc. 2006;12(4):549–58.

    Article  PubMed  Google Scholar 

  31. Bossuyt PM, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351:h5527.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Benedict RH, et al. Brief International Cognitive Assessment for MS (BICAMS): international standards for validation. BMC Neurol. 2012;12:55.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Thompson AJ, et al. Diagnosis of multiple sclerosis: 2017 revisions of the McDonald criteria. Lancet Neurol. 2018;17(2):162–73.

    Article  PubMed  Google Scholar 

  34. Kurtzke JF. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology. 1983;33(11):1444–52.

    Article  CAS  PubMed  Google Scholar 

  35. Benedict RH, et al. Revision of the Brief visuospatial memory test: studies of normal performance, reliability, and validity. Psychol Assess. 1996;8(2):145.

    Article  Google Scholar 

  36. Gronwall D. Paced auditory serial-addition task: a measure of recovery from concussion. Percept Mot Skills. 1977;44(2):367–73.

    Article  CAS  PubMed  Google Scholar 

  37. Benton AL, Sivan AB, Hamsher K, Varney NR, Spreen O. Contributions to Neuropsychological Assessment. 2nd ed. New York: Oxford University Press; 1994.

    Google Scholar 

  38. Delis DC, Kaplan E, Kramer JH. Delis-Kaplan executive function system. 2001.

    Google Scholar 

  39. Schagen S, Schmand B, de Sterke S, Lindeboom J. Amsterdam Short-Term Memory test: a new procedure for the detection of feigned memory deficits. J Clin Exp Neuropsychol. 1997;19(1):43–51.

    Article  CAS  PubMed  Google Scholar 

  40. Nauta IM et al. Performance validity in outpatients with multiple sclerosis and cognitive complaints. Mult Scler. 2021:13524585211025780.

  41. Lee GP, Loring DW, Martin RC. Rey’s 15-item visual memory test for the detection of malingering: normative observations on patients with neurological disorders. Psychol Assess. 1992;4(1):43–6.

  42. Baron-Cohen S, et al. The “Reading the Mind in the Eyes” Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. J Child Psychol Psychiatry Allied Disciplines. 2001;42(2):241–51.

    Article  CAS  Google Scholar 

  43. Rothlind J, Dukarm P, Kraybill M. Assessment of self-awareness of cognitive function: correlations of self-ratings with actual performance ranks for tests of processing speed, memory and executive function in non-clinical samples. Arch Clin Neuropsychol. 2017;32(3):316–27.

    PubMed  Google Scholar 

  44. Patel AA, Donegan D, Albert T. The 36-item short form. J Am Acad Orthop Surg. 2007;15(2):126–34.

    Article  PubMed  Google Scholar 

  45. Hobart J, et al. The multiple sclerosis impact scale (MSIS-29) a new patient-based outcome measure. Brain. 2001;124(5):962–73.

    Article  CAS  PubMed  Google Scholar 

  46. Soldatos CR, Dikeos DG, Paparrigopoulos TJ. Athens Insomnia Scale: validation of an instrument based on ICD-10 criteria. J Psychosom Res. 2000;48(6):555–60.

    Article  CAS  PubMed  Google Scholar 

  47. Soldatos CR, Dikeos DG, Paparrigopoulos TJ. The diagnostic validity of the Athens Insomnia Scale. J Psychosom Res. 2003;55(3):263–7.

    Article  PubMed  Google Scholar 

  48. Connor KM. Assessment of resilience in the aftermath of trauma. J Clin Psychiatry. 2006;67(Suppl 2):46–9.

    PubMed  Google Scholar 

  49. Pearlin LI, Schooler C. The structure of coping. J Health Soc Behav. 1978:2–21.

  50. Costa Jr PT, McCrae RR. The Revised Neo Personality Inventory (neo-pi-r). 2008: Sage Publications, Inc.

  51. Schwartz ES, et al. The NEO-FFI in multiple sclerosis: internal consistency, factorial validity, and correspondence between self and informant reports. Assessment. 2011;18(1):39–49.

    Article  PubMed  Google Scholar 

  52. Donnellan MB, Oswald FL, Baird BM, Lucas RE. The mini-IPIP scales: tiny-yet-effective measures of the Big Five factors of personality. Psychol Assess. 2006;18(2):192.

    Article  PubMed  Google Scholar 

  53. Brugha TS, Cragg D. The list of threatening experiences: the reliability and validity of a brief life events questionnaire. Acta Psychiatr Scand. 1990;82(1):77–81.

    Article  CAS  PubMed  Google Scholar 

  54. Reilly MC, Zbrozek AS, Dukes EM. The validity and reproducibility of a work productivity and activity impairment instrument. Pharmacoeconomics. 1993;4(5):353–65.

    Article  CAS  PubMed  Google Scholar 

  55. Honan CA, Brown RF, Hine DW. The multiple sclerosis work difficulties questionnaire (MSWDQ): development of a shortened scale. Disabil Rehabil. 2014;36(8):635–41.

    Article  PubMed  Google Scholar 

  56. van Egmond E et al. A Dutch validation study of the Multiple Sclerosis Work Difficulties Questionnaire in relapsing remitting multiple sclerosis. Disabil Rehabil. 2019:1–10.

  57. Benedict RH, et al. Negative work events and accommodations in employed multiple sclerosis patients. Mult Scler J. 2014;20(1):116–9.

    Article  Google Scholar 

  58. Foley T, McKinlay A, Warren N, Stolwyk RJ. Assessing the sensitivity and specificity of cognitive screening measures for people with Parkinson’s disease. NeuroRehabilitation. 2018;43(4):491–500.

  59. Dusankova JB, Kalincik T, Havrdova E, Benedict RH. Cross cultural validation of the Minimal Assessment of Cognitive Function in Multiple Sclerosis (MACFIMS) and the Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Clin Neuropsychol. 2012;26(7):1186–200.

    Article  PubMed  Google Scholar 

  60. Amato MP, Ponziani G, Siracusa G, Sorbi S. Cognitive dysfunction in early-onset multiple sclerosis: a reappraisal after 10 years. Arch Neurol. 2001;58(10):1602–6.

    Article  CAS  PubMed  Google Scholar 

  61. Bujang M, Adnan TH. Requirements for minimum sample size for sensitivity and specificity analysis. J Clin Diagn Res. 2016;10(10):Ye01–ye06.

    PubMed  PubMed Central  Google Scholar 

  62. Aarts J, Saddal SRD, Bosmans JE, de Groot V, de Jong BA, et al. Don’t be late! Postponing cognitive decline and preventing early unemployment in people with multiple sclerosis: a study protocol. BMC Neurology. 2024. https://doi.org/10.1186/s12883-023-03513-y.

  63. RStudio. TeamRStudio: Integrated Development for R. RStudio. Boston: PBC; 2020.

    Google Scholar 

  64. Corp I. IBM SPSS Statistics for Windows, Version 28.0. Armonk: IBM Corp; 2021.

    Google Scholar 

  65. Eijlers AJ, et al. Increased default-mode network centrality in cognitively impaired multiple sclerosis patients. Neurology. 2017;88(10):952–60.

    Article  PubMed  Google Scholar 

  66. Koo TK, Li MY. A Guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–63.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Akoglu H. User’s guide to correlation coefficients. Turk J Emerg Med. 2018;18(3):91–3.

Download references

Acknowledgements

Members of the Don’t be late! consortium are:

• Participating sites

 – Casper E.P. van Munster, Amphia Ziekenhuis, Breda, The Netherlands.

 – Renske G. Wieberdink, MS Centrum Stedendriehoek, Gelre, The Netherlands.

 – Jolijn J. Kragt, Reinier de Graaf Ziekenhuis, Delft, The Netherlands.

 – Judith Schouten, Rijnstate, Arnhem, The Netherlands.

 – Erwin L.J. Hoogervorst, St. Antonius Ziekenhuis, Nieuwegein, The Netherlands.

 – Paul A.D. Bouma, Tergooi Ziekenhuizen, Hilversum, The Netherlands.

 – Floris G.C.M. De Kleermaeker, Viecuri Medisch Centrum, Venlo, The Netherlands.

 – Meike Holleman, Jeroen Bosch Ziekenhuis, ’s-Hertogenbosch, The Netherlands.

 – Sofie Geurts, Canisius Wilhelmina Ziekenhuis, Nijmegen, The Netherlands.

 – Christaan de Brabander, Admiraal de Ruyter Ziekenhuis, Vlissingen, The Netherlands.

 – Nynke F. Kalkers, OLVG, Amsterdam, The Netherlands.

• Bram A.J. den Teuling, Pim van Oirschot, Sonja Cloosterman, Sherpa B.V., Nijmegen, The Netherlands.

• Jos Vermeer, Personal Fitness Nederland (PFN) B.V., Eindhoven, The Netherlands.

• Chris C. Schouten, Dutch MS Society, Den Donder, The Netherlands.

• Gerard J. Stege, Merck B.V., Schiphol-Rijk, The Netherlands.

• Thijs van‘t Hullenaar, Sanofi B.V., Genzyme Europe, Amsterdam, The Netherlands.

We would like to thank all members of the consortium for their contribution to the project.

Funding

This study is peer-reviewed and funded by the Dutch Research Council (NWO) as part of the Dutch National Research Agenda (NWA), file number NWA.1292.19.064. The funder does not have role in the design of the study and collection, analysis, and interpretation of the data and in writing the manuscript.

Author information

Authors and Affiliations

Authors

Consortia

Contributions

PW: Conceptualisation, Methodology, Investigation, Writing – original draft, Visualisation, Project Administration. BdJ: Conceptualisation, Methodology, Writing – Review & Editing, Supervision. BU: Conceptualisation, Methodology, Supervision. SS: Conceptualisation, Investigation, Project Administration. JA: Conceptualisation, Investigation, Project Administration. AR: Investigation, Project Administration. PO: Conceptualisation, Methodology, Software, Writing – Review & Editing. VG: Conceptualisation, Writing – Review & Editing, Supervision. FS: Conceptualisation, Supervision. KH: Conceptualisation, Writing – Review & Editing, Supervision. MR: Conceptualisation, Writing – Review & Editing, Supervision. MS: Conceptualisation, Supervision. GW: Conceptualisation, Writing – Review & Editing, Supervision. SV: Conceptualisation, Project Administration. ES: Conceptualisation. MK: Conceptualisation, Methodology, Writing – Review & Editing, Supervision. HH: Conceptualisation, Methodology, Writing – Review & Editing, Supervision, Funding Acquisition. All authors read and approved the final manuscript.

Authors’ information

See above.

Corresponding author

Correspondence to Pauline T. Waskowiak.

Ethics declarations

Ethics approval and consent to participate

The Medical Ethical Committee of the Amsterdam UMC, Vrije Universiteit Amsterdam has reviewed and approved this study (METC 2021.0707, protocol version 2, 4 May 2022). Any future substantial changes to the study protocol will undergo review and approval by the METC. Written informed consent is obtained from all participants upon enrolment in the study.

Consent for publication

Not applicable.

Competing interests

Pauline T. Waskowiak has no competing interests regarding publication.

Brigit A. de Jong has no competing interests regarding publication.

Bernard M.J. Uitdehaag has received research support and/or consultancy fees from Biogen Idec, Genzyme, Merck Serono, Novartis, Roche, Teva and Immunic Therapeutics.

Shalina R.D. Saddal has no competing interests regarding publication.

Jip Aarts has no competing interests regarding publication.

Aïda A.M. Roovers has no competing interests regarding publication.

Pim van Oirschot is employed by Sherpa BV.

Vincent de Groot has no competing interests regarding publication.

Frederieke G. Schaafsma has no competing interests regarding publication.

Karin van der Hiele has no competing interests regarding publication.

Marit F.L. Ruitenberg has no competing interests regarding publication.

Menno M. Schoonheim serves on the editorial board of Neurology, Multiple Sclerosis Journal and Frontiers in Neurology, receives research support from the Dutch MS Research Foundation, Eurostars-EUREKA, ARSEP, Amsterdam Neuroscience, MAGNIMS and ZonMW (Vidi grant, project number 09150172010056) and has served as a consultant for or received research support from Atara Biotherapeutics, Biogen, Celgene/Bristol Meyers Squibb, EIP, Sanofi, MedDay and Merck.

Guy A.M. Widdershoven has no competing interests regarding publication.

Sabina van der Veen has no competing interests regarding publication.

Esther C.F. Schippers has no competing interests regarding publication.

Martin Klein has no competing interests regarding publication.

Hanneke E. Hulst is an editor of the Multiple Sclerosis Journal controversies sections, receives research support from the Dutch MS Research Foundation and the Dutch Research Council. She has served as a consultant for or received research support from Atara Biotherapeutics, Biogen, Novartis, Celgene/Bristol Meyers Squibb, Sanofi Genzyme, MedDay and Merck BV.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Waskowiak, P.T., de Jong, B.A., Uitdehaag, B.M.J. et al. Don’t be late! Timely identification of cognitive impairment in people with multiple sclerosis: a study protocol. BMC Neurol 24, 26 (2024). https://doi.org/10.1186/s12883-023-03495-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12883-023-03495-x

Keywords