Skip to main content

A comparison of the brief international cognitive assessment for multiple sclerosis and the brief repeatable battery in multiple sclerosis patients

Abstract

Background

Recently, a Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) has been developed as an international and standardized brief cognitive test, which is easily performed in everyday clinical practice for neuropsychological assessment in multiple sclerosis (MS). However, we need to gather more information about this tool compared to other neuropsychological batteries. The aim of our study is to compare the performance of BICAMS and Brief Repeatable Battery (BRB) in MS subjects.

Methods

Tests of the BRB and BICAMS were administered to MS patients recruited from 11 Italian MS centres. Cognitive impairment (CI) was defined as the failure on at least two tests (scores below the fifth percentile) on the BRB and as the failure on at least one test of the BICAMS. The agreement between the performances on the two batteries was assessed through Cohen’s K statistic. Finally we calculated the effects sizes for each test of the two batteries using Cohen’s d.

Results

The two batteries were administered to 192 MS patients (142 women, 50 men; mean age 41.4 ± 10.8 years, mean education 12.3 ± 3.5 years). Mean scores of patients were lower compared to those of healthy subjects in all the cognitive measures examined. Forty-six MS patients were identified as impaired and 48 as unimpaired on both of the batteries, when the Symbol Digit Modalities Test (SDMT) was included in the analysis. Cohen’s K statistic was 0.46 which corresponds to a moderate accord. If the SDMT was excluded from the BRB, 37 MS patients were identified as impaired and 57 as unimpaired on both of the batteries. Cohen’s K statistic was 0.3 which corresponds to a poor accord. The SDMT, the Paced Auditory Serial Addition Test (PASAT) 3 and 2 yielded higher d values (SDMT 0.83, PASAT 3 0.65, PASAT 2 0.84).

Conclusions

This study confirms the feasibility of BICAMS in everyday clinical practice for the identification of CI and highlights the good psychometric properties of the SDMT.

Peer Review reports

Background

Cognitive impairment (CI) affects about 40–60 % of multiple sclerosis (MS) subjects [1]. It involves all the disease subtypes and it can be documented from the very beginning of the disease [1, 2]. Once established, it tends to progress over time, sometimes independently from the accumulation of physical disability [1]. Deficits of complex attention, information processing speed, episodic memory and executive functioning are prominent, whereas language and general intelligence are usually spared [3]. Also independently of physical disability, CI can have an important negative impact on patient performance in everyday activities, employment, social and recreational activities [1]. For this purpose, assessment of MS-related CI is strongly recommended. The most commonly used instrument to estimate cognitive dysfunction in MS patients, both for clinical practice and research purposes, is the Brief Repeatable Battery (BRB), that includes cognitive domains most frequently affected [4]. Normative values in the Italian population are available [5]. Despite its good psychometric properties, the implementation in clinical practice is limited by its time-consuming nature (about 45 min) and the need of administration and interpretation by experienced neuropsychologists. Therefore, there has been considerable effort over the past decade to streamline the neuropsychological assessment in MS, by developing brief assessment tools that can be incorporated in everyday patient assessment. In particular, recently, a Brief International Cognitive Assessment for MS (BICAMS) has been recommended as an international, validated and standardized brief cognitive test [6]. It is easily performed in everyday clinical practice as it can be completed in 15 min and can be administered by health care professionals who are not cognitive specialists. No special equipment (beyond pen, paper and stopwatch) is required [6, 7]. Translation and validation of the BICAMS is on going in several countries. It has been recently validated in the American [7], Czech [8], Iranian [9] and Italian populations [10]. We can consider BICAMS as a brief, practical and universal assessment tool for CI in MS subjects. However, little is known on its performance in comparison to other neuropsychological batteries. For this purpose, the aim of our study is to compare the performance of BICAMS and BRB as screening tools for cognitive impairment in MS patients [5, 10].

Methods

A total of 192 MS patients (142 women; 50 men), among those consecutively admitted to some of the major Italian MS centres (Bari, Barletta, Bologna, Cagliari, Chieti, Florence, Gallarate, Lanciano, Milan, Naples and Rome), were recruited. Inclusion criteria were diagnosis of relapsing-remitting (RR) MS [11] and age >18 years. Inclusion was restricted to RRMS subjects in order to avoid heterogeneity of cognitive profile due to MS course. Exclusion criteria were presence of current or past neurological disorder other than MS, major psychiatric illness, history of learning disability, serious head trauma, alcohol or drug abuse and relapse and/or corticosteroid use within 4 weeks preceding assessment. All the subjects had adequate vision and hearing to undergo the tests. All the participants in the study provided their informed consent and the study was approved by the ethic committee of the University of Florence.

Neuropsychological test procedures

At each site, patients were examined by the same neuropsychologist, who had participated to a common training session, in order to ensure uniform administration, data recording and scoring procedures. Tests of the BRB and BICAMS were administered in a standardized manner, during daytime, in a quiet room, and in a fixed order. We first administered the BRB and subsequently the BICAMS, in different sessions, within 1 week. The SDMT [4] was given only once. We used the validated Italian translation of both batteries [5, 10]. The BRB incorporates tests of verbal memory acquisition and delayed recall (Selective Reminding Test- SRT), visual memory acquisition and delayed recall (10/36 Spatial Recall Test-SPART), attention, concentration and speed of information processing (Paced Auditory Serial Addition Test –PASAT; SDMT) and verbal fluency on semantic stimulus (Word List Generation-WLG) [4]. The administration of the whole BRB battery takes about 45 min. The BICAMS includes the SDMT [4], California Verbal Learning Test, second edition (CVLT-II) first five trials, for assessing verbal memory [12] and Brief Visuospatial Memory Test-Revised (BVMT-R) first three recall trials, for visual-spatial memory [13]. Administration of the whole battery takes about 15 min.

Statistical analysis

Group comparisons were assessed through Student’s t test, Mann–Whitney U test and χ2 test, as appropriate. CI was defined as the failure on at least two tests (scores below the fifth percentile) on the BRB, based on the Italian normative data [5]. Failure on the BICAMS was defined as the failure in at least one test of the battery [10]. Performance on the SDMT was assessed using normative data from the Italian BICAMS validation [10]. The agreement between the performances of the two batteries was assessed through Cohen’s K statistic [1416]. The sensitivity, specificity and accuracy of the BICAMS against the BRB were assessed. Finally we calculated the effects sizes for each test of the two batteries using Cohen’s d (difference between means divided by pooled SD) separating MS patients and controls [17].

Results

The study sample consisted of 192 consecutive RRMS patients from 11 Italian MS Centres (Table 1). Table 2 shows mean scores of patients and normative samples on the neuropsychological tests [5, 10]. Mean scores of patients were lower compared to those of healthy subjects in all the cognitive measures examined. To compare the performance on the BICAMS and the BRB, we have repeated the analysis excluding SDMT, which was the only neuropsychological test included in both of the batteries. This exclusion is intended to avoid an overestimation of the accord between the two assessment tools. Forty-six MS patients were identified as impaired and 48 as unimpaired on both batteries, when the SDMT was included in the analysis. Cohen’s K statistic was 0.46 which corresponds to a moderate accord [1416]. The Cohen’s K statistic estimating the agreement between the SDMT alone and the BRB was comparable (0.42). As expected, the concordance decreased if we excluded the SDMT from the BRB. In this case, 37 MS patients were identified as impaired and 57 as unimpaired on both of the batteries. Cohen’s K statistic was 0,3 (0.26 for the SDMT alone) which corresponds to a poor accord [1416]. Using the whole BRB as the gold standard, overall BICAMS sensitivity was 58,2 %, specificity 86,7 %, with an accuracy of 75 %. Using the SDMT alone the sensitivity was 43 %, the specificity 95,6 % and accuracy 73,9 %. Table 3 shows Cohen’s d for different tests [17]. Overall, verbal memory tests of the BRB and BICAMS were comparable (SRT-LTS 0.55, SRT-CLTR 0.61, CVLT 2 0.61). Instead, the BVMT-R of the BICAMS showed a higher d value (0.60) as compared with the SPART test (0.38). Finally, the SDMT, PASAT 3 and 2 yielded higher d values (SDMT 0.83, PASAT 3 0.65, PASAT 2 0.84).

Table 1 Characteristics of the study sample (# 192 MS subjects)
Table 2 Mean scores (SD) of patients and normative samples on the neuropsychological tests [5, 10]
Table 3 Cohen’s d for each test of the two batteries separating MS patients and controls

Discussion

Cognitive assessment represents a key step in taking charge of MS patients. The most widely used neuropsychological battery is BRB [5]. Recently the BICAMS, a rapid tool, more suitable to be incorporated in everyday patient assessment, has been developed [6]. We recently published normative values for the BICAMS in the Italian population [10]. The implementation of BICAMS is still in its infancy; therefore we need to obtain more information about its performance in comparison to other neuropsychological batteries. In the present study we administered to MS patients both the BRB and the BICAMS. Mean scores of patients were lower compared to those of healthy subjects [5, 10] in all the cognitive measures examined. This is in line with the typical cognitive profile in MS patients [1]. In our study, tests with higher discriminating ability according to the d values were the SDMT, PASAT 2 and 3 s. This finding is in line with recent literature, identifying the SDMT as the test with higher ability in differentiating MS patients from healthy controls [18, 19]. On the basis of our results, the agreement between the BICAMS and the BRB is fair to moderate and mainly dependent on the inclusion of the SDMT, which is the only common test between the two batteries. We can hypothesize that the BICAMS and the BRB cannot be considered as equivalent in the assessment of CI in MS. The BRB investigates more comprehensively the cognitive profile, including wider assessment than the BICAMS. Therefore, we can consider the BRB a brief neuropsychological battery more complete than the BICAMS and more suitable to identify change over time [2022]. The BICAMS, on the other hand, can represent a valid alternative to a more comprehensive battery when available resources are scarce. Since in our study cognitive evaluations were performed by neuropsychologists, further analyses including the administration of tests by health care professionals who are not cognitive specialists are needed, in order to confirm the external validity of our findings.

Conclusions

In conclusion, on the basis of our results, we can consider the BICAMS a brief and feasible tool appropriate for the cognitive assessment of MS patients and for research use. It seems important to stress that, in optimal clinical conditions, where a neuropsychologist can take care of the cognitive assessment, it is preferable to achieve a more thorough cognitive evaluation using tools including neuropsychological tests that investigate several cognitive domains impaired in MS.

Abbreviations

BICAMS:

Brief international cognitive assessment for multiple sclerosis

MS:

Multiple sclerosis

BRB:

Brief repeatable battery

CI:

Cognitive impairment

SDMT:

Symbol digit modalities test

PASAT 3 and 2:

Paced auditory serial addition test 3 and 2

RR:

Relapsing-remitting

SRT:

Selective reminding test

SPART:

10/36 spatial recall test

WLG:

Word list generation

CVLT-II:

California verbal learning test, second edition

BVMT-R:

Brief visuospatial memory test-revised

References

  1. 1.

    Amato MP, Zipoli V, Portaccio E. Cognitive changes in multiple sclerosis. Expert Rev Neurother. 2008;8:1585–96.

    Article  PubMed  Google Scholar 

  2. 2.

    Zipoli V, Goretti B, Hakiki B, Siracusa G, Sorbi S, Portaccio E, et al. Cognitive impairment predicts conversion to multiple sclerosis in clinically isolated syndromes. Mult Scler. 2010;16:62–7.

    Article  PubMed  Google Scholar 

  3. 3.

    Amato MP, Langdon D, Montalban X, Benedict RH, DeLuca J, Krupp LB, et al. Treatment of cognitive impairment in multiple sclerosis: position paper. J Neurol. 2013;260:1452–68.

    CAS  Article  PubMed  Google Scholar 

  4. 4.

    Rao S. A manual for the brief repeatable battery of neuropsychological tests in multiple sclerosis. Milwaukee: Medical College of Wisconsin; 1990.

    Google Scholar 

  5. 5.

    Amato MP, Portaccio E, Goretti B, Zipoli V, Ricchiuti L, De Caro MF, et al. The Rao’s brief repeatable battery and stroop test: normative values with age, education and gender corrections in an Italian population. Mult Scler. 2006;12:787–93.

    CAS  Article  PubMed  Google Scholar 

  6. 6.

    Langdon DW, Amato MP, Boringa J, Brochet B, Foley F, Fredrikson S, et al. Recommendations for a Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Mult Scler. 2012;18:891–8.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  7. 7.

    Benedict RH, Amato MP, Boringa J, Brochet B, Foley F, Fredrikson S, et al. Brief International Cognitive Assessment for MS (BICAMS): international standards for validation. BMC Neurol. 2012;12:55.

    Article  PubMed  PubMed Central  Google Scholar 

  8. 8.

    Dusankova JB, Kalincik T, Havrdova E. Cross cultural validation of the Minimal Assessment of Cognitive Function in Multiple Sclerosis (MACFIMS) and the Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Clin Neuropsychol. 2012;26:1186–200.

    Article  PubMed  Google Scholar 

  9. 9.

    Eshaghi A, Riyahi-Alam S, Roostaei T, Haeri G, Aghsaei A, Aidi MR, et al. Validity and reliability of a Persian translation of the Minimal Assessment of Cognitive Function in Multiple Sclerosis (MACFIMS). Clin Neuropsychol. 2012;26:975–84.

    Article  PubMed  Google Scholar 

  10. 10.

    Goretti B, Niccolai C, Hakiki B, Sturchio A, Falautano M, Eleonora M, et al. The brief international cognitive assessment for multiple sclerosis (BICAMS): normative values with gender, age and education corrections in the Italian population. BMC Neurol. 2014;14:171.

    Article  PubMed  PubMed Central  Google Scholar 

  11. 11.

    Polman CH, Reingold SC, Banwell B, Clanet M, Cohen JA, Filippi M, et al. Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria. Ann Neurol. 2011;69:292–302.

    Article  PubMed  PubMed Central  Google Scholar 

  12. 12.

    Delis DC, Kramer JH, Kaplan E, Ober BA. California Verbal Learning Test, second edition (CVLT-II). San Antonio: Psychological Corporation; 2000.

    Google Scholar 

  13. 13.

    Benedict RHB. The Brief Visuospatial Memory Test Revised (BVMT-R). Lutz: Psychosocial Assessment Resources Inc; 1997.

    Google Scholar 

  14. 14.

    Fleiss JL. The measurement of interrater agreement. In: Statistical methods for rates and proportions. NewYork: Wiley; 1981.

    Google Scholar 

  15. 15.

    Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull. 1971;76:378–82.

    Article  Google Scholar 

  16. 16.

    Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.

    CAS  Article  PubMed  Google Scholar 

  17. 17.

    Cohen J. Statistical power analysis for the behavioral sciences (second ed.). Hillsdale: Lawrence Erlbaum Associates; 1988.

    Google Scholar 

  18. 18.

    Strober L, Englert J, Munschauer F, Weinstock-Guttman B, Rao S, Benedict RH. Sensitivity of conventional memory tests in multiple sclerosis: comparing the Rao brief repeatable neuropsychological battery and the minimal assessment of cognitive function in MS. Mult Scler. 2009;15:1077–84.

    CAS  Article  PubMed  Google Scholar 

  19. 19.

    Drake AS, Weinstock-Guttman B, Morrow SA, Hojnacki D, Munschauer FE, Benedict RH. Psychometrics and normative data for the multiple sclerosis functional composite: replacing the PASAT with the symbol digit modalities test. Mult Scler. 2010;16:228–37.

    CAS  Article  PubMed  Google Scholar 

  20. 20.

    Amato MP, Razzolini L, Goretti B, Stromillo ML, Rossi F, Giorgio A, et al. Cognitive reserve and cortical atrophy in multiple sclerosis: a longitudinal study. Neurology. 2013;80(19):1728–33.

    Article  PubMed  Google Scholar 

  21. 21.

    Patti F, Amato MP, Bastianello S, Caniatti L, Di Monte E, Ferrazza P, et al. Effects of immunomodulatory treatment with subcutaneous interferon beta-1a on cognitive decline in mildly disabled patients with relapsing-remitting multiple sclerosis. Mult Scler. 2010;16(1):68–77.

    CAS  Article  PubMed  Google Scholar 

  22. 22.

    Amato MP, Portaccio E, Goretti B, Zipoli V, Battaglini M, Bartolozzi ML, et al. Association of neocortical volume changes with cognitive deterioration in relapsing-remitting multiple sclerosis. Arch Neurol. 2007;64(8):1157–61.

    Article  PubMed  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Maria Pia Amato.

Additional information

Competing interests

MPA received personal compensation from Merck Serono, Biogen, Genzyme, Teva and Novartis for serving on scientific advisory board and for speaking, received financial support for research activites from Almirall, Merck Serono, Biogen Idec, Bayer Schering, Genzyme, Novartis, Genzyme and Teva. BG serves on a scientific advisory board for Biogen, received honoraria for speaking from Biogen-Idec and Teva. LP and MG received article grants from Almirall and Biogen. MF has received travel grants from Fondazione Italiana Sclerosi Multipla (FISM). EM has received travel and research grants from Fondazione Italiana Sclerosi Multipla (FISM) and serves as consultant for “Fondazione Cesare Serono”. RGV serves on a scientific advisory board for Biogen. E Po serves on a scientific advisory board for Biogen, Merck Serono and Bayer, received honoraria for speaking from Biogen-Idec and Teva, and receives research support from Merck Serono. MT has received honoraria for consultancy or speaking from Biogen, Sanofi-Aventis, Merck Serono and Bayer-Schering and research grants from Merck Serono, Biogen and Novartis. VM has received speaker honoraria or funding for travel expenses from Biogen-Idec, Merck Serono, Bayer Schering, Novartis, Teva Pharmaceutical Industries and Genzyme. UN has served as consultant for Biogen Idec, Sanofi Aventis, Teva Pharmaceuticals, Novartis Pharma, Boehringer-Ingelheim and has received travel and research grants by Merck Serono, Novartis Pharma and Biogen Idec. GF has received honoraria for consultancy or speaking from Merck Serono, Novartis and Teva. EC has received honoraria for consultancy or speaking from Biogen, TEVA, Novartis, Sanofi-Aventis, Merck Serono and Bayer. MGM has received honoraria for consultancy or speaking and research grants from Biogen, Sanofi-Aventis, TEVA, Novartis, Merck Serono and Bayer. EPi has received travel grants from Bayer Schering, Biogen Idec, Merck Serono, Novartis, Sanofi-Aventis and TEVA and has also received travel and research grants from the Fondazione Italiana Sclerosi Multipla. AL was a Bayer, Biogen Idec, Merck Serono, Novartis and Genzyme Advisory Board Member, received travel grants and honoraria from Bayer, Biogen Idec, Merck Serono, Novartis, Sanofi and Teva and research grants from Bayer, Biogen Idec, Merck Serono, Novartis, Sanofi and Teva, she also received travel and research grants from the Associazione Italiana Sclerosi Multipla and was a Consultant of “Fondazione Cesare Serono”. AG has served on scientific advisory boards for Merck Serono, Novartis, Biogen Idec, Teva Pharmaceutical Industries Ltd., and has received speaker honoraria from Merck Serono, Biogen Idec, Bayer Schering Pharma, Sanofi-Genzyme, Novartis, Serono Symposia International and Almirall. MR has received honoraria for consultancy from “Fondazione Cesare Serono”.

CN, BH, IR, CI, EG, FIA, MM, MC, NAL, GBZ, LP, DC and SS have no competing interests.

Authors’ contributions

CN and BG drafted/revised the manuscript, participated in study concept or design and carried out acquisition of data. E Po and MPA drafted/revised the manuscript, participated in study concept or design, analysis or interpretation of data and carried out study supervision. BH, MG, LP, IR, MF, EM, VM, CI, UN, GF, EC, MGM, EG, FIA, MM, MC, RGV, MT, NAL, GBZ, E Pi, AL, LP, MR, AG, DC and SS participated in study concept or design and acquisition of data. All authors read and approved the final manuscript.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Niccolai, C., Portaccio, E., Goretti, B. et al. A comparison of the brief international cognitive assessment for multiple sclerosis and the brief repeatable battery in multiple sclerosis patients. BMC Neurol 15, 204 (2015). https://doi.org/10.1186/s12883-015-0460-8

Download citation

Keywords

  • Multiple sclerosis
  • Assessment tool
  • Cognitive impairment
  • Brief international cognitive assessment for multiple sclerosis