Skip to main content

Validity and reliability of the Georgian-language brief international cognitive assessment for multiple sclerosis (BICAMS)

Abstract

Background

Cognitive impairment is one of the common features of multiple sclerosis (MS). Despite high prevalence, cognitive decline is often overlooked by neurologists. The Brief International Cognitive Assessment for MS (BICAMS) was therefore introduced by the international expert committee as a brief and effective tool for the assessment and monitoring of cognitive functions in patients with MS. The validity and reliability of BICAMS have been demonstrated in many countries. Our aim was to validate the BICAMS in Georgian patients with MS.

Methods

A total of 68 patients with MS and 68 matched controls were assessed by the Georgian-language BICAMS. All healthy controls and seven patients were re-evaluated with identical tests to assess retest reliability.

Results

In comparison to healthy controls, patients with MS performed significantly worse on all tests in the assessment battery. Test–retest reliability measures were good for all tests. The prevalence of cognitive impairment in patients with MS was 43%.

Conclusion

The Georgian-language BICAMS is a reliable and valid battery for the assessment of cognitive function in patients with MS.

Peer Review reports

Introduction

Multiple sclerosis (MS) is a chronic, inflammatory, demyelinating, and degenerative disease of the central nervous system [1]. MS usually affects individuals between the ages of 20 and 40 and is the leading cause of non-traumatic disability in young adults [2]. Over the past few decades, cognitive impairment (CI) has been recognized as an important feature of the disease, presenting in up to 65% of patients [3]. Information processing speed and episodic memory are the most commonly impaired cognitive functions [4]. CI presents even at early stages of the disease but is more prominent and prevalent in patients with progressive MS [5, 6]. In a recent multi-center study, the overall prevalence of CI in patients with secondary and primary progressive MS was 79.4% and 91.3%, respectively [7]. Planche et al. found that patients with progressive MS have more frequent and severe impairment of information processing speed, executive function, verbal episodic memory, visuospatial abilities, verbal fluency and working memory, compared to patients with RRMS [8]. Increasing evidence indicates that CI substantially impacts working ability and activities of daily living in patients with MS. Moreover, it is the leading predictor of occupational disability in these patients [9].

Cognitive dysfunction should be identified as early as possible to ensure timely intervention and adequate management. However, if not assessed with standardized neuropsychological tools, it can be overlooked during routine neurological assessments [10].

In 2012, the Brief International Cognitive Assessment for MS (BICAMS) was introduced by the international committee as an effective tool for the assessment and monitoring of cognitive function in patients with MS [10, 11]. Three tests are included in this battery. The first is the Symbol Digit Modality Test (SDMT), which assesses information processing speed. The test consists of nine digits paired with nine geometric symbols. A sample of nine symbol-digit pairs is followed by rows filled with random symbols. The patient writes or names the number corresponding to each consecutive symbol. Patients have 90 s in which to assign as many numbers as possible [12, 13]. The second is the California Verbal Learning Test 2nd edition (CVLT-II), which evaluates verbal memory. The test is composed of 16 words from four different categories. The examiner reads aloud the same list of words five times. After each reading, the patient recalls as many words as possible [14]. The third is the Brief Visual Memory Test-Revised (BVMT-R), which assesses visual memory. A sheet of paper displaying six unique geometric figures is presented to the patient for 10 s three times and is then removed each time. After each learning trial, the patient attempts to draw the figures in the correct position [15]. The BICAMS requires 15 min to administer and its validity and reliability have been demonstrated in many countries [16]. The battery can be used in routine clinical practice for monitoring cognitive function in patients with MS by a neurologist or other healthcare professional [11].

The objective of this study was to validate the BICAMS in Georgian patients with MS.

Materials and methods

The study was conducted at the S. Khechinashvili University Hospital from March 1, 2019 to October 1, 2020. The study protocol and informed consent form were approved by two ethics committees: the local S. Khechinashvili University Hospital committee and the Tbilisi State Medical University committee.

Prior to enrollment, the three tests and the administration manual of the BICAMS were translated into the Georgian language in accordance with the guidelines of the International Test Commission [17]. We did not make major changes to any of the tests.

Subjects

In total, 246 subjects were recruited for the study: 68 patients with MS and 178 healthy controls (HC). Inclusions were: (for patients with MS) willingness and ability to give informed consent; a diagnosis of MS confirmed according to the McDonald criteria (2017 revision); age ≥ 18 y; no evidence of relapse in at least the month preceding the evaluation; no history of medical conditions, other than MS, that could affect cognitive abilities; native speaker of the Georgian language.

MS patients admitted to the neurology outpatient clinic at S. Khechinashvili University Hospital from March 1, 2019 to October 1, 2020 were offered an opportunity to participate in the study and 68 accepted.

Two hundred and forty-six HCs were randomly selected from two regions in Georgia. The inclusion criteria for this pre-screening group were age ≥ 18 y and no history of neurological disease or severe head trauma. Neuropsychological assessment was performed by a trained psychologist and a neurologist. An age-, sex-, and education-matched group of 68 participants was then sub-selected from this group.

Testing

We collected demographic data on age, education, employment status, disease duration and subtype, and scores on the expanded disability status scale (EDSS). Neurological and neuropsychological assessment was conducted the same day. All assessments in the MS group were performed by the same neurologist. The BICAMS battery was applied in the recommended sequence, namely: SDMT; CVLT-II, first five learning trials; and BVMT-R, first three learning trials. We have used written version of SDMT, although original BICAMS battery includes oral version of the test [11]. After completion of the BICAMS, patients filled out the Beck Depression Inventory (BDI) [18].

To assess retest reliability, 68 HCs and 7 MS patients were re-evaluated with identical tests, with a mean interval between evaluations of 18.0. days (± 7.1.).

Patients were classified as cognitively impaired if their score on any BICAMS test was below 1.5 SD of the mean score of the control group.

Statistical analysis

Values are presented as means ± SD and percentages. Statistical analysis was performed with SPSS V26 software. Statistical significance was set at p < 0.05. Pearson’s correlation coefficient was used for the measurement of correlations and of test–retest reliability. Comparisons between different groups employed the paired-samples t-test.

Results

All 68 patients with MS completed the SDMT. The oral version was administered to four patients who scored ≥ 2 points on the pyramidal or cerebellar functional scales involving the dominant hand. The BVMT-R results were missing for two patients. The BDI was administered to 55 (81%) patients.

The main characteristics of the study population are outlined in Table 1. The mean age of the MS group was 39.2 ± 9.9. The mean age of the HC group was 38.5 ± 9.9. In the HC group, 84% were employed, whereas in the MS group, 57% were employed. Forty-nine HC participants and Fifty MS participants had ≥ 15 y of education. The proportion of patients with ≤ 14 y of education was slightly higher in the control group (28% vs. 26%).

Table 1 Characteristics of the study population

The MS group comprised 52 patients (78.8%) with relapsing–remitting MS (RRMS), 12 with secondary progressive MS (SPMS), and 4 with primary progressive MS (PPMS). The mean disease duration in patients was 7.0 ± 5.7 y. Forty-nine percent of patients had a ≤ 5-y history of MS. Ten percent of patients had newly diagnosed MS. The mean BDI score was 12 ± 9.0 points. Clinically significant moderate to severe depression (≥ 19 points) was identified in 14 patients (25%).

As shown in Table 2, the test–retest reliability coefficient was adequate to good for all three tests. The SDMT showed the highest reliability.

Table 2 BICAMS test–retest reliability

Scores are mean ± SD; SDMT, Symbol Digit Modality Test; CVLT-II, California Verbal Learning Test 2nd edition; BVMT-R, Brief Visual Memory Test-Revised.

The mean scores on all cognitive tests were lower in the patient group. The greatest discrepancy between the MS and HC groups was seen on the SDMT (Table 3).

Table 3 Comparison of mean ± SD test scores between patients with MS and healthy controls (HC)

Age was negatively correlated with all test scores in the HC group (Table 4). Such a correlation was observed with only the SDMT scores in the patient group (Table 5). In the HC group, we found a significant positive correlation between years of education and scores on all three tests. In the patient group, an analogous correlation was identified for the BVMT-R and SDMT, but not for the CVLT-II. Among 18 patients with fewer than 14 y of education, CI was identified in 14 (78%). In contrast, among 19 HCs with fewer than 14 y of education, CI was identified in only 2 (10.5%).

Table 4 Correlations of BICAMS scores with age and education in the HC group
Table 5 Correlations of BICAMS scores in patients with MS

Table 6 shows comparisons of patients with shorter and longer disease duration. The mean scores on the BICAMS tests were higher in patients with shorter (≤ 10 y) disease durations, i.e., 31.0 ± 12.6 vs. 37.3 ± 11.8 for the SDMT, 47.6 ± 10.5 vs. 52.2 ± 11.1 for the CVLT-II and 21.8 ± 8.4 vs. 22.4 ± 7.5 for the BVMT-R. The mean age was higher in patients with longer disease durations (41.0 ± 8.0 vs. 38.4 ± 10.0). On the other hand, the proportion of individuals with fewer than 14 y of education was higher in patients with a disease duration ≤ 10 y (31.5% vs. 14.3%).

Table 6 Comparison of patients with shorter and longer disease duration

The overall prevalence of CI in our MS sample was 43%. Cognitive dysfunction was found in 67% of patients with SPMS and in 75% of patients with PPMS. In the RRMS subgroup, CI prevalence was 34%. Cognitive decline was identified in 19 patients (28%) by the SDMT, in 22 patients (32%) by the CVLT-II, and in 13 patients (19%) by the BVMT-R. Among patients with MS who had CI, 31% performed below the cutoff score (i.e., showed CI) on all three tests, 24% showed CI on two of the tests, and 41% showed CI on one test.

Discussion

CI is an important feature of MS, and a reliable neuropsychological screening instrument for the identification and monitoring of cognitive dysfunction in patients with MS is essential. The presence of CI may indicate progression of the disease despite stable physical symptoms [19]. Patients with CI can benefit from early medical intervention and rehabilitation. There is a growing evidence that disease modifying therapies (DMT) can prevent or reduce CI in patients with MS, although this subject has not been studied extensively [20, 21]. Recent data suggest that cognitive rehabilitation programs that specifically target distinct cognitive domains, such as working memory, attention or information processing speed, have positive effect on CI [22, 23].

Despite the high prevalence, cognitive decline is often overlooked by neurologists. The BICAMS battery was therefore introduced by the international expert committee as a brief and effective tool for the assessment and monitoring of cognitive functions in patients with MS. The original BICAMS battery recommends administration of oral SMDT to minimize the effect of physical disability on performance [11]. However, oral motor slowing has also been shown to influence performance on SDMT [24]. Additionally, written version of SDMT is easier to administer.

This is the first study to evaluate the validity and reliability of the Georgian-language BICAMS in compliance with international validation recommendations [11]. We found that 43% of our patients with MS had CI; the prevalence of CI in the control group was only 14%.

We found that age, fewer years of education, and a greater disability status are the main predictors of CI in MS. As expected, unemployment was more common in patients with CI (50% vs. 24%). We found a negative correlation between the SDMT results and scores on the BDI. CI was more prevalent in patients with clinically significant depression (33.3% vs. 8.3%).

In comparison to other validation studies [25,26,27], our study found significantly lower scores on the SDMT for both patients and HCs. The large discrepancy with the reported mean scores should be explained by the fact, that we administered written version of SDMT. However, our results are compatible with the Brazilian BICAMS validation study [28], although the mean years of education was significantly higher in our study.

We found no correlation between the BICAMS scores and disease duration. However, the prevalence of CI was significantly higher in patients with longer (≥ 11 y) disease duration (57% vs. 39%).

One of the limitations of this study is the small sample size. Unlike the original BICAMS battery we used written version of SDMT that might have an impact on correlations with other studies. Further, we did not exclude patients with clinically significant depression, and this could have influenced the test results. However, two of the three tests showed no correlation with the BDI scores. Additionally, 4 out of the 68 patients completed the oral version of the SDMT. Despite the high internal consistency of the SDMT, the mean scores of the two forms may differ [29].

Conclusion

Overall, our results are consistent with other validation studies, confirming that the Georgian-language BICAMS can be used in clinical practice as a reliable tool for the monitoring of cognitive function in patients with MS.

Availability of data and materials

The dataset used during the current study are available from the corresponding author on reasonable request.

Abbreviations

BDI:

The Beck Depression Inventory

BICAMS:

The Brief International Cognitive Assessment for MS

BVMT-R:

The Brief Visual Memory Test-Revised

CI:

Cognitive impairment

CVLT-II:

The California Verbal Learning Test 2nd edition

EDSS:

Expanded disability status scale

HC:

Healthy controls

MS:

Multiple sclerosis

PPMS:

Primary progressive MS

RRMS:

Relapsing–remitting MS

SD:

Standard deviation

SDMT:

The Symbol Digit Modality Test

SPMS:

Secondary progressive MS

References

  1. Peterson LK, Fujinami RS. Inflammation, demyelination, neurodegeneration and neuroprotection in the pathogenesis of multiple sclerosis. J Neuroimmunol. 2007;184:37–44. https://doi.org/10.1016/j.jneuroim.2006.11.015.

    Article  CAS  PubMed  Google Scholar 

  2. Filippi M, Bar-Or A, Piehl F, Preziosa P, Solari A, Vukusic S, et al. Multiple sclerosis. Nat Rev Dis. 2018;4:49. https://doi.org/10.1038/s41572-018-0041-4.

    Article  Google Scholar 

  3. Amato MP, Zipoli V, Portaccio E. Multiple sclerosis-related cognitive changes: a review of cross-sectional and longitudinal studies. J Neurol Sci. 2006;245:41–6. https://doi.org/10.1016/j.jns.2005.08.019.

    Article  PubMed  Google Scholar 

  4. Benedict RH, Zivadinov R. Risk factors for and management of cognitive dysfunction in multiple sclerosis. Nat Reviews Neurol. 2011;7:332. https://doi.org/10.1038/nrneurol.2011.61.

    Article  Google Scholar 

  5. Denney DR, Sworowski LA, Lynch SG. Cognitive impairment in three subtypes of multiple sclerosis. Arch Clin Neuropsychol. 2005;20:967–81. https://doi.org/10.1016/j.acn.2005.04.012.

    Article  PubMed  Google Scholar 

  6. Brochet B, Ruet A. Cognitive impairment in multiple sclerosis with regards to disease duration and clinical phenotypes. Front Neurol. 2019;10:261. https://doi.org/10.3389/fneur.2019.00261.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Ruano L, Portaccio E, Goretti B, Niccolai C, Severo M, Patti F, Cilia S, Gallo P, Grossi P, Ghezzi A, Roscio M, Mattioli F, Stampatori C, Trojano M, Viterbo RG, Amato MP. Age and disability drive cognitive impairment in multiple sclerosis across disease subtypes. Mult Scler. 2017;23(9):1258–67. https://doi.org/10.1177/1352458516674367 (Epub 2016 Oct 13 PMID: 27738090).

    Article  PubMed  Google Scholar 

  8. Planche V, Gibelin M, Cregut D, Pereira B, Clavelou P. Cognitive impairment in a population-based study of patients with multiple sclerosis: differences between late relapsing-remitting, secondary progressive and primary progressive multiple sclerosis. Eur J Neurol. 2016;23(2):282–9. https://doi.org/10.1111/ene.12715 (Epub 2015 Apr 22 PMID: 25903918).

    Article  CAS  PubMed  Google Scholar 

  9. Clemens L, Langdon D. How does cognition relate to employment in multiple sclerosis? A systematic review. Mult Scler Relat Disord. 2018;26:183–91. https://doi.org/10.1016/j.msard.2018.09.018.

    Article  PubMed  Google Scholar 

  10. Langdon DW, Amato MP, Boringa J, Brochet B, Foley F, Fredrikson S, et al. Recommendations for a brief international cognitive assessment for multiple sclerosis (BICAMS). Mult Scler. 2012;18:891–8. https://doi.org/10.1177/1352458511431076.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Benedict RH, Amato MP, Boringa J, Brochet B, Foley F, Fredrikson S, et al. Brief International Cognitive Assessment for MS (BICAMS): international standards for validation. BMC neurol. 2012;12:55.

    Article  Google Scholar 

  12. Smith A. The symbol-digit modalities test: a neuropsychologic test of learning and other cerebral disorders. In: Helmuth J, editor. Learning disorders. Seattle: Special Child Publications; 1968. p. 83–91.

    Google Scholar 

  13. Smith A. Symbol digits modalities test. Los Angeles: Western Psychological Services; 1982.

    Google Scholar 

  14. Delis DC, Kramer JH, Kaplan E, Ober BA. California Verbal Learning Test-Second Edition (CVLT-II). San Antonio: The Psychological Corporation; 2000.

    Google Scholar 

  15. Benedict RH. Brief Visuospatial Memory Test-Revised (BVMT-R). Odessa: Psychological Assessment Resources; 1997.

    Google Scholar 

  16. Corfield F, Langdon D. A systematic review and meta-analysis of the Brief Cognitive Assessment for Multiple Sclerosis (BICAMS). Neurol Ther. 2018;7:287–306. https://doi.org/10.6084/m9.figshare.6490256.

    Article  PubMed  PubMed Central  Google Scholar 

  17. International Test Commission. The ITC Guidelines for Translating and Adapting Tests. 2nd ed. 2017.

    Google Scholar 

  18. Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Arch Gen Psychiatry. 1961;4:561–71. https://doi.org/10.1001/archpsyc.1961.01710120031004.

    Article  CAS  PubMed  Google Scholar 

  19. Johnen A, BĂ¼rkner PC, Landmeyer NC, Ambrosius B, Calabrese P, Motte J, et al. Can we predict cognitive decline after initial diagnosis of multiple sclerosis? Results from the German National early MS cohort (KKNMS). J Neurol. 2019;266:386–97. https://doi.org/10.1007/s00415-018-9142-y.

    Article  PubMed  Google Scholar 

  20. Patti F, Leone C, D’Amico E. Treatment options of cognitive impairment in multiple sclerosis. Neurol Sci. 2010;31(Suppl 2):S265–9. https://doi.org/10.1007/s10072-010-0438-7 (PMID: 20953811).

    Article  PubMed  Google Scholar 

  21. Landmeyer NC, BĂ¼rkner PC, Wiendl H, Ruck T, Hartung HP, Holling H, Meuth SG, Johnen A. Disease-modifying treatments and cognition in relapsing-remitting multiple sclerosis: A meta-analysis. Neurology. 2020;94(22):e2373–83. https://doi.org/10.1212/WNL.0000000000009522 (Epub 2020 May 19 PMID: 32430312).

    Article  CAS  PubMed  Google Scholar 

  22. DeLuca J, Chiaravalloti ND, Sandroff BM. Treatment and management of cognitive dysfunction in patients with multiple sclerosis. Nat Rev Neurol. 2020;16(6):319–32. https://doi.org/10.1038/s41582-020-0355-1 (Epub 2020 May 5 PMID: 32372033).

    Article  PubMed  Google Scholar 

  23. D’Amico E, Leone C, Hayrettin T, Patti F. Can we define a rehabilitation strategy for cognitive impairment in progressive multiple sclerosis? A critical appraisal Mult Scler. 2016;22(5):581–9. https://doi.org/10.1177/1352458516632066 (Epub 2016 Feb 26 PMID: 26920381).

    Article  PubMed  Google Scholar 

  24. Nygaard GO, de Rodez Benavent SA, Harbo HF, Laeng B, Sowa P, Damangir S, Bernhard Nilsen K, Etholm L, Tønnesen S, Kerty E, Drolsum L, Inge Landrø N, Celius EG. Eye and hand motor interactions with the symbol digit modalities test in early multiple sclerosis. Mult Scler Relat Disord. 2015;4(6):585–9. https://doi.org/10.1016/j.msard.2015.08.003 (Epub 2015 Aug 15 PMID: 26590666).

    Article  PubMed  Google Scholar 

  25. O’Connell K, Langdon D, Tubridy N, Hutchinson M, McGuigan C. A Preliminary Validation of the Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) Tool in an Irish Population With Multiple Sclerosis (MS). Mult Scler Relat Disord. 2015;4:521–5. https://doi.org/10.1016/j.msard.2015.07.012 (Epub 2015 Aug 7).

    Article  CAS  PubMed  Google Scholar 

  26. Sandi D, Rudisch T, FĂ¼vesi J, Fricska-Nagy Z, Huszka H, Biernacki T, et al. The Hungarian validation of the Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) battery and the correlation of cognitive impairment with fatigue and quality of life. Mult Scler Relat Disord. 2015;4:499–504. https://doi.org/10.1016/j.msard.2015.07.006.

    Article  PubMed  Google Scholar 

  27. Ozakbas S, Yigit P, Cinar BP, Limoncu H, Kahraman T, KösehasanoÄŸulları G. The Turkish validation of the brief international cognitive assessment for multiple sclerosis (BICAMS) battery. BMC Neurol. 2017;17:208. https://doi.org/10.1186/s12883-017-0993-0.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Spedo CT, Frndak SE, Marques VD, Foss MP, Pereira DA, Carvalho LDF, et al. Cross-cultural adaptation, reliability, and validity of the BICAMS in Brazil. Clin Neuropsychol. 2015;29:836–46. https://doi.org/10.1080/13854046.2015.1093173.

    Article  PubMed  Google Scholar 

  29. Sheridan LK, Fitzgerald HE, Adams KM, Nigg JT, Martel MM, Puttler LI, et al. Normative Symbol Digit Modalities Test performance in a community-based sample. Arch Clin Neuropsychol. 2006;21:23–8. https://doi.org/10.1016/j.acn.2005.07.003.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

Authors

Contributions

NB: Concept of the study, design of the research, data collection, data analysis, draft of the manuscript. NS: Statistical analysis and interpretation of data. NM: Concept of the study, analysis. AD: Interpretation of data, draft editing. NM: Data collection. MT: Data collection. MJ: Concept of the study, design of the research, draft editing. All authors have read and approved submitted version of the manuscript.

Corresponding author

Correspondence to Nazibrola Botchorishvili.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the S. Khechinashvili University Hospital ethics committee and the Tbilisi State Medical University ethics committee. Informed consent was obtained from every participant. All methods were performed in accordance with the relevant guidelines and regulations.

Consent for publication

Consent for publication was obtained from every participant of the study.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Botchorishvili, N., Shiukashvili, N., Mikeladze, N. et al. Validity and reliability of the Georgian-language brief international cognitive assessment for multiple sclerosis (BICAMS). BMC Neurol 21, 218 (2021). https://doi.org/10.1186/s12883-021-02249-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12883-021-02249-x

Keywords