- Research article
- Open Access
Assessment of cognitive functions in patients with multiple sclerosis applying the normative values of the Rao’s brief repeatable battery in the Portuguese population
BMC Neurology volume 21, Article number: 170 (2021)
The Brief Repeatable Battery of Neuropsychological Tests (BRBN-T) is one of the most sensitive and used measures for detecting cognitive impairment in Multiple Sclerosis (MS).
The aim of this study was to adapt and validate this battery to the Portuguese population of MS patients.
The Portuguese version of the BRBN-T was applied to a stratified control national sample of 326 individuals considering sex, age, educational level and geographic location and also a clinical sample of 115 MS patients from several national hospitals. Through the exploration of its psychometrics properties, the Portuguese BRBN-T norms were produced.
The normative data is presented as a regression-based formula to adjust test scores for gender, education and age, and the results reveal the BRBN-T has the ability to differentiate between MS patients and healthy participant’s cognitive performance.
This study demonstrated in our clinical population a good ability to detect cognitive impairment. Its clearly contributed to reinforcing the neuropsychological assessment in Portugal in patients with MS, by providing a new set of instruments, which can be used in the clinical practice, and in future studies. Moreover, it will allow a rigorous and precise support in relation to neuropsychological assessment for future longitudinal studies and clinical trials.
Multiple Sclerosis (MS) is a chronic inflammatory and demyelinating disorder of the central nervous system that can lead to a variety of clinical manifestations, including to a decline of cognitive abilities . Yet, cognitive symptoms may go unnoticed by patients and remain undetected when performing the routine neurological evaluation, unless specific assessment tools are applied.
Cognitive impairment has been reported to affect between 34 to 65% of MS patients [1,2,3], irrespective of age and gender [4, 5]. It can occur at all stages of the disease since its onset [6, 7], and it tends to progress over time, creating a significant impact on patients´ functioning and quality of life [3, 8,9,10,11].
In 1990 the Cognitive Function Study Group of the American National MS Society published the Brief Repeatable Battery (BRB) , later simplified as the Brief Repeatable Battery of Neuropsychological Tests (BRBN-T) , and through this screening tool a characteristic pattern of cognitive impairment in MS was defined: memory, information processing efficiency, executive functioning, attention and processing speed being the most commonly compromised functions [13, 14]. Other neuropsychological evaluation tests were later designed for MS patients, as the Minimal Assessment Cognitive Function in Multiple sclerosis (MACFIMS) [15, 16], and most recently the Brief International Cognitive assessment Multiple Sclerosis (BICAMS) , the latter having been recently validated for use in the Portuguese population by our research group .
The BRBN-T, the most widely used battery in MS, is made up of five tests that were found to be the most sensitive for detecting cognitive impairment in MS: the Selective Reminding Test (SRT), the 10/36 Spatial Recall Test (SPART), the Symbol Digit Modalities Test (SDMT), the Paced Auditory Serial Addition Test (PASAT) and the Word List Generation test (WLG) . It is noteworthy that cognitive impairment, as measured by BRBN-T, has been associated with long-term progression of MS . Although the importance of this battery is unquestionable, as testified by its usage in many MS centres and by the inclusion of some of its tests in clinical trials, the unavailability of normative values for specific populations limits a thorough utilization in clinical practice.
In this study we aimed to evaluate the cognitive functions of a representative sample of MS patients in several centers of our country with the BRBN-T and to compare the results with normative values obtained in the Portuguese population that have been partially published, namely the PASAT  and SRT tests .
The Portuguese language is the 7th most spoken language in the world with more than 242 million native speakers and therefore, this study is of particular importance since it advances the introduction of MS neuropsychological evaluation within Portuguese speaking countries. Furthermore, to our knowledge, this is the first study addressing the neuropsychological evaluation of Portuguese patients with MS and healthy subjects employing the BRBN-T.
Material and methods
The BRBN-T was applied to a group of 115 MS patients (clinical sample) with MS diagnosed according to the McDonald criteria , and to a group of 326 healthy control subjects that were matched for age, gender and educational status. (HC sample) (Table 1). Inclusion criteria for both samples were: being aged 20 years or older, having Portuguese as the native language, having attended school in Portugal and having finished primary school.
The MS patients were consecutively recruited at MS Clinics from six hospitals located in three different regions of the country: North - Centro Hospitalar Universitário São João (Porto) and Unidade Local de Saúde do Alto Minho (Viana do Castelo); Centre - Centro Hospitalar Universitário de Coimbra (Coimbra) and Centro Hospitalar de Entre o Douro e Vouga (Feira), South: Hospital de Santo António dos Capuchos (Lisboa) and Hospital Beatriz Ângelo (Loures), whereas the HC group was recruited from the community and among relatives and friends of MS patients. Exclusion criteria were current or past neurological disorder other than MS, presence of major psychiatric illness, history of learning disability, history of serious head trauma, presence of alcohol or drug abuse, relapse and/or corticosteroid use within 4 weeks preceding the neuropsychological assessment. The majority of MS patients had the relapsing-remitting subtype (MS RR 92.2%), the mean disease duration was 2.4 years and the median EDSS (Expanded Disability Status) score  was 1.0 (range: 0–6).
The HC sample distribution is representative of the Portuguese population, according to the 2011 Census , and regarding geographical, gender and educational variables. Exclusion criteria were also included for this group: history of neurological disorder, history of serious head injury, presence of a major psychiatric illness, history of alcohol or drug abuse, history of learning disability and of regular usage of antidepressants or anxiolytics. This study sample had been included in previous studies of our group that were conducted with the specific aim of validating individual tests in the Portuguese population, such as in the previously mentioned PASAT  and SRT tests .
All participants volunteered to participate in this study and gave written informed consent in accordance with the Helsinki Declaration. The Portuguese Data Protection Authority and the ethics committee of all hospital centers involved in this study approved the data collection.
Neuropsychological test procedures
An initial demographic interview was conducted that was based on a common script that included a demographic questionnaire, medical history, drinking and drug habits and present health status. Well-trained clinical psychologists performed the neuropsychological assessment, in a standardized way in a fixed order under the same circumstances (e.g. interview during the day in a quiet room).
The BRBN-T  was administered in the following order: Selective Reminding Test (SRT), 10/36 Spatial Recall Test (SPART), Symbol Digit Modalities Test (SDMT), Paced Auditory Serial Addition Test (PASAT), Word List Generation (WLG), Delayed Recall of the SRT, Delayed Recall of SPART. Administration of the test battery lasted from 28 to 35 min.
The Selective Reminding Test (SRT)  evaluates verbal learning and memory using a learning task in a paradigm of multiple trials. The list includes 12 words, which the examiner reads at a rate of one word per 2 seconds and the participant is instructed to recall all 12 words. In every consecutive trial only the words that are missed in the preceding one are given to the participant. After 15 min, the participant is asked to recall the 12-word list. The test was scored according to the rules of Buschke and Fuld (1974) procedures and allows for the distinction between short-term and long-term memory components, examining also the consistency of retrieval from long-term memory. The SRT indexes used in this study were: Total Recall, Long Term Storage (LTS), Consistent Long Term Retrieval (CLTR) and Intrusions. Total Recall consists of the sum of the recalled items in each trial; LTS is defined as any word that is recalled in two consecutive trials, without the patient being reminded of the word between trials, regardless of him/her forgetting it in subsequent trials; if a word is LTS and is consistently recalled in all subsequent trials (but not just the last trial), then it is scored as an CLTR whereas intrusions of extra-list words are also recorded.
The 10/36 Spatial Recall Test (SPART)  assesses visual memory acquisition and delayed recall. It consists of a 6 × 6 checkerboard with 10 checkers randomly placed. The board is presented to the participant for 10 s. After the presentation, he/she attempts to reproduce the original design on an empty board. This process is repeated twice and after 15 min and in a delay recall trial, the subject is asked to recall the design again. The score is the total number of correct responses for the three trials and the delayed recall trial.
The Symbol Digit Modalities Test (SDMT)  examines sustained attention, concentration and processing speed. The participant examines a series of nine meaningless geometric symbols, which are labeled from 1 to 9. Then, during 90 s the participants are instructed to replace the symbols in a row by the corresponding number in a verbal manner. The score is the number of correct substitutions (SDMT).
The Paced Auditory Serial Addition Test (PASAT)  measures sustained attention and information processing speed. In the trial set, a standard series of 61 numbers from 1 to 9 is presented and single digits are shown every 3 s. The participant is instructed to add the number most recently presented to the one immediately preceding it. Thus, the second is added to the first, the third to the second, the fourth to the third, and so on. For example, given the series of numbers 1, 2, 3, 4, the participant must add and provide a verbal response to the digit pairs “1, 2”, “2, 3”, “3, 4”. The PASAT performance is scored manually upon test completion by listening to the participants’ responses and coding the correct responses. The participant is able to complete this test if he provided at least two correct answers in any of the three practice sequences. The test score is the number of correct answers.
The Word List Generation (WLG) [12, 27] is a phonemic verbal fluency test that evaluates the spontaneous production of words and mental flexibility, when given a letter from the alphabet and within a limited amount of time (1 minute). The scoring procedures consist of totaling the correct responses.
We followed the statistical procedure suggested by Diehr and colleagues (2003)  and Amato and colleagues (2006)  to generate the BRBN-T Portuguese norms. Mean values for each BRBN-T test were compared between the MS patients and the healthy controls with the Student’s t-test for independent samples. We also used t-test or one-way ANOVA to test differences between groups regarding their performance on the BRB based on demographic characteristics (gender, age, and educational level).
We analyzed the distribution of the row scores of each BRBN-T test in the two different samples using the Shapiro-Wilk test. For non-normal distributions, we applied an inversed logarithmic transformation before calculating the Z scores . We then transformed these distributions into normally distributed scale scores (Mean = 10; SD = 3).
Stepwise multiple regression analysis was used to screen for the demographic variables (gender, age and educational level) that significantly contribute to predict scale scores in each BRBN-T test in the two different samples. To generate the T-scores corrected for significant demographic variables we performed another multiple regression analysis with these significant variables and saved the unstandardized predicted values for each equation. These values were then rescaled to a T-score (mean = 50; SD = 10) [20, 28].
All statistical analyses were performed using the IBM SPSS version 23.0.
The results of the BRBN-T neuropsychological tests are presented in Table 2. The comparison of MS patients with healthy control participants showed that groups significantly differed on PASAT and WLG. We also investigated whether there were significant differences between groups regarding their performance on BRBN-T tests.
We tested the normality of the distributions and only the distribution of PASAT was not normal in both samples (MS Sample: Shapiro-Wilk = .957; df = 115; p < .001; HC Sample: Shapiro-Wilk = .928; df = 326; p < .001). Because the distribution of the raw scores was negatively skewed, we applied an inversed logarithmic transformation to reduce these skews using the formula: -log10*(61 – raw scores). We then transformed BRBN-T raw scores tests to scale scores for both samples. Results are presented on Table 3 for the MS patients’ sample and on Table 4 for healthy controls participants.
In Table 5 we present the results of stepwise multiple regression analysis for all BRBN-T. We used gender (0 for male and 1 for female), age (number of years), education (number of years of education) and also the quadratic terms of age and educations.
Based on the results of the stepwise multiple regression analysis we performed additional multiple regression analysis with BRBN-T (scale scores as dependent variable and significant demographics variables in each neuropsychological test as predictors) to generate corrected T-scores. Normative formulas for the Portuguese population and MS patients are presented in Table 6.
We adopted the same statistical procedure from a previous study  in order to determine the performance of each BRBN-T. Performance below the 5th percentile of the normative sample on at least two out of five tests of the BRBN-T was considered as criteria for cognitive impairment. Then, using the previously reported criteria of impairment defined by “two or more abnormal tests” , it was found that 30.43% of the MS sample was impaired at baseline.
In the current study we calculated the following scores: accuracy, sensitivity, specificity, area under the ROC curve (AUC), positive likelihood ratio (PLR), negative likelihood ratio NLR, positive predictive value (PPV) and negative predictive value (NPV). All the scores were calculated through the MedCalc statistical software. Accuracy represents the proportion of true positive scores in the studied population, considering both the true positive and the true negative values (Number of correct / positive assessments / Number of all assessments). Sensitivity is the ratio between the number of true positives and the number of total cognitive impaired patients (Number of true positive assessment / Number of all positive assessments). Specificity is the ratio between the number of true negatives and the number cognitive unimpaired patients (Number of true negatives assessment / Number of all negative assessments). The AUC allowed us to evaluate the accuracy of the diagnostic test. Large ROC areas represent more accurate diagnostic tests with values of 0.9 < AUC < 1.0 considered as excellent, 0.8 < AUC < 0.9 very good, 0.7 < AUC < 0.8 good, 0.6 < AUC < 0.7 sufficient, 0.5 < AUC < 0.6 bad, and AUC < 0.5 as not useful. Likelihood ratio is an important measure for ruling-in/out diagnosis. The positive likelihood ratio (PLR) explains how much more likely the positive test is to occur in patients with cognitively impaired from cognitively unimpaired patients with MS. The higher the test score, the more indicative it is of MS with values higher than 10 meaning that the test has a significant contribution to the diagnosis. Negative likelihood ratio (NLR) explains how much less likely is the negative test score to occur in a MS patient than in a healthy participant. Good diagnostic tests have scores lower than 0.1. Finally, PPV represents the ratio between the number of true positives and the number of positive tests, while NPV represents the ratio between the number of true negatives and the number of negative tests.
Results in Table 7 represent the performance of each of the seven BRBN-T tests, considering a sample of 110 patients with MS. Accordingly, there is evidence that SDMT and SRT are the most sensitive tests. Taking into consideration the area under the ROC curve (AUC), SRT has a classification of good (AUC = 0.74) and SDMT of very good (AUC = 0.82). Accordingly, these results support the accuracy of both tests, suggesting their discriminative power to distinguish cognitively impaired from cognitively unimpaired MS patients. If we take into account the positive likelihood ratio, four tests (SRT, RLP, SDMT, and RCLP) have a higher score than 10. Thus, results provide evidence that, for these tests (as expected), it is more likely that the positive test will occur in cognitively impaired than in cognitively unimpaired patients with MS. However, our findings concerning the Accuracy of single test parameters should not be overestimated. Findings from previous research regarding the BRBN-T state that the differentiation of cognitively impaired from cognitively unimpaired patients with MS is not feasible with one screening test alone (such as the SDMT) [30, 31]. Therefore, we advise that cognitive impairment in patients with MS should be assessed with BICAMS or a short version of the BRB to provide more accurate results.
In this multicentre study, we evaluated the cognitive functioning of Portuguese patients with MS and healthy controls with the BRBN-T battery. Both samples were collected in different regions of the country, and should therefore be representative of the entire population. This is the first study carried out in Portugal with the aim of validating the BRBN-T battery and its applicability in patients with MS, which is a relevant task since the prevalence of the disease in Portugal has recently been estimated at around 64.4 per 100,000 inhabitants .
Our study provides both normative data for the A version of the BRBN-T in a Portuguese population and also the gross scores corrected for the relevant demographic factors. This study reinforces the importance of using demographically corrected scores for discriminating between persons with and without cognitive impairment in MS. Considering the great contribution of demographic characteristics to cognitive performance, the literature suggests that regression-based norms do not categorize continuous variables, which tend to increase specificity and efficiency of estimation, allowing the adoption of reduced sample sizes [33, 34].
In order to better comprehend the performance of patients with MS in each BRBN-T test, we compared that performance with the results obtained in a normative sample covering a larger number of subjects, which was an important advantage, as normative studies conducted in other countries included smaller samples and only healthy participants [35, 36]. Boringa and colleagues were the first to investigate and document the impact of demographic factors, such as age, sex and education on the cognitive performance of BRBN-T in healthy participants . In our sample, the mean age (40.3 ± 14) was similar to that reported in the Italian (41.5 ± 9.8) and Dutch (45.8) groups [34, 36]. It is known that age is one of the main factors that influence test scores. In our study and in consistence with others [35, 36], increasing age was associated with poorer performance across the entire battery, except for the verbal fluency test, the WLG. The impact of age on MS patients has also been observed in SPART and SDMT , revealing that older patients presented with increased changes in the capacity of visuo-spatial memory and the speed of information processing. On the other hand, only few studies included young patients with MS and reveal that in early-onset MS population mostly failed in spatial memory and semantic fluency .
Regarding the educational level, we found that compared to other studies [39, 40] we obtained a similar higher average educational level with low standard deviation (12 + 4 vs. 12.3 + 3.6). We found in both groups a strong relationship between these variables, showing that the higher the individual’s level of education, the better the cognitive performance. This relationship was observed in the clinical group, with the exception of the SPART test. Better performance in PASAT 3.0 was observed in our MS patients with a higher level of education like in another research , perhaps because these patients are active professionally and are more likely to be more cognitively trained.
The gender ratio in normal individuals and patients with MS was matched. In a previous study , no gender differences were found, except for WLG, in which women were better than men. In the present study, we found gender to have an impact in the control group in all tests with the exception of CRLT and SDMT. In the clinical sample, differences between genders were found only in the selective memory test, with women performing better than men.
The analysis of our results in the MS population benefited from previous normative data obtained for the PASAT  and SRT  tests. The results show that SDMT and SRT are the most sensitive tests with the greatest discriminative power. This evidence is in line with previous studies  that showed that both tests are very sensitive for detecting cognitive impairment in patients with multiple sclerosis. In terms of validity, both tests measure the cognitive dimensions of verbal memory, concentration and sustained attention, which represent cognitive impairments in patients with MS .
BRBN-T tasks were able to identify cognitive impairment in 30.43% of patients with MS using the impairment criteria defined by two or more abnormal tests. Similar values were reported in other studies, ranging from 34 to 65% [1,2,3, 31]. Our results showed that 46.42% of patients not yet treated with disease-modifying drugs had cognitive impairment, which is in line with the findings of other studies which showed that cognitive symptoms are already present in the early stages of the disease and sometimes even before the onset of clinical symptoms and signs [6, 7]. The assessment of cognitive functioning at the beginning of the disease course has shown to identify individuals with cognitive impairment, as well as to predict future changes, limitations and disease progression. In this sense, the early identification of cognitive changes can lead to more timely and targeted interventions.
This study has some limitations. Firstly, the tests were carried out in a low number of different disease phenotypes; secondly, there was a low level of physical disability with a mean EDSS of 1.0 in the MS group and lastly, the study did not include early-onset MS patients and in both samples there was a low number of patients above 60 years of age.
These data should be extrapolated with cautions for other MS phenotypes, for patients with higher EDSS and early-onset patients and over 60 years of age because these groups are underrepresented in this study leading to bias in the determination of cognitive impairment.
Cognitive impairment has been increasingly investigated in MS patients, not only in the clinical setting as a means to reach a thorough and comprehensive understanding and management of the disease, but also as an important outcome in clinical trials. Early detection of cognitive impairment is particularly important in MS as it is a chronic and progressive neurological disease that usually begins in young age. Besides early treatment with disease-modifying drugs, MS patients with cognitive deficits certainly benefit from other therapeutic strategies, rehabilitation, compensatory measures and lifestyle adjustments to manage those disabling symptoms. To match that purpose, an essential prior stage of the rehabilitation process is the evaluation of cognitive functions using validated instruments in a specific population, as we have carried out in Portugal and herein describe. In addition, the present study enables the comparison with the results obtained in MS populations from other countries.
Availability of data and materials
The datasets used and analysed during the current study are available from the corresponding author on reasonable request.
Remitting-remission Multiple sclerosis
Clinically isolated syndrome
Primary progressive multiple sclerosis
Secondary progressive multiple sclerosis
Brief Repeatable Battery
Brief Repeatable Battery of Neuropsychological tests
Brief International Cognitive Assessment for Multiple Sclerosis
Selective Reminding Test
10/36 Spatial Recall Test
Paced Auditory Serial Addition Test
Symbol Digit Modalities Test
Word List Generation test
Minimal Assessment of Cognitive Function in Multiple Sclerosis
Expanded Disability Status Scale
- IBM SPSS:
Statistical Package for the Social Sciences
Rao SM, Leo GJ, Bernardin L, Unverzagt F. Cognitive impairment in multiple sclerosis. I. Frequency, patterns, and prediction. Neurology. 1991;41(5):685–91. https://doi.org/10.1212/WNL.41.5.685.
Amato MP, Zipoli V, Portaccio E. Multiple sclerosis-related cognitive changes: a review of cross-sectional and longitudinal studies. J Neurol Sci. 2006;245:1–2.
Langdon DW. Cognition in multiple sclerosis. Curr Opin Neurol. 2011;24(3):244–9. https://doi.org/10.1097/WCO.0b013e328346a43b.
Glanz BI, Holand M, Gauthier SA, Amunwa EL, Liptak Z, Houtchens MK, et al. Cognitive dysfunction in patients with clinically isolated syndromes or newly diagnosed multiple sclerosis. Mult Scler. 2007;13(8):1004–10. https://doi.org/10.1177/1352458507077943.
Amato MP, Hakiki B, Goretti B, Rossi F, Stromillo ML, Giorgio A, et al. Association of MRI metrics and cognitive impairment in radiologically isolated syndromes. Neurology. 2012;78(5):309–14. https://doi.org/10.1212/WNL.0b013e31824528c9.
Zipoli V, Goretti B, Hakiki B, Siracusa G, Sorbi S, Portaccio E, et al. Cognitive impairment predicts conversion to multiple sclerosis in clinically isolated syndromes. Mult Scler. 2010;16(1):62–7. https://doi.org/10.1177/1352458509350311.
Amato MP, Ponziani G, Siracusa G, Sorbi S. Cognitive dysfunction in early-onset multiple sclerosis: a reappraisal after 10 years. Arch Neurol. 2001;58(10):1602–6. https://doi.org/10.1001/archneur.58.10.1602.
Kobelt G, Thompson A, Berg J, Gannedahl M, Eriksson J, the MSCOI Study Group, et al. New insights into the burden and costs of multiple sclerosis in Europe. Mult Scler. 2017;23(8):1123–36. https://doi.org/10.1177/1352458517694432.
Sá MJ, Kobelt G, Berg J, Capsa D, Dalén J. European Multiple Sclerosis Platform. New insights into the burden and costs of multiple sclerosis in Europe: Results for Portugal. Multiple Sclerosis. 2017;23(2-suppl):143–54.
Costa D, Sá MJ, Calheiros JM. Social characteristics and quality of life of Portuguese multiple Sclerosis patients. Neurol Ther. 2013;2(1–2):43–56.
Amato MP, Ponziani G, Rossi F, Liedl CL, Stefanile C, Rossi L. Quality of life in multiple sclerosis: the impact of depression, fatigue and disability. Mult Scler. 2001;7(5):340–4. https://doi.org/10.1177/135245850100700511.
Rao SM, Cognitive Function Study Group N. A manual for the brief repeatable battery of neuropsychological tests in multiple sclerosis. New York: National Multiple Sclerosis Society; 1990.
Bobholz AJ, Rao SM. Cognitive dysfunction in multiple sclerosis: a review of recent developments. Curr Opin Neurol. 2003;16(3):283–8. https://doi.org/10.1097/01.wco.0000073928.19076.84.
Lensch E, Matzke M, Petereit HF, et al. Identification and management of cognitive disorders in multiple sclerosis: a consensus approach. J Neurol. 2006;253:S29–31.
Benedict RH, Fischer JS, Archibald CJ, et al. Minimal neuropsychological assessment of MS patients: a consensus approach. Clin Neuropsychol. 2002;16(3):381–97. https://doi.org/10.1076/clin.16.3.381.13859.
Benedict RH, Cookfair D, Gavett R, et al. Validity of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). J Int Neuropsychol Soc. 2006;12(12):549–58.
Langdon DW, Amato MP, Boringa J, Brochet B, Foley F, Fredrikson S, et al. Recommendations for a brief international cognitive assessment for multiple sclerosis (BICAMS). Mult Scler. 2012;18(6):891–8. https://doi.org/10.1177/1352458511431076.
Sousa C, Neves MR, Miranda T, Alegria P, Vale J, Passos AM, et al. Validation of the brief international cognitive assessment for multiple sclerosis (BICAMS) in the Portuguese population with multiple sclerosis. BMC Neurol. 2018;18:172. https://doi.org/10.1186/s12883-018-1175-4.
Moccia M, Lanzillo R, Palldino R, Chu-Mei Chang K, Costabile T, Russo C, et al. Cognitive impairment at diagnosis predicts 10-year multiple sclerosis progression. Mul Scler. 2016;22:659–67. https://doi.org/10.1177/2F1352458515599075.
Sousa C, Neves M, Passos AM, Ferreira A, et al. Paced auditory serial addition test (pasat 3.0 s): demographically corrected norms for the Portuguese population. Appl Neuropsychol. 2017;23:1–7. https://doi.org/10.1080/23279095.2017.1323752.
Neves M, Sousa C, Passos AM, Ferreira A, et al. Verbal Selective Reminding Test (six trial administration) Portuguese normative data. Appl Neuropsychol. 2017;30(6):1–9. https://doi.org/10.1080/23279095.2017.1336712.
Thompson AJ, Banwell BL, Barkhof F, Carroll WM, Coetzee T, Comi G, et al. Diagnosis of multiple sclerosis: 2017 revisions of the McDonald criteria. Lancet Neurol. 2017;17(2). https://doi.org/10.1016/S1474-4422(17)30470-2.
Kurzke JF. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology. 1983;33:1444–52.
Instituto Nacional de Estatística (2011). Portal de estatísticas oficiais (Portuguese Official Statistics). Retrieved from http://www.ine.pt.
Buschke H, Fuld PA. Evaluating storage, retention, and retrieval in disordered memory and learning. Neurology. 1974;24(11):1019–25. https://doi.org/10.1212/WNL.24.11.1019.
Smith A. Symbol digit modalities test: manual. Los Angeles: Western Psychological Services; 1982.
Neves M, Passos AM, Ferreira A, Sousa C, et al. BRBN-T validation: adaptation of the selective reminding test and word list generation. Arquivos Neuropsiquiatria. 2015;73(10):867–72. https://doi.org/10.1590/0004-282X20150134.
Diehr MC, Cherner M, Wolfson TJ, Miller SW, Grant I, Heaton RW. The 50 and 100-item short forms of the paced auditory serial addition task (PASAT): demographically corrected norms and comparisons with the full PASAT in Normal and clinical samples. J Clin Exp Neuropsychol. 2003;25(4):571–85. https://doi.org/10.1076/jcen.25.4.571.13876.
Portaccio E, Stromillo ML, Goretti B, Zipoli V, Siracusa G, Battaglini M, et al. Neuropsychological and MRI measures predict short-term evolution in benign multiple sclerosis. Neurology. 2009;73(7):498–503. https://doi.org/10.1212/WNL.0b013e3181b351fd.
Portaccio E, Goretti B, Zipoli V, Siracusa G, Sorbi S, Amato MP. A short version of Rao’s brief repeatable battery as a screening tool for cognitive impairment in multiple sclerosis. Clin Neuropsychol. 2009;23(2):268–75. https://doi.org/10.1080/13854040801992815.
Hansen S, Muenssinger J, Kronhofmann S, Lautenbacher S, Oschmann P, Keune P. Cognitive screening tools in multiple sclerosis revisited: sensitivity and specificity of a short version of Rao’s brief repeatable battery. BMC Neurol. 2015;15(1):246. https://doi.org/10.1186/s12883-015-0497-8.
Branco M, Alves I, Silva AM, Pinheiro J, Sá MJ, Correia I, et al. The epidemiology of multiple sclerosis in the entre Douro e Vouga region of northern Portugal: a multisource population-based study. BMC Neurol. 2020. https://doi.org/10.1186/s12883-020-01755-8.
McGee S. Simplifying likelihood ratios. J Gen Intern Med. 2002;17(8):647–50. https://doi.org/10.1046/j.1525-1497.2002.10750.x.
Marrie RA, Whitehouse CE, Patel R, et al. Performance of regression-based norms for cognitive functioning of persons with multiple Sclerosis in an independent sample. Front Neul. 2020;11:621010. https://doi.org/10.3389/2Ffneur.2020.621010.
Boringa JB, Lazeron RHC, Reuling IEW, Adèr HJ, Pfennings LEMA, Lindeboom J, et al. The brief repeatable battery of neuropsychological tests: normative values allow application in multiple sclerosis clinical practice. Mult Scler. 2001;7(4):263–7. https://doi.org/10.1177/135245850100700409.
Obradovic D, Petrovic M, Antanasijevic I, Marinkovic J, Stojanovic T, Obradovic S. The brief repeatable battery: psychometrics and normative values with age, education and gender corrections in a Serbian population. Neurol Sci. 2012;33:1369–74. https://doi.org/10.1007/s10072-012-1099-5.
Yoshii F, Takagi S, Nagata E, Hasegawa Y, Kunika N, Maki F, et al. Cognitive impairment of Japanese multiple sclerosis patients: follow-up study using BRB-N assessment tool. J Neurol Sci. 2015;359(2015):323–7. https://doi.org/10.1016/j.jns.2015.11.002.
Falco F, et al. Normative values of the Rao’s brief repeatable battery in an Italian young adolescent population: the influence of age, gender, and education. Neurol Sci. 2019. https://doi.org/10.1007/s10072-019-3712-3.
Scherer P, Baum K, Bauer H, Gohler H. Normalization of the brief repeatable battery of neuropsychological tests (BRBN) for German speaking regions. Application in relapsing remitting and secondary progressive multiple sclerosis patients. Nervenarzt. 2004;75:984–90.
Amato MP, Portaccio E, Zipoli V, Ricchiuti L, De Caro MF, Patti F, et al. The Rao’s brief repeatable battery and Stroop test: normative values with age, education and gender corrections in an Italian population. Mult Scler. 2006;12(6):787–93. https://doi.org/10.1177/1352458506070933.
The authors would like to thank all participants of this study. The neurologists from the MS Clinics of all Hospitals where the Project for validation of the BRBN-T for the Portuguese MS population was conducted, for making the clinical data available: Rui Pedrosa, MD (Hospital de Santo António dos Capuchos, Lisboa); José Vale, MD and Paulo Alegria, MD (Hospital Beatriz Ângelo, Loures); Lívia Sousa, MD (Centro Hospitalar Universitário de Coimbra, Coimbra); Sandra Perdigão, MD (Unidade Local de Saúde do Alto Minho, Viana do Castelo) and José Leal Loureiro, MD, PhD (Centro Hospitalar de Entre o Douro e Vouga, Feira), and the psychologists that collaborated in the collection of cognitive data: Andreia Sá, Fanny Lima, Marta Santos, Telma Miranda.
This work was funded by an unrestricted educational grant from Biogen, which had any role in the study, namely in its design, sample collection, analyses and interpretation of data and in the writing of the manuscript.
Consent for publications
Ethics approval and consent to participate
All patients and all healthy control subjects provided written informed consent to participation in the study in accordance with the Helsinki Declaration. The Portuguese Data Protection Authority had approved data collection beforehand (N° 3313/2013). The Ethics Committees of Centro Hospital Universitário S. João, Porto (CE/63/2012) and all participating hospitals, approved the research project.
MJS has received consulting/speaker fees from Bayer, Biogen, CSL Behring, Merck, Novartis, Roche, Sanofi and Teva.
The other authors have nothing to disclose regarding this study.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Sousa, C., Rigueiro-Neves, M., Passos, A.M. et al. Assessment of cognitive functions in patients with multiple sclerosis applying the normative values of the Rao’s brief repeatable battery in the Portuguese population. BMC Neurol 21, 170 (2021). https://doi.org/10.1186/s12883-021-02193-w
- Multiple sclerosis
- Neuropsychological test
- Brief repeatable battery of neuropsychological tests
- Normative data
- Cognitive assessment