While mental comorbidity is considered common in multiple sclerosis (MS), its impact is poorly defined; methods are needed to support studies of mental comorbidity. We validated and applied administrative case definitions for any mental comorbidities in MS.
Using administrative health data we identified persons with MS and a matched general population cohort. Administrative case definitions for any mental comorbidity, any mood disorder, depression, anxiety, bipolar disorder and schizophrenia were developed and validated against medical records using a a kappa statistic (k). Using these definitions we estimated the prevalence of these comorbidities in the study populations.
Compared to medical records, administrative definitions showed moderate agreement for any mental comorbidity, mood disorders and depression (all k ≥ 0.49), fair agreement for anxiety (k = 0.23) and bipolar disorder (k = 0.30), and near perfect agreement for schizophrenia (k = 1.0). The age-standardized prevalence of all mental comorbidities was higher in the MS than in the general populations: depression (31.7% vs. 20.5%), anxiety (35.6% vs. 29.6%), and bipolar disorder (5.83% vs. 3.45%), except for schizophrenia (0.93% vs. 0.93%).
Administrative data are a valid means of surveillance of mental comorbidity in MS. The prevalence of mental comorbidities, except schizophrenia, is increased in MS compared to the general population.
Although depression and anxiety are considered common in MS [1, 2], population-based prevalence estimates for these conditions are rare. Even fewer prevalence estimates exist for bipolar disorder and schizophrenia in the MS population, and they vary widely [3, 4]. The paucity of population-based studies of mental comorbidity may reflect the challenges of conducting such studies. However, such studies are needed given the impact of mental comorbidity in MS, including lower quality of life and reduced adherence to treatment [5, 6]; and to minimize the biases from using clinic-based samples.
Studies of mental comorbidity could potentially use one of several data sources including medical records review, self-report, interviews, or administrative data. Administrative data are population-based in publicly funded health systems such as Canada and are cost-effective and accessible . Such data are useful for assessing the burden of disease at the population level, including health services use and costs . Mental comorbidities can be assessed in clinical samples using structured diagnostic interviews such as the Composite International Diagnostic Interview (CIDI) although these are time consuming and depend heavily on recall of past episodes . Administrative data have the advantage that they are recorded during an episode and need not be recalled later. Administrative data, however, are collected for health system management and are often inadequately validated [7, 10]. Indeed, few published case definitions for mental comorbidity have been validated, and efforts to develop and validate case definitions for depression have identified poor concordance with the CIDI Short Form , and difficulties distinguishing depression from anxiety .
We aimed to validate administrative case definitions for several mental comorbidities in MS, and to describe their prevalence among persons with MS versus a matched cohort from the general population. We hypothesized that the prevalence of depression, anxiety, bipolar disorder and schizophrenia would be higher in the MS population than in the general population.
We conducted this study in Manitoba, Canada, using anonymized administrative data obtained from Manitoba Health (MH) which provides health care services for more than 98% of Manitoba residents . Since 1984, MH has maintained computerized records of health services claims, which can be linked using a unique personal health identification number (PHIN) identifying the person who received the service. Physician claims include the PHIN, service date, and three-digit International Classification of Disease (ICD)-9-CM code for one physician-assigned diagnosis. Hospitalization records include the PHIN, admission and discharge dates, and up to 16 discharge diagnoses. Before 2004, diagnoses were listed using five-digit ICD-9-CM codes and since 2004 they have been listed using ICD-10-CA codes. Since 1996, the Drug Programs Information Network captures outpatient prescription drug dispensations including date, drug name, and drug identification number for Manitoba residents, regardless of payer. The population registry is updated when an individual migrates into or out of Manitoba, or dies.
Study populations and validation cohort
Using data from 1984 to 2006, we identified all Manitobans with MS using a previously validated administrative case definition . We identified up to 5 controls from the general population for each MS case, matched on sex, year of birth and region of residence (postal code), and excluding anyone with an ICD9/10-code for any demyelinating disease as previously described . As described in detail previously, the medical records of 430 persons with MS were reviewed by a trained abstractor using a standardized data collection form [14, 15]. Using each participant’s PHIN, these clinical data were linked with the administrative databases.
Administrative case definitions
We aimed to develop case definitions for depression, anxiety, bipolar disorder and schizophrenia using established approaches . Developing case definitions for mental comorbidity raised challenges. Although hospital claims provide 5-digit ICD codes, physician claims in Manitoba have only three digits, reducing the specificity of coding. For example, at the 3-digit level, the same code (296) describes bipolar I disorder, most recent episodic manic (296.4) and major depressive disorder recurrent episode (296.3). Therefore, we initially created an ‘omnibus’ definition for mental comorbidity to capture persons with any of the mental comorbidities of interest, followed by an ‘any mood or anxiety disorders’ definition which captured depressive disorders, anxiety disorders and bipolar disorder. Finally, we developed case definitions for individual mental comorbidities. We identified ICD-9/10 codes for the individual comorbidities and the combination definitions (Additional file 1: Table S1).
While incorporating prescription claims might improve specificity of the case definitions, many medications used for mental comorbidities are used off-label for other purposes, particularly in MS . To determine which medications to include in our definitions we compiled a list of antidepressants (N06A), anti-anxiolytics (N05B), anti-psychotics (N05A), combination agents (N06C), and mood-stabilizing agents including anticonvulsants (N05AN01, N03AG01, N03AX09, N03AX12) available in Canada based on the Anatomic Therapeutic Chemical Classification System . A multidisciplinary panel comprised of two pharmacists (CE, MM), a psychiatrist (SBP), a neuropsychologist (JDF), an epidemiologist (SW), and a neurologist (RAM) independently reviewed this list and indicated whether each medication was used (i) for each of the mental comorbidities of interest; (ii) other on-label uses including the specific condition; and (iii) off-label uses, especially for MS. To meet our goal of enhancing the specificity of case definitions with prescription claims, we selected medications considered to be moderately specific for the comorbidities of interest (Additional file 1: Table S1), meaning that the medication could not be used off-label for MS, and could not be used on or off-label for conditions other than mental comorbidities unless an ICD code could easily exclude the condition (e.g. epilepsy).
We developed several case definitions for each comorbidity by varying the number of physician, hospital and prescription claims required and the years of data required to classify a person as affected. Using our validation cohort, we compared the classification of mental comorbidity according to the administrative case definitions versus diagnoses based on medical records review for the 1 to 5 year periods ending in fiscal year 2005/06. We report a kappa (κ) statistic for agreement between administrative and medical records data , and the 95% confidence interval (CI) based on the normal approximation to the binomial distribution. We interpreted κ as follows: slight (0-0.20), fair (0.21-0.40), moderate (0.41-0.60), substantial (0.61-0.80), and almost perfect agreement (0.81-1.0) . Kappa indicates the proportion of agreement beyond chance and is calculated as (observed agreement – chance agreement) ÷ (1 – chance agreement). Kappa is affected by the prevalence of the condition of interest, however, such that if prevalence is very high or very low, chance agreement is high and kappa is reduced with a maximum value of less than one . Bias refers to the extent to which the raters (i.e. administrative data versus medical records data) disagree on the proportion of positive (affected) cases; greater bias, meaning a greater difference in the proportion (prevalence) of positive rates, is paradoxically associated with higher kappas. Because both prevalence and bias influence the magnitude of kappa, we also calculated the prevalence index, bias index, and the prevalence and bias-adjusted kappa for our preferred case definitions . We estimated that a sample of 400 persons can detect a k of ≥0.60 (substantial agreement) for comorbidities with ≥4% prevalence if the null hypothesis is k = 0.41, α = 0.05, and β = 0.20. Given the anticipated higher k (≥ 0.70) for bipolar disorder and schizophrenia we estimated that our sample would provide adequate precision for estimates of k with prevalences ≥ 3% for these disorders.
We computed sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for administrative definitions versus the “gold standard” of medical records review to identify whether an algorithm would be vulnerable to over- or under-estimating the prevalence of the comorbidity. Further, we explored the impact of these misclassifications on epidemiologic estimates by generating a range of ‘true’ prevalence estimates, and calculating the expected value of observed prevalence based on the sensitivity and specificity for the case definition of interest [7, 21].
For each comorbidity, we report the prevalence in the MS and matched cohorts. Once a person met the case definition, he or she was considered affected in all subsequent years while alive and resident in Manitoba. We estimated the point prevalence of the comorbidity on October 1, 2005 using mid-year population figures for denominators and also calculated prevalence ratios (PR) by dividing the prevalence in those with MS by that of the control group. To enhance comparability with other study populations, we age-standardized the findings to the 2001 Canadian population, and calculated 95% CIs using the exact binomial distribution. Using Poisson regression we calculated PRs and 95% CIs comparing the MS and general populations adjusting for age group (20-44, 45-59, ≥ 60 years) and sex. Cell sizes ≤ 5 were suppressed.
The University of Manitoba Health Research Ethics Board and the Manitoba Health Information Privacy Committee approved the study and data access. Participants in the validation cohort provided written informed consent. Statistical analyses were performed using SAS V9.2 (SAS Institute Inc., Cary NC).
The MS population included 4192 persons and the matched cohort included 20,940 persons (71.7% female). In the validation cohort most participants were White (91.6%), women (77.0%), with a mean (standard deviation) age at MS symptom onset of 33.2 (11.1) years . Mental comorbidity was common in the validation cohort, with 29.7% having any mental comorbidity, 29.2% having a mood or anxiety disorder, 27.5% having a depressive disorder, 6.5% having an anxiety disorder, 0.98% having bipolar disorder and 0.49% having schizophrenia.
Omnibus definition: any mental comorbidity
Agreement between the administrative case definitions (labeled A to Z) and medical records ranged from slight to moderate (k = 0.11 to 0.51, Additional file 2: Table S2). The definition with the highest level of agreement required ≥ 1 hospital or ≥ 5 physician claims or ≥ 1 physician claim and ≥ 4 prescription claims in 2 years (definition ‘Q’, k = 0.51); it had a sensitivity of 63% and a specificity of 86.8%. Using definition ‘Q’, the age-standardized prevalence of any mental comorbidity in 2005 was 33.9% (95% CI 32.0-35.9%) in the MS population and 21.9% (95% CI: 21.2-22.6%) in the general population (PR 1.55; 95% CI: 1.36-1.76). In both populations, the peak prevalence occurred in persons aged 45-59 years (Figure 1).
Any mood or anxiety disorder
Agreement between the case definitions (labeled A to Y) and medical records ranged from slight to moderate (k = 0.10 to 0.50, Additional file 3: Table S3). The highest level of agreement (k = 0.50) was achieved by several similar definitions, including definition ‘O’ (see below); all used prescription claims. Using definition ‘O’, (≥ 1 hospital or ≥ 5 physician or [≥ 1 physician AND ≥ 4 prescription] claims) the age-standardized prevalence of any mood or anxiety disorder in 2005 was 34.8% (95% CI 32.8-36.8%) in the MS population and 22.0% (95% CI: 21.3-22.7%) in the general population (PR 1.58; 95% CI: 1.39-1.80). The similarity of the estimates of any mood or anxiety disorder to those for any mental comorbidity reflects the predominance of mood and anxiety disorders. In both populations, the peak prevalence occurred in persons aged 45-59 years (Figure 1).
Agreement between the case definitions (labeled A to Y, Additional file 4: Table S4) and medical records ranged from slight to moderate (k = 0.11 to 0.49). The highest level of agreement for a definition which did not use prescription claims was moderate (k = 0.44), and used ≥ 1 hospital or ≥4 physician claims in 5 years. Among all case definitions the highest level of agreement (k = 0.49) was achieved by two similar definitions (‘G’ and ‘P’); both used prescription claims. Using definition ‘P’, which required (≥ 1 hospital or ≥ 5 physician claims) or (≥ 1 physician claim and ≥ 7 prescription claims) in 2 years, the age-standardized prevalence of depression in 2005 was 31.7% (95% CI 29.8-33.5%) in the MS population and 20.5% (95% CI: 19.8-21.2%) in the general population (PR 1.60; 95% CI: 1.41-1.82). In both populations, the peak prevalence occurred in persons aged 45-59 years (Figure 1).
Agreement between the administrative case definitions and medical records ranged from slight to fair (k = 0.02 to 0.23, Additional file 5: Table S5). The highest level of agreement for any definition was fair (definition ‘N’, k = 0.23), and required (≥ 1 hospital or ≥ 2 physician claims) or (≥ 1 physician and ≥ 2 prescription claims) in 2 years. Using definition ‘N’, the age-standardized prevalence of anxiety in 2005 was 35.6% (95% CI 33.7-37.7%) in the MS population and 29.6% (95% CI: 28.8-30.5%) in the general population (PR 1.24; 95% CI: 1.12-1.38). In both populations, the peak prevalence occurred in persons aged 45-59 years (Figure 1).
Agreement between the administrative case definitions (labeled A to U) and medical records ranged from slight to moderate (k = 0.20 to 0.42, Additional file 6: Table S6). Several definitions had the highest sensitivity of 75% with specificities of 97% or higher. Agreement for these definitions varied slightly, but confidence intervals overlapped. Using definition ‘W’, which required (≥ 1 hospital or ≥ 3 physician claims) or (≥ 1 physician and ≥ 3 prescription claims) in 5 years (k = 0.34), the age-standardized prevalence of bipolar disorder in 2005 was 5.83% (95% CI: 5.01-6.65%) in the MS population and 3.45% (95% CI: 3.17-3.73%) in the general population (PR 1.70; 95% CI: 1.55-1.87). Although the affected number of individuals was small, the prevalence of bipolar disorder was similar across age groups (Figure 1).
Agreement between all of the case definitions (labeled A to O) and medical records ranged from substantial to perfect (k = 0.67 to 1.0, Additional file 7: Table S7). Among case definitions with perfect agreement, the simplest definition with the highest sensitivity (100%) and specificity (99%) required ≥ 1 hospital or ≥ 2 physician claims in 2 years (definition ‘G’). Applying definition ‘G’, the age-standardized prevalence of schizophrenia in 2005 was 0.93% (95% CI: 0.61-1.26%) in the MS population and 0.93% (0.78-1.09%) in the general population (PR 0.95; 95% CI: 0.60-1.51). Although the small number of individuals affected requires cautious interpretation, the prevalence of schizophrenia was similar across age groups (Figure 1).
Table 1 shows the sensitivity, specificity, kappa, prevalence index and bias index for case definitions for which we presented prevalence estimates. The definitions for bipolar disorder, anxiety and schizophrenia have high values for the prevalence index indicating that kappa values will be reduced as compared to populations in which these conditions are more prevalent. Except for anxiety, the bias index was minimal. After adjustment for prevalence and bias, all κ increased except for schizophrenia which was already 1.0. Graphical analysis of misclassification bias suggested that the case definitions perform reasonably well in the expected range of prevalence for mental comorbidity in MS (Additional file 8: Figure S1) [1, 2, 22–28].
Impact of prevalence and bias on agreement (kappa) between administrative case definitions and medical records
Comparison to medical records
No. years of data
No. and type of claimsa
Sens (95% CI)
Spec (95% CI)
Observed kappa (95% CI)
Omnibus (Any Mental)
≥ 1 H or ≥ 5 P OR (≥ 1 P AND ≥ 4 Rx)
63.5 (54.0, 72.2)
86.8 (82.1, 90.6)
0.51 (0.41, 0.60)
Any Mood or anxiety disorder
≥ 1 H or ≥ 5 P OR (≥ 1 P AND ≥ 4 Rx)
62.8 (53.2, 71.7)
86.9 (82.2, 90.6)
0.50 (0.41, 0.60)
≥ 1 H or ≥ 5 P OR (≥ 1 P AND ≥ 7 Rx)
62.2 (52.4, 71.2)
86.7 (82.2, 90.4)
0.49 (0.40, 0.59)
≥ 1 H or ≥ 3P OR (≥ 1 P AND ≥ 3 Rx)
75.0 (19.4, 99.4)
97.5 (95.5, 99.4)
0.30 (0.036, 0.57)
≥ 1 H or ≥ 2 P OR (≥ 1 P AND ≥ 2 Rx)
42.3 (23.3, 63.1)
82.2 (78.0, 85.9)
0.23 (0.022, 0.23)
≥ 1 H or ≥ 2 P
1.0 (15.8, 100)
0.99 (98.6, 100)
1.0 (0, 1.0)
Few population-based studies have evaluated the prevalence of mental comorbidity in MS . To facilitate such studies, we validated case definitions for mental comorbidities based on hospital, physician and prescription claims. Our case definitions showed almost perfect agreement versus medical records for schizophrenia, and moderate agreement for any mental comorbidity, any mood or anxiety disorder, and depression. The case definition for bipolar disorder had lower agreement, but acceptable sensitivity of 75% and high specificity of 97.5%. The case definition for anxiety had the lowest agreement but a specificity of 82%. Further, persons with MS were at increased risk of depression, anxiety and bipolar disorder, but not schizophrenia when compared to the general population.
Previous validation studies of administrative case definitions for mental comorbidity were often disappointing, and have highlighted the challenges of distinguishing depression from anxiety when using 3-digit ICD codes [11, 12]. In a Manitoba study, agreement was only fair (k = 0.26) between surveys and administrative definitions for depression which used hospital, physician and prescription claims  although this lower agreement may reflected their use of survey data and a broader range of prescription claims than in our study. Among persons newly treated with antidepressants in Saskatchewan, Canada, agreement between depression identified on physician claims and medical records was moderate (k = 0.54), similar to our findings . We could not identify any published, validated case definitions for anxiety. Thus these validated case definitions augment the ability to conduct population-level surveillance of depression and anxiety. Despite challenges in developing case definitions for depressive and anxiety disorders sensitive and specific definitions were available for bipolar disorder (sensitivity 75%, specificity 97%). Among 225 Americans, inpatient diagnoses of bipolar disorder, outpatient diagnoses of bipolar disorder by mental health providers, and outpatient diagnoses of bipolar disorder by non-mental health providers that were accompanied by a prescription for lithium, carbamazepine or valproate, had false positive rates below 10% . However, we found that bipolar disorder could be identified without such claims. Consistently, administrative case definitions for schizophrenia have performed well, with agreement between hospital claims for schizophrenia and medical records of 93.9-100% [31, 32]. In American Medicaid data, the case definition that we validated of either one hospital or two physician claims for schizophrenia in two years identified only 6% false positives (k = 0.76) .Collectively, this suggests that administrative data can accurately identify bipolar disorder and schizophrenia in the MS and general populations.
Our approach is informative for researchers wishing to study mental comorbidity in other chronic neurologic diseases, which share the potential problem of under-reporting of comorbidities due to coding biases . While prescription claims may add sensitivity, their use for mental comorbidity is challenging in chronic neurologic diseases because of the frequent off-label use of therapies. By restricting the breadth of prescriptions used, and using them in combination with a physician claim for mental comorbidity we successfully created valid case definitions.
Prior studies suggest that the annual prevalence of depression in MS is up to 14% with a lifetime prevalence of up to 50% , and that anxiety disorders affect more than 30% of persons with MS [2, 22]. Our crude prevalence estimates of 33% for depression and 37% for anxiety based on two years of administrative data are consistent with those observations. The age-standardized prevalence of bipolar disorder in the MS population was 5.83% (crude prevalence 6.3%), 70% higher than in the general population. Studies in hospital or clinic populations suggested that bipolar disorder affects 0.30% to 13% of the MS population [24–28]. The only one of these studies that used a true general population control group reported that hospitalized persons with MS had bipolar disorder twice as often as hospitalized controls (1.97% vs. 0.92%) . Since that study was limited to hospitalized persons, the prevalence of bipolar disorder may have been underestimated, although the increased risk of bipolar disorder in MS was similar to our findings.
The prevalence of schizophrenia was the same in the MS and general populations (0.93%). Two population-based studies, both using administrative data, evaluated the prevalence of psychosis, not limited to schizophrenia. In Taiwan, psychosis affected 7.5% of the MS population and 2.0% of the general population (odds ratio 4.0) . In Alberta, Canada only 0.8% of MS patients had non-organic pyschoses including schizophrenia-spectrum disorders, and other non-organic psychoses, but this was more than in the general population . Our findings of an absence of an increased prevalence of schizophrenia in our MS population suggest a lack of increased risk which may reflect differences in the psychotic disorders studied (all versus schizophrenia alone), as well as the small number of persons with schizophrenia.
Medical records review for the validation cohort did not involve all records of all health care providers over the lifetime of study participants. For practical reasons we also compared medical records to administrative data for the 1 to 5 year period ending in fiscal year 2005/06, rather than from 1984 onward. Both factors may have reduced agreement between the data sources. Like medical records, administrative data only allow us to identify mental comorbidities for which the patient has been treated; undiagnosed mental comorbidity cannot be captured without a direct patient assessment. This study had several strengths, however. We validated the case definitions in a population similar to the one in which it was applied, the design was population-based, we used matched general population controls, and we used multiple types of administrative data.
Our findings suggest that administrative data can be used for surveillance for mental comorbidities in MS, and should facilitate studies of the impact of mental comorbidity on health outcomes captured by administrative data such as health care utilization. Our findings also provide population-based data emphasizing the increased prevalence of a range of mood and anxiety disorders in MS.
Composite International Diagnostic Interview
International Classification of Disease
Negative predictive value
Personal Health Identification Number
Positive predictive value
Prevalence ratio. CIHR Team in the Epidemiology and Impact of Comorbidity on Multiple Sclerosis includes. Ruth Ann Marrie (University of Manitoba), Bo Nancy Yu (University of Manitoba), Stella Leung (University of Manitoba), Lawrence Elliott (University of Manitoba), Patricia Caetano (University of Manitoba), James F Blanchard (University of Manitoba), Lawrence W. Svenson (University of Alberta), Joanne Profetto-McGrath (University of Alberta), Sharon Warren (University of Alberta), Christina Wolfson (McGill University), Nathalie Jette (University of Calgary), Scott B Patten (University of Calgary), Charity Evans (University of Saskatchewan), Helen Tremlett (University of British Columbia), John Fisk (Dalhousie University), Virender Bhan (Dalhousie University), Michelle Ploughman (Memorial University)
The results and conclusions presented are those of the authors. No official endorsement by Manitoba Health is intended or should be inferred. The authors thank Melanie McLeod, PharmD for her assistance in reviewing pharmacotherapies for this project.
This study was funded by operating grants and a Don Paty Career Development Award from the Multiple Sclerosis Society of Canada, the Manitoba Health Research Council, the Canadian Institutes for Health Research, and the Rx & D Health Research Foundation. The sponsors had no role in the design, in the collection, analysis or interpretation of the data; in the writing of the manuscript; or in the decision to submit the manuscript for publication.
Department of Internal Medicine, University of Manitoba
Department of Community Health Sciences, University of Manitoba
Health Sciences Centre
Departments of Psychiatry and Medicine, Dalhousie University
Faculty of Rehabilitation Medicine, University of Alberta
College of Pharmacy and Nutrition, University of Saskatchewan
Department of Epidemiology and Biostatistics and Occupational Health, McGill University
Research Institute of the McGill University Health Centre
Department of Community Health Sciences, University of Calgary
School of Public Health, University of Alberta
Surveillance and Assessment, Alberta Health
Department of Medicine (Neurology), University of British Columbia
Goldman Consensus Group: The Goldman Consensus statement on depression in multiple sclerosis.Mult Scler 2005, 11:328–337.View Article
Korostil M, Feinstein A: Anxiety disorders and their clinical correlates in multiple sclerosis patients.Mult Scler 2007, 13:67–72.PubMedView Article
Rodgers J, Bland R: Psychiatric manifestations of multiple sclerosis: a review.Can J Psychiatry 1996, 41:441–445.PubMed
Patten SB, Svenson LW, Metz LM: Psychotic disorders in MS: Population-based evidence of an association.Neurology 2005,65(7):1123–1125.PubMedView Article
Marrie RA, Horwitz R, Cutter G, Tyry T: Cumulative impact of comorbidity on quality of life in MS.Acta Neurol Scand 2012,125(3):180–186.PubMedView Article
Mohr DC, Goodkin DE, Likosky W, Gatto N, Baumann KA, Rudick RA: Treatment of depression improves adherence to interferon beta-1b therapy for multiple sclerosis.Arch Neurol 1997, 54:531–533.PubMedView Article
Patten S: Integrating data from clinical and administrative databases in pharmacoepidemiological research.Can J Clin Pharmacol 1998,5(2):92–97.
Asche CV, Singer ME, Jhaveri M, Chung H, Miller A: All-cause health care utilization and costs associated with newly diagnosed multiple sclerosis in the United States.J Manag Care Pharm 2010,16(9):703–712.PubMed
Haro JM, Arbabzadeh-Bouchez S, Brugha TS, De Girolamo G, Guyer ME, Jin R, Lepine JP, Mazzi F, Reneses B, Vilagut G: Concordance of the composite international diagnostic interview version 3.0 (CIDI 3.0) With standardized clinical assessments in the WHO world mental health surveys.Int J Methods Psychiatr Res 2006,15(4):167–180.PubMedView Article
Tricco AC, Pham B, Rawson NSB: Manitoba and Saskatchewan administrative health care utilization databases are used differently to answer epidemiologic research questions.J Clin Epidemiol 2008,61(2):192–197. e112PubMedView Article
Martens P, Fransoo R, McKeen N, Burland E, Jebamani L, Burchill C, Coster CD, Ekuma O, Prior H, Chateau D: Patterns of regional mental illness disorder diagnoses and service use in Manitoba: a population-based study. Winnipeg: Manitoba Centre for Health Policy, University of Manitoba; 2004.
Kisely S, Lin E, Gilbert C, Smith M, Campbell L-A, Vasiliadis H-M: Use of administrative data for the surveillance of mood and anxiety disorders.Aust N Z J Psychiatry 2009,43(12):1118–1125.PubMedView Article
Branch HIM: Population report. Winnipeg, Manitoba: Manitoba Health and Healthy Living; 2008.
Marrie RA, Yu N, Blanchard JF, Leung S, Elliott L: The rising prevalence and changing age distribution of multiple sclerosis in Manitoba.Neurology 2010,74(6):465–471.PubMedView Article
Horton M, Rudick RA, Hara-Cleaver C, Marrie RA: Validation of a self-report comorbidity questionnaire for multiple sclerosis.Neuroepidemiology 2010,35(2):83–90.PubMedView Article
Marrie R, Yu B, Leung S, Elliott L, Caetano P, Warren S, Wolfson C, Patten S, Svenson L, Tremlett H: Rising prevalence of vascular comorbidities in MS: validation of administrative definitions for diabetes, hypertension, hyperlipidemia.Multiple Sclerosis Journal 2012.
Landis JR, Koch GG: The measurement of observer agreement for categorical data.Biometrics 1977, 33:159–174.PubMedView Article
Sim J, Wright CC: The kappa statistic in reliability studies: use, interpretation, and sample size requirements.Phys Ther 2005, 85:257–268.PubMed
Kleinbaum D, Kupper L, Morgenstern H: Information bias. In Epidemiologic research: principles and quantitative methods. New York: Van Nostrand Reinhold; 1982:220–241.
Dahl O-P, Stordal E, Lydersen S, Midgard R: Anxiety and depression in multiple sclerosis. A comparative population-based study in nord-trondelag county, norway.Mult Scler 2009,15(12):1495–1501.PubMedView Article
Wood B, van der Mei I, Ponsonby A-L, Pittas F, Quinn S, Dwyer T, Lucas R, Taylor B: Prevalence and concurrence of anxiety, depression and fatigue over time in multiple sclerosis.Multiple Sclerosis Journal 2012. Epub ahead of print
Edwards LJ, Constantinescu CS: A prospective study of conditions associated with multiple sclerosis in a cohort of 658 consecutive outpatients attending a multiple sclerosis clinic.Mult Scler 2004, 10:575–581.PubMedView Article
Fisk J, Morehouse SA, Brown MG, Skedgel C, Murray TJ: Hospital-based psychiatric service utilization and morbidity in multiple sclerosis.Can J Neurol Sci 1998, 25:230–235.PubMed
Joffe RT, Lippert GP, Gray TA, Sawa G, Horvath Z: Mood disorder and multiple sclerosis.Arch Neurol 1987, 44:376–378.PubMedView Article
Jacobs LD, Wende KE, Brownscheidle CM, Apatoff B, Coyle PK, Goodman A, Gottesman MH, Granger CV, Greenberg SJ, Herbert J: A profile of multiple sclerosis: the New york state multiple sclerosis consortium.Mult Scler 1999,5(5):369–376.PubMed
Marrie RA, Horwitz RI, Cutter G, Tyry T, Campagnolo D, Vollmer T: The burden of mental comorbidity in multiple sclerosis: Frequent, underdiagnosed, and under-treated.Mult Scler 2009,15(3):385–392.PubMedView Article
West SL, Richter A, Melfi CA, McNutt M, Nennstiel ME, Mauskopf JA: Assessing the Saskatchewan database for outcomes research studies of depression and its treatment.J Clin Epidemiol 2000,53(8):823–831.PubMedView Article
Unützer J, Simon G, Pabiniak C, Bond K, Katon W: The use of administrative data to assess quality of care for bipolar disorder in a large staff model HMO.Gen Hosp Psychiatry 2000,22(1):1–10.PubMedView Article
Rawson NS, Malcolm E, D'Arcy C: Reliability of the recording of schizophrenia and depressive disorder in the Saskatchewan health care datafiles.Soc Psychiatry Psychiatr Epidemiol 1997,32(4):191–199.PubMedView Article
Kashner TM: Agreement between administrative files and written medical records: a case of the Department of Veterans Affairs.Med Care 1998,36(9):1324–1336.PubMedView Article
Lurie N, Popkin M, Dysken M, Moscovice I, Finch M: Accuracy of diagnoses of schizophrenia in Medicaid claims.Hosp Community Psychiatry 1992,43(1):69–71.PubMed
Elixhauser A, Steiner C, Harris DR, Coffey RM: Comorbidity measures for use with administrative data.Med Care 1998, 36:8–27.PubMedView Article
Kang J-H, Chen Y-H, Lin H-C: Comorbidities amongst patients with multiple sclerosis: a population-based controlled study.Eur J Neurol 2010,17(9):1215–1219.PubMedView Article
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.