Process for developing rehabilitation practice recommendations for individuals with traumatic brain injury
© The Author(s). 2017
Received: 18 July 2016
Accepted: 3 March 2017
Published: 20 March 2017
Attempts at measuring quality of rehabilitation care are hampered by a gap in knowledge translation of evidence-based approaches and lack of consensus on best practices. However, adoption of evidence-based best practices is needed to minimize variations and improve quality of care. Therefore, the objective of this project was to describe a process for assessing the quality of evidence of clinical practices in traumatic brain injury (TBI) rehabilitative care.
A multidisciplinary team of clinicians developed discipline-specific clinical questions using the Population, Intervention, Control, Outcome process. A systematic review of the literature was conducted for each question using Pubmed, CINAHL, PsychInfo, and Allied Health Evidence databases. Team members assessed the quality, level, and applicability of evidence utilizing a modified Oxford scale, the Agency for Healthcare Research and Quality Methods Guide, and a modified version of the Grading of Recommendations, Assessment, Development, and Evaluation scale.
Draft recommendations for best-practice were formulated and shared with a Delphi panel of clinical representatives and stakeholders to obtain consensus.
Evidence-based practice guidelines are essential to improve the quality of TBI rehabilitation care. By using a modified quality of evidence assessment tool, we established a process to gain consensus on practice recommendations for individuals with TBI undergoing rehabilitation.
KeywordsTraumatic brain injury Brain injury Rehabilitation Delphi Clinical practice guideline Evidence based medicine
For patients who suffer significant neurologic and functional deficits due to traumatic brain injury (TBI), inpatient rehabilitation centers provide comprehensive post-injury care that has been shown to improve functional outcomes and successful reintegration into the community . The structures and processes of care during inpatient rehabilitation can highly influence patient outcomes [2, 3]. However, there is a growing recognition that there is variability in application of evidence-based guidelines, demonstrating a need for a standard treatment approach . For example, amongst TBI Model System (TBIMS) rehabilitation centers, which provide a state-of-the-art multidisciplinary system of rehabilitation care , significant differences in risk-adjusted functional outcomes of TBI patients have been observed [5–7]. In the Traumatic Brain Injury-Practice Based Evidence study, it was determined that variations in care among TBIMS centers are due to differences in hospital characteristics, patient characteristics, and the experience of the clinicians [6–8]. Differential patient outcomes across inpatient TBIMS centers may also relate to processes of care (e.g., goal and treatment planning, selection of therapeutic interventions) [5, 6, 9]. These variations in care may result from suboptimal adoption of evidence-based best practices in routine clinical care, with a little over half of adult patients in the U.S. receiving recommended preventative, acute, and chronic disease care [10, 11]. Similar gaps in the acute management of patients at Level 1 trauma centers also exists for people following TBI [11, 12]. Although it has been shown that improved compliance with recommended care is associated with improved outcomes following TBI in the trauma setting, compliance with the limited evidence-based guidelines available in the rehabilitation milieu remains suboptimal [11, 12].
To further compound lack of a uniform approach to rehabilitation, the current literature supporting care practices in TBI rehabilitation does not allow for the identification of evidence-based care guidelines for optimal dose or intensity of therapy, the ideal timing of therapy in the recovery process, or the necessary modifications for subpopulations. This is due to variability in care practices and a lack of rigorously executed research including randomized controlled trials [6, 9, 13]. Thus, there is a critical need to resolve the gaps that exist in generating TBI clinical care guidelines. In order to address this gap, we formed a multidisciplinary group of clinicians and researchers to identify and evaluate evidence-based best practices for patients with TBI undergoing rehabilitation. The manuscript describes the process we have developed, so that it may be replicated by other groups as practitioners and researchers in the field of TBI continue to strengthen the knowledge base.
Methods – Fig. 1
Develop PICO questions
A team of research personnel and multidisciplinary clinicians who specialize in treating patients with TBI was established including representatives from the following disciplines: physiatry, neuropsychology, physical therapy, occupational therapy, speech language pathology, and therapeutic recreation. Clinicians created a list of keywords specific to their discipline based on common clinical problems or symptoms that they treat in their day-to-day practices. For example, key words chosen by physical therapists included hemiparesis, spasticity, balance, and ataxia. Research personnel then conducted a preliminary literature search using the keywords combined with “traumatic brain injury” to identify existing literature. Based on the initial evidence collected, clinicians were then asked to form research questions in PICO format (Population, Intervention, Comparison, Outcome) related to their clinical discipline and TBI rehabilitation interests (e.g., “In patients with traumatic brain injury, does prolonged positioning such as casting versus other therapies improve spasticity?”). The goal of this exercise was to generate a list of PICO questions in order to evaluate current literature and develop evidence-based recommendations.
We conducted a systematic literature review for each individual PICO question using search terms identified by the clinician in conjunction with the corresponding “brain injury” medical subject heading. PubMed, CINAHL, PSYCInfo, and Allied Health Evidence databases were utilized because they were projected to include the most content related to TBI rehabilitation research across clinical disciplines. Inclusion criteria for articles were limited to clinical trials, randomized controlled trials, comparative studies, observational studies, and case-series. Meta-analyses were also reviewed for relevant papers. Articles were included if they pertained to adults (≥18 years of age) with TBI and were published in English from January 1, 1987 – current date. Additionally, studies with a patient population of mild through severe and acute and chronic TBIs were considered. Relevant articles cited within manuscripts were also reviewed to find additional evidence to support the PICO question. Finally, articles were collated and sent to the clinician to review for applicability.
Quality of evidence
After identifying relevant literature for PICO questions, the next step was to assess the quality of evidence for the purpose of developing clinical practice recommendations . Research personnel were trained to use the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) methodology to evaluate the quality of studies pertaining to each PICO question . GRADE is a transparent system for rating quality of evidence of systematic reviews and journal articles and grading the strength of recommendations of guidelines . However, the group found it difficult to apply GRADE methodology to evaluate the level and quality of evidence of rehabilitation data. Specifically, the team found that the TBI rehabilitation research identified through the systematic review did not meet GRADE criteria because of the variations in design of the clinical trials, variation in outcome measures, and/or heterogeneous patient populations (i.e., other acquired brain injury mixed with TBI) and care settings . As a result, GRADE criteria such as inconsistency, indirectness of evidence, imprecision, effect, and dose–response relationship were difficult to assess.
Oxford Centre for Evidence-Based Medicine 2011 Levels of Evidence 
Systematic review of randomized trials or n-of-1 trials
Randomized trial or observational study with dramatic effect including crossover studies
Non-randomized controlled cohort/follow-up study
Case-series, case–control, or historically controlled studies
Assessment of Study Applicability based on Agency for Healthcare Research and Quality Methods
Sample is representative of the entire traumatic brain injury (TBI) population or the results are applicable to the entire TBI population
Sample is representative of a relevant subgroup of the target TBI population (i.e., patients <1 year post-injury, patients <65 years of age, etc.)
Sample is only representative of a narrow subgroup of the target TBI population and not well generalizable to other subgroups
Quality Assessment based on Grading of Recommendations, Assessment, Development and Evaluation (GRADE) – Modified
Good (low risk of bias)
These studies have the least bias and results are considered valid. A study that adheres mostly to the commonly held concepts of high quality including the following: a formal randomized controlled study; clear description of the population, setting, interventions, and comparison groups; appropriate measurement of outcomes; appropriate statistical and analytic methods and reporting; no reporting errors; low dropout rate; and clear reporting of dropouts.
These studies are susceptible to some bias, but it is not sufficient to invalidate the results. They do not meet all the criteria required for a rating of good quality because they have some deficiencies, but no flaw is likely to cause major bias. The study may be missing information, making it difficult to assess limitations and potential problems.
Poor (high risk of bias)
These studies have significant flaws that imply biases of various types that may invalidate the results. They have serious errors in design, analysis, or reporting; large amounts of missing information; or discrepancies in reporting.
The multidisciplinary team assessed the quality of evidence for one PICO question at a time. The team members individually reviewed the articles and completed the quality of evidence assessment table. Afterwards, the clinician who developed the PICO question led the discussion, and as a group, level of evidence, applicability, and quality of evidence were discussed and scored for each article. Additionally, footnotes were produced to indicate the rationale for how the quality of the articles was assessed (e.g., potential confounding treatment effects or risk of bias that lowered quality of evidence).
Delphi panel and making a recommendation
IRB approval was obtained to recruit Delphi panel members consisting of clinical professionals who treat patients with TBI and stakeholders (i.e., patients, caregivers) with a waiver of informed consent. Panel members were recruited through TBI professional organizational membership rosters for each disciplinary field via email invitation. Additionally, TBI survivors and their family members who participate in our local TBI advisory council were recruited to gauge the relevance of the research for real-world application.
Quality of evidence
Grading of Recommendations, Assessment, Development and Evaluation (GRADE) grid for the strength of the recommendation
Assessors’ view of the balance of desirable and undesirable consequences of the intervention
Desirable consequences clearly outweigh undesirable consequences
Desirable consequences probably outweigh undesirable consequences
Undesirable consequences probably outweigh desirable consequences
Undesirable consequences clearly outweigh desirable consequences
We recommend to “do something”
We suggest/conditionally recommend to “do something”
We suggest/conditionally recommend to “not do something”
We recommend to “not do something”
Delphi panel and making a recommendation
Developing evidence-based clinical practice guidelines is critical to improving quality of care and rehabilitation outcomes for TBI survivors. The process described in this paper provides a comprehensive approach to addressing the issue. However, there were several constraints that were noted throughout the process that warrant discussion as a means of refining the process. First, several attempts were made trying to utilize GRADE methodology in its original form for each PICO question before the quality of evidence review process was reorganized. After reorganization, all PICO questions that were reviewed under GRADE methodology were reevaluated using the modified ranking methodology. In addition, due to a lack of research articles for many of the original PICO questions created, several meetings were dedicated to generating new PICO questions. Furthermore, it was difficult for clinical staff to fit literature reviews and meetings times related to the recommendation process into their clinical load. Likewise, the time and process to recruit Delphi panel members was also challenging given personal and professional demands on time.
While GRADE is deemed the gold standard for evaluating evidence and generating recommendations, it was not applicable for TBI rehabilitation research due to the lack of published TBI rehabilitation articles that had used robust methodologies (e.g., randomized controlled trials) within each discipline of rehabilitation . In addition, outcomes in rehabilitation were not as concrete as the mortality and morbidity outcomes in medical literature, with GRADE appearing to be more relevant to medical and pharma-related outcomes. Despite the initial challenges of using GRADE methodology, the modified GRADE-Oxford did account for the unique design, outcome, and injury-related factors inherent to TBI rehabilitation research, thereby enabling this group to develop clinically relevant recommendations. By combining classification schemes into a unified quality evaluation tool, we were able to create a standardized process allowing for the aggregate review of heterogeneous studies (e.g., combination of randomized controlled trials and observational studies) with small patient populations and variations in outcome measures.
There is a need for more evidence-based practice guidelines to improve quality of rehabilitation care for TBI survivors. Recommendations were created across physiatry; neuropsychology; and speech, physical, occupational, and recreational therapy disciplines using the process described above. In creating the process, it became evident that there is a need for higher-quality research for individuals with TBI in the rehabilitation setting. Additionally, it is important to develop and standardize utilization of common data elements and outcome measures for research studies to enable researchers to more adequately compare different therapies and identify best practices. Finally, it is valuable and necessary to include the input of TBI survivors and family members when selecting the best therapies to treat TBI sequelae and help survivors achieve their goals .
Agency for Healthcare Research and Quality
Grading of Recommendations, Assessment, Development, and Evaluation
Health Insurance Portability and Accountability Act
Institutional Review Board
Population, Intervention, Control, Outcomes
traumatic brain injury
Traumatic Brain Injury Model System
The authors would like to thank all the clinicians and Delphi panel members who participated in the study and were gracious enough to give us their time.
All personnel and the contents of this manuscript were developed under a grant from the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR grant number 90DP0045-01-0). NIDILRR is a Center within the Administration for Community Living (ACL), Department of Health and Human Services (HHS). The contents of this manuscript do not necessarily represent the policy of NIDILRR, ACL, HHS, and you should not assume endorsement by the Federal Government. The project was also supported by the Ginger Murchison Foundation Traumatic Brain Injury Research Fund.
Availability of data and materials
The only dataset used in this study are the results of the Delphi panel vote. These can be obtained from the contributing author upon request.
LC and RB participated in the study design, conducted literature searches, facilitated group meetings and wrote the manuscript. SS designed the study and edited the manuscript. SD edited the manuscript. MD participated as a lead clinician for clinical questions, Delphi panels, and edited the manuscript. AC participated in the study design and conducted literature searches. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
Baylor Research Institute IRB approval was obtained to recruit Delphi panel members with a waiver of informed consent.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Hall KM, Cope DN. The benefit of rehabilitation in traumatic brain injury: A literature review. J Head Trauma Rehabil. 1995;10(1):1–13.View ArticleGoogle Scholar
- Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966;166–206.
- Donabedian A. The quality of care: How can it be assessed? JAMA. 1988;260(12):1743–8.View ArticlePubMedGoogle Scholar
- Centers for Disease Control and Prevention. Get the Stats on Traumatic Brain Injury in the United States. https://www.cdc.gov/traumaticbraininjury/pdf/blue_book.pdf. Accessed 19 Feb 2016.
- Dahdah MN, Barisa MT, Schmidt K, et al. Comparative effectiveness of traumatic brain injury rehabilitation: differential outcomes across TBI model systems centers. J Head Trauma Rehabil. 2014;29(5):451–9.View ArticlePubMedGoogle Scholar
- Seel RT, Barrett RS, Beaulieu CL, et al. Institutional variation in traumatic brain injury acute rehabilitation practice. Arch Phys Med Rehabil. 2015;96(8):S197–208.View ArticlePubMedGoogle Scholar
- Corrigan JD, Horn SD, Barrett RS, et al. Effects of patient preinjury and injury characteristics on acute rehabilitation outcomes for traumatic brain injury. Arch Phys Med Rehabil. 2015;96(8):S209–21. e206.View ArticlePubMedPubMed CentralGoogle Scholar
- Horn SD, Corrigan JD, Bogner J, et al. Traumatic Brain Injury–Practice Based Evidence study: design and patients, centers, treatments, and outcomes. Arch Phys Med Rehabil. 2015;96(8):S178–96. e115.View ArticlePubMedPubMed CentralGoogle Scholar
- Beaulieu CL, Dijkers MP, Barrett RS, et al. Occupational, physical, and speech therapy treatment activities during inpatient rehabilitation for traumatic brain injury. Archives of Physical Medicine and Rehabilitation. 2015;96(8):S222–34. e217.View ArticlePubMedPubMed CentralGoogle Scholar
- McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348(26):2635–45.View ArticlePubMedGoogle Scholar
- Shafi S, Rayan N, Barnes S, Fleming N, Gentilello LM, Ballard D. Moving from “optimal resources” to “optimal care” at trauma centers. J Trauma Acute Care Surg. 2012;72(4):870–7.View ArticlePubMedGoogle Scholar
- Shafi S, Barnes SA, Millar D, et al. Suboptimal compliance with evidence-based guidelines in patients with traumatic brain injuries: Clinical article. J Neurosurg. 2014;120(3):773–7.View ArticlePubMedGoogle Scholar
- Centers for Disease Control and Prevention. Report to Congress on Traumatic Brain Injury in the United States: Epidemiology and Rehabilitation. https://www.cdc.gov/traumaticbraininjury/pdf/tbi_report_to_congress_epi_and_rehab-a.pdf. Accessed 19 Feb 2016.
- Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ. What is “quality of evidence” and why is it important to clinicians? BMJ. 2008;336(7651):995–8.View ArticlePubMedPubMed CentralGoogle Scholar
- Group GW. Grading quality of evidence and strength of recommendations. BMJ. 2004;328(7454):1490.View ArticleGoogle Scholar
- Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94.View ArticlePubMedGoogle Scholar
- Group OLoEW. The Oxford 2011 levels of evidence. UK: Oxford Centre for Evidence-Based Medicine Oxford; 2011.Google Scholar
- Atkins D, Chang SM, Gartlehner G, et al. Assessing applicability when comparing medical interventions: AHRQ and the Effective Health Care Program. J Clin Epidemiol. 2011;64(11):1198–207.View ArticlePubMedGoogle Scholar
- Barbui C, Dua T, van Ommeren M, et al. Challenges in developing evidence-based recommendations using the GRADE approach: the case of mental, neurological, and substance use disorders. PLoS Med. 2010;7(8):980.View ArticleGoogle Scholar
- Eckardt K-U, Kasiske BL. Kidney disease: improving global outcomes. Nat Rev Nephrol. 2009;5(11):650–7.View ArticlePubMedGoogle Scholar