Skip to main content
  • Technical advance
  • Open access
  • Published:

Process for developing rehabilitation practice recommendations for individuals with traumatic brain injury



Attempts at measuring quality of rehabilitation care are hampered by a gap in knowledge translation of evidence-based approaches and lack of consensus on best practices. However, adoption of evidence-based best practices is needed to minimize variations and improve quality of care. Therefore, the objective of this project was to describe a process for assessing the quality of evidence of clinical practices in traumatic brain injury (TBI) rehabilitative care.


A multidisciplinary team of clinicians developed discipline-specific clinical questions using the Population, Intervention, Control, Outcome process. A systematic review of the literature was conducted for each question using Pubmed, CINAHL, PsychInfo, and Allied Health Evidence databases. Team members assessed the quality, level, and applicability of evidence utilizing a modified Oxford scale, the Agency for Healthcare Research and Quality Methods Guide, and a modified version of the Grading of Recommendations, Assessment, Development, and Evaluation scale.


Draft recommendations for best-practice were formulated and shared with a Delphi panel of clinical representatives and stakeholders to obtain consensus.


Evidence-based practice guidelines are essential to improve the quality of TBI rehabilitation care. By using a modified quality of evidence assessment tool, we established a process to gain consensus on practice recommendations for individuals with TBI undergoing rehabilitation.

Peer Review reports


For patients who suffer significant neurologic and functional deficits due to traumatic brain injury (TBI), inpatient rehabilitation centers provide comprehensive post-injury care that has been shown to improve functional outcomes and successful reintegration into the community [1]. The structures and processes of care during inpatient rehabilitation can highly influence patient outcomes [2, 3]. However, there is a growing recognition that there is variability in application of evidence-based guidelines, demonstrating a need for a standard treatment approach [4]. For example, amongst TBI Model System (TBIMS) rehabilitation centers, which provide a state-of-the-art multidisciplinary system of rehabilitation care [5], significant differences in risk-adjusted functional outcomes of TBI patients have been observed [57]. In the Traumatic Brain Injury-Practice Based Evidence study, it was determined that variations in care among TBIMS centers are due to differences in hospital characteristics, patient characteristics, and the experience of the clinicians [68]. Differential patient outcomes across inpatient TBIMS centers may also relate to processes of care (e.g., goal and treatment planning, selection of therapeutic interventions) [5, 6, 9]. These variations in care may result from suboptimal adoption of evidence-based best practices in routine clinical care, with a little over half of adult patients in the U.S. receiving recommended preventative, acute, and chronic disease care [10, 11]. Similar gaps in the acute management of patients at Level 1 trauma centers also exists for people following TBI [11, 12]. Although it has been shown that improved compliance with recommended care is associated with improved outcomes following TBI in the trauma setting, compliance with the limited evidence-based guidelines available in the rehabilitation milieu remains suboptimal [11, 12].

To further compound lack of a uniform approach to rehabilitation, the current literature supporting care practices in TBI rehabilitation does not allow for the identification of evidence-based care guidelines for optimal dose or intensity of therapy, the ideal timing of therapy in the recovery process, or the necessary modifications for subpopulations. This is due to variability in care practices and a lack of rigorously executed research including randomized controlled trials [6, 9, 13]. Thus, there is a critical need to resolve the gaps that exist in generating TBI clinical care guidelines. In order to address this gap, we formed a multidisciplinary group of clinicians and researchers to identify and evaluate evidence-based best practices for patients with TBI undergoing rehabilitation. The manuscript describes the process we have developed, so that it may be replicated by other groups as practitioners and researchers in the field of TBI continue to strengthen the knowledge base.

Methods – Fig. 1

Fig. 1
figure 1

Recommendation development process PICO indicates Population, Intervention, Comparison, Outcome; AHRQ, Agency for Healthcare Research and Quality; GRADE, Grading of Recommendations, Assessment, Development and Evaluation; TBI, traumatic brain injury

Develop PICO questions

A team of research personnel and multidisciplinary clinicians who specialize in treating patients with TBI was established including representatives from the following disciplines: physiatry, neuropsychology, physical therapy, occupational therapy, speech language pathology, and therapeutic recreation. Clinicians created a list of keywords specific to their discipline based on common clinical problems or symptoms that they treat in their day-to-day practices. For example, key words chosen by physical therapists included hemiparesis, spasticity, balance, and ataxia. Research personnel then conducted a preliminary literature search using the keywords combined with “traumatic brain injury” to identify existing literature. Based on the initial evidence collected, clinicians were then asked to form research questions in PICO format (Population, Intervention, Comparison, Outcome) related to their clinical discipline and TBI rehabilitation interests (e.g., “In patients with traumatic brain injury, does prolonged positioning such as casting versus other therapies improve spasticity?”). The goal of this exercise was to generate a list of PICO questions in order to evaluate current literature and develop evidence-based recommendations.

Systematic review

We conducted a systematic literature review for each individual PICO question using search terms identified by the clinician in conjunction with the corresponding “brain injury” medical subject heading. PubMed, CINAHL, PSYCInfo, and Allied Health Evidence databases were utilized because they were projected to include the most content related to TBI rehabilitation research across clinical disciplines. Inclusion criteria for articles were limited to clinical trials, randomized controlled trials, comparative studies, observational studies, and case-series. Meta-analyses were also reviewed for relevant papers. Articles were included if they pertained to adults (≥18 years of age) with TBI and were published in English from January 1, 1987 – current date. Additionally, studies with a patient population of mild through severe and acute and chronic TBIs were considered. Relevant articles cited within manuscripts were also reviewed to find additional evidence to support the PICO question. Finally, articles were collated and sent to the clinician to review for applicability.

Quality of evidence

After identifying relevant literature for PICO questions, the next step was to assess the quality of evidence for the purpose of developing clinical practice recommendations [14]. Research personnel were trained to use the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) methodology to evaluate the quality of studies pertaining to each PICO question [15]. GRADE is a transparent system for rating quality of evidence of systematic reviews and journal articles and grading the strength of recommendations of guidelines [16]. However, the group found it difficult to apply GRADE methodology to evaluate the level and quality of evidence of rehabilitation data. Specifically, the team found that the TBI rehabilitation research identified through the systematic review did not meet GRADE criteria because of the variations in design of the clinical trials, variation in outcome measures, and/or heterogeneous patient populations (i.e., other acquired brain injury mixed with TBI) and care settings [13]. As a result, GRADE criteria such as inconsistency, indirectness of evidence, imprecision, effect, and dose–response relationship were difficult to assess.

We then reviewed other methodologies used to generate evidence-based recommendations and developed a modified ranking methodology. This combined the level of evidence methodology from the (1) Oxford Centre for Evidence-Based Medicine 2011 Levels of Evidence [17], (2) study population applicability from Agency for Healthcare Research and Quality Methods Guide for Effectiveness and Comparative Effectiveness reviews [18], and (3) a modified quality of evidence assessment and recommendation format from GRADE methodology shown in Tables 1, 2, and 3, respectively [1518].A modified-GRADE methodology has been used successfully to create recommendations for other neurological disabilities and kidney disease [19, 20]. Table 3 displays the modified quality of evidence assessment criteria that we used. We found this modified approach more manageable, appropriate, and pertinent to TBI literature when compared to using only the GRADE-based methodology.

Table 1 Oxford Centre for Evidence-Based Medicine 2011 Levels of Evidence [17]
Table 2 Assessment of Study Applicability based on Agency for Healthcare Research and Quality Methods
Table 3 Quality Assessment based on Grading of Recommendations, Assessment, Development and Evaluation (GRADE) – Modified

The multidisciplinary team assessed the quality of evidence for one PICO question at a time. The team members individually reviewed the articles and completed the quality of evidence assessment table. Afterwards, the clinician who developed the PICO question led the discussion, and as a group, level of evidence, applicability, and quality of evidence were discussed and scored for each article. Additionally, footnotes were produced to indicate the rationale for how the quality of the articles was assessed (e.g., potential confounding treatment effects or risk of bias that lowered quality of evidence).

Delphi panel and making a recommendation

IRB approval was obtained to recruit Delphi panel members consisting of clinical professionals who treat patients with TBI and stakeholders (i.e., patients, caregivers) with a waiver of informed consent. Panel members were recruited through TBI professional organizational membership rosters for each disciplinary field via email invitation. Additionally, TBI survivors and their family members who participate in our local TBI advisory council were recruited to gauge the relevance of the research for real-world application.


Quality of evidence

Based on the results of the quality of evidence assessment table, a draft recommendation was created using GRADE recommendation formatting (e.g., weak/conditional in favor of prolonged positioning to improve spasticity in patients with TBI undergoing rehabilitation). Table 4 shows the GRADE grid for the strength of the recommendation. The draft recommendation was then posed to the Delphi panel to achieve consensus.

Table 4 Grading of Recommendations, Assessment, Development and Evaluation (GRADE) grid for the strength of the recommendation

Delphi panel and making a recommendation

The quality of evidence assessment table and corresponding articles were shared with Delphi panel members for review. However, footnotes and additional qualifying information by our group of clinicians was excluded to avoid biasing Delphi panel members. A conference call was then held to discuss the impetus of the study, history regarding the TBIMS, background information regarding the quality of evidence classification schemes employed in this study, and the process of draft recommendation development. Panel members were then directed to a HIPAA compliant web-based survey tool where they rated on a 4-point Likert scale the degree to which they agree with the draft recommendation (Figure 2). If 70% of the panelists agreed (to accept or reject the draft recommendation), then participation was complete, and the recommendation was finalized and disseminated via email back to the panel with manuscript publication to follow. Results and feedback were collated and sent to panel members via email. If 70% of panelists did not agree, panel members were asked to review the results and feedback of the first vote, and re-rate their agreement or disagreement with the draft recommendation based on the new information. Following the second round of rating (if necessary), participation was complete. If a 70% agreement was achieved following the second round of ratings, then the recommendation was finalized disseminated. If agreement was not reached, this finding was still reported. The implication of an inability to reach agreement was that the quality of evidence for that particular clinical (PICO) question is inconclusive, which suggests that further research is required.

Fig. 2
figure 2

Example of Delphi panel Likert survey


Developing evidence-based clinical practice guidelines is critical to improving quality of care and rehabilitation outcomes for TBI survivors. The process described in this paper provides a comprehensive approach to addressing the issue. However, there were several constraints that were noted throughout the process that warrant discussion as a means of refining the process. First, several attempts were made trying to utilize GRADE methodology in its original form for each PICO question before the quality of evidence review process was reorganized. After reorganization, all PICO questions that were reviewed under GRADE methodology were reevaluated using the modified ranking methodology. In addition, due to a lack of research articles for many of the original PICO questions created, several meetings were dedicated to generating new PICO questions. Furthermore, it was difficult for clinical staff to fit literature reviews and meetings times related to the recommendation process into their clinical load. Likewise, the time and process to recruit Delphi panel members was also challenging given personal and professional demands on time.

While GRADE is deemed the gold standard for evaluating evidence and generating recommendations, it was not applicable for TBI rehabilitation research due to the lack of published TBI rehabilitation articles that had used robust methodologies (e.g., randomized controlled trials) within each discipline of rehabilitation [13]. In addition, outcomes in rehabilitation were not as concrete as the mortality and morbidity outcomes in medical literature, with GRADE appearing to be more relevant to medical and pharma-related outcomes. Despite the initial challenges of using GRADE methodology, the modified GRADE-Oxford did account for the unique design, outcome, and injury-related factors inherent to TBI rehabilitation research, thereby enabling this group to develop clinically relevant recommendations. By combining classification schemes into a unified quality evaluation tool, we were able to create a standardized process allowing for the aggregate review of heterogeneous studies (e.g., combination of randomized controlled trials and observational studies) with small patient populations and variations in outcome measures.


There is a need for more evidence-based practice guidelines to improve quality of rehabilitation care for TBI survivors. Recommendations were created across physiatry; neuropsychology; and speech, physical, occupational, and recreational therapy disciplines using the process described above. In creating the process, it became evident that there is a need for higher-quality research for individuals with TBI in the rehabilitation setting. Additionally, it is important to develop and standardize utilization of common data elements and outcome measures for research studies to enable researchers to more adequately compare different therapies and identify best practices. Finally, it is valuable and necessary to include the input of TBI survivors and family members when selecting the best therapies to treat TBI sequelae and help survivors achieve their goals [13].



Agency for Healthcare Research and Quality


Grading of Recommendations, Assessment, Development, and Evaluation


Health Insurance Portability and Accountability Act


Institutional Review Board


Population, Intervention, Control, Outcomes


traumatic brain injury


Traumatic Brain Injury Model System


  1. Hall KM, Cope DN. The benefit of rehabilitation in traumatic brain injury: A literature review. J Head Trauma Rehabil. 1995;10(1):1–13.

    Article  CAS  Google Scholar 

  2. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966;166–206.

  3. Donabedian A. The quality of care: How can it be assessed? JAMA. 1988;260(12):1743–8.

    Article  CAS  PubMed  Google Scholar 

  4. Centers for Disease Control and Prevention. Get the Stats on Traumatic Brain Injury in the United States. Accessed 19 Feb 2016.

  5. Dahdah MN, Barisa MT, Schmidt K, et al. Comparative effectiveness of traumatic brain injury rehabilitation: differential outcomes across TBI model systems centers. J Head Trauma Rehabil. 2014;29(5):451–9.

    Article  PubMed  Google Scholar 

  6. Seel RT, Barrett RS, Beaulieu CL, et al. Institutional variation in traumatic brain injury acute rehabilitation practice. Arch Phys Med Rehabil. 2015;96(8):S197–208.

    Article  PubMed  Google Scholar 

  7. Corrigan JD, Horn SD, Barrett RS, et al. Effects of patient preinjury and injury characteristics on acute rehabilitation outcomes for traumatic brain injury. Arch Phys Med Rehabil. 2015;96(8):S209–21. e206.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Horn SD, Corrigan JD, Bogner J, et al. Traumatic Brain Injury–Practice Based Evidence study: design and patients, centers, treatments, and outcomes. Arch Phys Med Rehabil. 2015;96(8):S178–96. e115.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Beaulieu CL, Dijkers MP, Barrett RS, et al. Occupational, physical, and speech therapy treatment activities during inpatient rehabilitation for traumatic brain injury. Archives of Physical Medicine and Rehabilitation. 2015;96(8):S222–34. e217.

    Article  PubMed  PubMed Central  Google Scholar 

  10. McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348(26):2635–45.

    Article  PubMed  Google Scholar 

  11. Shafi S, Rayan N, Barnes S, Fleming N, Gentilello LM, Ballard D. Moving from “optimal resources” to “optimal care” at trauma centers. J Trauma Acute Care Surg. 2012;72(4):870–7.

    Article  PubMed  Google Scholar 

  12. Shafi S, Barnes SA, Millar D, et al. Suboptimal compliance with evidence-based guidelines in patients with traumatic brain injuries: Clinical article. J Neurosurg. 2014;120(3):773–7.

    Article  PubMed  Google Scholar 

  13. Centers for Disease Control and Prevention. Report to Congress on Traumatic Brain Injury in the United States: Epidemiology and Rehabilitation. Accessed 19 Feb 2016.

  14. Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ. What is “quality of evidence” and why is it important to clinicians? BMJ. 2008;336(7651):995–8.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Group GW. Grading quality of evidence and strength of recommendations. BMJ. 2004;328(7454):1490.

    Article  Google Scholar 

  16. Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94.

    Article  PubMed  Google Scholar 

  17. Group OLoEW. The Oxford 2011 levels of evidence. UK: Oxford Centre for Evidence-Based Medicine Oxford; 2011.

    Google Scholar 

  18. Atkins D, Chang SM, Gartlehner G, et al. Assessing applicability when comparing medical interventions: AHRQ and the Effective Health Care Program. J Clin Epidemiol. 2011;64(11):1198–207.

    Article  PubMed  Google Scholar 

  19. Barbui C, Dua T, van Ommeren M, et al. Challenges in developing evidence-based recommendations using the GRADE approach: the case of mental, neurological, and substance use disorders. PLoS Med. 2010;7(8):980.

    Article  Google Scholar 

  20. Eckardt K-U, Kasiske BL. Kidney disease: improving global outcomes. Nat Rev Nephrol. 2009;5(11):650–7.

    Article  PubMed  Google Scholar 

Download references


The authors would like to thank all the clinicians and Delphi panel members who participated in the study and were gracious enough to give us their time.

Funding statement

All personnel and the contents of this manuscript were developed under a grant from the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR grant number 90DP0045-01-0). NIDILRR is a Center within the Administration for Community Living (ACL), Department of Health and Human Services (HHS). The contents of this manuscript do not necessarily represent the policy of NIDILRR, ACL, HHS, and you should not assume endorsement by the Federal Government. The project was also supported by the Ginger Murchison Foundation Traumatic Brain Injury Research Fund.

Availability of data and materials

The only dataset used in this study are the results of the Delphi panel vote. These can be obtained from the contributing author upon request.

Authors’ contributions

LC and RB participated in the study design, conducted literature searches, facilitated group meetings and wrote the manuscript. SS designed the study and edited the manuscript. SD edited the manuscript. MD participated as a lead clinician for clinical questions, Delphi panels, and edited the manuscript. AC participated in the study design and conducted literature searches. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Baylor Research Institute IRB approval was obtained to recruit Delphi panel members with a waiver of informed consent.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Librada Callender.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Callender, L., Brown, R., Driver, S. et al. Process for developing rehabilitation practice recommendations for individuals with traumatic brain injury. BMC Neurol 17, 54 (2017).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: