Systematic Reviews to Answer Health Care Questions

187

Chapter 12 • Assessing and Rating the Strength of the Body of Evidence

TABLE 12.3 OVERALL RATING OF THE STRENGTH OF EVIDENCE (GRADE AND AHRQ EPC) GRADE 5,11 AHRQ EPC 3 High Considerable confidence in estimate of effect High

Very confident that the estimate of effect lies close to the true effect; evidence has few or no deficiencies; findings are stable

Moderate Further research likely to have an impact on con fidence in estimate, may change estimate Low Further research is very likely to have an impact on confi dence in estimate, likely to change the estimate

Moderate Moderately confident that the estimate of effect lies close to the true effect; evidence has some deficiencies; findings are likely to be stable

Low

Limited confidence that the estimate of effect lies close to the true effect; evi dence has major and/or numerous defi ciencies; additional evidence is needed to make conclusions

Very Low Any estimate of effect is very uncertain Insufficient No evidence, unable to estimate an effect, or no confidence in the estimate Abbreviations: AHRQ EPC, Agency for Healthcare Research and Quality Evidence-based Practice Centers; GRADE, Grading of Recommendations Assessment, Development and Evaluation.

to each. The overall rating then considers the ratings of all study designs. This approach can also be taken when both direct and indirect evidence are available for an outcome. For exam ple, a systematic review comparing the effectiveness of two interventions may include results of meta-analyses of placebo-controlled trials in addition to head-to-head trials. The ratings of strength of evidence for the placebo-controlled and head-to-head trials can be determined sep arately and then combined. In most cases, ratings will be made regarding the evidence for specific outcomes and inter vention/comparison pairs. The systematic reviewers must determine which outcomes will be graded based on the research questions and purpose of the systematic review. In general, these are the outcomes most relevant to decision making and reflect the input of clinical experts, stakeholders, and users and consider patient preferences and values. The number of outcomes may vary by the scope, research questions, and intent of the systematic review. Study Limitations The study limitations domain, also referred to as risk of bias, incorporates both study design and study quality ( risk of bias , internal validity ). 3,14 Based on the hierarchy of evidence (Chapter 8), a body of evidence composed entirely of RCTs has a higher strength of evidence rating than one with only observational studies. 29 Also, among observational studies, cohort studies are generally ranked higher than case–control studies because they are inherently less biased, whereas before–after (pre–post) and time-series studies are ranked lower. To determine the rating for this domain, the quality of the studies is considered collectively while considering the study design. Earlier in the systematic review process, individual studies were evaluated for quality using prespecified criteria (Chapter 8) resulting in a rating assigned to each study. In this domain, the reviewer makes an assessment about the quality of the entire group of studies. This is a subjective assessment, where outliers may be given less weight. For example, if a group of RCTs includes four moderately sized fair-quality trials and one small poor-quality trial, the reviewer may weigh the larger studies more heavily. The body of evidence would then get a rating of medium for study limitations using the AHRQ EPC method indicat ing moderate methodological limitations. The GRADE method would downgrade the trials from the no limitations to the serious limitations rating.

Copyright © 2024 Wolters Kluwer, Inc. Unauthorized reproduction of the content is prohibited.