Systematic Reviews to Answer Health Care Questions

192

Chapter 12 • Assessing and Rating the Strength of the Body of Evidence

In a similar way, studies demonstrating dose–response associations also overcome bias. In these studies, effects are greater with increasingly higher levels of interventions or exposures. For example, in studies of smoking and lung cancer, relative risks are greatest for smokers with the highest packs per day of use compared to smokers with lower use. The concept of plausible confounders describes the situation where confounding factors lead to an underestimate of the effect of an intervention. These biases create a situation that makes it less likely to find an effect, and an effect found under these circumstances is more likely to be real. For example, the association of smoking and heart disease could be confounded in an observational study comparing smokers and nonsmokers if age was not considered because age is also associated with heart disease. However, if the smokers in this study were younger than the nonsmokers, the association between smoking and heart disease would be biased against an effect. The estimate of the effect of smoking from this study could be considered an underesti mate of the true effect. To increase the rating of strength of evidence for observational studies based on a large mag nitude of effect (strength of association), dose–response association, or plausible confounders, the body of evidence must adequately meet criteria for the other domains. Therefore, it is inten tionally difficult for observational evidence to be rated as high using the GRADE method. The AHRQ EPC method rates observational studies according to how the domain criteria are fulfilled without starting at a specific level based on study design. In this approach, obser vational evidence for harms is considered stronger evidence than for benefits. 3 In some cases, observational studies of harms are superior to RCTs because they represent more real-world situations and can include much larger sample sizes. Overall Assessment The overall rating of the strength of evidence considers the assessment of all the individual domains. In the GRADE and AHRQ EPC methods, final ratings include high, moderate, low, and either very low or insufficient (Table 12.3). 3,20 The overall ratings are not necessarily cumulative. Issues contributing to the ratings of individual domains may overlap, and specific domains may be given more or less weight in individual situations. Additionally, the overall rating must take into account all bodies of evidence relating to the outcome being assessed, including direct and indirect evidence, trials and observational studies, and other relevant evidence. The final rating is qualitative and dependent on an in-depth knowledge of the evi dence and understanding of the domains. 35 However, despite efforts to standardize the rating process, it has been found to be subjective and highly variable across systematic reviewers. 36 Dual, independent assessment of ratings and descriptions of their rationale improve the trans parency of this process. Information relating to the body of evidence, domain ratings, and the overall strength of evidence rating are typically summarized in tables. 5 Although the structure and format of the summary tables may vary, tables that concisely summarize key information in a clear and trans parent manner are most useful to users. An example of a strength of evidence table outlining the domains and ratings for a systematic review is described in Box 12.1. 37 BOX 12.1 Example of Determining Strength of Evidence Grades for Studies of Patient Navigation to Increase Cancer Screening A systematic review of the effectiveness of health system interventions to reduce dispari ties in preventive health services included studies of patient navigation versus usual care to increase rates of screening for colorectal, breast, cervical, and lung cancer. 37,38 Patient

Copyright © 2024 Wolters Kluwer, Inc. Unauthorized reproduction of the content is prohibited.