Systematic Reviews to Answer Health Care Questions

190

Chapter 12 • Assessing and Rating the Strength of the Body of Evidence

For any chosen relative risk reduction, the available evidence meets optimal information size criteria if the number of events is above the associated line

RRR = 20%

400 Events

RRR = 25%

300 Events

RRR = 30%

200 Events

200 300 400 500 600 700

100 Events

Total number of events needed

0 100

0.0

0.2

0.4

0.6

0.8

1.0

Control group event rate

■■ FIGURE 12.2 Optimal information size calculations. Number of events given alpha of 0.05 and beta of 0.20 for varying control event rates and RRR (relative risk reduction) of 20%, 25%, and 30%. Source: Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 6. Rating the quality of evidence—imprecision. J Clin Epidemiol . 2011;64(12):1283–1293. Reprinted with permission.

Precision Precision is the degree of certainty surrounding an estimate of effect for a specific outcome. 16 For a meta-analysis of studies, precision is reflected in the width of the confidence interval. For studies that cannot be combined in a meta-analysis, precision can be determined qualitatively. The first step in assessing precision is to determine whether the studies in a systematic review collectively have adequate power to show a statistically significant difference where one exists. 33 Adequate power is estimated from the number of participants enrolled in the studies and the number of outcome events. The GRADE method refers to this as the optimal information size (OIS) , and is similar to a sample size calculation for an individual trial. A sample size calcu lation estimates the number of study subjects required for a prespecified effect size, whereas the OIS estimates the number of study subjects required for a prespecified number of outcome events. The required number of outcome events varies with the baseline risk of the outcome and the prespecified effect size, but 200 to 300 events are typically required (Figure 12.2). 16 If the OIS is met, precision can be determined from the confidence interval of an estimate from a meta-analysis of studies, or from studies with very large sample sizes and adequate fol low-up periods. For dichotomous outcomes, the systematic reviewer must determine acceptable thresholds for an appreciable benefit and an appreciable harm , for example a 25% increase or decrease in relative risk. Confidence intervals that extend beyond either threshold and cross the line of no effect (ie, are not statistically significant) are imprecise (Figure 12.3). Confidence intervals that reflect a statistically significant difference between groups are considered precise. A confidence interval that is not statistically significant but does not cross the preestablished threshold for appreciable benefit or harm is also precise. For continuous outcomes, thresholds for benefits or harms are determined by the minimum change in the outcome that is clinically important, such as a change in score on a symptom scale (ie, minimal important difference). Similar to the approach for dichotomous outcomes, confidence intervals for continuous outcomes that are not statistically significant and cross the minimal important difference thresholds are not precise. Thresholds for appreciable benefits and harms and minimal important differences should be prespecified, and the rationale for these decisions should be clearly described in the systematic review.

Copyright © 2024 Wolters Kluwer, Inc. Unauthorized reproduction of the content is prohibited.