Systematic Reviews to Answer Health Care Questions

188

Chapter 12 • Assessing and Rating the Strength of the Body of Evidence

Directness Directness describes the relevance of the evidence to the PICOTS elements of the research ques tion. 3,18 Evidence is direct when the interventions and comparisons in studies are the same as those specified by the research question. For example, if the research questions are about the comparative benefits and harms of Drug A versus Drug B, direct evidence would consist of studies comparing the two drugs against each other (eg, head-to-head trials). Indirect evidence would consist of studies of Drug A and Drug B compared to placebo, but not to each other. In this case, the systematic reviewer would make indirect comparisons across the placebo-con trolled trials to compare the two drugs, often using statistical indirect comparison meta-analysis, if feasible. Directness also describes how well the study population compares with the target pop ulation of the systematic review. For example, in a systematic review of treatments for osteoarthritis, the long-term cardiovascular harms of nonselective nonsteroidal anti-in flammatory drugs (NSAIDs) were determined in placebo-controlled trials of NSAIDs for the prevention of Alzheimer disease. 30 Although results of trials to prevent Alzheimer dis ease may provide insights into possible adverse outcomes in patients with osteoarthritis, the two populations are markedly different. These studies would be considered indirect evidence. Evidence is also indirect when intermediate or surrogate outcomes are used instead of the intended health outcome. This form of indirectness is presumed in systematic reviews that explicitly include intermediate or surrogate outcomes in the study selection criteria. For example, to determine whether statin drugs reduce cardiovascular disease events in high-risk patients, selection criteria could include studies with blood lipid measures as outcomes in addition to studies with cardiovascular event outcomes. The inclusion of studies with inter mediate measures could be justified by the established relationship between lipid levels and cardiovascular events. However, this evidence would be considered indirect if cardiovascular events are the specified health outcomes of the research question. This approach becomes more complicated when the relationships between the intermediate and health outcomes are not established, or the relationships vary across interventions or populations. For example, directness would be more difficult to determine when the relationships between lipid levels and cardiovascular outcomes differ between the various types of statin drugs. These issues need to be considered early in the systematic review process and involve technical and clinical experts. The applicability of individual studies was described in Chapter 8 as the extent to which the effects of an intervention observed in a study are likely to reflect the expected results when the intervention is applied under real-world conditions. 31 Other terms used when referring to appli cability include external validity , generalizability , and relevance . Many of the issues consid ered when determining directness also concern applicability, and the GRADE method refers to applicability as another dimension of directness. 18 A body of evidence is applicable if it focuses on the specific condition, patient population, intervention, comparators, and health outcomes that are the focus of the systematic review’s research protocol. Applicability is considered sepa rately from strength of evidence in the AHRQ EPC method. Consistency Consistency refers to the degree of similarity of results of different studies in a body of evi dence. 3,17 This concept is important because a body of evidence is stronger when studies agree with each other. The AHRQ EPC method distinguishes between consistency in direction of effect and magnitude of effect and requires the systematic reviewer to determine when the mag nitude of effect is important based on the underlying research questions. In assessing consistency of the direction of effect, the primary consideration is whether the point estimates are on the same side of the point of no effect (1.0 for relative measures,

Copyright © 2024 Wolters Kluwer, Inc. Unauthorized reproduction of the content is prohibited.