Nelson Systematic Reviews to Answer Healthcare, Questions, 2e

71

Chapter 6 • Selecting Studies for Inclusion

OBSERVATION

REVIEWERS AGREED

REVIEWERS DISAGREED

• Studies of healthcare delivery systems are complex and multifaceted • Standard taxonomy for describing

Include studies that explicitly described multiple components of interest (eg, small panel sizes; enhanced support by interdisciplinary teams; inte gration with affiliated phar macy, mental health, home health, and community-based and inpatient geriatric care service; and 24-hour access)

Reviewers with clinical exper tise in primary care were more likely to regard studies of singular enhancements as an “intense” level of service based on personal clinical experiences

intensive primary care programs has only recently been published

Intervention and comparator

Regardless of the approach used to involve a second reviewer, the intent is to identify dis crepancies in judgments about whether a study meets criteria for inclusion. In many cases, the cause of a discrepancy is simply a random error on the part of a single reviewer. In other cases, it is because of differences in the interpretation of the eligibility criteria and relevant study char acteristics. For example, in a review of pharmacologic treatments for adults with fibromyalgia, a reviewer who is a practicing rheumatologist may feel strongly about excluding studies that do not use the most recent American College of Rheumatology diagnostic criteria. A reviewer who is not a content expert may not make this distinction and include studies using other criteria. Discrepancies based on differences in the interpretation of the eligibility criteria can be useful in identifying important ambiguities that may require the development of supplemental crite ria or lead to valuable refinements to the protocol. All discrepancies should be discussed and resolved by a consensus process, sometimes requiring additional reviewers or experts. As with the approach to involving a second reviewer, the approach to handling discrepancies should be explicitly described in both the study protocol and the review report. Two-Stage Strategy to Selecting Studies The study selection process is typically conducted in two stages . The first stage involves the assessment of only the titles and abstracts from the database searches. The purpose of the first stage is to efficiently eliminate all obviously ineligible publications. For example, if the system atic review evaluates the effects of using antiepileptic drugs for treatment of bipolar disorder, a reviewer could confidently exclude a study entitled, “Drug Therapy for Epilepsy” after review ing only the title. The second stage involves a detailed assessment of the subset of full-text publications that were determined to be likely or possibly eligible based on the review of their titles and abstracts. An alternative to using a two-stage process for study selection is to use a single-stage process in which the full-text papers are reviewed for all studies identified from the database searches. Systematic review standards endorse use of both the two- and single-stage approaches and acknowledge that the single-stage approach is more expensive and time-consuming. 1 When using the two-stage strategy to selecting studies, the systematic reviewers may con sider using different approaches to involving the second reviewer within each stage. The impor tance of involving an independent second reviewer at the full-text stage has been emphasized in systematic review methods guidelines. 10–12 Decisions about using more than one reviewer and how independent they will be may depend on the volume of citations, number of reviewers, and reviewer experience level, among other factors. For example, AHRQ guidance on avoiding bias in study selection involves the use of a second reviewer to assess only the citations and abstracts that the first reviewer deemed ineligible. 12 However, AHRQ acknowledges a number of disadvantages to this approach, including the lack of empiric evidence to support its reliability.

Copyright © 2024 Wolters Kluwer, Inc. Unauthorized reproduction of the content is prohibited.