McMaster Premium Literature Service (Plus) Performs Well for Identifying New Studies to be Included in Updated Cochrane Systematic Reviews

Abstract

<p>Objective: We compared the performance of PLUS to that ofthe Cochrane Trial Registry (CCTR), Medline and EMBASE for locating studies added during an update of Cochrane Systematic reviews. We investigated the effect of excluding studies not found in PLUS on meta-analysis results.</p> <p>Study Design and Setting: A sample of new studies in updated Cochrane Systematic Reviews was used to establish a reference standard. Searches were performed for each study in PLUS, CCTR, Medline and EMBASE. Where a primary study was not indexed in PLUS, we examined the effect on the review of excluding the study. We compared the result of each selected meta-analysis including only new studies obtained in PLUS to the result using only those not found in PLUS (non-PLUS) via ratio of odds ratios (ROR). The sensitivity of Hedges filters for randomized trials was measured.</p> <p>Results: Ninety-eight updated reviews were identified and 87 included a meta-analysis suitable for calculation of RORs. The relative recall rates for PLUS, CCTR, Medline and EMBASE were 23%, 95%, 90% and 86%, respectively. PLUS contained all new studies for 13 of74 reviews. No statistically significant difference between PLUS and non-PLUS new studies was found when RORs were pooled across 39 reviews (ROR 0.929; 95% CI, 0.79-1.093). Nineteen updated reviews had no new studies indexed in PLUS. Hedges filters for Medline demonstrated 99.4% sensitivity to detect new trials.</p> <p>Conclusions: PLUS included less than a quarter of the new studies in Cochrane Review updates but the majority of reviews in our sample appeared unaffected by the use of PLUS as a sole source of literature. This may be because PLUS captures the most important studies although no predictors of PLUS retrieval performance were found. Reviewers should consider adopting PLUS and Hedges filters to keep their reviews up to date.</p>