CLINICAL PRACTICE AND PUBLIC HEALTH GUIDELINES: THE MAKING OF APPROPRIATE STRONG RECOMMENDATIONS WHEN THE CONFIDENCE IN EFFECT ESTIMATES IS LOW OR VERY LOW (DISCORDANT)

Abstract

Clinical practice, public health, and policy guidelines should be developed based on a systematic approach that uses the best available evidence. The advent of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework has facilitated this, resulting in a transparent approach to guideline development. GRADE suggests that guideline developers seldom make strong recommendations based on low or very low confidence in effect estimates (strong l/vl). The World Health Organization (WHO) produces recommendations that guide public health policy and, in 2003, WHO adopted the GRADE approach to guideline development. Initial anecdotal evidence suggested that WHO issues a large number of strong recommendations and particularly strong l/vl. Our research team evaluated the nature of WHO recommendations and conducted a qualitative study using interviews of guideline panel members. Key findings included: i) WHO makes a large proportion of recommendations as strong l/vl ii) many strong l/vl are inconsistent with GRADE guidance iii) reasons guideline panel members offered for strong l/vl included skepticism about the value of making conditional recommendations; political considerations; a high confidence in benefits despite formal ratings of low confidence; and long-standing practices, funding, and policy; iv) methodologist interviewees indicated panelists’ lack of commitment to conditional recommendations; a perceived tension between methodologists and panelists due to resistance to adhering to GRADE guidance; both financial and non-financial conflicts of interest among panel members as explanations of strong l/vl; and the need for greater clarity of, and support for, the role of methodologists as co-chairs of panels. The understanding of when and why strong l/vl are formulated at WHO is an important methodological issue that has implications not just for WHO, but for a wide range of guideline developers elsewhere. Our findings offer insights that may guide interventions to enhance trustworthiness of practice guidelines.