METHODOLOGIC ISSUES IN THE REPORTING AND EVALUATION OF QUALITY IMPROVEMENT STUDIES IN HEALTHCARE

Abstract

Introduction: Quality improvement (QI) encompasses a wide range of healthcare studies and activities with the common goal of improving patient outcomes, healthcare system performance, and professional development. QI is characterized by a diversity of definitions, stakeholders, clinical fields and study designs, which creates challenges for rigorous reporting and evaluation of these studies. Understanding and addressing the methodological issues that arise from conducting QI studies from multiple clinical disciplines is critical for generating good evidence for healthcare improvement to tackle health system challenges. Objectives: This thesis addressed three independent objectives: (i) Determine the quality of reporting of QI studies in neonatology. (ii) Compare different statistical methods that can be used to analyze data from a cluster randomized controlled trial with repeated measures data and examine how the estimate of intervention effects varies between these approaches. (iii) Determine the cost-effectiveness of providing timely surgery or timely rehabilitation for patients with hip fracture. Methods: Objective 1: We conducted a systematic survey of quality improvement studies in neonatology to examine the extent to which these publications adhered to SQUIRE 2.0, the guidelines for reporting studies that sought to improve the quality, safety, and value of healthcare. Using the same set of articles, we examined how various methodological attributes, such as stakeholder engagement, outcome measures, and statistical process controls, are reported in these studies. Objective 2: To compare the differences in the statistical estimates of intervention effects between linear mixed models and Generalized Estimating Equations, for the CP@Clinic Program cluster randomized RCT, which contains routinely collected monthly outcome data aggregated at the cluster level. Objective 3: We constructed a Markov cohort model to estimate the cost-effectiveness of receiving timely surgery within 24 hours of admission to the emergency department, receiving immediate admission to inpatient rehabilitation following acute care discharge, receiving both, or none. Results: Objective 1: In our assessment of reporting quality, we found that adherence to SQUIRE 2.0 guidelines was inadequate and that journals should endorse the SQUIRE 2.0 guideline for improvement publications to alleviate this issue. We found that process measures was the most frequently reported methodological attribute (89%), while stakeholder engagement with leadership (32%) or caregivers (10%) were infrequently reported or conducted.

Objective 2: In comparing statistical methods for analyzing a cluster randomized controlled trial with correlated data, we found that it was critical to apply a correction to the variance estimator of generalized estimating equations to produce robust estimates of the intervention effects.

Objective 3: In our economic evaluation, both timely surgery alone and the combination of timely surgery and timely rehabilitation yielded cost-effective improvements in the quality-adjusted life-years of patients with hip fracture. However, the combination of receiving timely surgery and timely rehabilitation requires a high willingness-to-pay threshold, above $128,000 per quality-adjusted life-years, to be considered cost-effective. Conclusions: Overall, understanding the state of reporting and the broad spectrum of methods and methodologic issues for evaluating quality improvement initiatives will advance its rigorous research, evaluation, reporting, and contribution towards informed decision-making for tackling pressing healthcare issues.