Representation Learning for Interpretation of Chest X-rays

Abstract

Chest X-ray imaging is one of the most commonly performed diagnostic procedures in radiology, playing a critical role in detecting chest pathologies, monitoring disease progression, and guiding treatment decisions. This thesis investigates the application of representation learning as an upstream modality to enhance various downstream tasks in chest radiograph analysis, including localized disease classification, progression tracking and automated radiology report generation. To achieve this, we utilize the Chest ImaGenome dataset, a subset of MIMICCXR, which comprises 242,072 scene graphs that describe individual chest X-rays. These scene graphs contain automatically extracted information from radiology reports, including patient demographics, anatomical bounding boxes, pathological findings, and progression of disease in each anatomical region. This structured information serves as supervisory labels for training models. For the upstream representation learning task, we employ the DEtection TRansformer (DETR), a transformer-based object detection framework, to identify anatomical structures in chest X-rays and generate meaningful feature representations. These learned features are subsequently leveraged for multiple downstream tasks, including localized classifications via specialized classifiers and radiology report generation using a large language model. Our approach achieves strong performance across these tasks, with an average ROC of 89.1% over nine disease categories in localized disease detection. Additionally, our method demonstrates effectiveness in tracking localized disease progression, achieving an average accuracy of approximately ∼67% and an average F1 score of ∼71%. Furthermore, it produces clinically relevant radiology reports. The results highlight the effectiveness of a unified transformer-based architecture for chest X-ray interpretation, demonstrating its capability to achieve competitive performance across multiple tasks while minimizing reliance on handcrafted features or task-specific models. This work underscores the potential of representation learning to enhance automated chest radiograph analysis and improve clinical decision support.