MASK-FREE SHADOW REMOVAL VIA FREQUENCY DISCRIMINATION AND FOUNDATION MODEL FUSION

Abstract

Shadow Removal is a significant topic in image restoration, which in recent years has seen vast progress thanks to the wide application of deep learning on image processing. However, many of the shadow removal methods nowadays relies on the ex istance of masks during training phase, which adds to the human resources needed to annotate shadows and limit the dataset availability. In this thesis, we propose a novel mask-free shadow removal framework utilizing both Vision Transformer and CNN based network. Specifically, a foundation model based on ConvNextV2 network learns the local texture, restores realistic lightning and a tweaked Vision Transformer called FFTFormer removes global shadow artifacts by separating it based on Fast Fourier Transform(FFT) . In addition, we conducted detailed ablation study experiments by replacing modules and comparing the different metric results, both statistically and graphically.