ATTENTIVE MULTI-BRANCH ENCODER-DECODER NETWORK FOR ADHERENT OBSTRUCTION REMOVAL

Abstract

With the rapid development of image hardware, outdoor computer vision systems, for instance, surveillance cameras, have been extensively utilized for various applications. These systems typically equip a protective glass layer installed in front of the camera. How- ever, during inclement weather conditions, images captured through such glass often suffer from obstructions adhering to its surface, such as raindrops or dust particles. Consequently, this leads to a degradation in image quality, which significantly affects the performance of the system. Existing obstruction removal algorithms attempt to resolve these issues using deep learning techniques with synthetic data, which may not achieve a good visual result for complex real-world situations. To solve this, some studies employ real-world data. How- ever, they tend to focus on a singular type of obstruction, such as raindrops. This thesis addresses the more challenging task of restoring images taken through glass surfaces, which are impacted by various adherent obstructions such as dirt, raindrops, muddy raindrops, and other small foreign particles commonly found in real-life scenar- ios, including stone fragments and leaf particles. This work introduces an encoder-decoder network that incorporates auxiliary learning and an attention mechanism. During the test- ing phase, the auxiliary branch updates the shared internal hyperparameters of the model, enabling it to restore images from not limited to known categories of obstructions from the training dataset, but also unseen ones. To better accommodate real-world situations, this work presents a dataset comprising real-world adherent obstruction pairs, which cov- ers a large variety of common obstructions along with their corresponding clean ground truth images. Experimental results indicate that the proposed technique outperforms many existing methods in both quantitative and qualitative assessments.