Algorithmic Rectification of Visual Illegibility under Extreme Lighting

Abstract

Image and video enhancement, a classical problem of signal processing, has remained a very active research topic for past decades. This technical subject will not become obsolete even as the sensitivity and quality of modern image sensors steadily improve. No matter what level of sophistication cameras reach, there will always be more extreme and complex lighting conditions, in which the acquired images are improperly exposed and thus need to be enhanced.

The central theme of enhancement is to algorithmically compensate for sensor limitations under ill lighting and make illegible details conspicuous, while maintaining a degree of naturalness. In retrospect, all existing contrast enhancement methods focus on heightening of spatial details in the luminance channel to fulfil the goal, with no or little consideration of the colour fidelity of the processed images; as a result they can introduce highly noticeable distortions in chrominance. This long-time much overlooked problem is addressed and systematically investigated by the thesis. We then propose a novel optimization-based enhancement algorithm, generating optimal tone mapping that not only makes maximal gain of contrast but also constrains tone and chrominance distortion, achieving superior output perceptual quality against severe underexposure and/or overexposure.

Besides, we present a novel solution to restore images captured under more challenging backlit scenes, by combining the above enhancement method and feature-driven, machine learning based segmentation. We demonstrate the superior performance of the proposed method in terms of segmentation accuracy and restoration results over state-of-the-art methods.

We also shed light on a common yet largely untreated video restoration problem called Yin-Yang Phasing (YYP), featured by involuntary, intense fluctuation in intensity and chrominance of an object as the video plays. We propose a novel video restoration technique to suppress YYP artifacts while retaining temporal consistency of objects appearance via inter-frame, spatially-adaptive optimal tone mapping. Experimental results are encouraging, pointing to an effective and practical solution to the problem.