MACHINE LEARNING AND DEEP LEARNING ALGORITHMS IN THERMAL IMAGING VEHICLE PERCEPTION SYSTEMS

Abstract

Modern Advanced Driver Assistant Systems (ADAS) focus more on daytime driving and primarily use daylight cameras as the main vision sources to detect, classify, and track objects. However, evidence has proved that autonomous driving using such a setup is compromised in the dark, and consequently, resulting in accidents. The hypothesis is that adding an infrared camera to the existing ADAS will boost the detection rate and accuracy, and further enhance the overall safety. This thesis investigates how well a standalone infrared camera performs onboard vehicle perception tasks such as object detection and classification using both machine learning and deep learning algorithms. Given a custom labeled infrared driving dataset that contains 4 classes of objects, “People”, “Vehicle”, “Bicycle”, and “Animal”, multiple attempts and improvements of training a supervised learning model, namely the linear multi-class Support Vector Machine (SVM) has been made by using various image preprocessing and feature extraction methods to detect the objects. During training, hard example mining is used to reduce the number of false classifications. This SVM employs a One-Against-All (OAA) styled approach and uses the image pyramid technique to enable multi-scale detection. On the deep learning side, a Convolutional Neural Network (CNN) based state-of-the-art detector, the YOLOv4 family including the full-sized and tiny YOLOv4 has been selected, trained, and tested at different input sizes using the same dataset. Labeling format conversion is performed to make this work. The results show that using bilateral filtering with the Histogram of Oriented Gradients (HOG) feature to train an SVM is preferable and is more accurate than the YOLOv4 family. However, the YOLOv4 networks are significantly faster. Overall, a standalone infrared camera cannot provide dominant detection results, but it can definitely supply useful information to the ADAS and complement other sensory devices for improved safety.