Convolutional Neural Network Detection and Classification System Using an Infrared Camera and Image Detection Uncertainty Estimation

Abstract

Autonomous vehicles are equipped with systems that can detect and track the objects in a vehicle’s vicinity and make appropriate driving decisions accordingly. Infrared (IR) cameras are not typically employed on these systems, but the new information that can be supplied by IR cameras can help improve the probability of detecting all objects in a vehicle’s surroundings. The purpose of this research is to investigate how IR imaging can be leveraged to improve existing autonomous driving detection systems. This research serves as a proof-of-concept demonstration. In order to achieve detection using thermal images, raw data from seven different driving scenarios was captured and labelled using a calibrated camera. Calibrating the camera made it possible to estimate the distance to objects within the image frame. The labelled images (ground truth data) were then used to train several YOLOv2 neural networks to detect similar objects in other image frames. Deeper YOLOv2 networks trained on larger amounts of data were shown to perform better on both precision and recall metrics. A novel method of estimating pixel error in detected object locations has also been proposed which can be applied to any detection algorithm that has corresponding ground truth data. The pixel errors were shown to be normally distributed with unique spreads about different ranges of y-pixels. Low correlations were seen in detection errors in the x-pixel direction. This methodology can be used to create a gate estimation for the detected pixel location of an object. Detection using IR imaging has been shown to have promising results for applications where typical autonomous sensors can have difficulties. The work done in this thesis has shown that the additional information supplied by IR cameras has potential to improve existing autonomous sensory systems.