Roadside Fisheye Vision for Detection, Localization, and Movement Classification of Road Users at Intersections

Abstract

This thesis addresses key challenges in intersection traffic monitoring using overhead fisheye cameras, focusing on object detection, localization, vehicle maneuver classification, and traffic violation detection. A data augmentation technique was developed to improve the performance of deep learning-based object detection algorithms for fisheye images. By fine-tuning these models, significant improvements in Average Precision (AP) were achieved for vehicle and pedestrian detection, effectively addressing object orientation and size variability.

A novel calibration method was introduced to mitigate the effects of road surface elevation changes on object localization. This method accurately translates image coordinates into geographical coordinates by incorporating 3D road characteristics. The proposed localization algorithm, validated through field tests, demonstrated high accuracy in localizing both cars and pedestrians. Furthermore, Kalman filtering techniques were integrated to enhance object tracking, providing precise localization even in complex environments like sloped streets.

In addition, a self-learning vehicle maneuver classification and counting algorithm was developed, capable of recognizing various vehicle movements such as turns and U-turns. The algorithm’s performance was validated in real-world scenarios, where it successfully classified and counted vehicle maneuvers at multiple intersections. Moreover, a traffic violation detection system was designed on top of the maneuver classification algorithm to identify common infractions like box-blocking and illegal turns at intersections.

The outcomes of this research contribute to a comprehensive system that enhances traffic monitoring, safety enforcement, and operational efficiency at intersections, offering practical solutions to modern traffic management challenges.