Robust Sequential View Planning for Object Recognition Using Multiple Cameras

Abstract

<p> In this thesis the problem of object recognition/pose estimation using active sensing is investigated. It is assumed that multiple cameras acquire images from different view angles of an object belonging to a set of a priori known objects. The eigenspace method is used to process the sensory observations and produce an abstract measurement vector. This step is necessary to avoid the manipulation of the original sensor data, i.e. large images, that can render the sensor modelling and matching process practically infeasible.</p> <p> The eigenspace representation is known to have shortcomings in dealing with structured noise such as occlusion. To overcome this problem, models of occlusions and sensor noise have been incorporated into the probabilistic model of sensor/object to increase robustness with respect to such uncertainties. The active recognition algorithm has also been modified to consider the possibility of occlusion, as well as variation in the occlusion levels due to camera movements.</p> <p> A recursive Bayesian state estimation problem is formulated to model the observation uncertainties through a probabilistic scheme. This enables us to identify the object and estimate its pose by fusing the information obtained from individual cameras. To this end, an extensive training step is performed, providing the system with the sensor model required for the Bayesian estimation. In order to enhance the quality of the estimates and to reduce the number of images taken, we employ active real-time viewpoint planning strategies to position cameras. For that purpose, the positions of cameras are controlled based on two different statistical performance criteria, namely the Mutual Information (MI) and Cramér-Rao Lower Bound (CRLB).</p> <p> A multi-camera active vision system has been developed in order to implement the ideas proposed in this thesis. Comparative Monte Carlo experiments conducted with the two-camera system demonstrate the effectiveness of the proposed methods in object classification/pose estimation in the presence of structured noise. Different concepts introduced in this work, i.e., the multi-camera data fusion, the occlusion modelling, and the active camera movement, all improve the recognition process significantly. Specifically, these approaches all increase the recognition rate, decrease the number of steps taken before recognition is completed, and enhance robustness with respect to partial occlusion considerably.</p>