Selection of Video Descriptors: Generating Compact Descriptor Sets for Video Pairwise-Matching

Abstract

This thesis presents several descriptor selection schemes for video content pairwise-matching tasks. Those proposed schemes attempt to leverage two significant properties of videos, temporal correlation and motion information.

Aiming to find an efficient and descriptive representation for a video sequence, the concept of descriptor persistency is defined. Those descriptors that satisfy this definition are called persistent descriptors. In order to exploit descriptor persistency, an encoder is proposed.

The proposed encoder consists of five main components. First, keyframe labelling is introduced to reduce complexity and ensure a reasonable size of persistent sets. After that, persistent descriptor detection is performed on each group of pictures (GOP) separately. The second component is the standard SIFT descriptor extraction. The third part is to identify persistent descriptors from each GOP, called persistent descriptors extraction. In this stage, three different methods are proposed: The direct method and two approximation approaches. Persistent descriptors selection, which is the fourth stage, is carried out to control the size of the persistent set. For this stage, three selection methods are proposed. All of them attempt to utilize the motion information to select more descriptive descriptors among all the persistent descriptors in the GOP. In order to perform pairwise-matching, in this thesis, a simple but efficient pairwise-matching method is proposed.

Experiments are carried out to evaluate the performance of the proposed schemes. The datasets used for performance evaluation are subsets from the categories that describe in [1]. Two metrics de ned in [2], namely false positive rate (FPR) and true positive rate (TPR), are used for the performance evaluation.