Weighted Feature Classification

Abstract

<p>Most existing classification algorithms either consider all features as equally important (equal weights), or do not analyze consistency of weights assigned to features. When features are not equally important, assigning consistent weights is a not obvious task. In general we have two cases. The first case assumes that a given sample of data does not contain any clue about the importance of features, so the weights are provided by a pool of experts and they are usually inconsistent. The second case assumes that the given sample contains some information about features importance, hence we can derive the weights directly from the sample. In this thesis we deal with both cases. Pairwise Comparisons and Weighted Support Vector Machines are used for the first case. For the second case a new approach based on the observation that the feature importance could be determined by the discrimination power of features has been proposed. For the first case, we start with pairwise comparisons to rank the importance of features, then we use distance-based inconsistency reduction to refine the weights assessment and make comparisons more precise. As the next step we calculate the weights through the fully-consistent or almost consistent pairwise comparison tables. For the second case, a novel concept of feature domain overlappings has been introduced. It can measure the feature discrimination power. This model is based on the assumption that less overlapping means more discrimination ability, and produces weights characterizing the importance of particular features. For both cases Weighted Support Vector Machines are used to classify the data. Both methods have been tested using two benchmark data sets, Iris and Vertebal.</p> <p>The results were especially superior to those obtained without weights.</p>