Robust Discriminant Analysis With Asymmetric Classes

Abstract

Discriminant analysis uses labelled observations to infer the labels of unlabelled observations in a population. Despite many advances in unsupervised and, to a lesser extent, semi-supervised learning over the past decade, discriminant analysis is often employed using approaches that date back to very well-known work of Fisher in the 1930s. One notable exception is mixture discriminant analysis, where the labels are estimated using parametric finite mixture models, commonly the Gaussian mixture model. The supposed advantage with mixture discriminant analysis is that multiple Gaussian components can be used for each class, hence providing a work around when a class is not Gaussian. This thesis makes several contributions to ``modern" discriminant analysis. Three robust discriminant analysis methods are introduced using mixtures of multivariate t-distributions, mixtures of multivariate power exponential distributions, and mixtures of contaminated Gaussian distributions, respectively. This provides an appealing framework for handling varying tail-weights and peakedness in the classes that may also contain mild outliers. To facilitate the modelling of asymmetric classes, we also explore robust discriminant analysis via finite mixtures of generalized hyperbolic distributions and mixtures of multivariate skew-t distributions. These approaches are tailored towards skewed classes but also have the added advantage of modelling symmetric classes where necessary. Finally, we introduce an approach that combines support vector machines with mixture discriminant analysis. This approach defines class boundaries in the labelled observations and, in some sense, improves mixture discriminant analysis performance. Crucially, in all of our mixture modelling work, we consider the case where the number of components per class is one. The utility of the approaches introduced is demonstrated on simulated and real data sets.