Frequentist Model Averaging for ε-Support Vector Regression

Abstract

This thesis studies the problem of frequentist model averaging over a set of multiple $\epsilon$-support vector regression (SVR) models, where the support vector machine (SVM) algorithm was extended to function estimation involving continuous targets, instead of categorical ones. By assigning weights to a set of candidate models instead of selecting the least misspecified one, model averaging presents a strong alternative to model selection for tackling model uncertainty. Not only do we describe the construction of smoothed BIC/AIC model averaging weights, but we also propose a Mallows model averaging procedure which selects model weights by minimizing Mallows' criterion. We conduct two studies where the set of candidate models can either include or not include the true model by making use of simulated random samples obtained from different data-generating processes of analytic form. In terms of mean squared error, we demonstrate that our proposed method outperforms other model averaging and model selection methods that were tested, and the gain is more substantial for smaller sample sizes with larger signal-to-noise ratios.