Improving Cross-validation Based Classifier Selection using Meta-Learning

In this paper we compare classifier selection using cross-validation with meta-learning, using as metafeatures both the cross-validation errors and other measures characterizing the data. Through simulation experiments we demonstrate situations where metalearning offers better classifier selections than ordinary cross-validation. The results provide some evidence to support meta-learning not just as a more time efficient classifier selection technique than cross-validation, but potentially as more accurate. It also provides support for the usefulness of data complexity estimates as metafeatures for classifier selection.