Display Publications by [Year] [Type] [Topic]
In difficult classification problems, the performance of a single classifier is often inadequate. In such cases, several researchers have combined the outputs of multiple classifiers to obtain better performance. This dissertation provides an analytical framework to quantify the improvements due to combining in classification problems. We show that combining classifiers in output space reduces the variance of the actual decision region boundaries around the optimum boundary. For linear combiners, we show that, in the absence of classifier bias, the added classification error, or the error due to the selection of a particular classifier, is proportional to the boundary variance. In the presence of bias, the error reduction is shown to depend on two factors, and in general is less than the reduction obtained in the absence of bias. Furthermore, we introduce the order statistics combiners, a family of non-linear combiners. We show analytically that the selection of the median, the maximum, and in general the ith order statistic improves classifier performance.
Another aspect of pattern classification that is of paramount importance is determining the fundamental limits on classification performance. The Bayesian error is such a limit, but is generally difficult to estimate. We provide a method based on linear combining theory that estimates the Bayesian error rates. Then, we present a more general method that provides decision confidences and error bounds based on different error types arising from the training data. Experimental results on both a difficult underwater acoustic data set and several public domain data sets are provided to confirm that both linear and order statistics combiners lead to improved classification performance, and that, in most cases, the results are near the fundamental limits.
(unavailable)
@phdthesis{tumer_phd96,
author="K. Tumer",
title="Linear and Order Statistics Combiners for Reliable Pattern
Classification",
department={Electrical and Computer Engineering},
school={The University of Texas},
address={Austin, TX},
month={May},
abstract ={In difficult classification problems, the performance of a single classifier is often inadequate. In such cases, several researchers have combined the outputs of multiple classifiers to obtain better performance. This dissertation provides an analytical framework to quantify the improvements due to combining in classification problems. We show that combining classifiers in output space reduces the variance of the actual decision region boundaries around the optimum boundary. For linear combiners, we show that, in the absence of classifier bias, the added classification error, or the error due to the selection of a particular classifier, is proportional to the boundary variance. In the presence of bias, the error reduction is shown to depend on two factors, and in general is less than the reduction obtained in the absence of bias. Furthermore, we introduce the order statistics combiners, a family of non-linear combiners. We show analytically that the selection of the median, the maximum, and in general the ith order statistic improves classifier performance.
<p> Another aspect of pattern classification that is of paramount importance is determining the fundamental limits on classification performance. The Bayesian error is such a limit, but is generally difficult to estimate. We provide a method based on linear combining theory that estimates the Bayesian error rates. Then, we present a more general method that provides decision confidences and error bounds based on different error types arising from the training data. Experimental results on both a difficult underwater acoustic data set and several public domain data sets are provided to confirm that both linear and order statistics combiners lead to improved classification performance, and that, in most cases, the results are near the fundamental limits.},
bib2html_pubtype = {Thesis},
bib2html_rescat = {Classifier Ensembles, Bayes Error Estimation},
year ={1996}
}
Generated by bib2html.pl (written by Patrick Riley ) on Wed Apr 01, 2020 17:39:43