Kagan Tumer's Publications

Display Publications by [Year] [Type] [Topic]


Linear and Order Statistics Combiners for Reliable Pattern Classification. K. Tumer. Ph.D. Thesis, The University of Texas, Austin, TX, 1996.

Abstract

In difficult classification problems, the performance of a single classifier is often inadequate. In such cases, several researchers have combined the outputs of multiple classifiers to obtain better performance. This dissertation provides an analytical framework to quantify the improvements due to combining in classification problems. We show that combining classifiers in output space reduces the variance of the actual decision region boundaries around the optimum boundary. For linear combiners, we show that, in the absence of classifier bias, the added classification error, or the error due to the selection of a particular classifier, is proportional to the boundary variance. In the presence of bias, the error reduction is shown to depend on two factors, and in general is less than the reduction obtained in the absence of bias. Furthermore, we introduce the order statistics combiners, a family of non-linear combiners. We show analytically that the selection of the median, the maximum, and in general the ith order statistic improves classifier performance.

Another aspect of pattern classification that is of paramount importance is determining the fundamental limits on classification performance. The Bayesian error is such a limit, but is generally difficult to estimate. We provide a method based on linear combining theory that estimates the Bayesian error rates. Then, we present a more general method that provides decision confidences and error bounds based on different error types arising from the training data. Experimental results on both a difficult underwater acoustic data set and several public domain data sets are provided to confirm that both linear and order statistics combiners lead to improved classification performance, and that, in most cases, the results are near the fundamental limits.

Download

(unavailable)

BibTeX Entry

@phdthesis{tumer_phd96,
	author="K. Tumer",
	title="Linear and Order Statistics Combiners for Reliable Pattern 
		Classification",
	department={Electrical and Computer Engineering},
	school={The University of Texas},
	address={Austin, TX},
	month={May},
	abstract ={In difficult classification problems, the performance of a single classifier is often inadequate.  In such cases, several researchers have combined the outputs of multiple classifiers to obtain better performance.  This dissertation provides an analytical framework to quantify the improvements due to combining in classification problems.  We show that combining classifiers in output space reduces the variance of the actual decision region boundaries around the optimum boundary.  For linear combiners, we show that, in the absence of classifier bias, the added classification error, or the error due to the selection of a particular classifier, is proportional to the boundary variance.  In the presence of bias, the error reduction is shown to depend on two factors, and in general is less than the reduction obtained in the absence of bias.  Furthermore, we introduce the order statistics combiners, a family of non-linear combiners.  We show analytically that the selection of the median, the maximum, and in general the ith order statistic improves classifier performance.
<p> Another aspect of pattern classification that is of paramount importance is determining the fundamental limits on classification performance. The Bayesian error is such a limit, but is generally difficult to estimate.  We provide a method based on linear combining theory that estimates the Bayesian error rates.  Then, we present a more general method that provides decision confidences and error bounds based on different error types arising from the training data.  Experimental results on both a difficult underwater acoustic data set and several public domain data sets are provided to confirm that both linear and order statistics combiners lead to improved classification performance, and that, in most cases, the results are near the fundamental limits.},
	bib2html_pubtype = {Thesis},
	bib2html_rescat = {Classifier Ensembles, Bayes Error Estimation},
	year ={1996}
}

Generated by bib2html.pl (written by Patrick Riley ) on Tue Jun 26, 2018 19:10:42