194 Pages
English

Benchmark experiments [Elektronische Ressource] : a tool for analyzing statistical learning algorithms / vorgelegt von Manuel J. A. Eugster

-

Gain access to the library to view online
Learn more

Description

Benchmark ExperimentsA Tool for Analyzing Statistical Learning AlgorithmsManuel J. A. EugsterMunchen 2011Benchmark ExperimentsA Tool for Analyzing Statistical Learning AlgorithmsManuel J. A. EugsterDissertationzur Erlangung des akademischen Gradeseines Doktors der Naturwissenschaftenam Institut fur Statistikan der Fakultat fur Mathematik, Informatik und Statistik der Ludwig-Maximilians-Universitat Munchen Vorgelegt vonManuel J. A. Eugsteram 25. Januar 2011in MunchenErstgutachter: Prof. Dr. Friedrich Leisch, LMU MunchenZweitgutachter: Prof. Dr. Achim Zeileis, LFU InnsbruckRigorosum: 16. Marz 2011AbstractBenchmark experiments nowadays are the method of choice to evaluate learn-ing algorithms in most research elds with applications related to statisticallearning. Benchmark experiments are an empirical tool to analyze statisticallearning algorithms on one or more data sets: to compare a set of algorithms, tond the best hyperparameters for an algorithm, or to make a sensitivity analy-sis of an algorithm. In the main part, this dissertation focus on the comparisonof candidate algorithms and introduces a comprehensive toolbox for analyzingsuch benchmark experiments.

Subjects

Informations

Published by
Published 01 January 2011
Reads 7
Language English
Document size 4 MB

Benchmark Experiments
A Tool for Analyzing Statistical Learning Algorithms
Manuel J. A. Eugster
Munchen 2011Benchmark Experiments
A Tool for Analyzing Statistical Learning Algorithms
Manuel J. A. Eugster
Dissertation
zur Erlangung des akademischen Grades
eines Doktors der Naturwissenschaften
am Institut fur Statistik
an der Fakultat fur Mathematik, Informatik und Statistik
der Ludwig-Maximilians-Universitat Munchen
Vorgelegt von
Manuel J. A. Eugster
am 25. Januar 2011
in MunchenErstgutachter: Prof. Dr. Friedrich Leisch, LMU Munchen
Zweitgutachter: Prof. Dr. Achim Zeileis, LFU Innsbruck
Rigorosum: 16. Marz 2011Abstract
Benchmark experiments nowadays are the method of choice to evaluate learn-
ing algorithms in most research elds with applications related to statistical
learning. Benchmark experiments are an empirical tool to analyze statistical
learning algorithms on one or more data sets: to compare a set of algorithms, to
nd the best hyperparameters for an algorithm, or to make a sensitivity analy-
sis of an algorithm. In the main part, this dissertation focus on the comparison
of candidate algorithms and introduces a comprehensive toolbox for analyzing
such benchmark experiments. A systematic approach is introduced { from ex-
ploratory analyses with specialized visualizations (static and interactive) via
formal investigations and their interpretation as preference relations through
to a consensus order of the algorithms, based on one or more performance mea-
sures and data sets. The performance of learning algorithms is determined by
data set characteristics, this is common knowledge. Not exactly known is the
concrete relationship between characteristics and algorithms. A formal frame-
work on top of benchmark experiments is presented for investigation on this
relationship. Furthermore, benchmark experiments are commonly treated as
xed-sample experiments, but their nature is sequential. First thoughts on a
sequential framework are presented and its advantages are discussed. Finally,
this main part of the dissertation is concluded with a discussion on future
research topics in the eld of benchmark experiments.
The second part of the dissertation is concerned with archetypal analysis.
Archetypal analysis has the aim to represent observations in a data set as
convex combinations of a few extremal points. This is used as an analysis
approach for benchmark experiments { the identication and interpretation of
the extreme performances of candidate algorithms. In turn, benchmark ex-
periments are used to analyze the general framework for archetypal analyses
worked out in this second part of the dissertation. Using its generalizability,
the weighted and robust archetypal problems are introduced and solved; and
in the outlook a generalization towards prototypes is discussed.
The two freely available R packages { benchmark and archetypes { make the
introduced methods generally applicable.