Automatic regression benchmark system
28 Pages
English

Automatic regression benchmark system

-

Downloading requires you to have access to the YouScribe library
Learn all about the services we offer

Description

Automatic regression benchmark system
Nicolas Desprès
LRDE seminar, June 8, 2005, revision 601
http://www.lrde.epita.fr/ Copying this document
Copying this document
cCopyright 2005 LRDE.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.2
or any later version published by the Free Software Foundation; with the
Invariant Sections being just “Copying this document”, no Front Cover
Texts, and no Back Cover Texts.
A copy of the license is provided in the file COPYING.DOC.
Automatic regression benchmark system, Nicolas Desprès LRDE seminar, June 8, 2005 1 Introduction
Introduction
A regression benchmark system is about keeping track of program
performance through different versions.
Automatic regression benchmark system, Nicolas Desprès LRDE seminar, June 8, 2005 2 Table of Contents
Table of Contents
Generalities.......................................................................................... 4
Regression benchmark system ......................................................... 9
Ranch.................................................................................................... 14
Precision............................................................. 21
Automatic regression benchmark system, Nicolas Desprès LRDE seminar, June 8, 2005 3 Generalities
Generalities
Automatic regression benchmark system, Nicolas Desprès LRDE seminar, June 8, 2005 4 Generalities What does benchmark ...

Subjects

Informations

Published by
Reads 62
Language English
AutomaticregressionbenchmarksystemEDRLNicolasDesprèsseminar,June8,2005,revisionhttp://www.lrde.epita.fr/106
CopyingthisdocumentCopyingthisdocumentCopyright c2005LRDE.Permissionisgrantedtocopy,distributeand/ormodifythisdocumentunderthetermsoftheGNUFreeDocumentationLicense,Version1.2oranylaterversionpublishedbytheFreeSoftwareFoundation;withtheInvariantSectionsbeingjust“Copyingthisdocument”,noFront-CoverTexts,andnoBack-CoverTexts.AcopyofthelicenseisprovidedintheleCOPYING.DOC.Automaticregressionbenchmarksystem,NicolasDesprès-LRDEseminar,June8,20051
IntroductionIntroductionAregressionbenchmarksystemisaboutperformancethroughdifferentversions.keepingAutomaticregressionbenchmarksystem,NicolasDesprès-LRDEseminar,June8,2005kcartfoprogram2
TableofContentsTableofContentsGeneralities..........................................................................................Regressionbenchmarksystem.........................................................Ranch....................................................................................................Precision..............................................................................................Automaticregressionbenchmarksystem,NicolasDesprès-LRDEseminar,June8,20054941123
GeneralitiesAutomaticregressionbenchmarkGeneralitiessystem,NicolasDesprès-LRDEseminar,June8,20054
GeneralitiesAbenchmarkis:.1.2Whatdoesbenchmarkmeans?Whatdoesbenchmarkmeans?Anypermanentmarktowhichotherlevelsmaybereferred.(dic)Atestorseriesoftestsdesignedtocomparethequalitiesorperformanceofdifferentdevicesofthesametype.Certainsetsofcomputerprogramsaremuchusedasbenchmarksforcomparingtheperformanceofdifferentcomputers,especiallybycomparingthetimeittakestocompleteatest.(dic)Automaticregressionbenchmarksystem,NicolasDesprès-LRDEseminar,June8,20055
GeneralitiesDifferentbenchmarkcategoriesDifferentbenchmarkcategoriesWecandistinguishdifferentkindsofbenchmarks:regression,scalability.Automaticregressionbenchmarksystem,NicolasDesprès-LRDEseminar,June8,20056
GeneralitiesDifferentmeasurecategoriesSeveraltypesofmeasuresseveralunits.Wewanttohandledifferentkindsofmeasures:DifferentmeasurecategoriesHowlongdoestheprogramstayintheusercontext?Howmuchmemorydoesituse?Howmanypacketsdoesitsendthroughthenetwork?Howbigarethey?Howmanypixelsdoesitaccess?andsoon...Automaticregressionbenchmarksystem,NicolasDesprès-LRDEseminar,June8,20057
GeneralitiesDifferentinputdatatypes/sizesDifferentinputdatatypes/sizesSeveralinputdatatypesseveralunits.AninputsizeN=200maymean:anautomatonof200statesinVaucanson;animageof200×200pixelsinOlena.Automaticregressionbenchmarksystem,NicolasDesprès-LRDEseminar,June8,20058
RegressionbenchmarksystemRegressionbenchmarksystemAutomaticregressionbenchmarksystem,NicolasDesprès-LRDEseminar,June8,20059
RegressionbenchmarksystemWewantto:MotivationsdetectperformanceregressionASAP;automatebenchmarkexecutions;provideagenericresultanalyzer/visualizer;centralizeresultstorage.Automaticregressionbenchmarksystem,NicolasDesprès-LRDEseminar,June8,2005Motivations01