293 Pages
English

Extraction of information from the dynamical activities of neural networks [Elektronische Ressource] / von David Rotermund

-

Gain access to the library to view online
Learn more

Description

Universität BremenExtraction of information from thedynamical activities of neural networksDavid RotermundSeptember 2007iiExtraction of information from thedynamical activities of neural networksVom Fachbereich fur¨ Physik und Elektrotechnikder Universit¨ at Bremenzur Erlangung des akademischen Grades einesDoktor der Naturwissenschaften (Dr. rer. nat.)genehmigte DissertationvonDipl. Phys. David Rotermundaus Delmenhorst1. Gutachter: Prof. Dr. rer. nat. Klaus Pawelzik2. Gutachter: Prof. Dr. rer. nat. Andreas KreiterEingereicht am: 11. September 2007Datum des Kolloquiums: 29. November 2007iiiiiAbstractInteracting with our dynamic environment requires to process huge amounts of sensorydata in short time. This incoming stream of information is combined with internalstates (e.g. memories or intentions) and results in actions. The fundamental mech-anisms behind this fast information processing are still not understood. Even howinformation is stored in, and transmitted with sequences of action potentials is stillunder heavy debate. This thesis provides novel ideas to accomplish fast informationprocessing, to understand adaptive coding strategies, and to perform unsupervised on-line learning of non-stationary representations.In its first, genuinely theoretical part (chapter 3 - Information Processing Spike bySpike) this thesis develops a new concept in the field of fast information processingwith single action potentials.

Subjects

Informations

Published by
Published 01 January 2007
Reads 32
Language English
Document size 10 MB

ersitätUniv

Extractiondynamical

vidDa

Bremen

aofctivitiesinformationofneuralfromnettheworks

undotermR

bSeptem

re

2007

ii

dynamicalExtractionaofctivitiesoinformationfneuralfromnetwtheorks

VomFachbereichf¨urPhysikundElektrotechnik

derUniversit¨atBremen

zurErlangungdesakademischenGradeseines
DoktorderNaturwissenschaften(Dr.rer.nat.)

rtationesDisgenehmigte

nvoDipl.Phys.DavidRotermund

elmenhorstDaus

1.Gutachter:Prof.Dr.rer.nat.KlausPawelzik

2.Gutachter:Prof.Dr.rer.nat.AndreasKreiter

Eingereichtam:11.September2007

DatumdesKolloquiums:29.November2007

ii

Abstract

iii

Interactingwithourdynamicenvironmentrequirestoprocesshugeamountsofsensory
datainshorttime.Thisincomingstreamofinformationiscombinedwithinternal
states(e.g.memoriesorintentions)andresultsinactions.Thefundamentalmech-
anismsbehindthisfastinformationprocessingarestillnotunderstood.Evenhow
informationisstoredin,andtransmittedwithsequencesofactionpotentialsisstill
underheavydebate.Thisthesisprovidesnovelideastoaccomplishfastinformation
processing,tounderstandadaptivecodingstrategies,andtoperformunsupervisedon-
linelearningofnon-stationaryrepresentations.
Initsfirst,genuinelytheoreticalpart(chapter3-InformationProcessingSpikeby
Spike)thisthesisdevelopsanewconceptinthefieldoffastinformationprocessing
withsingleactionpotentials.Theframeworkisbasedonstochasticgenerativemodels
usingPoissonianspiketrainsasinput.Itiscapableofrealizingarbitraryinput-output
functions,updatinganinternalrepresentationwitheachincomingspike,forperform-
ingcomputationsasfastaspossible.
Leavingthosepurelytheoreticalconsiderationsbehind,thesecondpartofthisthesis
(chapter4-SelectiveVisualAttentioninV4/V1)investigatesprinciplesofadaptive
neuralcodinginrealdata,focusingonthequestionhowaninternalcorticalstate,
evokedbyselectivevisualattention,modifiesinformationprocessinginthebrain.In
collaborationwithmonkeyneuro-physiologistswestudiedtheinfluenceofattention
onthediscriminabilityofvisualstimulithroughtheirneuronalcorrelatesrecordedas
epiduralfieldpotentials.
Thefinalpartinthisthesis(chapter5-StabilizingDecodingAgainstNon-Stationaries)
takesustowardsamedicalapplicationforextractinginternalbrainstatesfromneu-
ronalactivities.Forcontrollingprostheticdeviceswithbrainsignals,reliablealgo-
rithmsforestimatingtheintendedactionsofapersonarerequired.Amethodwas
designedwhichallowstostabilisetheestimatorofaneuro-prosthesisagainstdisrup-
tionsfromnon-stationaritiesinthecharacteristicsofcodingtheintendedactions,and
fromchangesintheirrepresentationsinthemeasuredneuronalcorrelates.
Takentogether,thisthesispresentedthreenewcontributions:
Atheoreticalmethodofprocessinginformationspikebyspikeinafastandefficient
fashion.Thisstudyalsoshowedthatitissufficienttouseneurons,generatingPoisso-
nianspiketrains,forperformingfastandefficientinformationprocessing(Ernstetal.,
2007b).Anewmechanism,producedthroughselectivevisualattention,wasrevealedthatren-
dersinformationaboutdifferentvisualstimuli,representedinγ-bandoscillatoryac-
tivityofneuronalpopulations,moredistinctforanexternalobserverandprobably
forthebrainitself.Italsoshowedthatinternalstatesofthebraincanaltertheneu-
ronalactivitypatterninacomplexmanneranditdemonstratedthatthepowerofthe
γ-bandcontainssignificantinformationaboutvisuallyperceivedshapes(Rotermund
2007a).al.,te

iv

A

mdetho

rmae(lik

describing

2006a).

orf

neuronal-prostheses

apablec

of

protecting

estimators

of

movements)againstnon-stationaries,forthecostofan

the

hismatcm

b

eenwet

the

iendedtn

nda

executed

ctiona

tendedin

ctionsa

errortraex

undRoterm(

ignals

et

al.,

sttenCon1Introduction1
2TheoreticalandBiologicalBackground7
2.1Encodinginformationintosequencesofactionpotentials........7
2.2Reconstructinginformationfromsequencesofactionpotentials....12
2.2.1Probabilities.............................13
2.2.2Informationmeasuresandlossfunctions.............16
2.2.3Propabilitybasedestimators....................20
2.2.4Discriminationandclassification.................25
2.3Modelingofneurons............................35
2.3.1Measuringneuronalresponses...................36
2.3.2Integrate-and-fireneurons.....................37
2.4Learningandusing(neuronal)networks.................42
2.4.1Feedforwardnetworks.......................43
2.4.2Bayesiannetworks.........................46
2.4.3MonteCarlomethodsandexpectationmaximisationalgorithm49
2.4.4Reinforcementlearning......................54
3InformationProcessingSpikebySpike59
3.1Motivation..................................59
i

ii

NTENTSCO

3.2ASpike-BasedGenerativeModel.....................
3.2.1BasicModel.............................
3.2.2FromPoissontoBernoulliProcesses...............
3.2.3FromDeterministictoProbabilisticDecomposition.......
3.2.4EstimationandLearningSpikebySpike.............
3.2.5Simplifiedalgorithmwithbatchlearning.............
3.3Results....................................
3.3.1ASimpleExample.........................
3.3.2Pre-Processing,Training,andClassification...........
3.3.3Booleanfunctions..........................
3.3.4HandwrittenDigits.........................
3.3.5HierarchicalNetworks.......................
3.3.6Stepstowardbiologicalplausibility................
3.3.7Artificialandnaturalimages....................
3.4SummaryandDiscussion.........................

4SelectiveVisualAttentioninV4/V1
4.1Motivation..................................
4.2Thevisualsystem..............................
4.2.1Retina................................
4.2.2Pathwaystoandthroughthevisualcortex............
4.2.3Visualattention..........................
4.3ExperimentalSetting,PreparationsandMethods............
4.3.1Theexperimentalsetting......................
4.3.2DataPreprocessing.........................
4.3.3DiscriminatingStimuliwithSVMs................

363636466586960717274757679889

103301601601701311611611911121

NTENTSCO

iii

4.4Results....................................122
4.4.1Discriminatingshapes.......................122
4.4.2Improvementofclassificationperformancesthroughattention.127
4.4.3Stimulus-specificsignalsandcoding................132
4.4.4Attentioninducedstimulus-specificsignalschanges.......135
4.4.5AttentioneffectsinV1.......................143
4.4.6Modellingstimulus-specificsignals.................146
4.4.7DiscriminatingtheAttentionalCondition.............152
4.4.8AttentiononMorphingShapes..................157
4.5SummaryandDiscussion..........................164

5StabilizingDecodingAgainstNon-stationaries
5.1Motivation............................
5.2NeuronalandComputationalBackground..........
5.2.1Motorsystemandmovementsofarms........
5.2.2Errorsignalsinthebrain...............
5.2.3Braincomputerinterfaces...............
5.3Themodelforthesimulations.................
5.3.1NeuralEncodingofIntendedMovement.......
5.3.2EstimationofIntendedMovement...........
5.3.3NeuralEncodingofPerceivedError.........
5.3.4Adaptation.......................
5.3.5ChoiceofParameters..................
5.4ResultsfromtheSimulations..................
5.5ConclusionandSummary...................

..............................................................................

169961071071571971481681781881981291391891

iv

onclusionCandSummary6

AAdditionalBackground
A.1Modelingofneurons............................
A.1.1HodgkinandHuxleymodel....................
A.1.2McCullochandPittsneurons...................
A.2Propabilitybasedestimators........................
A.2.1Minimummeansquarederrorestimator.............
A.2.2Linearminimummeansquarederrorestimator.........
A.3Recurrentnetworks............................
A.3.1Hopfieldnetworks..........................
A.3.2Boltzmannmachines........................
A.3.3Liquidstatemachine........................
A.4Generativemodels.............................
A.4.1HiddenMarkovmodel.......................
A.4.2Helmholtzmachines........................

NTENTSCO

203

213312231512712712022222222422522622622032

BInformationprocessingspikebyspike233
B.1PatternPre-Processing..........................233
B.2TrainingProcedures............................234
B.3ClassificationandComputationProcedures...............234
B.4DetailsandParametersfortheComputationofBooleanFunctions..235
B.5DetailsandParametersfortheClassificationofHandwrittenDigits..235

CStabilizingdecodingagainstnon-stationaries
C.1Theestimatorforthevelocity.......................
C.2Parameteradaptation...........................

237273932

NTENTSCO

dditionalAD

tuteraiLre

ionsublicatP

information

Acknowledgment

nslaufeLeb

/

sources

anksagungD

v

241

241

275

279

281

vi

NTENTSCO

1Chapter

ductiontroIn

Standinginthekitchenwhilecuttingvegetables,observingcookingpots,andtelephon-
inginparallelisanormalsceneinourdailylives.Inthisbusysituationaglassfilled
withwaterismovedaccidentallyovertheedgeofthetableandfallstowardthefloor.
Beforehittingtheground,theglassiscaughtbyafastarmmovement.Thiseveryday
situation,inwhichevensophisticatedrobotswouldfail,demonstratestheenormous
flexibilityandreliabilityofthehumanbraininprocessinghighamountsofinformation
withinveryshortperiodsoftime.

Letusexaminethisexampleinmoredetailforunderstandingwhythereisactuallyno
technicalcomplementthatcouldcompetewiththecomputationalcapabilitiesofthe
brain:Atypicalkitchencomprisesplentyandpartiallyoccludedobjects,butweare
Thecapablelighttoinginmateractybewithbrighandtswunlighithintthisorendimvironmencandlelightt,underorlweargelymayvaobservryingetheconditions.scene
fromdifferentpointsofview,butstillweareabletoexecuteourtask.Theperception
ofoursurroundingisfilteredinsuchawaythatweareabletofocusonlyontools
inandptegratesartsoftheitheenncomingvironmenmultipletsnecessarytreamsoforficnformationompletinginourtotoneask.cOuroherenntervpousercept,systemlike
comfrombtheiningkthenifevinisualourimagehand,ofinourhorderandtcoauttingvoidthevcuttingegetablesourselvweiths.theInthishapticfeedbacsituation,k
thesubjgectivlasselyslidingmoreoivmpertheortanttable’sproblem,edgedrarequiringwstheourbattenraintiontotfoocusatitsotallyresourcesdifferenquictaklynd
ontomanner.thenHoewwthesituation.brainpThiserformstaketsheplaceinnecessarylessinvthanarianastoecondbjectinanrecognition,extremelyinefficientegratest
theperceivedmultimodalinformation,andisabletoswitchbetweendifferenttasksis
largelynotunderstood,nordoweknowhowtoimplementallitscapabilitiesintoa
analogon.inehmac

Forunderstandingthesefunctionsofthebrainonehastoinvestigatethefundamental
principlesofinformationprocessingbasedonlargenetworksofnervecellswithvast
interconnections,subjectedtothetypicalphysiologicalconstraintsrevealedbyempir-

1

2

Chapter1:Introduction

studies.icalAprerequisitetoidentifytheseprinciplesistounderstandthe’language’usedbynerve
cellsforexchanginginformation.Sincetheimportanceofelectriccurrentforthecen-
tralnervoussystem(CNS)wasdiscoveredbyGalvani200yearsago,anunbelievable
amountofdetaileddatawascollected,butourunderstandingofhowobjects,internal
states,orpropertiesofsensoryinputarerepresentedinthepatternsofneuronalac-
tivitiesisstillfarfromcomplete.Forinferringcodingprinciplesfrommeasureddata,
informationextractionmethodsarenecessary.Thus,besidesawell-educatedguess
aboutthetypicalfeaturesofthedatawhichcarrythemaximumamountofinforma-
tion,thedevelopmentofalgorithmsandtoolsforextractingthisinformationfrom
observedneuronalactivitieshasbecomeanimportantpartofbrainresearch.These
methodscaninadditionbeusedtoquantifyhowinformationcodingortheneuronal
representationchangesindependenceontheactualsituation,task,orduringlearning.
Thiscanherebyfurtheradvanceourunderstandingofbrainfunctions.
InmythesisIwillanalysedifferentaspectsofinformationcodingandinformation
processinginthebrain.Theseinvestigationscovertherangefromtheoreticalstudies
uptothedevelopmentofalgorithmsforrealapplications.Beforefocusingonthe
differentpartsofmythesis,Iwillexplainthegeneralframeworkforthesestudies
whichcanbecondensedintothefollowingscheme:

InformationSentersthebrain,e.g.throughsensorsontheretinaforvisualstimulior
throughhaircellsforauditorystimuli.Theincominginformationisthentranscoded
intoaneuronalresponseXviatheencodingfunctionft(.).
Thefunctionft(.)isnotnecessarilystationaryovertime.Forseveralreasons,likee.g.
adaptationorlearningprocessesinthebrain,quitesubstantialchangesinft(.)can
occur.ButthisisnottheonlyfactorwhichcouldchangethemappingofSontothe
neuronalresponseX:Internalbrainstates,whicharedenotedbyVinthisframework,
cansubstantiallyalterthecoding.Thisincludesconditionsofattention,memories,or
othernon-observable(hidden)variablesofthesystem.
Asthelastcomponentinthisframework,S^(X)interpretsorperformsacomputation
ontheneuronalresponsesX.ThiscomputationcanbeimplementedintheCNSitself
orbeingperformedbyanexternalobserver.Forexample,S^(X)couldbeahigherbrain
areainferringthepresenceofacontourfromactivitiesXinprimaryvisualcortexoran
estimatorattemptingtoreconstructthevisualinformationS,respectively.Inhigher
brainareas,XoftenbarelydependsonS:suchatypicalsituationarisesinthecontext
of’readingthemind’usingbrainprosthesis,wheretheintentionV(whichisahidden
variabletous)ofahandicappedpersonV^(X)needstobeestimated.
Ofcourse,noisemaycompromiseeachoneoftheseprocessingandtransmissionsteps,
essentiallymakinginferenceahardtaskforboththebrainandtheresearcher.

3

Anastonishingfeatureofthemammalianbrainistherapidnessofhandlinglarge
amounwhethertsaonfainimalnformation.ispresenEtxporerimenabsentstsinhoawepdticturehatacanbecomplexexecutedtaskwlikeithind150etectingms
(Thorpeetal.,1996)andthataskilledballgameplayer(e.g.tabletennisorcricket)
needsonlyafewhundredsofmillisecondsforextractingandprocessingalltheneces-
expsaryerimentinformationsandotherfromeaxpperceiveriencesedbfromall’sdatrailyjectorylife,lik(ecLandatchandingaMfcLeoallingd,glass,2000).Tsuggesthese
suthatggestedinformation(Thorpehasettoal.,be2001)transmittedthatrapidandproinformationcessedrpapidlyrocesbsiyngourcanbbrain.eexItphaslainedbeenby
rank-ordercoding,butitislargelyunclearhowthisideamightworkunderrealistic
assumptionsonneuronalnoiseandinsituationswherestimuliarenotflashed,but
changinginacontinuousfashion.Inmythesis,Iwillinvestigateanalternativecoding
schemewhererapidprocessingandtransmissionofinformationcanbeimplemented
obfyactionbiological-potenmtiaeans,lsraeceivessumingdasstheensoirynformatievidence.onisstoredonlyinthe(relative)number

Thefindingasptheectooffastptimalcominformationbinationofftransmission(S)andS^corresp(X),ondsunderinstheelectedpresentedbiologicalframewborkoundaryto
conditionsdefinedthroughboundsonSandX.Inaprecursoryworkfocusingon
informationtransmissiononly,itwaspossibletoshowthatoptimalcodingstrategies
oflevel.storingAnalyticalinformationcintonsiderationsofiringrevratesealedofnphaseeuronsaretransitionsverysbetensitivweenetothefamiliesporesenfotptimalnoise
codingstrategiesfordifferentnoiselevels(Bethgeetal.,2003a;Bethgeetal.,2003b;
impBethgeortanettaspal.,ectof2002a).thewHoholewever,problem,fastiwhichnformationisattheprocescoresiofngourisiansvecond,estigationseveninmorethis
designthesis:anovAssumingelthatinformationthebproraincessingbuildsapalgorithmrobabilisticbasedrontheepresen(relativtatione)nfromumbitserofinput,singlewe
action-potentialsreceivedfromtheinputneurons.Thealgorithmallowstoextract
commonstructures,likesetsofbasisfunctions,frompresentedinputdistributions.
BoAlsoioleantisfpounctions.ssibletoWeusewillitfseeorpathattterntherrecoesultinggnitionorinformationtoperformprocessingcalculations,methodlikiseveery.g.
fastandextractsefficiently,incomparisonwithotherbenchmarkmethods,information
fromtheincomingaction-potentials.

Akeychallengeinbrainresearchistounderstandhowinformationaboutobjectsis
encodedintoneuronalactivities.Overtheyears,empiricalstudiescollectedvaluable
informationandestablishedseveralconceptsinunderstandingtheneuronalrepresen-
tationintheearlyvisualsystem(Carandinietal.,2005).However,themajorpart
ofthisworkconcentratesonlyonsingle,localizedaspectsofobjects,liketheorienta-
tionofanedgewithinasmallregioninvisualspace.Theneuronalrepresentationof
wholeobjectsorspatiallyextendedstimuliislargelyunknown,partlybecausereliable
multi-electroderecordingsbecameavailableonlyduringthepast10years,andpartly
becauseencodingquicklybecomesnon-linearwhenstimuliextendbeyondtheclassical
receptivefields.Althoughveryspecialisedneuronalpopulationwerefoundthatencode
complexobjectlikefamiliarpersons(Quirogaetal.,2005),itseemsthatthegeneric

4

Chapter1:Introduction

caseisthatobjectsarerepresenteddistributedoverlargeneuronalpopulationsoreven
overmultiplebrainareas.Thisfactprovokesthequestionhowdifferentattributesof
onesingleobjectsare’glued’togetherintoacoherentpercept(oftenlooselytermed
asthe’bindingproblem’).Ithasbeensuggestedthatthisactofcompositioncanbe
providedbysharedoscillatorybehaviourofneuronalpopulations(Friesetal.,2007).
Furthermore,itseemsthattherepresentationofobjectscanbeenhancedorsuppressed
byadaptingthecodingtoactualdemands.Visualselectiveattention,whichforex-
ampleallowsustoselectbehaviourallyrelevantpartsofavisualscenery,fallsunder
thiscategory.Inthisthesisitisstudiedhowdifferentvisualobjectsarerepresentedin
collectiveoscillatoryneuronalactivitypatternsandhowselectivevisualattentioncan
alterthesepatterns.Inastudyapplyingdataanalysismethodstoelectro-physiological
datafromanimalexperiments,wewillevaluatedifferentschemesofcodingobjectin-
formation,andcomparethemtoprinciplesdiscussedintheliterature.

Analysingneuronalactivitypatternstouncoverprinciplesofinformationcodingused
bythenervoussystemarenotsolelyperformedforacademicalreasons.Oneconcrete
applicationisfoundcurrentmedicalresearch:thedevelopmentofbrainprostheses.
Theideaofsuchaneuro-prostheticdeviceistohelphandicappedorparalysedpeople
antoarmregainmaovutonomement,y.fromThisthecouldpbatienet’saccomplisbrainahedctivitbyy,iwhicnferringhisansinubsequentendedtalyction,usedlikbye
amechanicaldeviceforexecutingthisaction.Manyresearchersareworkinginthis
fieldoffunctionalneuro-prosthetics,andtheirresults(Wolpawetal.,2002;Wolpaw
etsuggestal.,2000;thatsuchKueblerametaedicall.,2001;deviceCumarranybaendavaStokilablees,2003;withinHothechnbeextrgyetears.al.,In2006)a
brainprosthesis,hiddenstatesV,representingintendedactionslikearmmovements,
areextractedfrommeasuredneuronalactivitiesXbyanestimatorV^(X).Whilethe
performanceofthedevelopedestimatorsattainedovertheyearsareasonablelevel,
harvhighsestingpatialtheandntempecessaryoralramounestolutionofsdatatillfromremainstheCacNSovhallengeeral(Pongawpelzikeriodetofal.,time2006b).with
TheusedestimatorsV^(X)dependheavilyoncorrectknowledgeaboutthecoding
schemeft(V)forreconstructionmeaningfulinformationfromtheobservedneuronal
responses.Wronginformationaboutthecodingschemeleadstobadreconstruction
pmaeyberformances.subjecttoLearningongoingft=c0(hV)angesatonedueptoointnon-intimestationariesmaynotinsidebesuthefficienbraint,borecausatetheit
interfacebetweenbrainandmachine.Forlong-termmedicalapplicationsitwillbeim-
portanttofindstrategiesofcounteractingthesenon-stationaries,allowingtostabilise
thecontrolofneuro-prostheses.Inthisthesisanewon-lineadaptationstrategyis
presenstabilisationtedwihicsthoismableeasuretoancounadditionalteracttheseneurponalserturbations.ignalrelatedThetonovtheeliadeactualbpehinderceivethisd
errorbetweentheuser’sintention,andtheactionexecutedbytheprostheticdevice.
prosApplyingtheticthisdevicesovadaptationertime,astrategytthecohassttohefporequiringtentialtotoirecordncreaseastecohendstabilitsignaylosofsurce.uch

Thisthesiswillbesplitintodifferentparts,addressingthethreemaintopicsoutlined

above.Insummary,thesepartswillfocuson

5

•AtChapterheoretical3study(InformationispresenProtedcessingwhereSpikebyinformationSpike):aboutunderlyingsignalsSis
reconstructedasfastaspossiblefromneuronalresponsesX,whereXiscausedby
thesignalsSandincludesnoisegeneratedbyaPoissonianprocess/multinomial
process.Themodelisbasedontheassumptionthatthebrainusesaprobabilistic
representationofacombinationofcauseswhichgeneratedthereceivedinput.For
thisalgorithmreconstructionwillbedevperolopeceduredftromhefirstestimatorprinciples,S^(X)whichashtoulpdearnatesf(S,theVp).Arobabilisticsuitable
berepresenanalysed,tationwithwithfocuseachonhosinglewfastspikaendreceivpreciseed.thisPropertiesreconstructionofthisorialgorithmnformationwill
processingbasedonthisinputcanbedone.
•Chapter4(SelectiveVisualAttentioninV4/V1):
Inlearnthismorepart,abaoutthohoroughwobjdectsataaarenalysrepresenisisptedresinenoted,scillatorywiththesignalsfultimateromngoaleuronalto
populations.Thisincludestoinvestigatehowselectivevisualattentionenhances
the(theioutlinenformationofcondifferententtosfthapheess)ignal.presenThetedsitognalsmSonkaeryseininthisancaexpseevisuarimenlt.stimTheuli
neuronalresponsesXarerecordedfromthevisualcortexoftheanimals,while
vatheyriabalreesrVequiredthatcantopbeeirceivdenetSified.Xwisiththegeneratedtwodifferenundertdcifferenonditionststatesofofattenhtiddenion.
ThestimuliSwillbereconstructedfromX.Furthermore,itwillbeanalysedhow
thestateofthehiddenvariableVchangesf(S,V).
•TheChapterfinal5partof(StabilizingmytDhesisecowdingilldeveAgainstlopaNnaon-spproachtationaries):towardsasystemforpro-
tectingdataextractionalgorithmsoffunctionalneuro-prostheticdevicesagainst
non-stationaries.ThesignalsVareintendedactionsforcontrollingfunctional
neuro-prostheses.Theseintendedactions(e.g.armmovements)aredecoded
frothemesttheimator.neuroDnaluertoespcohnsesangesX.ofFfor(S,decoV)odingverV,time,ft(S,theV)knohaswtoledgebeolfeathernedesti-by
tformatorupdabatingoutfthist(S,V)information,hastobeabaseddaptedonaforfdditionalollowingneuronalthesercesphanges.onsesAdescribingstrategy
theactualperformanceoftheprostheticdevice,isdeveloped.

6

Chapter

:1

tronI

duction

2Chapter

BackTheoreticalgroundandBiological

2.1Encodinginformationintosequencesofaction
ialsttenop

hTheumanbbrainrainisonecomprisesoftheabmostout1012complexcellssandystemsthatineachnature.oftIthesewascellstyestimatedpicallythatreceivtehes
signals14from15hundredsuptothousandsofothernervecells,amountingtoapproximately
10to10connections(throughsynapses)intotal(Hubel,1989).Alongthesecon-
nections,informationismainlytransmittedbyactionpotentials.Anactionpotentialis
andgeneratedoutsidebyofantheexcnervhangeeocellfitonshat(e.g.traveplsoalongtassiumtheandsmemodbiumrane.ions)Sincebetweendifferenthetiansidection
ispobetentlievialsedt(alsohattcheiralled’shapspikeces’)earriesmittednofrominformation.oneneuronarenearlyindistinguishableit
Thereexistseveralhypotheseshowinformationcanbecarriedbyasequenceofspikes
(’spiketrain’).Forashortintroductionofthesehypothesesthedetailedshapeofthe
NoactionwwepocantentialsdescribwillebaesequenceignoredaofndspiktheesspikbyeswillbesubstitutedbyDiracδfunctions.
s(t)=δ(t−ti),
i

whereidenotesthei-thspikeattimeti.
In1926Adrian(MaassandBishop,2000)showedthattheactivityofmusclesand
themeannumberofspikesinagiventimeintervalarecorrelated.Sincethattime,
thepossibilityofencodinginformationinthemeanactivitiesofneurons,called’rate’
coding(DayanandAbbott,2001)wasinvestigatedintensively.Forobtainingtherate

7

8

Chapter2:TheoreticalandBiologicalBackground

Figure2.1:Schematicviewofanactionpotential.Afterthemembranepotentialofthe
nervecellwasrisenaboveathreshold,anactionpotentialisreleased.Differenttypesof
ionsstarttoflowfromtheinsideofthecellthroughmembranechannelstotheoutside
ofthecell,andvisaversa.Throughthedifferencesinthedynamicsofmembrane
channelsfordifferenttypesofions,theexchangeofionscreatesacharacteristicrise
andfalloftheelectricpotentialatthemembrane.Thisexcitationtravelsalongthe
membraneintotheaxonanddendriticcompartment.Ionpumpsinthemembrane
restoretheconcentrationdifferencesofionsbetweentheinsideandtheoutsideofthe
celllikeitwasbeforetheinitialisationoftheactionpotential.(Thefigurewasadapted
fromwikipedia.org)

rfromagivenspiketrains(t)onecomputes:

t1r=n=1s(τ)dτ.
TTt0

nrepresentsthenumberofspikesinatimeintervalwithlengthT,startingatt0and
.tatending1Experimentalinvestigationsofspiketrainsfromcorticalneuronsrevealahighvari-
abilityofneuronalresponsesevenwhenrepeatedlyusingthesamestimuli(Tomkoand
Crapper,1974;Tolhurstetal.,1983;Snowdenetal.,1992;BurnsandWebb,1976;
Brittenetal.,1993).Thisobservationsuggeststodescribeneuronalresponsesasa
stochasticprocess.Experimentallyithasbeenfound(ShadlenandNewsome,1998)
thattheinter-spike-intervaldistributionoftenapproximatelyfollowsanexponential
probabilitydistribution,ifaneuronfireswithaconstantrateoveraperiodoftime.
SuchdistributionsmayarisefromPoissonianpointprocesses,whosespikecountdis-
tributionsarefullydeterminedbythetrialaveragedrate<r>andatimewindow

2.1Encodinginformationintosequencesofactionpotentials

9

Tp(k|r·T)=k1!(<r>·T)ke−<r>·T.(2.1)
Drawingsamplesfromthisprobabilitydistributionwillyieldspikecountskwitha
meanvalueof<r>·Tandavarianceof<r>·T.

Figure2.2:Sketchofatypicaltuningcurveforavisualcorticalregion.Afullfield
gratingwithorientationφispresentedonascreenandtheresultingneuronalspike
activityismeasured.Onthelefthandside,thetuningfunctionforthedifferences
betweentheorientationofthebarandthepreferredorientation(orientationwith
themaximalneuronalresponse)oftheneuronisdisplayed.Ontherighthandside,
exemplaryspiketrainsevokedbythepresentedstimuliareshown.

Tonythepicallys,timtuhelusbactiviteingyofpresenanted.euronWdeepcanendsdonescribitseitnput,hiswdephicehndencymaybvayryallodepwingendingthe
averagedfiringrate<r>tobeafunctionofthestimulusx.Theaveragedrate
<rexample(x)>seeisFig.called2.2.nTeuronalhertuningesponsecurveintuningFig.curv2.2eis(orafshortunction’tuningdependingfunction’).solelyfonorthean
one-dimensionalvariableφ−φPreferred.Inthebrain,neuronalresponsesmaydependon
manyselectivities.featuresoSincefanstimeuronsulus.areOvftenastlytuningconnectedfunctionstocotherapturenonlyeurons,isingletispaspossibleectsoftthathesea
sounddetectedbyhailcellsintheearcanlaterinfluencetheactivityofneuronsinthe
ofvisualtheccortex.ompleteThustuningtuningfunctioncurvestothecalculatedtestedfsromtimmuluseasuredspace.dataToaremakonlyetheapsroituationjection
evenmorecomplicated:Stimulusx(t)maychangeovertime,responsesmaydepend
onthehistoryofactivities(e.g.fatigueandadaptationeffects),andresponsesmay
becorrelatedtotheactivityofneighbouringneurons(e.g.synchronisationeffects).

10

Chapter2:TheoreticalandBiologicalBackground

Nevertheless,theconceptoftuningfunctionsisoftenusedandinmanycasesvery
helpfull.Anotherimportantconceptforsensoryneuronsare’receptivefields’.Thereceptivefield
(RF)ofaneurondescribestheregionsinstimulusspacewheretheneuronreactstoa
stimuli(e.g.regionsontheretina).Forneuronsprocessingvisualsensoryinformation,
thiswoulde.g.bethespatialpositionofastimulusinthevisualfield.
Thefiringratecapturesonlytheinformationofhowmanyspikesoccurinsideagiven
intervaloftime.Informationregardingthetimingoftheactionpotentialsisignored,
butitisknownthatfluctuatinginputorstimuliwitharichtemporalstructurecan
generatespiketrainswithaprecisionofmilliseconds(MainenandSejnowski,1995;
Buracasetal.,1998;AzouzandGray,2000).Motivatedbythisevidencealternative
codinghypotheseswereproposed,e.g.’timingcode’and’rankordercode’.

Theideaofatimecodeistostoreinformationintheprecisetimingofspikesrelative
toareferenceevent(’latency’).Forexample,onecouldimaginethatthemeasured
timeinmillisecondsbetweentheoccurrenceofaspikeandareferenceeventrepresents
anumericalvalue.Theamountofinformationtransportedbyonespikeisthendeter-
minedbytheprecisionofthelatency(seeFig.2.3asanexample).Theoretically,ifa
periodoftimecouldbemeasuredwithinfiniteprecisionandaneuroncouldproduce
spikeswithinfiniteprecision,onespikewouldcarryaninfiniteamountofinformation.
Thiscodingschemehasthedrawbackthatthedecodingmechanismrequirestheprecise
latencyofthespike,implyingthatthedecodingneuronneedsaccesstotheincoming
spikeaswellastoreferenceevents(Thorpeetal.,2001).Nevertheless,experimental
evidenceforatimecodewasfound.Experimentsonhumanvolunteersshowedthat
responselatenciesareusedasacodeinsomatosensorytasks(JohanssonandBirznieks,
2004).Aparadigmwasusedwherethesubjecthadtoestimatethedistancebetween
twomechanicalstimulationsontheskin.Inthevisualsystemofblowfliesalatency
codingformotion-sensitiveneuronswasfound(WarzechaandEgelhaaf,2000),where
thelatencydecreaseswithincreasingcontrastandtemporalfrequencyofamoving
pattern.Additionallylatencycodingwasfoundinratbarrelcortex(Petersenetal.,
2002).

Sofar,wehavediscussedcodingschemesonlyforsingleneurons.Usingmorethan
oneneuronforcodingallowsfortheuseof’populationcodes’,whichallowtostore
additionalinformationbyusingcombinatorics.The’rankordercode’(Thorpeetal.,
2001;vanRullenandThorpe,2002)isoneofthesecombinatorialcodingstrategies
thatencodesinformationintotheorderofincomingactionpotentials(seeFig.2.3).
Fordecoding,thesequenceinwhichtheneuronswereactiveistakenandhastobe
comparedtoadictionary.Thisdictionaryallowstodecoderankordersequencesby
selectingthecorrespondingvaluesfortheobservedsequenceofactivities.Theband-
widthofthiscodingstrategyismainlydefinedbythenecessarytimeintervalbetween
tospikesforpreventingunwantedpermutationsduetonoise.Aneurophysiologically
plausibleimplementationofrankordercoding(vanRullenandThorpe,2002)seemsto

2.2Reconstructinginformationfromsequencesofactionpotentials

11

Figure2.3:Exampleofencodinginformationintotherankorderandthetimingof
actionpotentials.Thedottedverticallinesrepresenttheunitsofmeasuringlatencies
duetoprecision.(Thefigurewasadaptedfrom(Thorpeetal.,2001))

bemoresimplethanarealisationofatimingcodingwithbiologicalplausiblemeans.
Rankordercodingrequiresareferenceevent,otherwisethedecodingneuronwouldn’t
beabletodeterminewhenanewsequenceofspikesstarts.Forthevisualsystem,itwas
suggestedthatsaccadic(vanRullenandThorpe,2002)ormicro-saccadic(Martinez-
Condeetal.,2000)eyemovementscanbeusedassuchreferenceeventsforrankorder
ding.coimetand

Fromdetectingtheowhetherbservaationnathatnimal(hissuman)hownonbrainsascancreenreactvrequireseryfastlesstothanvisual150msstimuli(Thorp(e.g.e
etal.,1996)),itwasconcluded(Thorpeetal.,2001)thatratecodingistooslowfor
suchsuggestedoperationsthatbsomethingecauseoflikteheatimerankorderrequiredcodeforschouldountbingespikused.es.InTchus,hapterit3hasofbteenhis
thesisitwillbedemonstatedthatevenwithinformationprocessingbasedonrate
coding,complexcalculationscanbeperformedwithfewsinglespikes.

1997)Otherhandypcodothesesesuswingeresypncropohronsedy),(we.g.hichusingwewillcomplexnotdsispikcuseshere.patterns(Warlandetal.,

12

Chapter2:TheoreticalandBiologicalBackground

stimulusencodingnoisemodelresponsedecoding
x{f(x)}p(k|{f})k{g(k)}

Figure2.4:Schematicdiagramofencodinganddecodinginformationwithactionpo-
pten(kt|fials.)T(e.g.hePstimoissoulusnpxroiscess)intranslatedtoatviaempaosraletofandstuningpatialpfunctionsatternf(ofx)acandtionnpoiseotenmotialsdel
.k

2.2Reconstructinginformationfromsequencesof
actionpotentials

Thelastsectionwasconcernedabouthypotheseshowinformationcanbestoredinspike
trains.Fig.2.4showsaschematicillustrationofastimulusxthatiscodedinaneuronal
responsek.Thisresponsekcontainsmoreorlessinformationaboutthestimulusx.
Itisinterestingforseveralpurposes(e.g.functionsinneuro-prosthetics)toextractthe
storedinformationfromk,butitisimportanttounderstandthattheneuronalresponse
maynotallowtoreconstructxwithoutanyinaccuracies.Onereasoncanbethatthe
amountoftransmittedinformationisboundedbyconstraints,likee.g.limitednumber
ofneurons,limitationsontheavailabletimeforinformationtransmission,limitations
onavailableenergy,andneuronalactivityofneurons.Takentogetherthecapacityof
the’channel’(Shannon,1948),theinformationhastopass,maybelimited.

ReconstructiontransmissionAfterSourceFigure2.5:Exampleoftransmittinginformationthroughalimitedandunreliablechan-
nel.ThepixelvaluesI(x,y)∈[0,...,1]oftheoriginalimage(left)weretransmitted
throughchannelwithPoissoniannoise(withameanvalueofI(x,y)·5).Theoutput
ofthechannelisshowninthemiddleimage.Usinga’MinimumMeanSquaredError-
fromEstimator’thenois(seeydata.section(T2he.2.3o;wriginalithpflatictureprior)wasacreatesdaptedthefromrightwiikimagepeasdia.org)reconstruction

mationAdditionalpunreliable.roblemsAncaneaxamplerisefromisshtheowfnactintFig.hatt2he.5.chLoanneloselymasypeaking,transmitthethelimitedinfor-

2.2Reconstructinginformationfromsequencesofactionpotentials13

capacityofthechannelcanbeunderstoodasareductiononthenumberoftransmitted
symbols(e.g.sendingamessagebyusingonlythenumbers1,2,3and4)pertimeunit.
Theunreliabilityofthechannelmaygeneratemisinterpretationsresultingfromthere-
ceiverroredcsymorrectionbols.sIcthisemespossibleintototheureducesedcothede.Forunreliabilitexytractingbyintroinformationducingfromredundancyneuron-or
to-neuroncommunicationitisnecessarytotakeunreliabilities(e.g.bysynapses)into
account.Manydecodingstrategiesusemethodsfromstochasticsfordealingwiththese
thenproblems.omenclatureBeforeofstoexplainingchasticsthesewillbemethogivdesn.inmoredetail,ashortintroductioninto

Probabilities2.2.1

Expectationvalueandvariance

Assumingthatabutteredtoastmayhavetheprobabilityρoffallingonthesidewith
butterandtheprobability1−ρoffallingontheotherside,wecanusethebinomial
distribution(MacKay,2003)
p(r|ρ,N)=Nρr(1−ρ)N−r(2.2)
rNforcalculatingtheprobabilitypthatthebutteredsidewillhitthegroundrtimesout
ofNtimestossingthetoast.r=(N−Nr!)!r!arecalledbinomialcoefficients.
ThemeanvalueE[r]andthevarianceVar[r]forthebinomialdistributionaredefined
by

and

NE[r]=p(r|ρ,N)r=N·ρ
0=r

Var[r]=E(r−E[r])2
N=Er2−(E[r])2=p(r|ρ,N)r2−(E[r])2
0=r=Nρ(1−ρ).

withMoreapgeneral,robabilittheyfexpunctionectationp(r)valuewithEr[fas(ra)]mandvulti-dimensarianceiVaonalr[f(conrt)]inofuousafvaunctionriablef(arer)

14

bdefinedy

Chapter2:TheoreticalandBiologicalBackground

(2.3)(2.4)(2.5)

E[f(r)]=f(r)p(r)dr(2.3)
Var[f(r)]=(f(r)−E[f(r)])2p(r)dr(2.4)
1=p(r)dr(2.5)
orwithrasamulti-dimensionaldiscretevariableby
E[f(r)]=f(r)p(r)(2.6)
rVar[f(r)]=(f(r)−E[f(r)])2p(r)(2.7)
r1=p(r).(2.8)
rEq.(2.5)andEq.(2.8)representthenormalisationequations,whichallprobabilitydis-
fullfill.stumtributionsFurthermore,iftworandomvariablesr1andr2arestatisticallyindependentthenthe
followingrelationcanbeused:
E[r1+r2]=E[r1]+E[r2]
Var[r1+r2]=Var[r1]+Var[r2]

BayesTheorem

Aprobabilitydistributionforanensembleofrandomvariablesrands,istermedjoint
probabilitydistributionp(r,s).Ajointprobabilitycanbereduced(’marginalised’)to
amarginalprobabilitydistribution(MacKay,2003),byaveragingoveroneormore
.g.eriablesavp(r)=p(r,s).
sAlsoitispossibletousethe’productrule’(oralsocalled’chainrule’)forexpressinga
jointprobabilitywithaconditionalprobabilitydistribution
p(r|s)p(s)=p(r,s).
Asacombinationoftheseprocedureswegainthefollowingrules:
p(r,s)=p(r|s)p(s)=p(s|r)p(r)
p(r)=p(r,s)=p(r|s)p(s)
ss

2.2Reconstructinginformationfromsequencesofactionpotentials

AnotherderivationoftheproductruleistheimportantBayes’theorem:
p(r|s)=p(s|r)p(r)
)s(p

Bayesianinference

15

(2.9)

UsingtheBayesiantheorem,itispossibletocalculatetheconditionalprobability
ofunobservedvariables(e.g.parametersofthedistributionλ)giventheobserved
variables(e.g.measureddataD).ThiscomputationiscalledBayesianinference.
p(λ|D)=p(D|λ)·p(λ)
)D(pThespecificprobabilitiesintheBayestheoremareoftentermedlikelihoodofthe
parametersforp(D|λ),priorp(λ),evidence(ormarginallikelihood)forp(D)and
posteriorp(λ|D)(MacKay,2003).Wecanassigninterpretationstotheseprobabilities,
whichwillshowmoreclearlywhattheyrepresent:

•Posteriorp(λ|D)
Theposteriorisaconditionalprobabilityandallowstojudgehowprobablea
hypotheses(e.g.asetofparameters)aregivenasetof(observed)data.
•Priorp(λ)
Thepriorallowstointroduceknowledgeorhypothesesaboutparameters.A
simpleandfrequentlyusedchoiceisthe’uniformprior’.Itallowstoconstraina
parameteronaninterval(e.g.[α,β]):
1p(x)=β−0αforotherwiseα<x<β(2.10)
Formostproblems,thepriorwilldependonassumptionsandthusitwillalways
besubjective.
•Likelihoodp(D|λ)
Theconditionalprobabilitiesp(D|λ)characterisehowlikelyitisthatthisset
ofdatawillbegeneratedbyarandomprocesswiththeseparameters.This
conditionalprobabilitycanalsobeinterpretedasthelikelihoodL(λ|D)=p(D|
λ).Lisafunctionoftheparametersforagivensetofdata.Itisnotnormalised
overparameterspace.
•Marginallikelihoodp(D)
ThemarginallikelihooddescribestheprobabilitythatthissetofdataDwill
begeneratedbyagivenrandomprocess.Itcanbecalculatede.g.fromthe
likelihoodandpriorby
p(D)=p(D|λ)p(λ).
λ

16

Chapter2:TheoreticalandBiologicalBackground

2.2.2Informationmeasuresandlossfunctions

Aftertheoverviewregardingprobabilitiesandusingthemforcalculations,itisalso
importanthowtomeasuredistancesorsimilaritiesbetweenprobabilitydistributions
ortoquantifytherichnessofthestructureofadistributions.
Alossfunction(orutilityfunction)canbedemonstratedbythefollowingexample
(Bernardo,1979):Letusassumethatanexperimentisperformedinwhicharandom
variablexisobserved.Throughthisexperimentwewanttolearnmoreaboutthe
functionΨ=Ψ(θ).Aftertheexperiment,wecanexpressalltheknowledgegathered
aboutΨbytheexperimentinp(Ψ|x).Asourhypothesisaboutthedistribution
ofΨ(the’unknown’truevalue)weusep†(Ψ).Ifp†(Ψ)representsΨbest,thenthe
expectationvalueoftheutilitywillbeatitsoptimum.Thisexpectationvalueofthe
utilitycanbecalculatedby
D(p†(Ψ)Ψ)=d(p†(Ψ)Ψ)p(Ψ|x)dΨ,
wheredistheutilityfunction.Itwasshown(Bernardo,1979)thatthelogloss
d(p†(Ψ)Ψ)=Alogp†(Ψ)+B(Ψ),
whereAisaconstantandB(Ψ)anarbitraryfunctionofΨ,hasgoodpropertiesas
utilityfunction(itistheonlysmooth,properandlocallossfunction).

ytropention-theoreticInforma

Theinformation-theoreticentropyH(X)isaninformationmeasure,basedonthelog-
loss.Itisoftencalled’Shannon-Entropy’asreferencetoShannon’sarticlefrom1948
byPa(Shannon,uliin19331948)(Pbutauli,itis1933)saidto(Smith,problems2001)ofstthatatisticaltheemnectrophanicsyw.asThaelreadyinformation-applied
theoreticentropyisdefinedby
H(X)=−p(x)logp(x).(2.11)
xForunderstandingpropertiesoftheentropy,letusassumeanexamplewherearandom
theprobcessaggisenerafilledteswtithwotohebservsameablenumclabersses,ofe.g.elemendratswingforobraothngcesalassesndathenpplesitfisromabamaximallyg.If
mauncertainximum.whichIncreaclasssingwillthebneumdraberwnofneext.lemenFtsorfotrhisonecsituationlassatnd/heorentropdecreayissingattheits
numberofelementsfortheotherclasswillreducetheuncertaintyregardingwhich
classwillbedrawnnext.Inthiscasethevaluefortheentropywilldecrease.Inthe
0,textreme,hereiswnoheretheuncertainptyrobabilitleftyandfortheoneenclasstropygoesapproacto1hesand0.fortheotherclassgoesto

2.2Reconstructinginformationfromsequencesofactionpotentials

Kullback-Leiblerdivergence

17

DerivatedfromEq.(2.11)fortheentropy,thecrossentropyH(XP,XQ)fortwoproba-
bilitydistributionsp(x)andq(x)canbedefinedby
H(XP,XQ)=−p(x)logq(x).
xThecrossentropyisnotasymmetricmeasurewithrespecttobothdistributions,which
meansthatexchangingthetwoprobabilitydistributionswillresultinadifferentvalue
ofH.ThecrossentropyispartofthefrequentlyusedKullback-Leiblerdivergence
(introducedbySolomonKullbackandRichardLeiblerin1951(KullbackandLeibler,
1951)),whichsharesthesameasymmetry:
)x(pDKL(XPXQ)=p(x)logq(x)(2.12)
x=p(x)logp(x)−p(x)logq(x)
xx=−H(X)+H(XP,XQ)
Inthecasewherep(x)=q(x)(forallx),thesum−xp(x)logq(x)matchesthe
entropyandthusthevaluefortheKullback-Leiblerdivergencewillbe0.Forall
othercombinationsofp(x)andq(x),theKullback-Leiblerdivergencewillbepositive.
WecaninterprettheKullback-Leiblerdivergenceasthenecessaryamountofextra
informationwhenacodebookbasedonp(x)isrewrittenbyusingq(x)instead.

ioninformatualMut

Figure2.6:TherelationshipbetweenjointentropyH(X,Y),(marginal)entropiesH(X)
andH(Y),conditionalentropiesH(X|Y)andH(Y|X)andmutualinformationI(X;Y).
(Figuretakenfrom(MacKay,2003).)

inGiventerestingatorandompdeterminerocesshothatwmuchgeneratestinformationworandomonevvariableariables(carriesXandabYout),tithemayotherbe

18

Chapter2:TheoreticalandBiologicalBackground

variable.Orinotherwords,ifwehaveabagfulloftomatoesandapples(whichboth
wcanehaobtainvetifhewecolourdrawagreentoromatored)outweofcantheabskag.howObvmiucoushlyi,thenformationanswerabdepouttendsheoncolourthe
probabilitiesp(green|apple)andp(green|tomato).
Theentropyforbothrandomvariablestogetheriscalledjointentropy.InFig.2.6it
isshownhowthejointentropy
H(X,Y)=−p(x,y)log(p(x,y))
x∈Xy∈Y
=H(X|Y)+H(Y)=H(Y|X)+H(X)
=H(X)+H(Y)−I(X;Y)
canbebrokendownintothe(marginal)entropyEq.(2.11),theconditionalentropy
H(X|Y)=−p(x|y)logp(x|y),
yxandthemutualinformationor’transinformation’
p(x,y)
I(X;Y)=x∈Xy∈Yp(x,y)logp(x)·p(y).(2.13)
Inourexample,themutualinformationEq.(2.13)measuresthecommoninformation
aboutX(tomatoorapple)thatissharedbyY(greenorred).Inthecase,ifandonly
ifXandYareindependentrandomvariables,whichmeansthatbothrandomvari-
ablesarenotsharinginformation,thenp(x,y)=p(x)·p(y)andthetransinformation
becomeszero.IncontrasttotheKullback-Leiblerdivergence,themutualinforma-
tionissymmetricinXandY.Leavingthetomatoesandapplesbehind,themutual
informationcanalsobeusedastoolforanalysingneuronalcodes,e.g.(Eckhornand
aPoneppeuronalel,r1974).espFonseorexsharesample,withwithatshistimulusmeasureandwcecanompareevaluatedifferenhotwhmypuchothesesionformationfhow
stimulimaybecodedbyneuronalresponses.

Meanrroresquared

Adescriptivemeasureisthemeansquarederror(’MSE’).Itquantifiesthedistance
betweentwofunctionsbytheexpectationvaluecalculatedoverthesquareddifferences
oftheircomponents

MSE(x)=E(f(x)−g(y))2|x
=p(y|x)(f(x)−g(y))2.(2.14)
yLetusassume,thataresponsekwasgeneratedbyanoisemodelp(k|x)basedon
thequantityx,whichisnotdirectlyobservable.Forreconstructingxoutofk,an

2.2Reconstructinginformationfromsequencesofactionpotentials

19

estimatorx^isused.TheseassumptionsallowustodecomposetheMSEinto’bias’and
variance(seesection2.2.1)bythesocalled’bias-variance-decomposition’:
E(x−^x(k))2|x=p(k|x)(x−^x(k))2
k=p(k|x)(x−E[^x(k)|x]+E[^x(k)|x]−x^(k))2
k=p(k|x)(x−E[^x(k)|x])2+(E[^x(k)|x]−x^(k))2
k+2(x−E[^x(k)|x])·E[^x(k)|x]−p(k|x)^x(k)
k22E[^x(k)|x]−E[^x(k)|x]=0
k=(x−E[^x(k)|x])+p(k|x)(E[^x(k)|x]−x^(k))
Bias[^x(k)|x]2Var[^x(k)|x]
=Bias[^x(k)|x]2+Var[^x(k)|x](2.15)
Thebiasquantifieshowfartheexpectationvalueoftheestimatordiffersfromthe
unobservablequantityx.Eq.(2.15)showsthatthemeansquarederroriscomposedby
varianceandbias.Furthermore,itshowsthatanunbiasedestimator(biasequalszero)
canstillhaveameansquarederrorlargerthanzero.Ingeneral,itcanbeaproblem
toreduceoneofthesequantitiesindependentlyfromeachother(thesocalledbias-
variancetradeoff,e.g.(Gemanetal.,1991)).

FisherInformationandCramer-Raobound

propUsingertiestheofFisherestimatorsinformationandtI(xuning)(Fisher,functions1922)ispforeopular.g.ienxptheloringfieldofptheoreticallyopulationthe
coding(Pougetetal.,2003;WilkeandEurich,2002).
2
∂xI(x)=E∂log(p(k|x))x(2.16)
CTheramerreason-Rfaoorbothisundp(Copularitramer,yis1946;theRao,linkbe1946).tweenThetheCrFamerisher-Raobinformationoundproandvtideshe
anbiasoftasymptoticheeslotimatorweralimitndtheforaFisnyherestimatorinformationx^(k)I(asx)af(Pougeunctiontetofatl.,woq2003).uantities:the
Var[^x(k)|x]≥∂x∂E[^x(k)|x]2=1+∂x∂Bias[^x(k)|x]2.(2.17)
I(x)I(x)
Itulationisnotcogding,uaranttheeedFisthatheraninformationestimatore(forxistsawmhichculti-dimensanreacihontvhisersbionoound.ftheForFispop-her

20

Chapter2:TheoreticalandBiologicalBackground

informationsee(Blahut,1987;LehmannandCasella,1999))canbeusedasameasure
thatrepresentstheexpectationhowstronglytworecordedactivitiesfortwoslightly
tiondifferenquantstimtifiesulithewillexpdifferected(Pcurvougeteaturetaol.,fthe2003)likorelihoinodother.Fworordsany:uTnhebiasFisedherestinforma-imator,
theCramer-RaoboundissolelyafunctionoftheFisherinformation
1Var[^x(k)|x]≥I(x).(2.18)
ItmaximisknuomwnliktelihohatotdheesMStimatores(timatorSeung(andLehmannSompaondlinsCkya,sella,1993)(s1999)ection(s2ection.2.3)2are.2.3)asaymp-nd
Raototicallybound.efficienTht.us,forEfficienantmincreasingeansthatnumanberunofbiasedobservaestimatortionsthesMSEaturatesatheCsymptoticallyramer-
theapproacFisherhesII(1x)fornformation.allx.FItoralsosexample,houldwbeednotedemonstratedthatttherehattexistheusepitfallsofFinishertheiusenfor-of
depmationendsforoncthepharacterizingarticularctheodpingrecisionscheme.ofacWoedehafvoreashogivwnenthatdecothedingotimeptimalTwsidthtronglyof
apFisher-oopulationptimalofcoGdaesussianarealwtuningayscurvindepesendendepetndsofTonthe(Bethgeavaetilableal.,deco2002c).dingtime,while

2.2.3Propabilitybasedestimators

Minimummeansquarederrorestimator

Afunctionthatmapstheneuronalresponsebacktotheunderlyingstimuliusing
rorgive(nkMMSE)nowledgeestimator(seeiFig.sb2ased.4)ionscthealledpestosteriorimator.riskTandhethuminimsonumthemeanBayessqtuaredheorem.er-
SometimesitisalsocalledBayesestimator,butinfactitisjustonememberofthe
wholefamilyofBayesestimators,whicharedefinedby
x^Bayes(k)=x^(argmink)rPL(x,x^(k)|k)(2.19)

withrPL(x,x^(k)|k)=rL(x,x^(k)|x)p(x|k)dx.(2.20)
Asasimpleconsequence,theBayesestimatoralsominimizestheaveragedrisk(also
calledBayesrisk)(Bethge,2003).InthegeneralcaseofBayesestimators,theloss
functionrLneedsnottobetheMSEbut,asthenamesuggests,fortheMMSEthe
lossfunctionisdefinedbythemeansquarederrorandsoEq.(2.19)andEq.(2.20)are
ybnegiv

2x^Bayes(k)=argminEPosterior(x−^x(k))|k.(2.21)
)k(x^

2.2Reconstructinginformationfromsequencesofactionpotentials

21

(2.24)

withEPosterior(x−^x(k))2|k=E(x−^x(k))2|xp(x|k)dx.(2.22)
TheBayesriskisdefinedfortheMSEthrough
χ2=E(x−^x(k))2(2.23)
=EE(x−^x(k))2|x=EE(x−^x(k))2|k.

ThebestBayesestimatorunderthislossfunctioncanbeformulatedby
x^(k)MSE=E[x|k]=xp(x|k)dx(2.24)
whichallowstorewriteχ2as
χ2=E(x−^xMSE(k))2(2.25)
=Ex2−Ex^MSE(k)2.(2.26)
Itisoftendifficulttocomputex^(k)MSEbutsometimesitispossibletofindsolutions
inclosedform.Inthefollowing,Iwillpresentanexamplewherethisispossibleand
theresultwillbeneededlaterinsection5.TheMMSEestimatorforthefunction
flin(x)=N·((fmax−fmin)x+fmin)(2.27)
willbecalculated.ThetuningfunctionEq.(2.27)describesthemeanfiringrateofa
population,composedofNneurons,whicharecodingthecontinuousvaluex∈[0,1]
withthesamelinearmappingforallneurons.Thedynamicrangeofeachsingle
neuronliesbetweenfminandfmax.Thechannelthatwillbeusedinsection5for
transmittingthisinformationissubjecttoPoissonnoise(seeEq.(2.1))anddeliversas
neuronalresponsethespikecountK(containingthenumberofactionpotentialsfrom
allneuronsofthepopulation)foratimewindowT.
ThesimplicityofthistuningfunctionincombinationwiththePoissoniannoisemodel
allowstocomputetheoptimalBayesestimatorusingtheexpression
1f^(K)=0p(K|Tflin(x))xdx
10p(K|Tflin(x))dx
01x·((fmax−fmin)x+fmin)Ke−N((fmax−fmin)x+fmin)Tdx
(2.28).=01((fmax−fmin)x+fmin)Ke−N((fmax−fmin)x+fmin)Tdx
Evaluationoftheintegralsyieldsthefinalexpression
Γ(2+K,Fmin,Fmax)Fmin
f^(K)=(Fmax−Fmin)Γ(1+K,Fmin,Fmax)−(Fmax−Fmin)(2.29)

22

Chapter2:TheoreticalandBiologicalBackground

withFmin=NTfmin,Fmax=NTfmaxandΓ(k,a,b)denotingtheincompleteGamma
functionbΓ(k,a,b)=Γa,b(k)=xk−1e−xdx.(2.30)
aItshouldbenotedthatthisestimatorwasderivedfromauniformpriordistribution
ofvaluesxintheinterval[0,1]Eq.(2.10).Anydeviationoftherealp(x)fromthis
assumptionleadstoabiasinestimation.

Figure2.7:Generalpopulationcodingscheme.Thestatisticalpropertiesofthesource
forthestimulusxaredescribedbyp(x).Thestimuluswillbeencodedbyasetoftuning
functions{f(x)}intoameanfiringrateandthencodedbyanoisemodelp(x|{f(x)})
tointoanreconstructeuronalofrespthesonsetimuk.lusTxhis.Theneuronalestimaterespx^onseandisthedecorealdedstimbytulushexareestimatorcomparex^(kd)
withrespecttoagivenlossfunction,herethemeansquarederror.(Thefigurewas
adaptedfrom(Bethge,2003))

Fortheoreticalanalysesofcodingstrategies,theMMSEestimatorandtheMSEcanbe
usede.g.tofindoptimaltuningfunctions.Schematicallysuchanoptimisationtaskis
showninFig.2.7.Foragivensetofnoisemodel,stimulusstatistics,typeofestimator,
andlossfunction,thetaskistofindtheoptimaltuningfunctionwhichminimisesthe
lossfunctionunderthegivenconstrains.AnexampleisshowninsectionA.2.1.

Linearminimummeansquarederrorestimator

onThetoMMSEestimatesebutstimatortherehasarenofreasonsunctional(likee.g.limitationsbiologicalformcappingonstraints,neuronallimitedrespcompu-onses

2.2Reconstructinginformationfromsequencesofactionpotentials

23

tationalpowerandthepossibilitytoobtainanalyticalsolutionsinclosedform)which
makeitnecessarytorestricttheestimatortotheclassoflinearfunctions(Salinasand
Abbott,1994).ThisoptimallinearBayesestimator(OLE)isdefinedbytheequation
Nx^(k)=kiAi+B,
1=iwiththeparametervectors{Ai}i=1,...,NandBandk={k1,...,kN}denotingthesignal
(e.g.thespikecountvectoroftheneurons).
TheoptimalchoiceofparametersforanOLEdependsontheerrorfunction,whichin
thiscaseisdefinedby
N2
χ2=ρ(x)p(k|x,P)x−kiAi+B,(2.31)
x1=ikwithPbeingthesetofparametersdescribingthepropertiesofthesystem.
TheoptimalAiandBcanbecomputedfromχ2usingthecalculusofvariationsas
exemplifiedin(SalinasandAbbott,1994),whichyieldsthefollowingfunctionbased
.(2.31)qEon

(2.32)

Nx^(k)=(kj−Mj)Dj+Z,(2.32)
1=jusingthefollowingabbreviations
Dj=(Li−MiZ)R−1i,j,R={Qi,j−MiMj}i,j=1,...,N,
iMi=ρ(x)gi(x),Li=ρ(x)gi(x)x,
xxZ=ρ(x)x,Qi,j=ρ(x)p(k|x,P))kikj,
xxkandgi(x)=p(k|x,P)ki.(2.33)
k

Insection5wewillmakeextensiveuseoftheOLE.

Maximumlikelihoodestimator

Anothermethodofconstructingestimatorsisthemaximumlikelihoodmethod(MLE).
TheideabehindthistypeofestimatorwasdevelopedbyFisher(Fisher,1922)inthe
yearsbetween1912and1922(Aldrich,1997).TheMLEiscommonlyusedanda

24

Chapter2:TheoreticalandBiologicalBackground

Figure2.8:Comparisonbetweenthemaximumlikelihoodestimatorandthemaxi-
mumaposterioriestimator.Thefigureonthelefthandsideshowsthelikelihood
Llik(Paelihoorameterd|functionDatawith)=presp(Dataectto|Patheprameteraram).eter.TheTheMLEotherpicfiksguretheshomwsaximincumofompari-the
sontinformationhepoabsterioroutptherobabilitparameteryd(istributionshowni(nshoblacwnki)nwrasedcomcolour).binedThewithathedditionallikelihoporiord
(showninredcolour),usingtheBayestheorem,toobtaintheposteriorprobability
distribution.TheparameternowselectedbytheMAPdiffersfromthatselectedby
LE.Mthe

lotofstudiesinthefieldofneurosciencearebasedonthismethod,e.g.(Seungand
Sompolinsky,1993;Deneveetal.,1999;Schulzke,2006).
AsexplainedinthepreviouspartaboutBayesinference,thelikelihoodisdefinedasa
functionofparametersandafixedsetofdata
L(Parameters|Data)=p(Data|Parameters)
L(x|k)=p(k|x).
Themaximumlikelihoodestimatorsimplychoosesthexwhichmaximisesthelikelihood
function,

x^MLE(k)=argmaxL(^x(k)|k).
k(x^)ForanexampleseeFig.2.8(left).

Maximumaposterioriestimator

(2.34)

ThMLE.emaxInsteadimumofamposaximisingterioriesthetlikimatoreliho(ModAP;functionseeFig.L(x2|k.8)),itshecloseMAPlymrelatedaximizestotonhe

2.2Reconstructinginformationfromsequencesofactionpotentials

distributionosteriorpthex^MAP(k)=argmaxpposterior(^x(k)|k)
)k(x^withp(k|x)pprior(x)
pposterior(x|k)=p(k|x)pprior(x).
xSincethedenominatordoesnotdependontheparameter,wecanalsowrite
x^MAP(k)=argmax{p(k|x^(k))pprior(^x(k))}.
)k(x^

lassificationcandDiscrimination2.2.4

25

(2.35)

(2.36)(2.37)

Figure2.9:Chartillustratingdiscriminability.Thetaskistodiscriminatewhethera
valuexwasdrawnfromtheGaussianprobabilityfunctionwithmeanvalue<xA>or
asfromdistheocriminationthercGaussianriteria,wcurveecanwlithabmeleanthevaluedifferen<xtrB>egions.If:aThethresholdredriegionsinistrocalledduced
’truenegatives’andtheblueregionisnamed’truepositives’.Thegreenareaistermed
’falsenegatives’andthelastregion(yellow)isdenotedas’falsepositives’.(Thefigure
wasadaptedfromwikipedia.org)

Letusimagineanexperiment,wherewecanobserveaneuronalresponsex(e.g.the
firingratemeasuredinafixedtimewindow).Weknowthatxrepresentstwodifferent
classesbuttheregardedneuronalresponsesaredrawnfromtwooverlappingprobability
distributions(seeFig.2.9).Givenasamplex,wewanttodiscriminatewhetherxwas
generatedbyclassAorclassB.ForGaussianfunctionswiththesamevarianceσ,the
dydiscriminabilitd=<xA>−<xB>(2.38)
σ

26

Chapter2:TheoreticalandBiologicalBackground

Figure2.10:ExamplesforROCcurves.Threeexamplesareshownwheretheperfor-
Themancecoforresphowondingwellsetsamplesoffromdistributionstwoaresdistributionshownoncantheberdighthandistinguished,side.areBothdifferendistri-t.
inbutionstheRforOCcasediagram.AareiTdenheticalanddiscriminationthusnpotedisrformancetinguisishable.betterTheforresultcaseisBaanddiagonaleven
beresultstterfiornathescorrespetofondingdistributionsmovementmarkofedthegwithreenC.dotAconhangetheiRnOtCchresholdurve.i(nTheFig.figure2.9
wasadaptedfromwikipedia.org)

allowstomeasuretheseparability.ThedifficultytodistinguishbetweenclassAand
Bandthusthediscriminability,isrelatedtothedegreetowhichbothdistributions
overlap.Thelargerthevalueofdis,themoreseparablethedistributionsare(Dayan
2001).ott,bbAandOnewayofperformingthediscriminationistointroduceathresholdϑ.Ifx≥ϑ,we
decidethattheresponsewasgeneratedbyclassBandifx<ϑthenwedecidethatx
wasgeneratedbyclassA.Asaconsequenceofthisthreshold,wewillguesscorrectly
withaprobabilityofp(x≥ϑ|ClassB)whenx≥ϑwasgeneratedbyclassB,andwe
willmakeawrongdecisionwithaprobabilityofp(x≥ϑ|ClassA)whentheresponse
wascreatedbyclassA.Theseprobabilitiescanbecomputedusingthenormalisation
probabilitiesofertiespropp(x<ϑ|ClassA)=1−p(x≥ϑ|ClassA)
p(x<ϑ|ClassB)=1−p(x≥ϑ|ClassB).
Insignaldetectiontheoryxiscalledtest,α(ϑ)=p(x≥ϑ|ClassA)denotedasthe
sizeorfalsealarmrateofthetestandβ(ϑ)=p(x≥ϑ|ClassB)isthepoweror
hitrateofthetest(DayanandAbbott,2001).Inthecontextofdiscriminabilitywith
binarydecisions,classBisoftensymbolisedby’+’andclassAby’-’.Followingthis
terminology,inFig.2.9differentregionsarelabelandmarkedbycolours.Theregion
’truepositives’representcaseswheretheneuronalresponsesweregeneratedbythe’+’
classandwerediscriminatedcorrectly,the’truenegatives’correspondtothe’-’class
thathavebeenclassifiedcorrectly,and’falsepositives’aswellas’falsenegatives’are
representingthecorrespondingerrors.

2.2Reconstructinginformationfromsequencesofactionpotentials27
FromFig.2.9werealisethattheperformanceofthediscriminationisdirectlyrelated
tothethresholdϑ.Selectingthethresholdsuchthatα=0andβ=1isingeneral
notpossible,soanoptimalsolutionwouldbetofindaϑwhichmaximizesβ.

CRO

Thereceiveroperatingcharacteristics(ROC)allowstomeasurethediscrimination
performancedependingonthethresholdϑ.EachpointontheROCcurve,withαon
theabscissaandβontheordinate,correspondstoadifferentthresholdϑ(seeFig.
2.10andFig.2.9).TheareaundertheROCcurveisrelatedtotheperformanceof
howwellsamplesfromtwodistributionscanbedistinguishedinadiscriminationtask
(MasonandGraham,2002).Foratwo-alternativeforcedchoicetask,theareaunder
theROCcurvecorrespondsdirectlytotheperformance.
1p(Correct)=β(α)dα.(2.39)
0

ForGaussianfunctions,wherebothcurvesshareacommonvarianceσ2,theROCcurve
dependsonlyond
p(Correct)=1erfc−d
22withthecomplementaryerrorfunction
∞erfc(x)=√2e(−y2)dy.
πxItiscommontocalculatedeveniftheunderlyingprobabilitydistributionsareother
thanGaussian(DayanandAbbott,2001).

NearestNeighbourMethod

Arelativelysimple,butinmanycasessuccessfulmethodofreconstructing’hidden’
informationfromdataisthe’nearestneighbour’method.Givenalabeledset(with
Mmembers)ofN-dimensionaldatavectorsXi={Xi,1,...Xi,N}withacorresponding
setoflabelsLiforsettingthefreeparametersofthenearestneighbour(’training’the
nearestneighbourbymemorizingthewholetrainingdataXiina’dictionary’)anda
testdatavectorY={Y1,...YN}withunknownlabelU,thedistancecanbecalculated
betweenthetestvectorandthetrainingdata
ND(XiY)=|Xi,j−Yj|α.
1=j

28

Chapter2:TheoreticalandBiologicalBackground

Typicallyα=2,foranEuclideandistance,orα=1areused.Inanextstepthe
datavectorfromthetrainingdatasetwiththesmallestdistancetothetestvectoris
tedualaevη=argmin{D(XiY)}.
iAsaresultwegetU=Lηasestimateforthelabelofthetestvector.
Thismethodcanbeextendedbytakingmorethanthenearestneighbourintoaccount.
Aselectedsetoflabelsfromtheproximityofthesmallestdistancehavethentobe
reducedtoonevaluebye.g.aweightedsum.

MSV

Figure2.11:Exampleofahyperplane.Theplaneisgivenbythexvaluesthatfullfill
theequation<w,x>+b=0.Thevectorwisorthogonaltothehyperplane.All
pointsbelowtheplanearedefinedby<w,x>+b<0,andthepointsaboveare
determinedby<w,x>+b>0.(Thefigurewasadaptedfrom(Sch¨olkopfandSmola,
2001))

Inthebeginningofsection2.2.4,weintroducedthemethoddiscriminatingtwodistri-
butionsbyathreshold.Arelatedstrategyistheverypopularsupportvectormachine
(SVM)methodfromthefieldofmachinelearning(Iwillfollowthetextof(Sch¨olkopf
andSmola,2001)fortheexplanationofSVM’s).SVM’sarebasedontheideaofusing
hyperplanes

{x|<w,x>+b=0}

2.2Reconstructinginformationfromsequencesofactionpotentials

29

Figure2.12:Ahyperplaneseparatestwoclassesofdatapoints.1Thedistancebetween
thecanonicalhyperplaneandthenearestpointisgivenbyw.Themarginplanes
(orangelines)aredefinedby{x|<w,x>+b=−1}forthemarginbelowtheseparat-
inghyperplaneand{x|<w,x>+b=+1}theupperone.(Thefigurewasadapted
from(Sch¨olkopfandSmola,2001))

forseparatingmulti-dimensionaldata(seeFig.2.11).wisavectororthogonaltothe
hyperplaneand<w,x>isthedotproduct(inEuclideanspace).
Letusassumethatwehaveadatasetfortraining{xi,yi}i=1,...,m,whereyiarelabels
thatassignallvectorsxitooneoftwoclassesyi∈{+1,−1}.Ifitispossibletoisolate
alldatapointsofoneclassfromalldatapointsoftheotherclassbyahyperplane,then
wecallthedataseparable.Inthiscase,thedistancesdibetweenthehyperplane
bwdi=w,xi+w
multipliedwiththelabelyi,zi=yi·diareallpositiveforthewholedata.Apositive
zishowsthattheregardingdatapointliesonthecorrectsideofthehyperplane(see
Fig.2.12).Ifthedatapointsarenotseparable,thensomeziwillbesmallerthan0,
whichsignalisesthatthesedatapointsliebeyondthehyperplane.
fw,b(xi)=sgn(<w,xi>+b)(2.40)
canbeusedforcalculatingonwhichsideofthehyperplanethedatapointxilies.If
thehyperplane(^w,b^)fulfils
(w^,b^)=argmin{|<w,xi>+b|=1}∀i=1,...,m
,bwforalldatapointsofthetrainingset,thenthehyperplaneiscalledcanonicalandhas
adistanceof1w(xrepresentsthelengthofvectorx)tothenearestdatapoint

30

Chapter2:TheoreticalandBiologicalBackground

Figure2.13:Usingalinearhyperplane,itisimpossibletoseparatethedatapoints
ofthetwoclasses(upperfigure).Applyinganon-linearmappingφallowssucha
separationinahighdimensionalfeaturespace(lower,leftfigure)ofwhichasuitable
projectionisshown.Thiscorrespondstoanon-lineardecisionsurfaceininputspace
(lower,rightfigure).(Thefigurewasadaptedfrom(Sch¨olkopfandSmola,2001))

(called’margin’,seeFig.2.12).w1isanimportantmeasurefortheSVMbecauseit
iscorrelatedwiththerobustnessagainstperturbations(e.g.noise).Thelargerw1is,
etter.btheThegoalistofindtheoptimalmarginhyperplanewiththelargestw1.Todothis,we
havetocalculate(foraseparablesetofdata)
{wopt,bopt}=wmin,bτ(w)
withτ(w)=1w2(2.41)
2andalsosubjectedto
yi(w,xi+b)≥1∀i=1,...,m.(2.42)

1τ(w)=w2
2

(2.41)

(2.42)

2.2Reconstructinginformationfromsequencesofactionpotentials31

Orasanalternative,whichcanbemoreconvenient,itispossibletooptimisethe
Lagrangianm1L(w,b,α)=w2−αi(yi(k(xi,w)+b)−1)
21=iwhere{αi}i=1,...,mareLagrangemultipliers.Lhastobemaximisedwithrespecttoαi
andhastobeminimisedwithrespecttowandb(theoptimumwesearchforisa
saddlepoint).Wearelookingforxiwithαi>0becausethesevectorslieexactlyon
themarginhyperplane.Thesexiarecalledsupportvectors.
AsaresultoftheLagrangianoptimisation,wehavetofindtheαthatminimizesW(α),
ybnegiv

mm1W(α)=αi−αiαjyiyjk(xi,xj)(2.43)
i=12i,j=1
mandsubjecttoαi>0∀i=1,...,mandi=1αiyi=0.Usingtheαi>0,wethen
bywalculateccanmw=αiyixi
1=i

andwithb

mb=yi−yiαik(xi,xj)
1=iforallsupportvectors.Sometimesitmaybehelpfulnottousethisbvalueandutilise
bforadjustingthefalsepositivesandfalsenegatives.UsingEq.(2.43)inEq.(2.40)
givesusareformulateddecisionfunction
mf(x)=sgnαiyik(x,xi)+b.(2.44)
1=i

BeginningwiththeLagrangianL,theeuclideandotproduct<x,xi>isreplacedbya
distancemeasurementfunctionk(x,xi)=<φ(x),φ(xi)>(withφasnon-linearmap-
pingfrominputintofeaturespace),alsocalled’kernel’.Theprobabilityofreplacing
onepositivedefinitekernelbyanotherpositivedefinitekerneliscalled’kerneltrick’,or
inotherwords:TheSVMframeworkdoesn’tdependontheeuclideandistancemeasure
andcanbereplacedbyothersuitablekernels,e.g.thepolynomialclassifierofdegree
d

k(x,xi)=<x,xi>d,

32

Chapter2:TheoreticalandBiologicalBackground

ortheradialbasisfunctionclassificationwithGaussiankernel
2k(x,xi)=exp−x−xi(2.45)
c(withc>0),orthetanhactivationfunction
k(x,xi)=tanh(κ<x,xi>+Θ)(2.46)
thewitheκ>uclidean0asdotgainproandductΘcasannothorizonsolvtealsthehift.pInroblem,Fig.2while.13aanneon-xalinearmplekiseshrnelowncanwfindhere
aseparationbetweenthetwoclasses.
ThepresentedSVMcanonlydiscriminatetwoclasses,butitisoftennecessaryto
distinguishmorethantwoclasses(e.g.section4).Forthisproblemdifferentextensions
fortheSVMareavailablelike,e.g.

•Oneversustherest
Thedatafromoneclassisseparatedfromtherestandabinaryclassifieris
trainedonthesenewsets.ThusforMclassesoneobtainsMbinaryclassifiers.
Theresultoftheclassificationisdeterminedby
j=1argm,...ax,Mgj(x)

with

mgj(x)=yiαijk(x,xi)+bj
1=i

•Pairwiseclassification
Fortrainedthisontstrategyhese,tsub-hedatadataforsets,anytwobtainingoclasses(M−1a)reMclaselectedssifiersandforbMinaryclasses.classifiersFor
2classification,theclasswiththemostvotesisselectedasresult.

Itmightbepossiblethatthedatacannotbeseparatedperfectlyoritmaybeadvan-
tageoustoavoidtheuseofthisstrictseparation(becauseofe.g.outliersormislabeled
datainthedatasetandtoavoidoverfitting).Thisproblemcanbesolvedbyintroduc-
ing’slackvariables’ξi>0∀i=1,...,mandchangingEq.(2.41)byaddingapenalty
tointermmτ(w)=21w2+mCξi
i=1
penaltyterm
withaconstantC>0andEq.(2.42)into
yi(w,xi+b)≥1−ξi∀i=1,...,m.

2.2Reconstructinginformationfromsequencesofactionpotentials

33

Bothequationshavetobeusedfortheoptimizationprocessandforcalculatingtheαi
andthuswandb.TheconstantCiscontrollingthetradeoffbetweenminimizationof
thetrainingerrorandminimizationofthemargin,butitcanbeaproblemtoselect
ortofindanoptimalC.ThistypeofsupportvectorclassifieriscalledC-SVC.Other
methodsliketheν-SVCareavailable,fordetailssee(Sch¨olkopfandSmola,2001).

34

apterhC

:2

Theoretical

and

iologicalB

roundgkacB

Figure2.14:Morphologicalstructureofatypicalneuron.Thecentralpartofthecell
iscalledcellbody(soma).Atypicalnervecellcollectstheoutputofotherneurons
overabrancheddendritictree.Furthermore,thecellpossessesinmostcasesasignal
generativezone(axonhill)andoneaxonthatcarriestheactionpotentialtoother
connectedcells.Theendingsoftheaxon(axonterminal)andthedendritesarecoupled
byelectricorchemicalsynapses.(Figureadaptedfromwikipedia.org)

2.3Modelingofneurons

2.3Modelingofneurons

35

Figure2.15:Simplifiedillustrationofchemicalsynapses.Anincomingactionpotential
onthepre-synapticsidecancreateareleaseofneurotransmittersintothesynaptic
cleft.Forthisreleaseofneurotransmitters,asynapticvesiclefilledwithneurotrans-
mitterfuseswiththemembraneoftheaxonterminal.Iftheneurotransmittersreach
suitablereceptorsonthedendriticspinethenapost-synapticreactioncanbecreated,
representingtheincomingactionpotential.Theinfluenceofthereceivedneurotrans-
mittersonthepost-synapticcellcanvaryfromincreasingordecreasingtheexcitability.
Andfinallyitmayleadtoanewactionpotential.Thesynapticcleftisthencleaned
fromneurotransmittersby’re-uptake’pumps,whichrecyclethetransmitter.(Figure
adaptedfromwikipedia.org)

Realneuronsinthe(mammalian)brainshowanastonishinglyhighvariabilityintheir
morphology.Thequestion,howneuronsandtheirconnectingelements,thesynapses,
functionisstillunderheavyresearch.InFig.2.14andFig.2.15simplifiedillustrations
ofbothofthesebasicbuildingblocksinthebrainareshown.Thedetailedstructures
ofrealneuronsaretoocomplexformosttheoreticalcomputationalmodelsofneurons
andneuronalnetworks.

Forthisreasondifferenttypesofsimplifiedmodelshavebeendevelopedtodescribeonly
thenecessaryaspectsfortreatingaspecificproblemandignoretherestofphysiological
knowledge.Inthefollowingtextwewilltakealookonsomeofthesemodelsafter
discussinghowtomeasureneuronalactivities.

36

Chapter2:TheoreticalandBiologicalBackground

2.3.1Measuringneuronalresponses

Variousmethodsofrecordingneuronalactivitieshavebeendeveloped.Thesemethods
differintheirareaofapplicationandinthepropertiestheyobserve.Someofthemare
desigdynamicsnedfoofrshainglendlingcells.manyneuronsandothercanaccessthedetailedprocessesand
One1990).eAxampleneislectrothede’patc(glasshcpiplamp’ettemfilledethodw(ithcSakmannonductingandfluid)Neher,is1984;broughtEdwinatordsconetatactl.,
chwithannelsthemonemthebranememoftbraneheonervfnecell.eurons.ThisPatchmethodclampevhenasallosiwstomilaritiesinveinstigateitscsonceptingletiono
theoldervoltageclampmethod,whichusestwoelectrodesformeasuringe.g.voltage
andimpedance(ColeandCurtis,1939).
Itwithoutrequiresdhighamagingeffortsthecell.tobringOftenanitisselectroufficiendetintotoconbringtacttheweithlectrothedeinsurfacetotheofparoximitneurony
ofaneuron.Electrodesareusedforthispurposearetypicallymadeoutofmetaland
ofarethealmosetlectrocdeompletelyisused.coveTredhisbyalloisowslatorytorecordmaterial.theeForlectricmeasactivituremenytsof,othenlytheneurontip
extracellularly.Dependingontheattributesoftheelectrodeanditstip,theelectrode
showfrequencysdifferencomtpronenecordingtsofeproplectricaerties.laE.g.ctivity.electroTheydesadlsoiffercainndtheirifferainbilitiestheirtommaxeasureimal
radiusofdetectingtheactivityofcells.Inatypicalcaseanelectroderecordsspike
activdifferenitytofgmoreeneratingthanoneneuronsnbeuron.yanItaisppropriateoftenpossspikibleestoortingre-assignalgorithm,singlees.g.pik(esLewictokthei,
1998).Notonlysingleelectrodesareused,butalsoarraysofelectrodes.
Acofmprequencyonentwithrangeuputpoto200Hz20.000andHzaishigherrecordedfrequencyandthencompsonplittedentibnteyooandlo1wer.000fHz.requencyThe
rangehigheroffsucrequencyhanerangecxtracellularontainsethelectro-phtimeycoursesiologicalofrthespikecording,es.tInhethecomlowerbinedefrequencylectric
activitiesofmanyneuronsfromtheproximityoftheelectrodetipcanbefound.This
measuredvariableistermedlocalfieldpotential(LFP).Iftheelectrodeisshapedas
aloomeasured,pandwhicplacedhbonearsatoplotofofthedursimilaritiesamaterwiththentheLtheFP.eEpi-lectroduraldesfieldpplacedotenontialthecsancalpbe
recordedforElectroencephalography(EEG)measurements.
Someotherpopulartypesofmeasuringneuronalactivitiesare:

•fMRI(functionalmagneticresonanceimaging)
Thistechnologyallowstomeasurethedynamicsofthebloodflowanditsoxy-
genation.AMRI-tomographisalargeandexpensivesystem.Thetemporal
resolutionisnotfastenoughtoresolvesinglespikeeventsandthespatialres-
olutiondoesnotallowtoseesingleneurons.Furthermore,thedependenceof
themeasuredsignalontheneuronalactivityinthetissueisstillunderresearch

2.3Modelingofneurons

37

(HeegerandRess,2002).AcorrelationbetweenthefMRIsignal(BOLD)and
thelocalfieldpotentialwasfound(Logothetisetal.,2001).fMRIisoftenused
toobtainanoverviewabouttheactivitydistributioninthebrainemergingfora
specificcognitivetask.
magingioptical•Opticalimagingcanmeasurethroughchangesinthepropertiesofscatteredand
reflectedlighte.g.thehaemoglobinoxygenationlevel(likefMRI)andotherquan-
titiesofthecellwhicharecorrelatedwiththescatteringandreflectionproperties
ofthetissue(VillringerandChance,1997).Themeasurementscanbedoneon
a(sub-)millisecondtimescaleandthespatialresolutioncanresolvestructures
uptosinglesynapses(e.g.usingtwophotonmicroscopy).Inaddition,special
dyescanbeappliedthatallowtoaccessotherbiochemicalvariableslikee.g.
concentrationsofmoleculesandions.
•MEG(Magnetoencephalography)
TheMEGisbasedontheideatodetectthemagneticfieldscreatedthrough
theneuro-electricactivitiesinthebrainbySQUIDs(SuperconductingQuantum
InterferenceDevices)andthusgaininformationabouttheelectricactivityin
thebrain(H¨am¨al¨ainenetal.,1993).MEGsareevenmoreexpensivethanfMRI
systemsandveryrare.

2.3.2Integrate-and-fireneurons

R

C

I(t)compFigureos2ed.16:ofaCriesrcuitistoranddiagramacofanapacitor,inwhichtegrate-and-fireareconnectedneuron.inThparallel.emodelneuronis

Thedescription’integrate-oftwoand-otherfire’n(IAF)euronnmodeuronels,istheoneHoofdgkinthemandostHpuxleyopularmondel,euronandmodelsMcCullo(Thech
anddynamicsPittsoftneurons,hememcanbranebepfooundtentialinofasectionneuronA.1).simplyThebIyAFacneuronapacitorandrepresenatsresistorthe

38

Chapter2:TheoreticalandBiologicalBackground

(seeFig.2.16).Thecapacitorischargeduntilaspecificthresholdisreached,atthis
momentanactionpotentialisgeneratedandthemembranepotentialisreset.Theidea
ofintegrate-and-fireneuronswasapparentlyintroducedintheyear1907byLapicque
describ(Lapicqued.e,An1907),extensivwhicehrweviewasofroughlythein45ytegrate-earsband-eforefiretmheodHoeldgkcaninbeHfuxoundleyinmodel(Burkitt,was
follo2006a;wloosBurkelyitt,(Burki2006b).tt,2006a).HerewFeorwillmorefocusdetailsonlyononthisthebtopic,asicspleasofteheseemothesdeleraevndiewwsill.
Thedynamicsofthemembranepotentialv(t)isgivenby

Cdv(t)=Ileak(t)+Isyn(t)+Iinj(t),(2.47)
dtwhereCrepresentsthemembranecapacity,Ileak(t)describesthecurrentgenerated
bypassiveleakageofthemembrane,Isyn(t)modelsthesynapticinput,andIinj(t)
allowstointroduceacurrentthatisexternallyinjectedintothecell.Thisequationof
themembranepotential’sdynamicsdoesnotincludetheprocessofgeneratingspikes.
WhenthemembranepotentialcrossesthegiventhresholdVththenaspikeisgenerated
andthemembranepotentialissettoVreset.Thetimecourseofthespikehastobe
definedbyextraequations.InmanystudiesactionpotentialsarerepresentedbyDirac
deltafunctionsbecauseoftentheshapeofthespikesareconsideredtobeirrelevantfor
problem.turrencthe

Figure2.17:Membranepotentialofanintegrate-and-fireneuronunderinjectionof
constantcurrent.Afterthreespikes,whicharemarkedwithreddots,theinputwas
turnedoff(seethelowerfigureforthetimecourseoftheinput)andthemembrane
potentialdecaysexponentially.Thethresholdoftheneuronisrepresentedbyanorange
line.

2.3Modelingofneurons

4

3

tΔ / τ2

1

39

0012 R I / (Vth V30)45
Figure2.18:Firingrateofanintegrate-and-fireneuronwithconstantinput,see
Eq.(2.50).Therateisshowninunitsofthemembraneconstantτ,andtheinput
R·IinunitsofVth−V0.

TheleakcurrentinEq.(2.47)canbedescribedby
Ileak(t)=−τC(v(t)−V0).(2.48)
τisthe(passive)membranetimeconstant,definedbytheresistorandthecapacitor
τ=R·C.V0iscalledrestingpotentialandrepresentstheasymptoticvalueofthe
membranepotentialwhennoinputfromexternalsourcesisgiventotheintegrate-
and-fireneuron.Withoutinputtherelaxationprocessfollowsanexponentialdecay
describedbythemembranetimeconstant.
IfwerestraintheinputtoIinjection(t)andsetthemembranepotentialattheinitial
timet0torestingpotential(v(t0)=V0)thenEq.(2.47)togetherwithEq.(2.48)canbe
solvedtoexpressthemembranepotentialbelowthresholdby(Burkitt,2006a;Tuckwell,
1988)tv(t)=V0+e−τt1Iinjection(t)etτdt.(2.49)
Ct0ForconstantcurrentIinjection(t)=I,theEq.(2.49)canbesimplifiedto
v(t)=V0+I·R1−e−t−τt0.
InwhicthehicsasenwecessaryherethetotincreasehresholdtishefixedmemtobVranethwpeotencantcialfalculateromttheherestingdurationpoΔtten=tialt−tto0
hreshold:ttheΔt=−τlog1−Vth−V0.(2.50)
RI·

40

Chapter2:TheoreticalandBiologicalBackground

Figure2.19:Membranepotentialofanintegrate-and-fireneuronwithnoisyinput.
Spikesaremarkedwithreddotsandthelowerfigureshowsthetimecourseofthe
input.Thethresholdoftheneuronisrepresentedbyanorangeline.

Eq.(2.50)showsthatifI·R<Vth−V0thennospikecanbegeneratedbytheintegrate-
and-fireneuronbecausethedecaycausedbyleakingcurrentsisstrongerthanthein-
creaseofthemembranepotentialbytheinputcurrentIinsuchawaythatthethreshold
canneverbereached.Assumingthatafteraspikeisemitted,themembranepotential
isresetdirectlytorestingpotential(Vreset=V0),theIAF-neuronfiresregularlywith
aspike-rateΔt1andaninterspikeinterval(intervalbetweentwosubsequentspikes)of
Δt.Anadditional(absolute)refractoryperiodτref,wheretheneurondoesnotreact
toanyinput,canbeincludedintothisconsiderationandchangesthespike-rateto
τref1+Δt.
Wecandescribethesynapticcontributionstothemembranepotentialfromother
neuronsintwodifferentways.Letusassumethattheincomingneuronalactivitiesof
otherneuronsaregivenby
SE,k(t)=δt−tiE,k
tiSI,k(t)=δt−tiI,k.
tiSE,krepresentstheexcitatorypostsynapticspikesfromneuronkandSI,kdenotesthe
inhibitorypostsynapticinput.Theexcitatoryinputincreasesthemembranepotential,
whiletheinhibitoryinputdecreasesthemembranepotential.tiE,kandtiI,karedenoting
thetimeswhereneuronkfiresitsi-thspike.
Oneconvenientwayofquantifyingtheinfluenceofincomingspikesonthemembrane

2.3Modelingofneurons

41

pofothetentaialictualstommemodelbranep’currenotenttialsynapses’.andcanHbeere,desthecribedimpactbyoftheinputisindependent

NENI
Isyn(t)=CaE,kSE,k(t)+CaE,ISE,I(t),(2.51)
k=1k=1
whereaE,k>0andaI,k<0reflecthowstronglyonespikeaffectsthemembrane
potentialoftheintegrate-and-fireneuron.

Amorebiologicallyrealisticapproachistouse’conductancesynapses’(Burkitt,2006a;
Tuckwell,1979)

NEIsyn(t)=C(VE−v(t))gE,kSE,k(t)
1=kNI+C(VI−v(t))gI,kSI,k(t).(2.52)
1=kComparingwithA.1.1,theequationsshowsomesimilaritiestoEq.(2.52)becausethe
Hodgkin-Huxleymodelisalsoaconductance-based,one-compartmentalmodel.The
crucialdifferencebetweenthesetwomodelsisthemissingtemporaldynamicsofthe
conductancesgE,k>0andgI,k>0forthismodel.VEandVIrepresentthereversal
potentialsfortheexcitatoryandinhibitorysynapticconnections(VI≤Vreset)<Vth<
).VE

42

Chapter2:TheoreticalandBiologicalBackground

2.4Learningandusing(neuronal)networks

Theneuronsinthebrainarenotjusttransmittinginformationfromonepointto
another,bualsoprocessingthisinformation.Incomingdatafromsensorysystemsis
filteredbyneuronalprocesses.Thisdataisevaluated,includinglearned/memorized
information.Thereductionininformationstartsalreadyinthesensorysystemitself,
wheredifferentreceptorsareonlysensitivetoanarrowpartofthefullspectrumofa
signal.Forexample,forhumansitisimpossibletoseeinfra-redorultra-violetlight,or
tohearultra-sonicsoundslikeotheranimalscando.Someoftheinformationreduction
stepsinthebrainareleadingtoillusionaryeffectslikee.g.changeblindness(Simons
andRensink,2005).Asaresultofthiswholeinformationprocessingchain,typically
abehaviouralreactionisgeneratedthatinteractswiththeenvironment.Nevertheless,
thesestepsofreducingthecomplexityoftheinputareveryimportantfordecreasing
thecomputationaldemandsofprocessingthestreamofincominginformation.

Thecentralnervoussystemshowsahighdegreeofvariationandcomplexityinits
structures,interactionsandactivitypatterns.Thismakesitinterestingtosearchfor
fundamentalinformationprocessingmechanismsthatcanmimictheprocessingprop-
ertiesofrealnervoussystemsevenunderthesecircumstances.Oneoftheproposed
classesare’artificialneuronalnetworks’(orjust’neuronalnetworks’).

Neuronalnetworksaretypicallycomposedofinter-connectedartificialneurons.Com-
putationalrulesdescribinghowinformationistransmittedbetweentheneurons,how
inputisincluded,andoutputisgeneratedarealsopartoftheneuronalnetwork.Dif-
ferentneuronalnetworkmodelsmayvaryintheirdegreeofbiologicalplausibilityand
theincludedbiologicalconstraints.Someexamplesdemonstratingthevarietyofap-
proachesaredifferenttypesofinformationrepresentation(e.g.booleanvalues/action
potentials,realvalues),theresolutionoftemporaldynamics(fromtimestepstocon-
tinuoustime),thesizeofmemoryforpreviousinputsoroutputs,deterministicor
probabilisticinteractions,thecomplexityoftheconnectionstructure(e.g.recurrentor
purefeed-forwardconnections)andinformationflow,andthecomplexityoftheneuron
models(e.g.linearmodel,leakyintegrateandfiremodel,HodgkinHuxleymodel).

Animportantissueforneuronalnetworksislearning.Ifanartificialneuronalnetwork
shouldperformaspecificcomputationalfunction,thentypicallythestructureofcon-
nectionsandotherparametersofthemodelhavetobeadaptedtotheproblem.One
w(e.g.ayformeanfindingsquaredthebestdistancesetobfetcweenonnectionsthedandesiredpandarameters,actualatoutcomeargetorofltosshefmounctiondel)
isnecessaryforevaluatingtheperformance.Duringthisprocessitmaybehelpfulto
constrainparameters,e.g.thenumberandstrengthofconnections,duringlearning.
notpOtherwiseerformitwceanllhapp(’generalize’)enthatothenontheretworkdata.justmemorisesthetrainingdataandwill

ral)Anothercorrelationpossibilitofyistneuronalheuseoactivitflyearningpatternsmeclikehanismse.g.tthatheareHebb’sbasedruleonthe(Hebb,(temp1949;o-

2.4Learningandusing(neuronal)networks

43

SejnowskiandTesauro,1989),long-termpotentiation(LTP)(BlissandCollingridge,
1993)andlong-termdepression(LTD)(LindenandConnor,1995),orspike-timede-
pendentplasticity(STDP)(KistlerandvanHemmen,2000).Thistypeoflearningis
oftenreferredtoas’neuronsthatfiretogether,wiretogether’.Inmoredetail,givena
weightthatdescribesthepostsynapticresponsetoanincomingspike(e.g.thesynaptic
weighefficacy),tssucwehtcanhatsttorehepeinformationrformanceoinfathenetvwaorkluesforofatgheseivenwetaskightissorwoptimizedecanadjust(Gerstnerthe
andKistler,2002b).Thesechangescanbedrivenbye.g.thecorrelatedactivityof
pre-andpostsynapticneurons.
Indures.thefolloTheowing,vewrviewewillwillintrostartducewithssometyptructurallyesofsimpleneuronalnetneuronalworksnetandworklmearningodelspro(likce-e
perceptrons)willcontinuewithlearningstrategies.InsectionA.3andA.4aglanceon
morecomplexneuronalnetworkmodelswillbegiven.

2.4.1Feedforwardnetworks

Figure2.20:Illustrationofafeedforwardneuronalnetworkwithonehiddenlayer.

Inpurelyfeedforwardnetworks,theflowofinformationproceedsfromonelayertothe
nextlayer,withoutpropagatingbackwardstoalowerlayer,orexchanginginformation
betweenneuronswithinthesamelayer,andalsowithoutdirectconnectionsbeyond
thetheinextnputlalayyerer.(seeTheFig.last,2.20).output,Ilanformationyerofenthistersnetsucwhorkanreuronalepresentsnettheworrkatesultstheoffitrst,he
computation.Layersbetweeninputandoutputlayerareoftentermedhiddenlayers,
becausetheyhavenodirectaccesstotheinputoroutput.

44

Perceptrons

Chapter2:TheoreticalandBiologicalBackground

Oneprominentexamplerelyingonthisfeedforwardstructureareperceptrons,origi-
cannallybefpropoundosedinby(HertzRosenetblattal.,(Ros1991).enTblatt,hissu1958).mmaryAusesdetailedthisovereviewrviewasoftbasheis.peTherceptronsim-
pleperceptroniscomposedofonlyaninputandanoutputlayer.TheoutputΩiof
neuron(’unit’)iisdescribedby
Ωi=gwi,j·xj,
jwherexjrepresentstheinputfrominputunitjandwi,jthe’strength’oftheconnection
betweeninputneuronjandoutputneuroni.Thefunctiong(.)iscalled’activation
function’,typicalexamplesaretheHeavysidefunctionwithvariablethresholdϑi,the
signumfunction,sigmoid-likefunctions(includingparametersforshiftandsteepness)
function.linearajustorSimpleperceptronsshowsimilaritiestosupportvectormachines.Theperceptronde-
finesEuclideanahypespacerplanethatcanparametrisedbeusedbytothetseparatehresholdtwoanddifferenwetighctslassesinmofulti-inputdimensionaldata.A
simpleperceptroncanonlydefineonehyperplane.Ifadatasetwithtwoclassescan
beseparatedbyasimpleperceptron,theproblemiscalledlinearlyseparable(Hertz
etal.,1991;MinskyandPapert,1969).
Assumingalinearlyseparableproblemofthistype,theweightsandthresholdscan
belearnedbyfindingsupportvectorswith(non-soft)marginhyperplanesandlinear
kernel,seesection2.2.4fordetails,or(Sch¨olkopfandSmola,2001;Hertzetal.,1991).
Iftheactivationfunctiong(.)isdifferentiableandifanerrormeasureisgiven,the
weightscanbecomputedviaagradientdescentalgorithm(whichisalsocalled’method
ofsteepestdescent’).Oneexampleforanerrormeasureisthesum-of-squarescost
function

E=1(Oiμ−Ωiμ)2
2i,μ2=1Oiμ−gwi,j·xjμ,(2.53)
2ji,μwhereμidentifiestheinputpatternandOiμdefinesthedesiredresultforoutputneuron
iandinputpatternμ.
Thegradientdescentstartswithaninitialsetofweightswti,j=0andmoveswitheach
iterationstepindirectionofthelocaldownhillgradient−∂w∂Ei,juntilreachingasetof
weightsthat(locally)minimizetheerror.Theupdatesoftheweightsarecalculated

2.4Learningandusing(neuronal)networks

by

45

wti,j+1=wti,j+Δwi,j
=wti,j−η∂E,
∂wi,jwhereηisalearningparameter.Abadchoiceofηmayleadtoproblemsfindinga
localminimum.Oftenitishelpfultoreducethelearningparameterduringthelearning
process(simulatedannealing).ForEq.(2.53)thedownhillgradientisgivenby
μ−η∂E=ηOμ−gwi,k·xμxμ∂g(kwi,k·xk).(2.54)
∂wi,jμikkj∂wi,j
Itisimportanttonotethatingeneralagradientdescentmethodmayfindonlylocal
minima,dependingontheinitialsetting.
Sincesimpleperceptronscanonlyrepresentlinearseparablefunctions,itisdesirable
tointroducehiddenlayersforextendingtheirfunctionalcapabilities.Thisextension
causedproblemsforsometimebecauseitwasnotclearhowtotraintheweightsofthe
multiplelayers.Theinventionofthe’back-propagation’algorithm(BrysonandHo,
1969;Werbos,1974;Parker,1982;Rumelhartetal.,1986a;Rumelhartetal.,1986b)
solvedthisproblem.Initscore,theback-propagationmethodisalsoagradientdescent
performedontheweights.
Theoutputneuroniinlayerlisdefinedasfollows:Forthefirsthiddenlayer(the
inputlayerisdenotedbyasuperscript0)as
hμi,1=gw0i,j→1·xjμ,
jforallotherhiddenlayersas
hμi,l=gw(li,j−1)→l·hj,μ(l−1),
jandfortheoutputlayer,whichisdenotedbyindexMas
Ωiμ=gw(i,jM−1)→M·hj,μ(M−1).
jTheweightsw(i,jl−1)→linthisdefinitionhavebeenindexedwithsuperscriptsforthe
regardedlayer(upperlayer(l)andlowerlayer(l−1)).
Usingagradientdescentalgorithmonthesum-of-squarescostfunction,theupdatefor
theweightsis
Δw0i,j→1=ηδμi,1·xjμ(2.55)
μΔw(i,jl−1)→l=ηδμi,l·hj,μ(l−1)(2.56)
μ

46

with

Chapter2:TheoreticalandBiologicalBackground

μδμj,l=(∂hl−1j,l)→lwli,j→(l+1)·δi,μ(l+1)
∂wii,jμδμj,M=(M∂Ω−1j)→MOjμ−Ωjμ
∂wi,jWhilethederivatives∂w∂handthevaluesofhμμi,l(andΩiμ)arepropagatedwiththecurrent
tothenextlayer,thepropagatingerrorsδi,laresendintotheoppositedirection,thus
givingthealgorithmitsname.Themethodcanbeextendedtootherdifferentiable
seerrore(mHertzeasuetres.al.,For1991).moredetailsonback-propagationandvariationsofthismethod
Eq.(2.55)andEq.(2.56)canbecalculatedandsummedoverallinputpatternsatonce,
whichisthencalledlearninginbatchmode,orwithonlyoneinputpattern,which
chadvaangesntagesafterovereacthhebatciterationhmsodtep.e,Te.g.helatterequiringrisclesalledsimemoryncrementcapacitalupydateforeacandhhlasearningoften
step.

2.4.2Bayesiannetworks

Figure2.21:ExampleofaBayesiannetwork.Thenetworkcontainsthepriorand
conditionalprobabilities:p(a),p(f|a),p(b|a),p(c|b),p(d|b),p(e|c,d),and
p(g|f,e).(Theimagewastakenfrom(ZhangandPoole,1994))

2.4Learningandusing(neuronal)networks

47

Bayesiannetworks(alsocalled’Bayesianbeliefnetworks’or’beliefnetworks’)area
pobilisticpularexpmertethokdnotocwledgeapture(Hecinkermanteractionsetal.,betw1995).eenThisprobabilistictypeofvnetwariables,orkscanlikebpeuroba-sed
topredictionslearnaboonuttheunderlyingbasisofthisrelationsinformationhidden.inBaayesiandatanetsetw(Porksearl,canh2000)andleandcanincompletemake
thedatavsetsariableswhere(Heconekorerman,several1995).vRariableseviewaresonmissing,thisutopicsingcanlearnedbefcoundinorrelations(Hecbkeetwrman,een
1995;PearlandRussell,2003;ZhangandPoole,1994).
ABayesiannetworkℵconsistsofthreebasiccomponents.First,asetofstochastic
varother.iableAsV.Stogetherecond,withasVetofformsarcsaAthatdirectedspecifygraph,howe.g.tsheseeeFvaig.2riables.21.aIfrethisrelatedgraphtodeoacesh
notcontainloops,itiscalledadirectedacyclicgraph.Anundirectedgraphicalmodelis
andcalledtheMarkovconnectionsnetwork’edges’.(Pearl,V1988).erticesThewithoutrepresienncomingtationofedgesavareariablecalledis’calledsources’’vertexand’
verticeswithoutoutgoingedges’sinks’.A→BmeansthattheobservationofB
isdepaesndsetofonA,anconditionaldthatpAisarobabilitiesparentforofBall.Tvheariables,lastcompgivenonenttheirofarespBayeectivsianepnetarenwts.ork
Thus,p(v|πv)aretheconditionalprobabilitiesofvariablevconditionedbyitsparents
denotedbyπv.Thesetofparentsπvisemptyforsources(ZhangandPoole,1994).
Usingthisinformation,itispossibletocalculatethepriorjointprobability
Pℵ(V)=p(v|πv).
V∈vWecanmarginalisethisjointprobabilitydistributionthrough
Pℵ(X)=Pℵ(V).
X−VofThethesvumariableshastofbromecsetalculatedX.UosingvertheallBvaayesriablestheoremfromwtehecansetVcalculatewiththetheeposteriorxception
yprobabilit

Pℵ(X|Y)=Pℵ(X,Y).
)Y(PℵItisoftenhelpfultoremovenon-involvedvertices.Thisrequirestoremoveallarcsand
removeormodifytheconditionalprobabilitiesthatareconnectedwiththisvertex.At
least,thevertexitselfhastobepurged(ZhangandPoole,1994).
Furthermore,itwasshownthatthecalculationofposteriorprobabilitiesinthisframe-
workcanbeNP-hard(Cooper,1990).Inmanycasesusingtheparticularstructureof
theproblemcanhelptoreducethecomputationaleffort.Importantforthisreduction
istouseasmuchaspossiblefactorizationsofjointprobabilities.Thisallowstosum
outvariableswithlesscomputationaleffortandtocreateareplacingcompoundvertex.
Usingthisideaandsimilartricksmakesitpossibletoreducethecomputationtime

48

Chapter2:TheoreticalandBiologicalBackground

isneededalsopoforsssibleolvitongusethepdifferenroblemt(typShaferesofandMonteShenoCyarlo,1998;simulationZhangandmethoPodsole,(e.g.1994).GibbsIt
sampling)forapproximatingdesiredproperties(Neal,1993;Pearl,1988;Gemanand
Geman,1984;Madiganetal.,1995).

Bayesianlearning

SeveralstrategiesforlearningBayesiannetworks(Buntine,1994;Hintonetal.,2006)
fromcompleteorincompletedatasetsexist.Learningaimsatfindingconditional
thenprobabilitetwoyrkd(Fisriedman,tributions1998)(Jordan,thatex1999;plainBinderthedetataal.,best.1997)aswellasthestructureof
bGeivinenateresteddatainsetpredictingconsistingotherofevesamplesntspgroducedeneratedbybytheansameunknoswnystem.system,Ifweweassumemay
apriordistributionoverpossiblemodelsp(Model)andaprobabilitydistribution
p(Θ|Model)regardingtheparametersforthemodels,wecanexpresstheprediction
oftheprobabilityforaneventZthrough
p(Z|Data)=p(Z|Model,Data)p(Model|Data)
sledMo=p(Z|Model,Data)p(Data|Model)p(Model)
Modelsp(Data)

withp(Z|Model,Data)=p(Z|Θ,Model)p(Θ|Model,Data)dΘ
andp(Data|Model)=p(Data|Θ,Model)p(Θ|Model)dΘ.
Themodellikelihoodp(Data|Model)isoftenthebasisforBayesianmodelselection
(Buntine,1994).ForcomparingtwoBayesiannetworkmodels(MacKay,1995),wecan
.g.eusep(Modelα|Data)p(Modelα)p(Data|Modelα)
p(Modelβ|Data)=p(Modelβ)p(Data|Modelβ).
Alsoaverypopularscore(especiallyforflatpriorsp(Model))forcomparingtwo
modelsisthesocalledBayes-factor
p(Data|Modelα).
p(Data|Modelβ)
Whenitisineffective(orimpossible)tocalculatep(Z|Data),itmaybesufficient
touseanapproximationofp(Z|Data).Thiscanbedonee.g.byevaluatingthe

2.4Learningandusing(neuronal)networks

49

maximumaposteriorimodelorasumoveraselectionofthemodels.Usingmaximum
aposterioriparametersleadsto
p(Z|Model,Data)≈pZ|Model,Θ^,
beca1998).useΘ^p(isΘthe|pModelarameter,Datathat)ismaxapproimisxesimatedbyaDiracdeltadistribution(Friedman,
p(Θ|Model,Data)=p(Datap|(DataΘModel|)pModel(Θ|)Model).
ForfindingΘ^,wecanuseagradientdescentmethod(seesection2.4.1)ortheEM
.4.3).2section(seealgorithmWhenthestructureisgiven,theprocessoflearningisreducedtooptimizeconditional
probabilitiesp(v|πv).Itisimportanttoincorporatetheprobabilisticnatureofthese
variablesbyrequiring

and

0≥p(v|πv)≥1

p(v|πv)=1.
vTheseconstraintscanbesatisfiedbyprojectingthegradient(forfindingtheoptimal
conditionalprobabilities)onaconstrainedsurface(Binderetal.,1997;Lanterietal.,
2002;Lanterietal.,2001).

2.4.3MonteCarlomethodsandexpectationmaximisational-
gorithm

IntheprevioussectionaboutBayesianbeliefnetworks,weencounteredtheproblemof
findingamaximumaposteriorisolutione.g.forlearningBayesiannetworkstructures
andconditionalprobabilities.Differentstrategiesareavailableforcalculatinganap-
proximationofthesolution.Oneexampleisthesocalled’expectationmaximisation’
algorithm(inshortEM-algorithm).Sincethismethodhasalotincommonwiththe
’Gibbssampling’method(Buntine,1994),afterintroducingMonteCarlobasicswewill
discussGibbssamplingfirstandcontinuewiththeEM-algorithm.
GibbssamplingisonememberofthefamilyofMonteCarloalgorithms,ofwhicha
reviewcanbefoundin(Neal,1993)and(Walsh,2002),whichwewilluseasguideline.
First,asimpleMonteCarlomethod,rejectionsamplingisdescribed.

50

Chapter2:TheoreticalandBiologicalBackground

Figure2.22:Rejectionsamplingallowstodrawsamplesindirectlyfromacomplex
probabilitydistributionp(x).Insteadofusingp(x)wewillusef(x)(hasstillthesame
complexityasp(x)),whichisexactlyp(x)withoutnormalisation.Inaddition,weneed
anauxiliaryfunctiong(x)withf(x)≤c·g(x).g(x)hastobeselectedinsuchaway
thatitiseasytodrawsamplesfromit.Forconsideringtheoriginalcourseoff(x)
(andthusp(x))thesampleswillberejectedwithprobabilitypaccept(x)=c·fg((xx)).The
figureshowsanexampleforg(x)andf(x).x2islessoftenrejectedthanx1because
atx2f(x)isbetterrepresentedbycg(x).Theclosercg(x)approximatesf(x),theless
rejected.reasamples

Rejectionandimportancesampling

Ifitisnotpossibletodrawsamplesfromaprobabilitydistributionp(x1,···xN)itself
becausee.g.thecumulativedistributionfunctionisnotknownoritiscomputationally
toThisextensivmethoedtousesdrawans(moreamples,sthenimple)arejectionuxiliarysamplingfunctioncanforhdraelptwingosolvtheesthisamples.problem.For
bility,consideringwhichrtheoepresenriginaltsthecoursedifferenceofp(xb1e,tw···eenxN)p(eacx1,h···xsampleN)andistherejectedauxiliarywithapfunctionroba-
(seeFig.2.22).
Typically,MonteCarlomethodsareusedforevaluatingequationsfromthetype

<a>=a(x1,···,xN)·p(x1,···xN)
x1,···,xN

51

2.4Learningandusing(neuronal)networks51
byapproximating<a>through
<a>≈1ax1t,···,xtN,(2.57)
N−1
N0=twherex1t,···,xtNaredrawnfromthedistributionp(x1,···xN).
Giventhecasethatp(x1,···xN)cannotbeusedtogeneratethenecessarysamples
forEq.(2.57),weneedtousea’workaround’like’rejectionsampling’(Ripley,1987;
Devroye,1986).Forthisalgorithm,weneedtoassumeafunctionf(x)thatrepresents
by)x(pp(x1,···xN)=f(x1,···xN)
x1,···,xNf(x1,···xN)
andafunctiong(x)with
f(x)≤c·g(x)∀x
wherecisaconstant.Itmaynotbepossibletodrawsamplesxtfromf(x)orp(x)
directly,butitispossibleuseg(x)asasubstituteforf(x).Wedrawaxfromg(x)
andthissamplewillbeacceptedaspartofthesuminEq.(2.57)withtheprobability
f(x)
paccept(x)=c·g(x).
Theeffectivityofthismethoddependsonhowwellg(x)approximatesf(x).Thisis
expressedthroughthenumberofsampledxbeingaccepted.Forexample,inthecase
wheref(x)=g(x)withc=1,allxwillbeaccepted.
Thesameproblemofcalculatinganapproximationof<a>,canbeaddressedby
importancesampling(Hastings,1970).Here,thefunctiong(x)hastobeg(x)=0
wheref(x)=0.Thenwecanuse
ta(xt)g(xt)
f(xt)
<a>≈f(xt).
tg(xt)
Itmayhappenthatthisapproximationisnotveryaccurateforasmallnumberof
samplesbutthismethodisnotrejectinginformationfromanysamples.

Metropalgorithmolis-Hastings

TheMetropolis-Hastingsalgorithm(Walsh,2002;Hastings,1970;Metropolisand
Ulam,1949;Metropolisetal.,1953)isafurtherdevelopmentoftherejectionsam-
plingmethod,whichusesandgeneratesMarkovchainswhilesamplingtheprobability

52

Chapter2:TheoreticalandBiologicalBackground

distribution.IfthisMarkovchainis’ergodic’(Norris,1997)thenitisguaranteed
thatthewholedistributionisusedforsampling,independentlyofinitialvalues.This
algorithmcanbeusedtosamplefromp(x)when
p(x)=f(x),(2.58)
cwherecisanormalisationconstantthatishardtoevaluate.Whileapplyingthe
Metropolisalgorithm,aMarkovchain{xt}t=0,···,nisgenerated.SuchaMarkovchain
isbasedonaMarkovprocess,whichisarandomprocesswithmemoryonlyforthe
lateststate.Expressedinconditionalprobabilities
pxn+1|xn,{xt}t=0,···,(n−1)=pxn+1|xn,
wherexn+1representsthevaluesfortherandomvariablesXinthenextstep,andxn
fortheactualstep.UsingthisMarkovassumption,wecanbuildaMarkovchainwith
values{xt}t=0,···,nfortheregardedrandomvariables.
ThereareseveralwaysofproducingMarkovchains,onewaytogenerateaMarkov
chainistostartwithaninitialprobabilitydistributionp0(x)andtopropagatethe
iavdistributionpt+1(x)=pt(x)·Πt(x,x)
x(Chapman-Kolomogrovequation),whereΠt(x,x)representsthetransitionprobabili-
tiesbetweenstatexandx.Markovchainscanbe’ergodic’insuchawaythatthere
existsastationarystateψ(x)forwhich
ψ(x)=ψ(x)·Πt(x,x).
xIfaMarkovchainisergodic,thenaftera’burn-in’phase,itdoesnotdependonthe
initialsamplesanymoreanditwillshownoperiodicbehavior.Theuseofsuchaergodic
MarkovchainguaranteestosamplethewholespacewiththeMonteCarloalgorithm
independentofthestartvalues.FormoredetailsonergodicMarkovchainssee(Neal,
1993;Walsh,2002).
BacktotheMetropolisalgorithm,startingwithaninitialx0withf(x0)>0and
Π(x,x)(calledcandidate-generatingdistribution)fordescribingtheprobabilityfor
makingatransition(orjump)fromxtox.Forthisalgorithm,itisnecessarythat
Π(x,x)=Π(x,x).Beginningwithaninitialx,wesampleaxusingthecandidate-
generatingdistribution.Thiscandidatexwillbeacceptedwiththeprobability
p(x)
paccept=minp(xt−1),1
f(x)
=minf(xt−1),1.

2.4Learningandusing(neuronal)networks

53

IfthecandidateisacceptedthenitisstoredasanewelementintheMarkovchain.
Otherwisenewcandidatesaredrawnandtestedviapacceptuntilasuitablenewelement
isfound.Hastings(Hastings,1970)extendedthisproceduretocandidate-generating
distributionsthatdonotrequireΠ(x,x)=Π(x,x).Forsuchdistributionspacceptis
ybnegivf(x)Π(x,xt−1)
paccept=minf(xt−1)Π(xt−1,x),1.
ThismethodiscalledMetropolis-Hastingsalgorithm.Whileapplyingthismethod,
especiallyforbuildingagoodΠ(x,xt−1),severalproblemscanoccur,see(Walsh,
1993).al,Ne2002;

samplingGibbs

ofGibbsthesMaetropmplingo(Gemanlis-HastingsandaGlgorithm.eman,T1984;hecWalsandidate-h,2002;generatingNeal,1993)isdistributionaspecialΠ(x,casxe)
isgiventhroughunivariantconditionalprobabilitydistributionsinwhichallrandom
variablesdimensionalarepfixedrobabilitvaluesydwithtistributionheexceptionp(a,b,ofc),one.thenLetwuesacanssumecalculatethatwethehaveunivamarianulti-t
conditionaldistributionwiththefixedvaluesb=banda=aby
p(a|b,c)=p(a,b,c)
ap(a,b,c)
Furthermore,pacceptissetto1whichmakesthealgorithmacceptallthesamples
2002).,hals(WThewholeprocedureworksasfollows:Startingwithasetofinitialvalues(a0,b0)we
performone’scan’
ci←p(c|a=ai−1,b=bi−1)
ai←p(a|c=ci,b=bi−1)
bi←p(b|a=ai,c=ci)
bysamplingfromtheunivariantconditionaldistributionswhilesuccessivelyreplacing
i(aoldi,bvi,caluesi).tForhroughmorevnewaoriablesnes.theAtprotheecedurendcofanthebescan,extendedweaobtainccordinglyan.ewsetx=
Ifwecalculatedropwiththesthisetsfrominformationthetheburning-expinectationphasevandalueuseofaonlyfunctiontherf(xemaining)x,wecan
mE[f(x)]≈1f(xi)
m1=iorcancomputeamarginalizedprobabilitydistribution
mp(x)≈m1p(x|z=zi,y=yi).
1=i

54EM-algorithm

Chapter2:TheoreticalandBiologicalBackground

Letusassumethatajointprobabilitydistributionp(x,y|θ)isparametrizedbyθand
isdependingonasetofrandomvariablesz=(x,y),wherexisobservableandyisnot.
Thetaskistofindtheθ,whichmaximizesthemarginalisedprobabilitydistribution
p(x|θ)withgivenvaluesforx.Sometimesitishelpfulltousethelog-likelihood
L(θ|x)=log(p(x|θ))
insteadofp(x|θ)(Bilmes,1997),becauseproductscanbereplacedbysums.Thiscan
helptosimplifythe(analytically)handlingofequations.
ForfindingamaximumofLwecanusetheexpectationmaximisationalgorithm(or
’EM’-algorithm)(Dempsteretal.,1977;Buntine,1994;NealandHinton,1999;Bilmes,
1997).Thisalgorithmcompromisestwosteps,whichareappliedrepetitively:

1.Theexpectationstep,whichcomputestheexpectationfore.g.log(p(x,y|θ)),
givein−1theobservedvaluesforx(data),ui−sing1theprobabilitydistributionp(y|
x,θ),basedonthesetofparametersθfromthelastiteration
Π(θ,θi−1)=dylog(p(x,y|θ))·py|x,θi−1.
y2.Themaximisationstep,wheretheoptimalparameterθiscomputedfortheex-
,Πectationpθi=argmaxΠ(θ,θi−1).
θ

Eachiterationstepincreases(orconserves)thelog-likelihood(insteadofthelikelihood
alsomethothedisapguaranosterioriteed(Bundistributiontine,c1994).anbeIfaused).θionlyFindingincreasesalocaltheemaximxpuectationmbywthisith
respcalledectgtoeneralizedthelastEMsetof(NealandparametersHinton,andis1999).nottheAodisptimcussium,onofthenotherthevmarianethotsdoifs
theEMalgorithmcanbefoundin(NealandHinton,1999).Furthermore,EMcan
beunderstoodasadeterministicversionofGibbssampling(Buntine,1994;Sahuand
Roberts,1999).Asaconsequence,theexpectationstepcanbeapproximatedbyMonte
ds.ethomCarlo

2.4.4earningltReinforcemen

Letusassumethatwewanttobuildamachine(’agent’)thatinteractswithitsen-
vironment.Theagentisabletoobservedifferentstatesoftheenvironmentandis
abletoperformdifferentactionsthatchangetheagent’senvironment.Someofthe

2.4Learningandusing(neuronal)networks

55

agenFiguret2and.23:envIllusironmentrationtinofactheloseidntlooeractipsonituationandintduringerchangeofreinforcementinformationlearning.between

observedstateshaveaspecialmeaningfortheagent.Theagenttriestomanipulate
theenvironmentinsuchawaythatthesestatesshowanoptimalconfiguration.An
exampleforsuchasetupwouldbeacomputergamblingagainstahuman.Thecom-
puterisinstructedtooptimizetheamountofthegainedmoney(reward).Fordoing
sothecomputercamactivelychooseitsownmovesandcanobservethemovesofits
oppmaposnent.ituationsFig.on2to.23actionsillustratesinatwheaythatsituation.therewTheardcismomputationalaximized.problemishowto
Forthispurposereinforcementlearning(Hertzetal.,1991;SuttonandBarto,1998)
hasbeendeveloped,inspiredbyrewardsignalsfoundinthebrain(seesection5.2.2).
Inreinforcementlearning,theagentlearnsfromitsownexperiences.Thisisdifferent
fromsupervisedlearning,whereasupervisorprovidesknowledgeaboutthesituation.
Reinforcementlearningcanincludesuchinstructionsasinformationsourcebutrein-
peforcemenrvisedtllearningearningistyoftenpicallynotdepeapplicablendstoatohighersituations,degreewohicnhevdaepluativendeoninfeedbactk.eractions.Su-
Reinforcementlearningcanincludeplanning.Elementsofreinforcementlearningare
1998):Barto,nda(Sutton

liciesoP•Apolicycanbeunderstoodasamappingfromobservedstatesintheexternal
worldontoactions.Anothernameforpolicyisstimulus-responserule.Insimple
settingsthiscanbeastationaryfunctionorlook-uptable.Ingeneral,thepolicies
maybestochasticandverycomplex.Forexample,apolicyforanagentcanbe
describedbytheprobabilityπt(State,Action)forselectingthenextaction.
Reinforcementlearningmethodsactonthesepoliciesandchangethem,asa
resultoftheagent’sexperiences,suchthate.g.thetotalamountofaccumulated
maximized.isrdarewfunctionrdaRew•Therewardfunctiondefinesthegoalforthereinforcementlearningprocess.It

56

Chapter2:TheoreticalandBiologicalBackground

representstheworthofreachingstatevaluesfortheagentbyasinglenumber,
thereward.Maximzingtheactualrewardorthetotalaccumulatedrewardis
theaimofreinforcementlearning.Rewardsmaybestochasticwithrespectto
theperformedaction.Inaddition,therewardmaynotbedistributeddirectly,
butdelayed,aftertheperformedaction.Actionsmayalsoeffectallsubsequent
rds.arewfunctionlueaV•Alwaysselectingtheactionwhichyieldsmaximalreward(called’greedyalgo-
rithm’)maynotmaximizetheaccumulatedrewardovertime(seeFig.2.24).
Thevaluefunctionrepresentsthelong-termdesirabilityofastateincludingthe
expectationsofaccessibilityandrewardsforthefollowingstates.Rewardsare
providedbytheenvironment,thevaluemustbeestimated.Thevaluefunction
mayincludeinitsevaluationonlyagivennumberoffutureactions.
•Model(abouttheenvironmentusedbytheagent)
Amodelcanbeusedforestimatinghowabehaviorwillinteractwiththeen-
vironment.Modelsareespeciallyinterestingforplanning,e.g.combinationsof
action,andforinferringrewardsandvaluesofstates.Itispossibletoperform
reinforcementlearningwithouthavinganexplicitmodel.

Inreinforcementlearning,atrade-offcanarisebetweenexploitationbehavior,using
learnedactionstorealiserewards,andexplorationbehavior,searchingforbetteractions
yieldingmorerewardinthefuture.
Manydifferentapproachesofimplementinglearningstrategieswereproposed(Sutton
andBarto,1998;W¨org¨otterandPorr,2005).Forexample,the’classical’wayoftrain-
ingisdynamicalprogramming.Correspondingalgorithmsarebasedonevaluatingthe
optimalpolicyfromavaluefunctionfora(perfect)modeloftheenvironment.Dy-
namicalprograming(DP)algorithmsareknowntobeslowandtypicallyneedahigh
computationaleffort.MonteCarlo(MC)methods,ontheotherhand,donotrequire
completeknowledgeoftheenvironment.Thesemethodsarebasedonlyonexperi-
encesharvestedbysequencesofstates,actionsandrewardsfrominteractingwiththe
surroundings.Morepopularthandynamicalprogramingistemporal-difference(TD)
learning,whichcombinesideasfromdynamicalprogramingandMonteCarloalgo-
rithms(SuttonandBarto,1998).Itisalsopossibletouseexpectationmaximization
algorithmsforreinforcementlearning(DayanandHinton,1997).
IwanttobrieflyintroduceaninterestingideafromXieandSeungtrainingirregular
spikingneuralnetworksbyreinforcementlearningstrategies(XieandSeung,2004b).
Inthispublicationtheauthorsshowthatitispossibletotraintheweightsofaneuronal
network(forsolvingtheXOR-problem)withtherewardRandthecorrelationbetween
theinputandoutputofanetwork.Themodelisbasedonanetworkofneuronsthat
transmitsinformationbyPoissonianspiketrains.Theinstantaneousfiringratefor
neuroniisthengivenby
λi(t)=fi(Ii(t)),

2.4Learningandusing(neuronal)networks

57

withIi(t)asthetotalincomingsynapticcurrentfromneuroniandfi(.)asthemean
firingfrequencyofemittedspikes.Inturn,thetotalinputIi(t)isdefinedby
Ii(t)=wi,jhj(t),
jwherewi,jareweightsdescribingtheconnectionstructureandhj(t)arethesingle
synapticcurrents.Theauthorsmodelthetimecourseoftheincomingsynapticcurrents
throughthedifferentialequation
hi(t)=−τsyndhi(t)+δ(t−Tia).
dtaτsyndefinesatimeconstantandTiadescribesthetimestampforthea-thspikeof
.ineuronUsingthissetup,itispossibletobuildalearningrulethatworksontheactivityof
theneuronalnetworkduringatimeintervalT.Thelearningruleusesthe’eligibility
Ttrace’definedas
ei,j=dfi(Ii(t))hi(t)δ(t−Tia)−fi(Ii(t))dt
0dtfi(Ii(t))a
andchangestheweightsproportionallytotherewardRby
Δwi,j=ηRei,j,
whereηisapositivelearningrate.Thislearningruleusescorrelationsbetweenthe
rewardandthepresynapticandpostsynapticactivitiestoperformastochasticgradient
ascentontheexpectedreward.Thistemporallynon-locallearningrulecanbeextended
vialearninglineon-todwi,j=ηR(t)e¯i,j(t),
dtwhilee¯i,j(t)isdefinedbythedifferentialequation
τetde¯i,j(t)+e¯i,j(t)=dfi(Ii(t))hi(t)δ(t−Tia)−fi(Ii(t)),
dtdtfi(Ii(t))a
withτetastimeconstant.Itwasshownthatthistypeofcorrelation-basedalgorithm
canbeusedtotrainneuronalnetworkstoperformcertaincomputationslikesolving
theXORproblem.Whilethewholeideaisappealing,learninglargenetworkscanbe
veryslowandmaybenotapplicableforthisreason.
Regardingthisthesis,inchapter5reinforcementlearningwillreappear.There,I
willshowthatitisinterestingtocombinereinforcementlearning-likestrategieswith
neuro-prostheticcontrolsforcompensatingnon-stationariesintherecordings.Forthis
purposewewillusetheideaofcontrollingarandomwalkinparameterspaceofthe
prosthetic’scontrollerwithareward/errorsignalinaclosedloopsituation.

58

Chapter2:TheoreticalandBiologicalBackground

Figure2.24:Dependencyoftheaccumulatedrewardonthedecisionstrategy.Taking
alwaysthemaximalreward(greedystrategy)maynotbeagooddecision.Inthelong
runitmaybesometimesbettertousenon-optimaldecisionswhichmayallowabetter
0)accumtwoutyplatedesrofewsardotrategiesverlareongeravtaimeilableintervfor’als.DecisInionthis1’.Texamplehe(bgreenseginningtrategyathasdecisionthe
higheractualrewardandwouldbeselectedbyagreedystrategy.Selectingthegreen
orthebluestrategy,producestwonewpossibleoptions.Takingthesenewoptionsat
to’Decisigenerateon2’ainhtoigheraccounaccumt,uthenlatedtherewnardion-optimalnthelong(blue)run.strategyat’Decision1’allows

3Chapter

InformationProcessingSpikeby
eSpik

tionaMotiv3.1

Perceptionofourenvironmentandthusinformationprocessinginthebrainisknown
tobefast.TheworkofThorpeetal.(Thorpeetal.,1996)revealedthatthebrain
requiresonly150msforthedecisionwhetheranaturalscenecontainsananimalornot.
Assumingtypicalfiringfrequenciesofcorticalneurons(≈15-45Hz)andapproximating
thenumberofinvolvedprocessingstageswith10,itwasestimatedthataboutone
tothreespikesareusedperprocessingstepandneuronforthistask.Adifferent
exampleforthehighspeedofthemammalianbrainwasfoundincontourintegration:
macaquemonkeysweretrainedtocorrectlydetectalignededgeelementswithinafield
ofrandomlyorienteddistractoredgesfrompresentationslasting30-50msbeforea
maskappeared(MandonandKreiter,2005).Theseexperimentsdemonstratedthat
evenifcontourintegrationisperformedlocallywithintheprimaryvisualcortex,this
computationcanrelyonlyonveryfewspikesperneuron.Inmaskingexperimentsit
hasbeenshownthatgroupingprocessesinvolvedinGestaltperceptionarecapableof
bindingfeatures,whichwerepresentedforonly20-30ms(HerzogandFahle,2002).
Inadditiontotheaimtouselownumbersofspikes,thesespikesaretypicallyemitted
stochasticallyandshowahighdegreeofvariabilityintheirtiming(Insection2.1we
alreadydiscussedthistopicinmoredetail).Inthefollowing,wewillinvestigatethe
questionofhowinformationprocessingcanbeaccomplishedevenifonlyalownumber
ofstochasticallyemittedspikesareavailable(Ernstetal.,2007b).

Adeeperexaminationofthistopicrequiresahypothesisofhowinformationiscoded
intoaseriesofemittedactionpotentials.Oneapproachistoproposearank-ordercode
(seesection2.1),whoseappliancemayfailwhenthespikingprocessisnotstrictlyde-
terministic.Italsotypicallyrequiresadefinedtimeintervalforinformationprocessing.
Itisdoubtfulthattheserequirementsforarank-ordercodearefullfilledinhigherbrain

59

60

Chapter3:InformationProcessingSpikebySpike

areas.Iftheuseofstochasticspikesfortransmittinginformationisexplicitlyassumed,
thenseveralcodingprinciplesallowingforahighvelocityofsignaltransmissionhave
beenproposed.Examplesincludemassivepopulationratecodeswithandwithouta
balanceofexcitationandinhibition(Panzerietal.,1999;GerstnerandKistler,2002a;
FourcaudandBrunel,2002),orcodeswherethesignaliscodedintothevarianceof
inputsontoalargepopulationofneurons(Silberbergetal.,2004).Inaddition,the
shapeofneuronaltuningcurvesmightbeoptimizedtofacilitatefastinformationcod-
ing.Dependingonthetimeavailablefordecoding,eitherbinarytuningcurves(for
shorttimes)orgradualtuningcurves(forlongtimes)werefoundtobeoptimal(Bethge
etal.,2003a;Bethgeetal.,2003b)(seesection2.2.3).
Theseapproachesaloneprovidenoconclusiveexplanationoffastperceptionfortwo
reasons:First,theworkonpopulationcodinglargelyignorestheproblemofprocessing
informationwithrespecttoacertaintask.Second,itisdoubtfulifthemassiveamounts
ofneuronsrequiredforaccurateinformationtransmissionandinformationprocessing
areavailableinthebrain.Thisisthereasonwhythisworkwillbefocusedona
plausibleneuronalon-linealgorithmforsmallnetworks,whichmightexplainfastand
precisecomputationsdespiteahighdegreeofstochasticityinthespikingprocess.
InsectionA.4generativemodelswereintroducedasaframeworkthatiswellsuited
fordescribingprobabilisticcomputation.Intechnicalcontexts,generativemodelshave
beenusedsuccessfully,e.g.forthede-convolutionofnoisy,blurredimages(Richard-
son,1972;Lucy,1974).Sinceinformationprocessinginthebrain,too,seemstobe
stochasticinnature,generativemodelsarealsopromisingcandidatesformodelingneu-
ralcomputation.Inmanypracticalapplicationsand–becausefiringratesarepositive
–possiblyalsointhebrain,generativemodelsaresubjecttonon-negativityconstraints
ontheirparametersandvariables.Thesub-categoryofupdatealgorithmsforthese
constrainedmodelswastermednon-negativematrixfactorization(NNMF)byLeeand
Seung(LeeandSeung,1999).TheobjectiveofNNMFistoiterativelyfactorize’scenes’
(images)VμintoalinearadditivesuperpositionHμofelementary’features’Wsuch
thatVμ≈WHμ.Forthiswell-definedproblem,multiplicativeupdateandlearning
algorithmshavebeenderivedandanalyzedfordifferentnoisemodelsandvariousfur-
therconstraintsontheparametersWandinternalrepresentationHμ(LeeandSeung,
2000).Generativemodelscanbeseenasalgorithmicrealizationsofaprinciplestatedfirstby
Helmholtzwhichhypothesizes,thatthebrainattainsinternalstatesthataremaximally
predictiveofthesourceofsensoryinputs.Whilethisideahasoftenbeenusedin
phenomenologicalmodelstoexplainpsychophysicalandneurobiologicalevidence,it
istemptingtoapplyitalsotoneuralsubsystemsinthesensethatanetworkshould
evolveintoastatewhichis’thebestexplanation’ofallitsinputs.Thisperspectiveon
neuralsystemsmightappearratherphilosophical,butitturnsouttobeaveryuseful
frameworkforunderstandingneuronalcomputation.Generativemodelsalsocover
deterministiccomputationsbybeingabletopredictthemissingpartofatypical,but
incompleteinputpatternfromitsinternalrepresentation(foranexampleseeFig.3.1).

ationotivM3.1

61

Ingeneral,non-deterministiccasegenerativemodelsareobviouslyrelatedtoBayesian
estimation.Ifinputsignalsoriginatefromdifferentsensorymodalities,agenerative
modelcanperformoptimalmultimodalintegration,andtheinternalstateswillthen
3.1).represenRtecenoteptimalxpBaerimenytesianalevidenceestimatesdofemonstraunderlyingtesthatfeaturesthebra(Denevinceanetal.,indeed2001)adopt(Fig.a
Bayesianstrategywhenintegratingtwosourcesofinformation;inoneexampletactile
and(ErnsvtisualandiBanksnformation,2002),abwouthiletheinwidthanotherofeabarxampleisintuncertainegratedinaninformationoptimalabmouttannerhe
poshiftsitionintofheavhandirtualpositionhandis(K¨evordingaluatedandiWncorpolpoert,rating2004).priorInkanoddition,wledgeabmotionoutaprorandomcess-
inginareaMThasbeensuccessfullydescribedbyaBayesianapproach,replicating
theneurons’firingpropertiestovariousmovingrandomdotpatterns(Koechlinetal.,
wo1999).rkforTiaknvenestigationstogetherofgenerativneuralemodelscomputation.provideNevaeveryrthelesgs,eneralitisandanopplauseniblequestionframe-if
itispossibletoimplementabiologicallyplausiblegenerativenetworkwhichisableto
copewiththehighstochasticityofspiketrains,whileatthesametimebeingasfastas
pofothessibleeinxternalinwtegratingorld.theincominginformationintoasuitableinternalrepresentation

Inthischapter,agenerativenetworkwhichisdesignedtosolveatypicalfactorization
problemwithnon-negativityconstraintswillbeinvestigated.Incontrasttoprevious
workusingnoise-freeanaloginputpatterns(LeeandSeung,1999),thenetworkreceives
astochasticsequenceofinputspikesfromasceneVμ.Anovelalgorithmwhichupdates
theinternalrepresentationusingonlyasinglespikeperiterationstepwillbederived.
Itturnsouttobesurprisinglysimpleandefficient,andcanbecomplementedbyanon-
linelearningalgorithmforthemodelparametersthatcanbeinterpretedasaHebbian
rulewithweightdecay.Furthermore,modifyingthebasicframeworkofagenerative
modelallowstoimplementdeterministiccomputationsofarbitraryfunctions,which
willbeshowntobecloselylinkedtooptimalmultimodalintegration.Thedynamics
andperformanceofthisapproachisdemonstratedbyapplyingittotwoillustrative
cases:AnetworktrainedwithBooleanfunctionsdemonstratesthelearnabilityand
errorlessperformanceforarbitrarycomplexfunctions.Asmallnetworkforhandwritten
digitrecognitiondemonstratesthatlessthanonestochasticspikeperinputneuroncan
sufficetoachieveimpressingperformanceinavisualtaskexceedingthenearestneighbor
classifier(explainedinsection2.2.4).FurthermoretheBooleanparityfunctionisused
todemonstratethathierarchicalcombinationsofthebasicmodelcanperformefficient
spikebasedcomputationinaself-consistantmanner.Takentogether,theframework
providesanovelapproachforexplainingfastperceptionwithstochasticspikesina
generativenetworkmodelofthebrain.

62

Chapter3:InformationProcessingSpikebySpike

Figure3.1:Schemeofthegeneralframeworkinvestigatedinthischapter:AsceneV
composedasasuperpositionofdifferentfeaturesorobjectsiscodedintostochastic
spiketrainsSbythesensorysystem.Inthisexamplethescenecomprisesahouse,a
tree,andahorsetogetherwiththeirspokenlabels.Witheachincomingspikeatone
inputchannels,themodelupdatestheinternalvariablesh(i)ofthehiddennodes
i.Normally,thegenerativenetworkusesthelikelihoodsp(s|i)toevolveitsinternal
representationHtowards’thebestexplanation’whichmaximizesthelikelihoodofthe
observationS.Inthisexample,thenetworkseekstheoptimal,combinedexplanation
ofthespikesfromtheaudio-visualinputstream(blackarrows).Analternativeappli-
cationofagenerativemodelcanbeusedtoinferfunctionaldependenciesinitsinput.
Ineverynaturalenvironment,certainfeatureshaveahighlikelihoodofoccurrence
becausetheydescribetwodifferentaspectsofasingleobject.Inthisexample,the
impingingsignalscanbedividedintotwopartsSVandSAwhichrepresentthein-
putandoutputargumentsofafunctionSA=f(SV).Inourexample,thefunctionf
describesmeaningfulcombinationsofvisualandauditoryinput.Presentingonlythe
’visualpart’SVtoanappropriategenerativenetworkwillleadtoan’explanation’H∗
(greyarrowstotheright).ThisinternalrepresentationH∗inturncangenerateasharp
distributionoverthefunctionvalues,centeredatthecorrect’auditory’counterpartSA
totheinputSV.Inthissense,themodelcomputesthefunctionfmappingthecorrect
spokenlabeltoeachofthevisualobjects(greyarrowstotheleft).

3.2ASpike-BasedGenerativeModel
3.2ASpike-BasedGenerativeModel

63

Everynaturalenvironmentcomposedofvariousobjectsleadstoanabundanceofspikes
tivwhicehnetnwetworkosrkshouldintheinferbrain(or’receiverecognize’)fromtthesypicalensors’objorects’otherorbrain’features’regions.whichAgunderlieenera-
com(i.e.bgineeneratepriororkno’wpredict’)ledgeaabnoutotybservpicaledspikfeaturesetrainand(Fig.their3.1).frequencyHereby,oftohebrainccurrencemausts
wellaspreviouslyobservedsignalsandinputfromothernetworks.Thispriorknowl-
edgehastobeacquiredduringearlierlearningphases,wheretypicalcorrelationsin
thesrecognitiontreamsofpeedandinformationperformancefromallinsalourcesatertask.shouldbeextractedinordertoincrease

3.2.1BasicModel

Letusassumeastructurallysimplerealizationofagenerativemodel.Apartofthe
netarewcorkionnectedsgivenviabywaleighaytserofws,iinputtoanolaydeseriofndexed’hidden’bysno=des1i,...ndexed,S.Tbyhesei=input1,...no,Hdes.
weTheighhtsriddenepresennodtesholdinformationthemoabdel’souttinheternalinputstatesstatisticshμ,i(Fig.whent3y.1).picalThefeaturesconnectioniare
ispartsuppofoassedcenetoμ.explaAinadoptingnobtheservfedsramewceneorkVcμhosasenainlinear,(LeenandSonnegativeung,esup1999),erptohesimotiondelof
viaWeaturesfthe

Hvμ,s≈ws,ihμ,i.
1=i

(3.1)

Atafirstglance,thisapproachislimitedbecauseofitslinearityconstraintonthe
decompositionofanobservation.Nevertheless,suchaframeworkhasbeensuccess-
fullyappliedtomanyproblemsinnaturalimageprocessingwithanaloginputpatterns
(Hoyer,2004;OlshausenandField,1996;LeeandSeung,1999),andallowstocom-
parethedifferencesbetweenthisnovelspike-basedstochasticmodeldirectlytoknown
analogdeterministicalgorithms.Itwillbedemonstratedthatthisapproachissuffi-
cientlycomplextoallowforcomputationsofarbitrarydeterministicfunctionswithfew
stochasticneuronsinthebrain.Furthermore,themathematicalderivationofspike-
basedinformationprocessingcanbeappliedalsotonon-lineargenerativenetworks
hniques.ectsimilarusing

3.2.2FromPoissontoBernoulliProcesses

Becauseofthehighdegreeofstochasticityofspikeevents,networksinthebrainreceive
informationaboutasceneμthroughstochasticspiketrainsimpingingontheinput

64

Chapter3:InformationProcessingSpikebySpike

nodess.Inthismodel,weassumethatspiketrainsaregeneratedwithratesRμ∝Vμ
fromindependentPoissonianpointprocesses.Inthiscasethenumberofspikesineach
channelcountedwithinagiventimeintervalareasufficientstatistics,i.e.theyprovide
themaximalamountofinformationaboutVμavailablefromspikeobservation.
Forsettingupthemodel,theraterμ,sisreformulatedintermsoftheprobabilitypμ(s)
toreceivetherespectivenextspikeinchannels.Astraightforwardcalculationyields
SSp˜μ(s)=rμ,s/rμ,s=vμ,s/vμ,s.(3.2)
s=1s=1
Therepresentationofthesceneμthroughtheseeventprobabilitieshastheconvenient
propertiesthatitisinvariantwithrespecttothetotalrateandignoresthetruetiming
ofthespikes.ItchangesthedescriptionofthespikestatisticsbyPoissonianpoint
processestothemoretractableBernoulliprocessesandidentifieseachspikeeventwith
theindexofthechannelinwhichthisspikeoccurred.Thesepropertieswillstrongly
simplifythesubsequentconstructionofthismodel.
Insteadofrealtime,wewillnowusetherunningnumberofspikeevents,whichis
averageproportionaltothemeanoft/sS=1rμ,s.Thusglobalknowledgeaboutthe
denotedherebyt.Notethatthisspike-by-spikeclockingimpliesthatrealtimeison
totalinputrateallowstoestimatetherealtimeelapsedduringtevents,butisnot
necessaryforderivingthefollowingalgorithms.

3.2.3FromDeterministictoProbabilisticDecomposition

Similartothetransformationofvμ,sintop˜μ(s),theweightsws,iareproportionalto
theprobabilityp(s|i)ofaninputspikeinchannelsgiventhatthesceneμiscomposed
ofthesinglefeaturei,
Sp(s|i)=ws,iws,i.(3.3)
1=sReformulatingtheinputsceneVμintermsofthe(normalized)firingprobabilitiesp˜μ(s)
asdescribedinthelastsubsectiontransformsthefactorizationproblem(Eq.(3.1))into
theexpression
HHp˜μ(s)≈Hi=1p(sS|i)hμ,i=p(s|i)Hhμ,i.(3.4)
i=1hμ,is=1p(s|i)i=1i=1hμ,i
Itisobviousthatasimplevariabletransformationhμ(i)=hμ,i/iH=1hμ,ireduces
otproblemthisHp˜μ(s)≈p(s|i)hμ(i)=pμ(s),(3.5)
1=i

3.2ASpike-BasedGenerativeModel

withthenon-negativityandnormalizationconstraints

p(s|i)>0hμ(i)>0
SHs=1p(s|i)=1i=1hμ(i)=1.

65

(3.6)

Itshouldbementionedherethatthenewvariablesh(i)denotingthenormalized
superpositioncoefficientshaveacorrespondingstochasticμinterpretation.Toexplain
thedoubly-nextsatochctionasticppotenrotialcessarrivingunderlyingatonespikiegnputnoeneration:des,thefirst,aspgenerativecificemofeature,delaorcssumesausea
ifromisdrathewncforrespromtheondingprobabilitconditionalydpistributionrobabilithyμ(di).Inistributionasecopnd(s|i)step,,wahichspikiseiosbsderravewnd
inchannels.Whileatafirstglancethisinterpretationseemsratheracademical,itisa
supnaturalerimpdosedescriptionfeaturesofthecreatesphysicalphotopronscessofimpinginghowaonvisualretinalsceneganglioncompocells.sedoflinearly
Whileconnectingweininputterprettneuronhesconditionalwithhiddenpnrobabilitieseuroni,p(sthe|i)astosubemptionrelatedoftaocsontinynapticuouswineighter-ts
monalsdels.tateUvasuallyriablethetobmeodpreselsenutseintheeacahnctualeuronasctivitiesomewfhatorgoesrepresenbeytingondustheualnetnetwowrk’sork
state.Sincerealneuronsarespatiallyextendedobjectswithmanyinternalstatevari-
ables,weassumethatsomeofthemparametrizeneuronalexcitabilityratherindepen-
denderivtelydinfromtheitsanextctualsectionactivitmya.yThethencserveharacteristicasspecificdynamicspredictionsofintofernalthissmotatedel.variables

3.2.4EstimationandLearningSpikebySpike

bBaeymeusianltipliedwestimationiththeimpliesconditionalthatppriorrobabilityinformationofoabbservingoutttheheainctualternalexternalstatesinput,hould
giventheactualinternalstate.Therefore,generativemodelswithiterativeBayesian
algorithmsappeartobesuitablecandidatesforacomprehensive,abstractmodelfor
functionanddynamicsofnetworksinthebrain(Mumford,2002;LeeandMumford,
2003;Rao,2004).Inthecaseofperceptionsubjectedtotimeconstraints,priorknowl-
lineedgeasaboutefficienthetlyacausesspossibleunderlyingwhilethethepsreviousequencesofensoryobservainputtionshasoftothebeispikntesegratedarriveon-at
thedifferentinputchannels.Aswealreadydiscussed,previousmodelslargelyneglect
thisrealisticconditionbyusinganalog,noise-freeinputpatternsineachiterationstep
insteadofonlyasinglespike.
Anupdatealgorithmfortheinternalstatesandsynapticweightscanbederivedby
beconsideringwrittenasstotchehastictemporallyensemblesorderedofesxactlyequencesTincoofinputmingchspikesannelsforatawhicscenehtμ,whesehicspikhcesan

66

Chapter3:InformationProcessingSpikebySpike

,earrivSTμ=sμtt=1,...,T.(3.7)
Wenowseekaniterativealgorithmwhichwitheveryspikeencounteredtendstomax-
imizethelikelihood
PSTμμ{hμ(i)}μ,i,{p(s|i)}s,i(3.8)
ofobservingtheensembleofsequencesSTμμ=1,...,Moverthespaceofallinternal
states{hμ(i)}μ,iandmodelparameters{p(s|i)}s,i.With
Tp^μ(s)=1/Tδs,sμt(3.9)
1=tdenotingtherelativenumberofspikesinchannelsinoneobservationsequenceμ,this
likelihoodisgivenby
P=Πμ=1ΠS(Tp^μ(s))!Πs=1pμ(s)Tp^μ(s).(3.10)
MT!S
1=spμ(s)isdefinedasinEq.(3.5)1.Thestandardapproachtothisoptimizationproblem
istominimizethenegativelogarithmofthelikelihoodP,
SL=−1/TlogP=C−p^μ(s)logpμ(s).(3.11)
1=sundertheconstraintsmentionedinEq.(3.6).Cisaconstantwhichdoesnotdepend
ontheoptimizationvariables.Ifthereisnopriorknowledgeaboutthetypicalfeatures
ofwhichscenesμarecomposed,thegenerativenetworkwillhavetodoboth,estimate
hμ(i),andlearnthelikeliestsetoffeaturesp(s|i)inordertoexplaintheensembleof
observedspikesequences.Assoonassuitablep(s|i)havebeenfoundandcanbeheld
constant,minimizingLturnsintoaconvexoptimizationproblemforthehμ(i)only.
Unfortunately,serialalgorithmsupdatingtheirrepresentationswitheachincoming
spikefromoneinputpatterncannotbededuceddirectlyfromknownupdateequations
whichworkonμ=1,...,MfullobservationsequencesSTμinparallel(LeeandSeung,
1999).Thereasonsforthiswillbecomeclearduringthefollowingderivations,inwhich
wewillseekforarapidupdateruleforthehμ(i)’sandp(s,i)’sminimizingL,which
arebasedonasinglespikeobservationsμtineachiterationstept.
1Notethatp(s),p^(s),andp˜(s)alldenotedifferentvariables.Whilep˜(s)describes
thereal,butunknowninputstatisticsofascene,p^(s)denotesaparticularstochastic
realizationofthisscenemeasuredastherelativespikecountattheinputnodess.The
approximationofp^(s)byalinearsuperpositionofelementaryfeaturesisthendenoted
byp(s).

3.2ASpike-BasedGenerativeModel67
In(Lanteriparticular,etal.,we2002),willmaimaultiplicativtameualtiplicativlgorithmseaurepdknateowrnule,tocbonvecauseergeivnerymfanasytcwaseshen
comparedtosimpleadditivegradientmethods.Unfortunately,itwasnotpossibleto
findthefastestalgorithmpossiblebyderivingitdirectlyfromLanditseemsthatthis
cannotbedoneforthegeneralcase.
Fordevisingamultiplicativerulebymeansofagradientdescent,thederivatives−∇L
ofthelog-likelihoodhavetobecomputedfirst,
∂LSp^μ(s)
−∂hμ(i)=s=1pμ(s)p(s|i)(3.12)
∂LMSδs,sMp^μ(s)
−∂p(s|i)=μ=1s=1p^μ(s)pμ(s)hμ(i)=μ=1pμ(s)hμ(i).(3.13)
Forunconstrainedproblems,D∇LisavaliddescentdirectionifDisapositivedefinite
matrix(Bertsekas,1995).Withzdenotingtheiterationstep,thegradientdescentin
itsmostgeneralformreads
h˜μz+1(i)=hμz(i)−γhDhz∇hμLzi(3.14)
p˜z+1(s|i)=pz(s|i)−γpDpz∇pLzsi.(3.15)
Inthiscase,letuschooseDhzandDpzasdiagonalmatriceswithentries[Dhz]=
hz(i)andDpzsi,si=pz(s|i).DhzandDpzexplicitlydependontheupdatestepi,iz.
Furthermore,anupdateconstantsγh(’estimationrate’)andγp(’learningrate’)are
introduced.Sincetheupdateisalwayspositive,wehavetosatisfythenormalization
constraintsonly,whichisdonebyapost-updatenormalizationscheme
z+1h˜μz+1(i)
hμ(i)=jH=1h˜μz+1(j)(3.16)
p˜z+1(s|i)
pz+1(s|i)=sS=1p˜z+1(s|i).(3.17)
Combiningupdateandnormalization,andsubstituting=γh/(1+γh)andγ=
γp/(1+γp)yieldstheequations
Shμz+1(i)=hμz(i)(1−)+pp^zμ((ss))p(s|i)(3.18)
s=1μ
z+1pz(s|i)(1−γ)+γμM=1pp^μzμ((ss))hμ(i)
p(s|i)=(1−γ)+γμM=1hμ(i)sS=1pp^μzμ((ss))pz(s|i).(3.19)
TheseequationsstillusethefullsequencesSTμforallμ=1,...,Mpatternstodoone
updatestepfromztoz+1.IfagivencollectionofTspikesisconsideredafixedinput

68

Chapter3:InformationProcessingSpikebySpike

vector,algorithmsE(q.(3.18)LeeandandESeung,q.(3.19)2000)havwehicthehscanamebfiedexedrivpeoindtussaisngtheescorrestimation-maxpondingimizationNNMF
real(EM),time,andofornlyone→s1ceneandμγis→1presentheytedbaecondmeonlyequivonealoenrt.veryHofwewever,spikatesoanreeoinstabservnteind.
InalgorithmsordertotocononevertpEqatterns.(3.18)μatandat(ime.3.19)Ftoanurthermore,on-lineletupusdateassumerule,wethatresfromtrictthisthe
Thispattern,proonlycedureoneissepikquiveoalenbservttoeddattroppingimettheensterstheummationsupdoateveerμ,quationsreplacingforhzbyand/ort,anpd.
replacingthefullpatternp^μ(s)byδs,st.Thecorrespondingupdateequations(’on-line
learning’and’on-lineestimation’)resultingfromEqs.(3.18)and(3.19)thenread
tht+1(i)=ht(i)(1−)+pp(t(ss|ti))(3.20)
pt+1(s|i)=pt(s|i)1+(1−γ)pt(st)+γh(i)pt(st|i)δs,st−pt(st|i).(3.21)
γh(i)
Inthefollowing,thealgorithmdefinedbyequationEq.(3.20)willbetermed’spike-by-
spikeon-lineestimation’,whileEq.(3.21)willbereferredtoas’spike-by-spikeon-line
learning’.Eq.(3.20)andEq.(3.21)areon-linealgorithms,incontrasttoEq.(3.18)andEq.(3.19)
andtheNNMFlearningrules.Theirupdatereliesononlyonespike,butitisstraight-
forwardtoconstructversionswhichtakeintoaccountseveralspikesforeachiteration
step(thisissimplydonebychoosingTintheEq.(3.9)forp^μequaltothenumber
ofspikeswhichshouldbeconsideredinonestep).Perconstruction,theon-linever-
sionshavefinitememory,andalsoforstationaryinputpatterns,fluctuationsofthe
hiddenalgorithmsstatescanwatillinmostacgeneralhievreemainapproxfiniteimateduemtotheaximizationstochofasticthedlikrive.elihoTodhereforeforthebofullth
inputexamplesspiktehatswequence.ithsInuitablythecnhextosensupectiondateitwillbparametersedemonstrated∈[0,1]byandmγeans∈[of0,1],tsimplehis
approximateconvergenceissufficienttoachievehighcomputationalperformance.

3.2.5Simplifiedalgorithmwithbatchlearning

Theon-linelearningequationcanbetransformedtoreducecomputationalcomplexity.
Inthiscase,learningofthep(s|i)isassumedtotakeplaceonamuchlargertimescale
thantheestimationprocessoftheinternalstateh.
Inreality,learningisaslowprocessextendingoverthe(repeated)presentationof
manytrainingscenesμ.Eq.(3.21)herebyimposesahighcomputationalloadduring
on-linelearning.Thisloadcanbesignificantlyreducedbyassumingthatonafast
timescale,thehiddenrepresentationshμtforeachμ,μ=1,...,M,arecomputedwith
theon-lineestimationalgorithmEq.(3.20)forTspikesandtimestepseach.After
thesecomputations,nowonamuchslowertimescale,theaveragedhiddenactivities<

69esultsR3.3h>μΔ=1/ΔtT=T−Δ+1hμtareusedinparallelforasingleupdatestepoftheconditional
probabilitiesp(s|i)accordingtoEq.(3.19).Asuitablemultiplicativealgorithmforthis
updatehasbeenproposedbySeungetal.(LeeandSeung,1999)whichreads
MH
p˜z(s|i)=pz(s|i)p^μ(s)<h>μΔ(i)pz(s|i)<h>μΔ(i)(3.22)
Sμ=1i=1
pz+1(s|i)=p˜z(s|i)p˜z(s|i).(3.23)
1=sTheparameterz=1,...,Zcountsthelearningstepsandisidenticaltotheupdate
stupepdateusedofintheEqps(.s|i)(3.18)isandmade(eac3.19).hTMWithspikMes.Edifferenqst.(3.22)scenesand((stim3.23)uli)togeinthetotal,rwioneth
Eq.(3.20)definetheSpike-by-Spikebatchlearningalgorithm(short:SbS-batch).Note
thatEqs.(3.22)and(3.23)canbederiveddirectlyfromEq.(3.19)bysubstitutingthe
average<h>forh,andbychoosingthemaximumupdateconstantγ→1.
sultseR3.3Figure3.2:Spike-by-spikenetworkfortrainingaclassificationtask.Thetraining
clapatternssses,awreithpSresenpcotedmpaosnrenats,ndotogmlyetherdrawnwithspiktheiretracoinsrrecttoclaS=ssificaStio+nSintoinputoneonofdesSc
cpduringlearning.Thenetworktherebyfindsasuitablerepresentationp(s|i)ofthe
inputensemble,andestimatesaninternalstateh(i)foreachinputpatternaccording
toeithertheSbSon-lineortheSbSbatchalgorithm.
Inthissection,itwillbedemonstratedthattheSbS-onlineandSbS-batchalgorithms
optimizesuitablerepresentationsofsceneensemblesbypredictingthespikesarriving
attheinputnodes.ForlearningBooleanfunctions,thesceneensemblewillconsistof
allinputandoutputbitpatternsoftherespectivefunction.Forlearninghandwritten

70

Chapter3:InformationProcessingSpikebySpike

presenFigureted3.3:totSpikhee-bSpy-Spikinputennoetwdes,orkandforcanlasasificationppropriateofipntatternsernal.sTtatehethes(it)patterns’explaining’are
theclaassificactualtionpq(atternc)isisinferredestimatedandbytcomparedheSbStothealgorithm.correctFromclassification.theinternalstate,a

digits,thesceneensemblewillberepresentedbythepixelpatternsofthedigitimages
togetherwiththeircorrectclassificationsfromatrainingdataset.
Theperformanceofthelearnedrepresentationwillbeevaluatedinasubsequentcom-
putationorclassificationtaskonatestdataset.Resultswillbecomparedtoasimple
nearest-neighborclassifier(abbreviatedbyNNclassifier,fordetailsseesection2.2.4)
andtotheNNMF-algorithmbySeunget.al(LeeandSeung,1999).
Attheendofthissection,theSbSbatch-learningalgorithmwillbeappliedtonatural
uli.stim

ExampleimpleSA3.3.1

First,letusillustratecharacteristicpropertiesofthealgorithmforspike-by-spikeup-
dateofinternalstatesandtheresultingcodinginthenetworkusingaminimalmodel.
Forthispurposewewillconsiderahighlyover-completesituationwhereS=2input
neuronsprojecttoH=100hiddenneurons.Theoptimizationproblemistherefore
under-constrained,andaccordingly,theSeungmodelconvergestodifferentinternal
statesHfordifferentinitialconditions.Itappearsthatthealgorithmprefersmixtures
ofalmostallavailablefeaturesWasshowninFig.3.4(a).
Incontrast,thesparsesolutionHfoundbytheon-lineupdatealgorithmturnsouttobe
uniqueforalmostallinitialconditions(Fig.3.4(b)).Thedegreeofsparsenessistightly

esultsR3.3

71

Figure3.4:(a)DynamicsofinternalrepresentationHindependenceontheiteration
stepzfortheNNMF-modelbySeungetal.,and(b)forthespike-by-spikenetworkin
dependenceonthenumbertofincomingspikes.ThenetworkscompriseS=2input
nodesandH=100hiddennodes,withtheweightschosenasp(1|i)=(i+1)/H,
andp(2|i)=1−p(1|i).Theinitialstatewasrandom,butidenticalforbothmodels,
andtheinputwasgivenbythevectorV={0.42,0.58}.WhiletheNNMF-model
convergestoabroadmixtureofallfeaturevectorsp(s|i),theSbS-networkconverges
toasparsestatewheretheinternalrepresentationpeaksaroundthefeaturevector
p(s|42)={0.42,0.58}andaccuratelyreflectstheinputratedistribution.Theupdate
constantwas0.5.Inadditionweshowintherightplotof(b),thattheKullback-
Leibler(KL)-divergence,averagedover1000runswithdifferentV,decreaseswith
iterationtime,thusdemonstratingconvergenceofouralgorithm.

linkedtotheparameter:thelarger,thesparserbecomestheinternalrepresentation.
With=1,onlyonehiddenstatewillbeactivewhichhasthemaximumlikelihoodto
explaintheinputsequence.IntheexampleshowninFig.3.4(b),thehighvalueof
causestherepresentationtoconcentrateonthefeaturevectorwhichisinthiscasethe
’mostnatural’explanationoftheinputspikestatistics.

3.3.2Pre-Processing,Training,andClassification

Thesimulationsarecarriedoutintwostageswhichconsistofatrainingandtestrun,
respectively.Beforepresentingtheresultsonlearningandperformanceforspecific
thedatacomputations,intolettrainingusandbrieflytestinstroets.duceconstructing,pre-processingandpartitioning
Training.trForlearningaclassificationorcomputationtask,eachoftheMtrtraining
scenesVμcomprisesapatternorimagetogetherwithitscorrectclassificationintoone
oftransScclasses.formationlFirst,eadsttoheavnegativerageeispremoatternvevdafluesrom,eandachbpecausatternefi(orringimage).ratesareSincealwtahisys

72Chapter3:InformationProcessingSpikebySpike
positive,itisnecessarytodistributethecorrespondingvaluestotwicethenumberof
channels.Herebypositivevaluesaredirectlyassignedtotheoddchannels,whilethe
absolutevaluesofthenegativeentriesareassignedtotheevenchannels.Second,the
classificationisrepresentedbyavectorwithScentrieswithonlyonenon-zeroentryat
thepositioncorrespondingtothecorrectclass.Finally,thisvectorandtheprocessed
patternareweightedandcombinedtoformasingle,normalizedvectorVμtrfromwhich
theinputspikesaredrawn.Themathematicaldetailsofthisprocedurearedescribed
insectionB.1andB.2.
OnVμtr,thenetworknowlearnsasuitablerepresentationincludingthecorrectmatch
betweenpatternsandclassesinanunsupervisedmanner.Thisprocedureisschemati-
callyexplainedinFig.3.1,andemploysthenetworkstructuresketchedinFig.3.2.
Test.Fortestingthetrainedgenerativenetwork,theMtstestscenesVμtsconsist
onlyofthetransformed,positiveinputpatternswithouttheclassinformation(details
describedinsectionB.3,seealsoFig.3.3).Thecorrectclasscμtsiscomparedtothe
predictionc^μbeinginferredfromtheinternalstatesofthegenerativemodel.This
methodisdescribedinFig.3.1asinferenceprocedurewithpartialsensoryinputand
employsthenetworkstructuresketchedinFig.3.3.

foleanBo3.3.3unctions

ForaproofofprincipleweusedtheproblemoflearningandcomputingBooleanfunc-
tions.RealizingarbitraryBooleanfunctionsinnetworksthatreceiveonlystochastic
inputisparticularlynasty.Thespikenoiseneedstobecancelled,becauseBoolean
functionsarenon-smoothinthesensethatflippingofoneinputbitcanresultin
acompletechangeoftheoutputvalues.Ifanetworkiscapableofperformingthe
requiredoperation,thishasinterestingconsequencesformultimodalintegrationand
attention,whichwillbeexplainedinthesection3.4.Inanycase,efficientcomputation
ofBooleanfunctionswoulddemonstratethatstartingfromthewell-knownXORprob-
lem,inprincipleanypossiblecomputation,canberapidlyperformedinsmallnetworks
withstochasticspikes.
Forthesimulations,BooleanfunctionsofN=5inputbitsNmappedtooneoutput
bitwereselectedrandomly,leadingtoMtr=Mts=32=2input-outputpatterns2
foronespecificfunctionifoneunitisidentifiedwitheachinput-outputrelation.
Consequently,anumberofH=2NhiddenunitsshouldsufficetorepresentoneBoolean
functionofNbits.Inthiscase,eachweightvectorp(s|i)comprisingtheconditional
probabilitiesforafixedimatchesoneoftheMtstestpatterns.Consequently,for
2Thesamesetofpatternswereusedbothfortrainingandclassification.Note,
however,thattherealizationofthepatternswasdifferentinbothrunsbecausethe
patternsarerepresentedbyafinitenumberofinputspikes,whicharerandomlydrawn
ineachrepetitionofapresentation.

esultsR3.3

73

eachoftheinputpatterns,onlyoneofthehiddenunitshshouldbeactive(winner-
takes-allnetwork).Incontrast,itmightbepossiblethatlessthan2Nhiddennodes
inthenetworkaresufficienttoperfectlyrepresentaspecificBooleanfunction.In
this(softccase,ompeeachtitioninputnetwpatternork).Acansbweercepresenonsideredtedobynlyaonemixtureoutputofactivbit,etherehiddenarenoodnlyes
Sc=2differentclassestowhichaninputpatterncanbeassigned.Fordetailsofthe
simulation,seesectionB.4.

bits,Figurein3.5:depeMndenceeancolasnsitheficationnumeberrroroffornoBodesoleanHinftheunctionshiddenof5layerinputofandthe1netwooutputrk.
The(dashed-SbS-dottedonlineline),learningwhich(solidbarelyline)acpproaclearlyhesoutp20%erformserror.thenoiselessNNMF-algorithm

Performanceofthenetwork.Fig.3.5showsthemeanclassificationerrorofdif-
ferentalgorithmsindependenceonthenumberofhiddenunitsH.TheSpike-by-
Spikealgorithms(withon-linelearning)performsconsiderablybetterthantheNNMF-
algorithm.TheNNMF-algorithmtendstoexplaintheinputasaverybroadmixtureof
manybasicfeaturesasintheotherexampleshowninFig.3.4,whiletheSpike-by-Spike
algorithmconcentratesonthemostpredictivefeatures.WithSbStheerrordropsto
approximatelyzeroassoonasmorethanH=25hiddenunitsareinthenetwork.In
thesecases,theinputpatternsarereconstructedusingamixtureoftwoormorefea-
turesrepresentedbytheactivatedhiddennodes.Fig.3.6showshowtheclassification
errordecreaseswiththenumberofincomingspikes.Here,SbSleadstoaclassification
errorwhichreacheszeroafteronlyaboutthreespikesperinputnode.Incomparison,

74

Chapter3:InformationProcessingSpikebySpike

Figure3.6:ErrorrateeofSbSon-linealgorithm(solidline)forthetaskintroduced
inFig.3.5,comparedtotheerrorofaNN-classifier(dottedline).TheSbSon-line
algorithmisasfastandasaccurateastheNN-classifierwhichinthiscaserepresents
anoptimallook-uptableoffunctionvalues.Notethatameanof3spikesperinput
neuronissufficienttoachieveperfectclassification.

theNN-classifierisalsoastonishinglyfastandcloselyapproachestheerrorcurveof
theSbSalgorithm.Inthisspecificexample,thecomplexityofthe’codebooks’ofthe
SbSalgorithmandNN-classifierarecomparable.

Handwritten3.3.4igitsD

Asathirdexamplewewillselecttheproblemofhandwrittendigitrecognitioninorder
todemonstratethatreal-worldestimationproblemscanbesolvedefficientlyandvery
fastbyestimationwithsinglespikes.WewillusetheUSpostalservicedatasetwhich
servesasabenchmarkforclassificationalgorithmsandallowscomparisonwithhuman
performance.Forparametersofoursimulations,seesectionB.5.
Performanceofthenetwork.Fig.3.9showstheclassificationerrorofthenetwork
versusthemeannumberofspikesperinputnode.ItturnsoutthattheNN-classifier
isremarkablygood,reaching6.1percenterroronthetestsetafteraboutameanof
onespikeperinputnode.Thisperformancecomes,however,atthepriceofusingthe
fullMtr=7291trainingpatternsforclassification.Incontrast,anetworkwithonly
H=500nodesoptimizedonthetrainingsetwiththebatchalgorithmwithonespike

esultsR3.3

75

perinputneuronhasasuperiorperformance.Usinganannealingstrategyanincrease
inclassificationspeedandqualitycanbeobtainedbyadapting(’cooling’)during
learningandestimationoftheinternalstates:thecorrespondingcurve(dottedlinein
Fig.3.9)showsthatwithonlyH=100theclassificationisnearlyasfastandgood
astheNN-classifierwithits7291storedpatterns.Fig.3.8showsthecompleteweight
setforthecaseSbSwithH=500and=0.1.Abouthalfoftheweightsconsist
oftemplateswhich,however,becomecombinedwithparticularfeatureweightsduring
classificationrunsandarerequiredtoachievetheclassificationperformanceshown.
Withthesameparameters,theclassificationerrorofthenetworkindependenceofH
isshowninFig.3.10.UsingH=25hiddennodesisalreadysufficienttoobtainan
errorwhichisonlytwiceashighastheerroroftheNN-classifierusingthefullsetof
7291patterntemplates.Inaddition,theerrorcurveshowsnosignaturesofoverfitting
asitdecreasesmonotonicallyevenforverylargenetworkswithuptoH=7290nodes.
Robustnessagainstnoise.Whilethealgorithmbyconstructionisrobustagainst
noiseinthespikingprocess,thequestioniswhetherthisremainstrueforothertypesof
noise.Toaddressthisissue,thedigitpatternsweresubjectedtotwotypesofpertur-
bations:first,anincreasingnumberofverticalorhorizontallineswereoccludedinthe
digitpatterns(startingwiththetwocenterlines),andsecond,anincreasingamount
ofnoisetoeachpixelinapatternwasadded.Fig.3.11(a)showstheclassification
performanceindependenceonthenumberofcoveredlines.Uptoanumberof6lines,
classificationerrorstaysbelow20percent.Fig.3.11(b)demonstratesthatpixelnoise
mustincreasetoaconsiderableamountη=0.35inordertoraisetheerrorabove20
t.rcenep

3.3.5HierarchicalNetworks

vaLetriaubslesdh(i)emonstrateasprobabilitiesthatthisforbasicsendingnetwooutrknextcanbespikces,omi.e.binedwebtyurnutsinghemtheintohiiddennput
neuronsofsubsequentnetworks.Asasimpleexampleahierarchicalnetworkwillbe
presentedwhichsolvestheBooleanfunctionoftheparityproblemwhichrequiresthe
outputtobezerowheneveranunequalnumberofinputneuronsareNactiveandtobe
oneotherwise.WhilethisproblemcanbesolvedtriviallywithH=2hiddenneurons,
ahierarchicalarrangementofsimpleSbS-networkscandrasticallyreducethesizeofthe
netmanner,work.usingThisthenetwoorkutputproofvesonetshatimplethemoapproacduleahscaanbemeaningfulgeneralizedinputintoasanotherelf-consistensimplet
module.computations.Therefore,TogetheronecanwithctheoncluderesultsthatonBSbS-oonetleanworksfunctions,canbecspikome-bybined-spikforenetwiterativorkse
arethusabletocomputearbitrarycomplexfunctionswithanyrequiredprecisionina
finiteamountoftime.
bitsThebpeingarit1yisfanunctodiondnnumetbweor,rk.andThe0binaryotherwise.paritFoyranfunctionarbitraryis1ifnuthembnerumofberinputofinputbits,
theparityfunctioniseasilycomputedbyahierarchicaltreeofXOR-modules.Forthe

76

Chapter3:InformationProcessingSpikebySpike

simulations,N=16inputbitsarechosen,thushavingSp=2N=32on-offinput
channels,andtwooutputclassificationnodes(Fig.3.12).
Thefullnetworkconsistsof15spike-by-spikeXOR-modules,whoseweightswereset
upmanually(i.e.avalueof0.5wasassignedtoallconnectionsdrawninFig.3.12).In
everysinglemodule,input/outputnodesareshownaswhitediscs,andhiddennodes
asXORgra-myodulediscs.areTheupdhatediddenbynodeaEqs.(3.20)ctivitiesandh((i)B.8)andwithoutputeachvaincomingriablesp^(spikk,e.n)ofSpikesone
areelicitedbyeithertheexternalinput,orbytheactivitiesp^(k,n)ofaninternal
outputnoden,whichisatthesametimetheinputnodeforthenextXOR-module.
ofWiththepactualrobabilitiynputpIbit=0.05,pattern,aspikaendissendtratwnooneaccordingoftheto8XtheOpR-morobabilitdulesyidntheistributioninput
layer.WithprobabilitypH=^p(k,n)(1−pI)/14,aspikeisdrawnfromoneofthe
outputnodesofthe14XOR-modulesinthelowerlayersofthehierarchy,andpassed
ontothenextXOR-module.
ofIfthenetinput/outputworkpenorformsdessthehouldintendeddisplaycaweomputationll-definedpaerfectlyctivit,ybyforconseachtructioninputeachpattern.pair
Asanexample,considerthetwooutputnodesmarkedbyanarrowinFig.3.12.If
ntheumnbumer,bteherofoutputbitsvofavriablealueof1thewithinleftnothedefirstshould8bitstakoefatlheowerinputvaluepattern(ideallyis0an)tevhaenn
theoutputvariableoftherightnode(ideally1),thuscomputingtheparityfunction
forthefirst8inputbits.Fig.3.13displaysthecorrespondingerrorsofthenetworkfor
theoutputlayerandfortheinput/outputnodesinthehiddenlayersindependenceof
onthenumdifferenbetrtofimespikesscales.arrivingClearlyin,ithetciannputbelayseener.thatAlltheerrorsegorrorsdoinwnthetolozerowerlevlayel,ersbutof
thehierarchydecreaseearlierthanerrorsinthehigherlayers.Thisbehavioristobe
laexpyeriectedscorrect,fromctheansthetructureoutputoftheerrornetofwtheork:m-tonlyhlaywhenerthedecreaisenputto0.fromthem−1-th

3.3.6Stepstowardbiologicalplausibility

Inthischapter,analgorithmwaspresentedwhichiscapabletoperforminformation
processingonthebasisofsinglespikes.Itisstillunclear,howthisalgorithmcanbe
implementedbybiologicallyplausiblemechanisms.Inthefollowing,wewillseethat
spike-by-spikeinformationprocessingcanbeapartofaneuronalnetworksconsisting
ofleakyintegrate-and-fireneurons(seeFig.3.14).Inthistypeofnetwork,thespike-
by-spikealgorithmcanbeinterpretedasonecompartmentofamulti-compartment
neuronmodel.Itwillbeshownthatthiscombinationallowsaself-consistentwayof
informationtransmission.Theanalogh-valuesareconvertedintospikes.Thisallows
toconstructmulti-layernetworkswheretheinformationisonlytransportedbysingle
spikesfromoneneurontoanother.ThesimulationsdepictedinFig.3.15andFig.
3.16showthatthisispossible.Bothfiguresweregeneratedbysimulatinganetwork

3.3esultsR

77

builtwithleakyintegrated-and-fireneurons,describedbytheequation:
τmV˙i(t)=−Vi(t)+RIIi(t)+Rηηi(3.24)
ηemulatesnoiseonthemembranepotentialandisdrawnfromanormaldistribution
withmeanandvariance1.ThetworesistanceswereselectedwithRη=0.04and
RI=0.06.Thetimeconstantoftherelaxationofthemembranepotentialwassetto
τm=18ms.AspikewasbegeneratedifVi(t)waslargerthanthethresholdϑ=1.0.
Afteremittingaspike,themembranepotentialwasresettoVReset=0.
Thespike-by-spikealgorithmisusedtomodeltheinputcurrentIi(t).Ii(t)iscomposed
ofspikesreleasedatthetimesTS.Thesespikesarerepresentedbyexcitatorypost
synapticpotentials(EPSPs),thataremodelledbyasimpleexponentialdecaywith
aretimew-ceighonsttedantbτyEPStheP=h(i)2ms-v.Balueseforefromtsumminghespikupe-bythe-sspikingleedEPSPsynamicstosIuci(ht)t,thatheIi(tEPSPs)is
ybeddescrib

Ii(t)=exp−t−TsΘ(t−Ts)hTs(i).(3.25)
τPEPSsTheinputspikesforgeneratingIi(t)weredrawnfromaPoissondistribution,usingthe
inputvaluesfromtheactualtask.
Afteraspikeisreleasedfromoneoftheintegrate-and-fireneurons,itismultipliedby
weightvaluesandthenaccumulatedinthelayerofoutputneurons.Theoutputneurons
withthehighestaccumulatedvalueisusedasresultoftheinformationprocessing
process.Fig.3.15shows(withXOR=0.5fortheh-dynamics)thatitispossible
tobuildfunctioningXOR-gateswithsuchanetworkconfiguration.ForFig.3.16,
thesameframework(withUSPS=0.1)wasusedtoclassifyhandwrittendigitsfrom
theUSPSdatabase.ThecurveinFig.3.16showsthemeanperformanceofthis
classificationindependencyofprocessingtimeanditalsoshowsthatthistypeof
informationprocessingschemeworksstillwell.Bothsimulationswereperformedwith
0.1msforeachsimulationstep.
Anotheraspectofthespike-by-spikealgorithm,whichmakesproblemswiththebio-
logicalplausibility,isthedenominatorintheequationfortheh-dynamics.Itisan
unsolvedquestionhowthenecessaryinformationabouttheh-valuesisexchangedfor
theupdateprocedure.ThismayberegulatedbyextracellularGABA(γ-aminobutyric
acid).Anotherapproachistousepopulationsofneuronsforrepresentingtheproduct
hi·p(st|i)intermsoftheirspikingactivity.Forthissetup,theh-valuedynamicsis
ybtedlculaca

tht+1(i)=ht(i)(1−)+Ξp(s|it).(3.26)
jjhi·p(st|i)isusedasmeanvalueforaPoissonianprocess,wheretidenotesthenoisy
representationofhi·p(st|i).UsingaPoissonianrandomprocessmadeitnecessary

78

Chapter3:InformationProcessingSpikebySpike

toreplacetheh(i)=1conditionbyh(i)=Ξ.Thisnewparameterallowsto
controlthemeaninumberofspikesrepresenitinghi·p(st|i)andthustocontrolthe
precisionandnoiselevelofthenoisynominator.Fig.3.17showstheperformance
ofclassifyinghandwrittendigitsfromtheUSPSdatabaseindependencyoftheΞ-
Pvaolue.issonianΞcannbeeurons.intWitherpretedasincreasingparameterΞ,tfheormaximadescribinglpothessiblesizepeorfofrthemapnceopulationincreases.of
Feacuhrthermore,transmission,Fig.t3he.17psehowsrformancethatawithpproaconlyahesftewhespikerroresforrepresenthentingon-ntheoisynh-ominatordynamics.for

3.3esultsR

5045403530error[%]25201510501

2

5

10# of spikes

25

50

100

79

Figure3.7:ErrorrateeofSbSon-linealgorithm(solidline)andtheNN-classifier
(dottedline)fromFig.3.6incomparisonwiththeperformanceofthreetypesof
estimators.Thebluecurverepresentsanestimatorwhichdrawsrandomlyanoutput
valueaslongasallinputneuronsfiredatleastonce.Theredlinewasgeneratedby
ansub-optimalestimatorwhichcomplementsmissinginputchannelsrandomlyaslong
asallinputneuronsfiredatleastonce.TheusedrealisationoftheNN-classifierfinds
thenearestmemorizedsample.Ifmorethanonesamplewithsimilardistanceare
found,thenoneofthesesamplesischosenrandomly.Thisstrategyissimilartothe
redestimator.Thelast(green)curveistheerrorproducedbyanoptimalestimator.
Thisestimatoralsocomplementsmissinginputchannelsaslongasallinputneurons
firedatleastonce.Butitusesthemostlikelyinputbitconfigurationoftheactual
Booleanfunctionsetforthemissingbits.SimilarcanbeexpectedfromtheSbSon-line
algorithm.

80

Chapter3:InformationProcessingSpikebySpike

Figure3.8:Conditionalprobabilities(weightvectors)p∗(s|i)fortheSbSbatchlearning
algorithmusingH=500hiddennodes.Vectorsiforevenandoddinputnodessare
combinedandindividuallyre-scaledtoagreyvaluegi(s)∈[−1,1],andthendisplayed
ina16×16pixel∗raster.The∗gi(s)werecomputedasgi(s)=g˜i(s)/max{|g˜i(s)|}
withg˜i(s)=p(2s−1|i)−p(2s|i).Parametersforbatch-learningwerechosenas
describedinthesectionB.5.

esultsR3.3

81

numFigurebers3o.9:fnMeuronseancinlassithehficationiddenelarroryer,einfordtepheendenceUSPSodnatathebnaseumsbhoerwnofforinputdifferenspikest
(parametersasinFig.3.8).ThedashedlineshowstheerrorofaNN-classifierin
comparison.TheSbSalgorithmwith500hiddenneurons(solidline)isconsiderably
slotionwiser,pebutrformedexceedswtithheapnearfordaptivmaenceeoftstimationheNN-cconstanlassifier.t∝Ift−1/3learning(dottedandcline),lassifica-the
peNNrformance-classifier.withTheownlyeigh100tsinhiddenthisnexampleeuronswereapproactrainedhesspwitheedtheandpbatchlerformanceearningofrule.the

82

Chapter3:InformationProcessingSpikebySpike

Figure3.10:MeanclassificationerrorefortheUSPSdatabaseindependenceofthe
numberofhiddennodesafter10spikesperinputnode.Theweightsareattainedvia
thebatchlearningruleandtheotherparametersareaschosenforFigure3.8.

3.3esultsR

83

Figure3.11:ErrorrateoftheSbSalgorithm(a)fordigitpatternspartiallyoccluded
bygreybars,and(b)fordigitpatternssubjectedtoavaryingamountηofpixelnoise
(weightsandparametersasinFig.3.8).In(a),thegreybarsindicatetheerrorrate
underahorizontalocclusionofpixelrows(fordetailsseeB.5),whileblackbarsindicate
theerrorunderaverticalocclusionofpixelcolumnsindependenceonthenumberof
occludedrowsorcolumns.Foranumberoflessthan4occludedpixelrowsorcolumns,
andforanoiselevellessthanη=0.25,therecognitionerrorremainsbelow10percent.

Figure3.12:Structureofahierarchicalnetworkconstructedforsolvingtheparity
functionwith16inputbits.Thenetworkconsistsof15XOR-moduleslinkedtogether.
Greydiscsdenotethehiddenlayernodesofthesinglemodules,whileopencircles
denotetheinput/outputnodes.Connectionsbetweenthenodesaremarkedwithsolid
lines(allconnectionssettoaweightof0.5).Thearrowindicatestheoutputnodes
whichcomputetheparitysub-functionforthefirst8inputbits(fromleft).

84

Chapter3:InformationProcessingSpikebySpike

inFiguredepe3.13:ndenceontClassificationhemeanenrrorumobfertheofspikhierarcespehicalrinputSbS-netnowdeork(soliddisplaline).yediInnFaig.hierar-3.12
chtheicalonetutputwofork,thetheloawerccuracylayers.ofTthehisodeputputeofndenceahisigherdlayemonstratederreliesboynttheheerroraccuracycurveofs
forthefirsthiddenlayer(dashedline),thesecondhiddenlayer(dashed-dottedline),
anderrortihenthethirdn-hthliddenayerlayfallserbe(dottedlow10line).percentCorrespafterabondinglyout,1t−he2curvspikesespshoerwinputthatnothede
laterthaninthen−1thlayer.

esultsR3.3

85

Figure3.14:Anoverviewhowthespike-by-spikealgorithmiscombinedwithinanet-
workofleakyintegrate-and-fireneurons.Theinputisgeneratedlikeintheother
examplesforthespike-by-spikealgorithm.Thetypicalspike-by-spikelayer(denoted
asspike-by-spike’compartments’)usestheincomingspikestoupdateitsh-valuesvia
thespike-by-spikeh-updaterule.Inanewnextstep,thespike-by-spikecompartments
converttheincomingspikeintoanexcitatorypost-synapticpotential.ThisEPSP
decaysexponentiallyovertimeanditsheightitproportionaltohTs(i)forthespike-by-
spikeneuroni,whereTsdenotesthespikeattimeTs.ThisEPSPistheinputfori-th
leakyintegrate-and-fireneuron(withadditivenoiseηi).Ifthemembranepotential
oftheintegrate-and-fireneuronssurpassesthethreshold,adeltaimpulseissendto
theoutputneurons.Thedeltaimpulseismultipliedwiththerelativeweightsandis
accumulatedbytheoutputneurons.Theindexoftheoutputneuronwiththehighest
valueisselectedasresultofthecomputation.

86

Chapter3:InformationProcessingSpikebySpike

Figure3.15:ImplementationoftheXOR-gate(parityfunctionwith2inputbits)
throughacombinationofspike-by-spikeupdatealgorithmandleakyintegrate-and-
fireneurons.Oneofthefourpossibleinputstrings(leftmostcolumn)splittedinto
on/offcenterchannelsareusedforgeneratinginputspike-trainsdrawnfromPoisson
distributions.Theinputspikes(secondcolumn)wereconvolutedwithrealisticpost-
synapticpotentialsandthenweightedwiththeh-valuesfromthespike-by-spikealgo-
rithmbeforetheywereusedasinputcurrents(seeEq.(3.25))fortheleakyandnoisy
integrate-and-fireneurons(seeEq.(3.24)).WhenoneofthemembranepotentialsVi(t)
hitthethresholdϑ,themembranepotentialwasresettoVReset,aspikeweighted,and
senttotheoutputlayer.Asampletimecourseofthemembranepotentialisshownin
thethirdcolumn.Thelastcolumnshowstheaccumulatedoutput.Whentheredline
liesbelowthebluelinethentheoutputofthenetworkiscorrect.

R3.3esults

100

90

80

70

60error [%]5040

30

20

10

00

1000

30002000time [ms]

4000

87

5000

Figure3.16:Implementationoftherecognitionofhandwrittendigitsthroughacom-
binationofthespike-by-spikeupdatealgorithmwithleakyintegrate-and-fireneurons.
ThesetupissimilartoFig.3.15,usingtheweightsandinputstructureintroduced
pinoFig.ssible3.8.withinThisthiscurvnoisyeshoandwsthatbiologicallythecmorelassificationplausibleofthemodel.handwrittenThecurvdeigitsshoiwssathelso
erroraveragedover25differentinitialconditionsforthe2007handwrittendigitsfrom
theUSPStestdataset.

88

9080706050error [%]40

30

20

10

Chapter3:InformationProcessingSpikebySpike

10025050010001500Normal

01000100101# of spikesofFigurethen3um.17:beroClasfisinputficationspikesp(x-erformanceaxis)andofthereliabilitUySPSofhtheandwrittennominatord(igitscolorindcoepde).endencyThe
ttwithnominatormeanvwaasluehirepresen·p(stted|it)(seehroughiEq.(3.26)).i,wiΞth=i100dra(redwnfline)romraPesultsoissonin≈d0.03istributionmean
spikesforeachnormalisationstep.Withthislownumberofspikes,theperformanceof
thealgorithmisclosetochancelevel.UsingΞ=250(greenline)increasesthemean
numberofspikesto≈0.35andimprovestherecognitionperformancesignificantly.
WithΞ=500(blueline),ameannumberof≈2.5spikesisusedfortheinformation
casewtransmission.iththeTun-pheeperturbedrformancewinformationiththisΞvpropagation.alueapproacAfhesurtherthepincreaseetrformanceoΞ=oft1000he
coun(magentt≈aline,12.8)mreseanultssopiknlyeincounats≈mall5.5spikes)enhancemenandtΞin=p1500erformance.(cyanline,Them’eansnormal’pike
algorithm,withtheun-perturbednominator,isrepresentedbytheblackdashedline.

R3.3esults

3.3.7Artificialandnaturalimages

89

Seenasaputativemodelforinformationprocessinginthevisualcortex,itisinter-
estingtolookatthedynamicsofspike-by-spikenetworksdrivenbynaturalimages.
Beforeusingnaturalimages,inanprecedingstepaspike-by-spikenetworkwasfed
withartificialpicturescomposedofsuperpositionsofhorizontal,vertical,anddiagonal
rectangles.Theideabehindthisanalysisistoshowthatresultsandweightsfroma
spike-by-spikenetworkmakesense.Thisisimportanttoknowbeforeusingnatural
imagesasinputs.Fornaturalimageswedonotknowwhatstatisticalstructuresthe
picturesarecontaining.Thusitisnoteasytojudgeifthealgorithmdeliversusfor
naturalimagesmeaningfulinformation.Forshowingthatthenetworksisabletoex-
tractsuchmeaningfulinformation,thespike-by-spikealgorithmisappliedtoartificial
imageswereweknowallaspectsofthestructureoftheinputandalsoknowwhatthe
outputofthenetworkshouldbe.Fig.3.18showsthedetailsandresultsofthissimu-
lation.Thespike-by-spikebatchlearningrulewasabletoextractthebasisfunctions
ofwhichthetrainingdatasetwascomposed.Usingtheemergingweightset,itwas
possibletoselectoneofthoseartificialimagesasinputforthenetworkandtousethe
resultingh-valuedistributionforreconstructingtheinputimagesnearlyperfectly.It
shouldbenotedthatitwasnotnecessarytouseon/offchanneldecompositionlikeit
wasappliedfortheUSPShandwrittendigits.
Similartotheprocedurewiththeartificialimages,5000imagesfromtheCorelnatural
imagedatabaseweretaken.Fromeachoftheseimagesa12pixelx12pixelpatch
wascutoutandusedastrainingdata.Weightsfordifferent-valueswerelearned
bythespike-by-spikebatchlearningrule.Fig.3.19showsfoursetsoftheseweights.
Dependingonthe-value,thestructuralcomplexityoftheweightsdiffers.Forlarger
-valuesspatiallystructuredweightsweregenerated.Smaller-valuesleadtoweight
setsresultingapixel-basedrepresentation.
Usingtheseweightsetsforreconstructingimagesasamosaiqueofsmallerpatches
leadstotheproblem,thatthemethodremovestheabsoluterangeofeachtile.This
isduetothefactthattheinputpatternisconvertedintoaprobabilitydistribution
whichsumsuptothevalueof1.Ascountermeasure,oneextrachannelisaddedto
thesetofinputchannels.Forallpatches,theextrapixelvaluewassetto256,before
calculatingtheprobabilitydistributionfromthepixels∈[0,···,256].Usingthisextra
informationallowstoreconstructtheabsoluteluminancerangeforeachpatchfromthe
distributionofh-values.ForFig.3.20andFig.3.22thisideawasappliedtoanimage
fromtheCorelpicturedatabase(becauseofthelargernumberofinputchannels,the
weightsetshadtobelearnedagain).Severalreconstructions(patch-based)areshown
foragray-colourversionofthispictureinFig.3.20andfora24-bitcolourversionin
Fig.3.22.TheusedweightsetsaredisplayedinFig.3.21andFig.3.23.
Fig.3.24showsthequalityofthereconstructionsindependencyofthenumberof
spikes,usingtheweightsfromFig.3.22with432hiddenneurons.Aftereachofthe
inputneuronsfiredafewtimes,thesubjectoftheimagecanalreadyberecognised.

90

Chapter3:InformationProcessingSpikebySpike

Figure3.18:Reconstructionofartificialimagesviaspike-by-spikealgorithm.Asetof
3600imagescomposedofsuperpositionsofvertical,horizontal,anddiagonalbarswere
usedastrainingdataforaspike-by-spikenetworkwith24hiddenneuronsusingthe
batchlearningalgorithm.Anexamplefromtheseinputpatternsisshowninthetop
panel.Thealgorithmwascapable,afterseveralrepetitionsofdrawingabout30.000
spikesfromeachinputpatch,toextractthegeneratingweights(basisfunctions,center
panel).Usingtheseweights,aninputpatch(bottompanel,left)wasfedintothespike-
by-spikenetwork.Afteriterating4000inputspikeswiththespike-by-spikedynamic,
itwaspossibletoreconstructtheinputpatchalmostperfectlyfromtheh-valuesvia
ip(s|i)hi(bottompanel,right).An-valueof0.1wasusedforthissimulation.

esultsR3.3

91

Figure3.19:Foursetsofweightsobtainedbytrainingaspike-by-spikenetworkon
naturalimageswiththespike-by-spikebatchalgorithm.Eachsetofweightsconsists
of144vectorswith144entriescorrespondingto12x12pixelvalues,shownfor=0.5
(upperleft),=0.1(upperright),=0.01(lowerleft),and=0.005(lowerright).
Forthetrainingprocedure5000imagesfromtheCorelimagedatabasewereused.From
eachoftheseimagesone12x12patch(extractedwithoffset100x100fromtheupper
leftcorneroftheimage)wastaken.Thethreecolourchannelsofeachofthesepixels
wereaveragedforgeneratingagrayvaluebetween0and255.Foreachtrainingstep,
20.000spikesweredrawnfromeachpattern.Thetrainingprocedurewasrepeated300
timesincludingall5000imagetiles.Thefourexamplesshowthatwithdecreasing
theweightsemergeintoapixelbasis.Forlargervaluesmorespatiallystructured
weightsevolve.

92

Chapter3:InformationProcessingSpikebySpike

Figure3.20:Reconstructionofnaturalimages(consitingofgrayvalues)withspike-by-
spikenetworks(=0.005).Theoriginalinputpicture(titled’real’)wascutintoaset
of12x12pixelarrays.Weights(showninFig.3.21)for144hiddenneurons(second
row),72hiddenneurons(thirdrow),and12hiddenneurons(lastrow)similartothe
proceduredescribedinFig.3.19weretrained(with20.000spikesperarrayandtrial).
Sincethisproceduredestroystherelativecontrastbetweeneacharray,anextrainput
channelforeacharraywasintroducedandusedforcodingapixelvalueof256.Forthe
reconstruction,20.000inputspikesweredrawnfromeach12x12tile.Thegenerated
hi-valueswereusedforinferringtheinputtileviaip(s|i)hi.Theextrapixelwas
usedforcorrectingtherelativecontrastbetweenthetiles.Reducingthenumberof
hiddenneuronsresultsinareductionofreconstructionquality.Thepictureshown,
wasalsotakenfromtheCorelnaturalpicturedatabase.

esultsR3.3

Figure3.21:Weights(=0.005)forthereconstructions
hiddenneurons(fistrow),72hiddenneurons(secondrow),
.)wro

wohsn12nda

ig.Fnihidden

93

n3.20,euronsfor(last144

94

Chapter3:InformationProcessingSpike

eSpikyb

Figure3.22:Reconstructionofnaturalimagesspike-by-spike(=0.005).Thisfigure
issimilartoFig.3.20withthedifferenceofusingcolourimageswiththreecolour
channelsinsteadofonlyonechannelwithgrayvalues.Theimagewasreconstructed
using432hiddenneurons(secondrow),72hiddenneurons(thirdrow),and12hidden
neurons(lastrow).ThecorrespondingweightsareshowninFig.3.23.

3.3

esultsR

Figure3.23:Weights(=0.005)forthereconstructions,shown
hiddenneurons(fistrow),72hiddenneurons(secondrow),and12
.)wro

95

inhiddenFig.n3.22,euronsfor(last432

96

Chapter3:InformationProcessingSpikebySpike

Figure3.24:Reconstructionofnaturalimagesspike-by-spikeindependencyofthe
numberofspikeevents.ThisfigurewasgeneratedlikedescribedinthecaptionofFig.
3.23.22.2secoThendnetroww.orkwSnaascpshoomptsoofsedtheofr432ecohiddennstructionnaeuronsfterusingreceivingthew1,eigh5,ts10,sho25w,n5i0n,aFnig.d
100meanspikesperinputneuronandarray.Withincreasingnumbersofspikes,the
divqualityergence)oftheoftherreconstructioneconstructioniincreases.ndepFig.endency3.25oftshohewstusedhenuqualitmbery(ofKullbacinputk-spikLes.eibler

3.3

esultsR

KullbackLeibler divergence

1

0.1

0.011

Figure

3.25:

10010# of mean spikes / input neuron and patch

-LeiblerkKullbac

ergenceivd

fo

het

tructionrecons

wnhos

in

Fig.

97

1000

.24.3

98

Chapter3:InformationProcessingSpikebySpike

3.4SummaryandDiscussion

Agenerativeframeworkforneuralcomputationthatreliesonspikesignalsgivenby
Poissonianpointprocesseswaspresented.Inthisframeworkprobabilitiesforeliciting
spikescaptureneuralactivitiesandsynapticweightsarecorrespondinglyrelatedto
conditionalprobabilitiestoobservespikesgivenaputativeelementarycause.While
thisschemeisequivalenttonon-negativematrixfactorization(NNMF)whenusedfor
uptheadateforsymptoticsinglecaseactionwherepomtenteanialsratesleadstorepreseanttfundamenhetsignals,allythedifferenspikte-dby-synamicspikeoofn-linehid-
denhiddensltates.ayerInrepresenparticular,tationsforionvecontrastr-completetotherepresenbasictationsalgorithmstheforalgorithmNNMFfavdorserivedsparsein
(LeeWhileaadndhocSeung,spars1999),enesswchileonditionsthedforegreeoNNMFfshparsaveenesbseencandisbcusesedtunedin(bHoyyaer,2004),parameter.in
thespike-by-spikeframeworkitemergesfromtherequirementofrapidconvergence.A
differentideatolinksparsenesstofastvisualprocessingwasputforwardbyPerrinetet
al.pro(jPeectionrrinetofettheal.,best2004).matcHhingere,thefeatureideafroismtothesinputubtractwsceneith(eacmatchsphingikeanopursuit).rthogonalThis
methodhasbeenshowntoneedveryfewspikestoachieveagoodreconstructionof
cenes.snatural

Usingthenewalgorithmitwasdemonstratedonsimpleparadigmaticexamplesthat
verysmallnetworkscanrapidlyandaccuratelyperformcomplexcomputationsusing
onlyfewstochasticspikesperinputneuronforachievingmaximalperformance.

Asaproofofconcept,learningandcomputationofrandomBooleanfunctionswas
inoneimplemencthanneled.B(oobit)leanmayfunctionsswitcharefromasparticularlypecificcfunctionhallengingofbtheecauseraemainingdifferenbitsttoinputan
arbitrarydifferentfunction.Inaneuralenvironment,thisdynamicalpropertyallowsto
chThisangeconatcextcanomputationbeinpterpretederformedein.g.aasacertainttentbrainionalapreariming,depaendingdditionalonaspsensoryecificconcues,text.or
Ninformationprovidedbyotherbrainareas.ForBooleanfunctionswithNinputbits,
2thishiddenanalysisnoitdwesasarefioundngthateneraloptimalrequiredperforforamancecompleteisinmostrepresencasestation.possibleHowwitevher,ilessn
Ntothanfind,theseand2torhiddenepresennotdres.Thisedundanciesreductioninanisedueffectivtoethemcanner.apabilityTheofetrrorhevalgorithmanished
completelyindramaticcontrasttothealgorithmforNNMFbyLeeandSeung(Lee
spikandesSeung,effectiv1999),elyiwhicmplemenhftailedsaforMonthiste-tasCarlok.Istispamplingossibleoverthatthethepstoosteriorchasticitwhicyhofhelpsthe
btoyHofindyearabndetterHyv¨sarolutioninen(asHowyeithrandNNMF.Hyv¨arineThisn,isan2003).interpretationbeingputforward

TheabilityofthisspikebyspikenetworktolearnandcomputeBooleanfunctions
showsthathighlynonlinear,non-smoothfunctionscanefficientlybecomputedonthe
basisofveryfewspikeswhichunderlinesuniversalityofthisframework.

3.4SummaryandDiscussion

99

ApplicationsofthemodelwithH≈500hiddennodestohandwrittendigitsdemon-
stratethatonaveragelessthanonespikeleadstorecognitionrates(5.8%errorrate
)whichexceededthoseofthenearestneighborclassifierthatusedthefulltraining
setwithmorethan7000patterns(6.2%errorrate).Thishighspeedofrecognition
appearstobeageneralpropertyofourframework,anddependsonlylittleontheinput
stimulusensemble.FortheoriginalUSPSdataset,usingaSupportVectorMachine
resultsin4.0%errorrate(Sch¨olkopfetal.,1995)andhumanshavea2.5%errorrate
(BromleyandS¨ackinger,1991).
Alsothespikebyspikemodelwasstudiedonthereconstructionofnaturalimages.
Trainingthenetworkviathebatchlearningruleleadstoweightvectorsp(s|i)whose
appearancesstronglydependonthevalueof.Forverysmall,weightvectors
approximateorthogonalpixelbasisfunctions,specializinginexplainingthespikesof
fewneighboringinputchannels.Consequently,therepresentationofonenaturalimage
isalinear,non-sparsesuperpositionofthesebasisfunctions.Incontrast,valuesof
nearoneleadtoweightvectorsapproximatingtemplatesofimagepatches.The
representationthengetsmaximallysparsebyactivatingonlyonehiddennodewhose
weightvectormatchesmostcloselythepresentedinputpattern.Inbetweenthese
extremecasestheweightvectorshavestrongsimilaritiestothereceptivefieldsshown
inOlshausenanfField(OlshausenandField,1996).Aninterpretationoftheweight
vectorslearnedwithafixedisoneofarepresentationwhichisoptimizedtoexplain
anyinputpatternwithanaveragenumberofspikesbeingproportionalto1/.
Insummary,theresultschallengethenotion,thatthehighspeedofvisualperception
foundinhumanandmonkeypsychophysics(Thorpeetal.,1996;MandonandKreiter,
2005)canonlybeexplainedbyassumingthattheexacttimingofindividualspikes
conveystheinformationaboutthestimulus(vanRullenandThorpe,2001).The
recognitionofhandwrittendigitswasalsousedtoshowthattheframeworkisvery
robusttonoiseanddatacorruption.
Perconstruction,thespike-by-spikealgorithmisinvarianttoscalingsofoverallinten-
sities.Thispropertyhasbeenfoundinprimaryvisualcortexwherethetuningof
neuronsisinvarianttostimuluscontrast(Skottunetal.,1987),whichgivesahintto
thepotentialrelevanceofthisapproachforexplainingcorticalcomputation.Atfirst
sight,theequationsforupdateandon-linelearningEqs.(3.20)and(3.21)mightappear
biologicallyunrealistic.However,itmaybeexpectedthatthealgorithmscanbeimple-
mentedusingknownbiophysicalpropertiesofrealneuralcircuits.Thisimplementation
isnotcompletelydoneyet,butwewillgiveanoutlineoftheunderlyingideasinthe
followingparagraph(someoftheseideas,movingthespike-by-spikealgorithmastep
towardsabiologicallyplausibleimplementation,areshownintheResultssectionof
apter).hcthisFirst,letusassumethatthehiddenvariableht(i)isrepresentedintheexcitabilityof
aneuron,or,evenmorerealistic,intheexcitabilityofaneuronalgrouporpopulation
(WilsonandCowan,1972)whichwouldbeequaltotheaverageoverallmembrane
potentials.Second,weobservethattheupdateterminEq.(3.20)isproportionalto

100

Chapter3:InformationProcessingSpikebySpike

ttheffica(i)pcy(sp|(is)t.|i)O,wnehichcouldisminterpretultipliedtwhisithttermheasaexcitabilitspikyedht(elivi).eredInfovact,erasucshmynapseultiplica-with
ettiveal.,in2002).teractionsSpikhaesveinbtheeenpobservopulationsedinrrealepresenneuronstingasthee.g.ht(iin)’sCwillhancebeetal.elicited(Chancewitha
tandtprobabilitoaveyragepropothosertionalspiktoes,ahnd(i).feedOnethembacinhibitoryktoptheoopulationtherpowouldpulationssufficeasatordivisiveceivee
normalization,whichisaneuraloperationthathasbeenextensivelystudiedinearly
visualtransmissioncorticalisanotreasinstan(Heeger,taneous1992;butCathesrandiniimulationsandHshoeeger,wedt1994).hatisTnothisapinformationroblem.

Takentogether,thisoutlinedemonstratesthatabiophysicallyplausibleimplementa-
tionleakyofintthesegrate-pike-baynd--sfirepikeneuronsalgorithmwhereshtouldhebepneuronsossiable.reWecoupledusedbnyetawnorkscimplemenompotsedationof
ofthespike-by-spikeupdatedynamicsforinformationprocessing.Thesuccessfulim-
plementationofBooleanfunctionsandtherecognitionofhandwrittendigitsthrough
suchnetworks,showedthatthisapproachisselfconsistentandthatitcanbeusedfor
constructingcomplexnetworksbystackingmanylayersofneurons,whilecommuni-
cationalgorithmisstillcanbedoneibymplemensingletedspikases.onepFuartrthofaermoremuitsuggestslti-compartmenthattntheeuronspikme-obdyel.-sApikn-e
otherhandycapforimplementingthespike-by-spikealgorithminabiologicalplausible
fashionisthedenominatoroftheupdateterminEq.(3.20).Weshowedthatrecog-
nitionofhandwrittendigitsbyaspike-by-spikenetworkisstillpossibleevenwhen
thevalueofthedenominatorisestimatedfromPoissonianspiketrainssendfromthe
otherneurons.Thissuggeststhatabiophysicallyplausibleimplementationshouldbe
possibletoformulate.

Adifferentapproachtospike-by-spikeestimationwasstudiedbyDen`eve(Deneve,
2005),whoderivedadifferentialequationforthedynamicsofaBayesianneuronup-
thedatingreceptivaneestimatefield.ofThisthelstudyog-liktakelihoesaod-ratiodditionallyforthethetemporalpresence/absencedynamicsofoafsatimstimulusulusin
incausetoaaccounndnott,fhoorwever,mixturesinitsofccurrenauses.tDstateen`evoenlyalsofordtheemonstratedpresence/absencethattheofcorrespasingleond-
ingdifferentialequationis,assumingvariousapproximations,similartothoseofan
integrate-and-fireneuron.

Fordemonstratingthebiologicalplausibilityofon-linelearning,Eq.(3.21)canbein-
terpretedsuchthattheupdateoftheweightsiscomposedbytwoterms:onethatis
Hebbian,andanotheronewhichdescribesweightdecayproportionaltopostsynaptic
activity.Thepurposeofthesecondtermistoputlimitsonthegrowthoftheweights
andthustoprevent’weightexplosions’.
Δp(s|i)∝h(i)p(s|i)(δs,st−p(st|i)).(3.27)
Inpreviousapproaches,simplecellresponseswereexplainedfromecologicalrequire-
ments(Barlow,1960)likestatisticalindependenceofhiddennodeactivations(Bell
andSejnowski,1997).Fornaturalimagedataitwasarguedthatthisrequirementis

3.4SummaryandDiscussion

101

equivalenttosparsenessconstraintsontheactivities(OlshausenandField,1996).A
differentjustificationforsparsenessmaybefoundintheprincipleofminimalenergy
theconsinputumptiondata(LevareymoanddelledBaxbyter,alinear1996).Isupnefrpramewositionorkofsblikasisetvheectors,presenhotedwevoer,ne,impwhereos-
adinghoc.sparsenessIncontrastconditions,thiswonorkthepcooinetsfficientowtasrdsrighatnewfromethebxplanationeginningofspapparseearsnessrasathera
consequenceofconvergencetimeconstraints.InEq.(3.20)residualnoiseofthehidden
variableestimationscausedbynon-vanishingvaluesoftheupdatespeedparameter
becoinducesmesmossparsenesstclearofwhenhiddenconodnsideringeactivtheations.updateThealgorithmrelationofwithspeedtheeandxtremesvparsenessalue
of=1.Inthiscase,asymptoticallyonlythehiddenneuronobtainingthelargest
inputwillremainactive.Here,sparsenessbecomesmaximalandweeffectivelyhavea
thehwinner-takiddeners-it-allepresentnetwationork.isdWithominated<1byspatrhesenessnecessitisyenfotorced,explainhowtheever,dsata.parsenessof
Thesedependenciesmayalsobeinterpretedinadifferentfashion:theparameter
inantroinputducesspikateimechangesscalewthehicihnteernalffectivrelyepresendeterminestation.howSmallstronglyincreatheseothebserva’memotionry’of
toforanpreviousinputspikpatternes,thatustthehecinostternalofarepresenlargertationobservacantionbeatime.djustedItismoborevaiousccuratelythata
moreaccuraterepresentationneedsingeneralalsomorebasicfeaturestoexplainthe
inputsufficestodata.Inexplainconantrast,inputwithapattern.largeThisasparsrepresenereprestationenthasationaofhighfewvariabilitfeaturesyduealreadyto
theimagsemallconumpressiombernofusedobservincoedspikmputeres(Thisscience,tradeoffwherethashenaumlotbiernocfbaommonsisfwunctioithlnsossyis
restricted.).Takingalltheseobservationstogether,thisframeworkmightprovideanovelbasis
forunderstandingneuralcomputation:Comingfromarathertechnicalbackground
wheresimilaralgorithmswereintroducedtoachieveblindsourceseparationandblind
shode-conwnvtohatlutionthis(oftenapproacrheferredisqtouiteastunivhe’coersalcaktailndpcanartybeuproblem’),sedtotheefficienetlyxamplespherformave
generalcomputations.Inparticular,itcanperformmulti-modalneuralintegration
ofsensoryinputwithspikesequencesfromdifferentsensorymodalitiesorotherbrain
includeregions(statDenevisticaeletexpal.,ectat2001).ions,pAlsoarticularitistsasks,traighatndforwattenardttoion.extendtheapproachto

•Statisticalexpectations(priors)canbeincludedbyasetofextrainputneurons.
Thedistributionoverthespikingpropabilitiesoftheseneuronsrepresentsthe
prior.

•Sevlectioneralotfdasksifferencantatlsoasksbercandepresenonebtedydbyiffaerensettoafectivitxtraydinputistributionsneurons.overThethesese-
neurons.TheseissimilartoBooleanfunctionswereonebitcanchangetherest
unction.ftheof

102

iontttenA

the

prior

anc

or

eb

the

ducedtroin

selection

of

a

Chapter

thetoin

task.

nformationI:3

spike-by-spike

ProcessingSpikebySpike

wetnrko

elik

edescribd

for

4Chapter

SelectiveVisualAttentionin
V4/V1

tionaMotiv4.1

’Attention’isimportantforthevisualsystem(seesection4.2.3).Ifattentionisdirected
toanobjectembeddedinacomplexsensoryscenery,attentionisknowntoenhancethe
resprepresenonses,tationloweroftthehresholdsattendedandobbjetterect.Someediscriminabilitxamplesyfoforimproattendedvedinaspcectsaomparisonrefasterto
lyingnon-attendedneuronalobjrepresenects.Thesetationandpenhancemenrocessingtsofthroughinformationattentionsrelateduggesttoathatttendedtheobjunder-ects
areimproved.Inthischapterwewanttolearnmoreabouthowattentionmodifiesthe
improneuronalvementrepresenisbased.tationofvisuallyperceivedshapesandonwhichmechanismsthis

Forexample,selectivevisualattentionhasbeenfoundtoinducemodulationsofneu-
ronalfiringrates(seesection4.2.3)(McAdamsandMaunsell,1999b;MoranandDes-
imone,1985;Motter,1993;Reynoldsetal.,1999;Reynoldsetal.,2000;Treueand
Maunsell,1996).Furthermore,neuronsengagedinprocessingofanattendedobject
tendtoorganizetheirresponseintosynchronousfiringpatternswithoscillationfre-
quenciesinthegamma-band(Tayloretal.,2005;Steinmetzetal.,2000;Friesetal.,
2001).

Severalmechanismshowmodulationsoffiringratescaninfluenceandmightimprove
representationshavebeenproposed.Oneapproachforexplainingperceptualimprove-
mentsisbasedonanincreaseofmeanfiringratesbyattentionlikeitwasobserved
inseveralstudies(seesection4.2.3).Theseratemodulationsareaccompaniedby
improvedsignal-to-noiseratios,thusprovidingabasisforenhancingdiscriminabili-
tiesofdifferentstimuli(McAdamsandMaunsell,1999a).However,thecorrespond-
ingimprovementsofrepresentationsappeartobelimitedsinceforstochasticspike

103

104

Chapter4:SelectiveVisualAttentioninV4/V1

trainssignal-to-noiseratiosincreaseroughlywiththesquare-rootofthefiringrateand
attention-dependentincrementsoffiringrateareoftensmallorevenmissing(Reynolds
2004).helazzi,Cand

Inaddition,whenmultiple,spatiallynearbystimuliarepositionedwithinthesame
receptivefield(RF),itwasfoundthatneuronstendtoreactasifonlytheattended
stimuluswasshown(MoranandDesimone,1985;TreueandMaunsell,1996).While
thishelpstodisambiguatetherepresentationwithrespecttosuchstimuli,itdoesnot
necessarilyimplyanimprovedrepresentationofasingleattendedstimulusascompared
toasinglenon-attendedstimuluswithoutother,competingstimuliwithintheRF.

Inthethisratherchlapter,argepitewillrceptualbeinveffectseostigatedfattentwhetherion(Roacketdditionalal.,m1992;ecWhanismsolfeamandyeBennett,xplain
1997).Thefollowinganalysisisintendedtoimprovetheunderstandinghowselective
Theattenationnalysiscihangessbasedthedonfieldiscriminabilitpotentyialofsnignalseuronalwhicahctivitwyerepratternsecordedinthefromvisualanepiduralcortex.
electrotendeddetooarraneyofitwmplanostedpatiallyinwemacaquell-separatemondkeshapys.esDuringplacedtheinethexprigherimenttand,aleftnimalsvisualat-
hemifields.

Thegoalofthisanalysiswastoinfertheperceivedstimuli(shapes)fromthemeasured
neuronaldataandtousetheperformanceofthisinferenceastoolforlearningmore
abouttheneuronalrepresentationsofstimuli.

Therecordedelectrophysiologicalsignalswerepre-processed(e.g.current-source-density
method)anddecomposedintotheirfrequencycomponentsbywaveletanalysis.For
evaluatingthedegreeofdiscriminability,supportvectormachines(SVMs,seesection
2.2.4)asstateoftheartclassifier(Sch¨olkopfetal.,2000;Sch¨olkopfandSmola,2001)
wereused.TheSVMswereappliedtoidentifythedifferentclassesoftheshownshapes
basedonthemeasuredneuronalactivitypatterns.

ationotivM4.1

105

Figureinformation4.1:Oovveerrviewtheirofothepticalvisualnervepathfibwearsys.toBtheothlrateraletinascollectgeniculateandnutranscleusmit(LGvisN).ual
FromtheLGNistheinformationtransmittedtoV1.V1distributestheinformation
tootherareasofthebrain.ForamoredetaileddescriptionofV1andV4(notshown)
seesection4.2.2and4.2.2.V3AandMT/V5areprocessingmotioninformation.LOis
involvedinprocessinglarge-scaleobjectsandV8islinkedtocolorvision.(Thefigure
wasadaptedfrom(Logothetis,1999),c1999TereseWinslow)

106Chapter4:SelectiveVisualAttentioninV4/V1
4.2Thevisualsystem

Fevormolutionanyosfpbecies,iologicalprosyscessingtemsvisualbroughtupinformationmanydisifferencrucialttyfporesoftheirvisdualailysystsurvivemsa(l.LandThe
anddiscussFernald,some1992).featuresTofhisareaovV1erviewandwillV4foofcustheontvisualhecortexprimateinvismualoresdysetail.temandLaterwillin
section4wewilldescribeondatarecordedintheseareasandanalysetheinfluenceof
shoattenwtn.ionWeonwilltheirstartthedynamics.folloInwingFig.survey4.1awithnovetherrviewetina,ofctheonhtinueumanwithvtisualhesLGystemN,andis
finallydescribepropertiesofthevisualcortex.

4.2.1Retina

Theretina(Hubel,1989;Goebeletal.,2003)canbecharacterisedasthefirstpart
ofconvtheertsbrainimpingingthatcomeslightiwnavcesonintacttopwithatternvissuoalfneuronalinformationactivitiesfromathendenvtransmitsironmentt.heseIt
activitiesovertheopticalnervetofurtherprocessingstages.Inthestructureofthe
retinawefindseverallayersofdifferenttypesofcells.
Thefirstlayerofthehumanretinaconsistsoutof’retinalpigmentepithelium’cells
asbarriertothetissuewiththebloodvessels,followedbyalayerofphotoreceptors.
ThTheusdtheensitylighofthastreceptorsopassisallnotlacyersonstanbetforeoverthethephotonswholercanetina.reacAhthehigherdensitphotoreceptors.ycan
befoundPhotoreceptorsinthecancenbtraledpividedartofinttohetworetinadifferenintpcomparisonopulations,withrotdshepandecripheralones.pTheart.
rodsarespecializedforviewingintwilightandtheywillbedeactivatedbybrighter
illumination.Incontrasttotherods,theconesceasetoworkiftheilluminationis
tooprimatespoor.topTheerceivconesearecolors.spNecialisedormallytothreedifferentyptesspofectralconesbwithandwidths,differenwthicahbsorptionallows
spectracanbefoundintheretina.Theirmaximaofsensitivityarelocatedatblue(≈
420nm),green(≈540nm),andred(≈560nm).Youngpresentedfirstaconceptin
1802(Young,1802)howtocombinethesethreecolorsforexplainingcolorvision.This
basicconceptwasrefinedbyvonHelmholtz(vonHelmholtz,1860).
Inthenextretinallayer,threetypesofnervecellsarepresent:bipolarcells,horizontal
cells,andamacrinecells.Thebipolarcellsgettheirinputfromthephotoreceptorsand
Themanyhoforizonthemtalprocellsjectarepdirectlyositionedtotheinsparallelubsequentottlaheyelaryceronofrtainingeceptorsretinalandproganglionvidecells.long
rangeconnectionsbetweenreceptorsandbipolarcells.Theamacrinecellsprovidea
similarstructureforthebipolarcellsandtheganglioncells.Thelastrelevantretinal
layerforinformationprocessingconsistsofretinalganglioncells.Theaxonsofthese
tyretinalpicallyganglioncomprisescells125formmilliontheroodspticalandnervcones,e.Anbutonlyastonishingonemfactillionisretinalthateachganglioneye

4.2Thevisualsystem

107

procells.cessedThisbeproforevokitesistheroutedqouestionverhtheowothepticalpnerverceiveetdoitheLGnformationN.isselectedandpre-

On-CenterOff-Center
Figure4.2:Illustrationofcenter-surroundon-centerandoff-centerreceptivefields.The
regionsmarkedwithaplus-signmodifytheneuronalactivitythroughanexcitatory
influencewhenilluminated.Withoutillumination,theseregionshaveaninhibitory
influenceontheneuronalactivity.Theregionsmarkedwithaminussignreactsimilar
butwithoppositesign.

In1953Kuffler(Kuffler,1953)publishedastudyaboutdischargepatternofganglion
cellsintheretina.On-andoff-centerreceptivefields,withastructureasshownin
Fig.4.2,werefound.Thesecenter-surroundon-centerreceptivefieldsreactmost
receptivstronglyetofieldsstimrevuliealoflightheirthighestsurroundedrespbyonsedartoakness,darkandareathescenurroundedter-bsurroundylight.off-Hcenomo-ter
geneousbrightnessoverthewholevisualfieldwillnormallynotcauseastrongneuronal
response.Theganglioncellscanvaryintheirtemporalresponsebehaviour,sizeofre-
ceptivefieldsandsensitivitytocontrast,color,andmovement.Inaddition,theretina
showsextremelycomplexpatternsofneuronalactivityformorenaturalandespecially
transientvisualstimuli.Thequestion,whatneuronalresponsecanbeexpectedfora
etgiveal.,n(2005;complex)Festimrnandezulusetisal.,still2000;topicofBerryexpeetrimenal.,tal1997;andtGrescheoreticalhneretal.,research2002).(HosoTyhea
researchregardingthevisualsysteminspiredthefieldofimageprocessingandcom-
.npressio

4.2.2Pathwaystoandthroughthevisualcortex

Theaxonsofretinalganglioncellsprojectacrosstheinnersurfaceoftheretinatothe
opticaldiskwheretheopticalnerveisformed(Goebeletal.,2003).Bothopticnerves
fromtheretinaspasstheopticchiasm(Jeffery,2001)wherethenasalfibersofoneeye
arefusedwiththefibersfromthecontralateraleye.Thefibersarebundledintotwo
opticaltracts,ofwhich≈90%continuetothelateralgeniculatenucleus(LGN).

108

Lateralgeniculatenucleus

Chapter4:SelectiveVisualAttentioninV4/V1

TheLGN(MalpeliandBaker,1975;ConnollyandvanEssen,1984;Goodaleand
Milner,1992;Xuetal.,2001)consistsofsixlayersofneuronsseparatedbylayersof
verysmallnervecells(’koniocellular’layers(Casagrande,1999)).Thecellsinthefirst
andsecondlayerarerelativelylarge(’magnocellular’)whilecellsintheotherfourlayers
aresmaller(’parvocellular’).Thelayersone,four,andsixreceivetheirinputfromthe
contralateraleyewhilethelayerstwo,three,andfivegettheirinputfromtheipsilateral
eye.Thecellsofeachlayerformaretinotopicrepresentation,whichmeansthattwo
neighboringpointsontheretinaarerepresentedbytwoneighboringcellpopulations.
Evidencewasfoundthatthemagnocellular,parvocellularandkoniocellularneurons
arepartsofseparatevisualpathways(alreadystartingattheretinaganglioncells),
seee.g.(LivingstoneandHubel,1987;LivingstoneandHubel,1988;Xuetal.,2001).
Themagnocellular(M)pathwayisrelativefastandprocessesnotmanyaspectsfrom
theperceivedvisualinput(e.g.showsnowavelengthselectivity).Itseemsthatitis
devotedtostereovisionandmotionprocessing.Theparvocellular(P)pathway,which
isslowerthantheMpathwayduetomorethinaxons,isaccreditedtotransmitdetailed
informationwithhighspatialresolutionanditseemstoprocessshapeinformationand
color.Thelatterincooperationwiththekoniocellular(K)pathway.TheP-pathway
reactsonlyweaklytomotion.TheLGNprojectstotheprimaryvisualcortex(V1)and
theprimaryvisualcortexsendsconnectionsbacktotheLGN.Theconnectionsfrom
theLGNtoV1preservetheretinotopicrepresentation,buttheP-pathwayinnervates
theprimaryvisualcortexatadifferentlayerofV1thantheM-pathway.Thefunctional
roleoftheLGNisnotfullyunderstood.

V1Area

Figure4.3:Illustrationofthereceptivefields(RF)ofsimplecells.(Figureadapted
from(Hoyer,2002))

canAreabeV1understo(Goeboeldetasal.,gatewa2003),ytoalsohigherknovwnisualasaBroreasdasmannwellasareaan17imp(Broortantdmann,processing1909),
sCtaagellawofay,visu1998),alitistinformation.ypicallyInndecomposedeuro-anatomicallyinto9sub-ladesycers:riptionsI,II,II(GoI,eIbVeAl,eItaVB,l.,IV2003;Cα

4.2Thevisualsystem

109

Figure4.4:ExamplesofmeasuredreceptivefieldsinV1(Figureadaptedfrom(Ringach,
2002),usedwithpermissionfromTheAmericanPhysiologicalSociety)

(receivinginputfromtheMpathway),IVCβ(receivinginputfromthePpathway),
V,andVI.80%ofthecells(’excitatorypyramidalcells’)inV1providefeedforward
andfeedbackconnectionstoothercorticalareas.Theaxonsofthenon-pyramidalcells
(morethan40differenttypescanbedistinguished)donotleaveV1.Thecomplete
functionalityofV1isunknown.

Regardingthefunctionalarchitectureandneuronalresponsepropertiesof(monkey)
visualcortex,importantresearchwasdonebyHubelandWiesel(HubelandWiesel,
1968;HubelandWiesel,1977).IncontrasttoretinalganglioncellsandLGN,the
preferredstimuliofV1areorientedlinesegments.Aconcentrictuningtosimplelight
dotswasmainlyfoundinthesub-layerIVC,whichgetsdirectinputfromtheLGN.

Orientation-selectivecellsinV1aretypicallyclassifiedas’simplecells’or’complex
cells’.Thesecellsaretunedtotheorientationsofelongatedpatchesorlines,like
depictedinFig.2.2.Simplecellsfavorstationarylightbarsofcertainorientations.In
Fig.4.3schematicsforsimplecellsareshown.Thereceptivefieldsofthesecellsare
dividedintoexcitatoryandinhibitoryzones,similartothecenter-surroundreceptive
fieldorganisationoftheLGNorretinalganglioncells,butdifferentinshape.The
receptivefieldsofcomplexcellsarelargerthanthoseofsimplecellsandcanbeinvariant
totheexactpositionofthebarwithinthereceptivefield,aslongastheorientation
ismatchingthepreferredorientation.Insomecasesthestimulushastobespatially
non-stationaryforgeneratingstrongresponses.Furthermore,cells(e.g.hypercomplex)
exist,whichareonlyselectivetolineswithacertainlength.

110

Chapter4:SelectiveVisualAttentioninV4/V1

Figure4.5:Orientationmapfromcatvisualcortex.Thecolorsencodetheorientation
ofstimulithatgeneratethemaximalresponse.(Picturetakenfrom(Crairetal.,1997),
withpermissionfromElsevier)

inTher’corticaleceptivcefieldsolumns’,ofcellsalignedinthenpormalrimarytothevisualsurfacecortexoafretheorganizedcortex,sucsharehthatnearlyneuronsthe
samereceptivefieldproperties.Inorthogonaldirectiontocorticalcolumns,receptive
fieldfeaturesproplikeertiese.g.choangerientationgradually(seeandFig.f4orm.5,t(opBonhoographicefferandmapsG.rinMvapsald,for1991)),elemenoculartary
thatdominance,contextualdirection,effectsbyspatiasltimfulirequencyshow,naondutsidedisparitofythewe(reclassical)found.receptivEvidenceewasfieldsfoundcan
altertheresponsepropertiesofV1(Zipseretal.,1996).

Otherareasofthevisualcortex

Classicallytwodifferentvisualpathways(Mishkinetal.,1983;Goebeletal.,2003)are
distinguished,theventralandthedorsalstream.
Thecessingventralsystem.streamItiissmregardedainlyadrivsaen’wbyhat’theorP’v-Pisathionwfaorypeandprception’erformsianformationfine-grainedpro-
analysisofe.g.colorandshapeofvisualscenesaswellaspatternrecognitionand
formanalysisfromfacesorotherlocalspatialstructures.Anillustrationoftheventral
streamisshowninFig.4.6.Incontrasttotheventralstream,thedorsalstreamis
describedasinvolvedin’where’or’visionforaction’informationprocessingtasks.It
ismainlydrivenbytheM-pathwayandprocessesspatialinformationofvisualscenes,
visuallyguidedactionsandvisuomotortransformations.
Theareasalongbothpathwaysareorganizedhierarchicallyandlow-levelrepresen-
tationsarefollowedbymorecomplexrepresentations.ForexampleV2sharesmany

4.2Thevisualsystem

111

Figure4.6:Simplifiedschematicsofconnectionsintheventralloop.(Figureadapted
(Pfrom1999))ollen,

receptivefieldpropertieswithV1,butinadditiontheneuronsinV2canbesensitive
tobinoculardisparity(depthinformation)orcanhavelargerreceptivefields.V3is
highlysensitivetocontrastandselectivefororientations,whileV3Aismotion-sensitive
andV3Breactstomotionboundariesandkineticcontours.MT/V5istunedtodi-
rectionsandvelocitiesofmovingstimuli.Itwasshownthatifthisareaisdestructed,
deficienciesinmotiondirectiondiscriminationwilloccur(Goebeletal.,2003).

etTheal.,idea2005;ofsFerreeparateravandenMtralaunsandell,doral2005;pathZewkia,ysis1993)).aAllsimplificationofthose(vseeisuale.g.are(Tasoliasare
Thedirectlyforunctionalindirectlyroleof(ovtheerfothereedbackareas)linkconnectionsedbyisstillfeedforwunderarddorebatefeedbac(Pokllen,1999).connections.

112

V4Area

Chapter4:SelectiveVisualAttentioninV4/V1

ThequestionwhatinformationtheneuronsfromareaV4arerepresentingbytheir
activitypatternsisstillunderresearch.InthefollowingIwilldiscusssomeofthese
findingsfromexperimentsaboutthesensitivityofV4neuronstostimulusproperties.

FormanyV4neuronsatuningofneuronalresponsestorbarstimuliwasmeasured.
Selectivitiesfordifferentlengthandwidthaswellasdifferentorientationsandpolar-
itiesofcontrasthavebeenconfirmed.ManyreceptivefieldsofV4neuronsshowed
similaritiestocomplexreceptivefieldsinV1(DesimoneandSchein,1987).Theyshow
asensitivity(ofsomedegree)tocolourbutalsorespond,withaloweractivity,towhite
light(ScheinandDesimone,1990).Differentpopulationsofneuronsarealsoselective
tosinusoidalgratingsandchangesinthespatialfrequency,orientation,phase,andsize
ofthegratings(Gallantetal.,1996;DesimoneandSchein,1987).Furthermore,itwas
demonstratedthatneuronsinV4aremorestronglyactivatedbynon-sinusodialtypes
of’gratings’,likee.g.concentricandradialpatterns(Wilkinsonetal.,2000;Gallant
etal.,1996).ManyV4cellsdisplayasensitivitytotextureinformationofstimuli,
whichsuggeststhatthisvisualareamaybeinvolvedintheextractionoftexturein-
formation(HanazawaandKomatsu,2001).Ifstimuliaretoolarge(withrespectto
thereceptivefieldsize)thensomeofthecellsreactwithstrongsuppression.Optical
imagingdetectedafunctionalorganisationlinkedtostimulussize(GhoseandTs’O,
1997).Typesofneuronswerefound,whichweresensitivefordirectionofmotionand
kineticpatterns(Mysoreetal.,2006;DesimoneandSchein,1987).Inadditionare
V4neuronsalsoablecodetheabsolutedifferencesofobjects(Dobbinsetal.,1998),
3Dorientationsofslantedlines(HinkleandConnor,2002)anddisparity(Hegdeand
vanEssen,2005;HinkleandConnor,2001).Ithasbeenfoundthatinsidetherecep-
tivefield,theselectivityforbinoculardisparityisinvariantforthepositionofstimuli.
Neuronswithsimilardisparityselectivityareclustered(Watanabeetal.,2002b).

Espstimuli.eciallyIthasrelevbaneentfordiscothisvewredorktishattheV4wacellsyV4respneuronsondscotrongerdetoinformationcomplexabshapoutessthaphane
tocellssiinmpleV4bararesotimftenulituned(KobataktoceonandtourTanakfeaturesa,(1994).likeeF.g.aurthermore,nglesanditwcurvasreves),ewaledithtthathe
strongestvariationoftheneuronalresponsebeingcorrelatedtothecontourfeatures
oriendirectiontationofatndheconanglevexitofy.aneOrientdge-elemenationotfa(PaconstupathourymaeansndCinothisnnor,cont1999).ext,Le.g.atertheit
wasfoundthattuningfunctionsparametrisedbycurvatureandangularpositionfitted
orneuronalaxisorienrestponsation.eswForithmmanoreyacells,ccuracywhichthanresptuningondedtotfunctionshesetyppesoarametrisfstimeudli,byitewdgeas
patospssibleecifictochangulararacterizepositionsthes(seetrongestFig.4resp.7).onseOtherbyfeaturesfeaturesooffthethesshtimapehulus’adbonlyoundariesminor
ofV4influencescanbonemtheodneledeuronalbytrwespomonse.ultipliedInGaddition,aussianitseemsfunctions,thattonehesepkindarametrisedofbselectivitiesythe
securvgmenaturet(PofastheupathshyapaesndCegmenotnnor,andt2001)he(sothereeoFig.ne4by.8).theFuangularrthermore,posfitionorsofthisilhouette-likshapee

4.2Thevisualsystem

113

Figure4.7:Curvatureandangularpositionareimportantfeaturesfordescribingthe
responseofaV4neurontoashape.Insidetheblackcircle,anexampleshapeisshown.
Aroundtheblackcircle,theboundaryoftheshapeisshownincurvatureandangular
positionspaceaswhiteline.(Picturewastakenfrom(PasupathyandConnor,2002),
withpermissionfromMacmillanPublishersLtd/NaturePublishingGroup)

stimuliitwasshownthattheboundaryfeaturescanbepartiallyreconstructedfrom
theneuronalresponseofV4populations(PasupathyandConnor,2002).
TheneuronalresponsesofV4neuronsaremodulatedbyvisualattention(seesection
4.2.3fordetailsaboutvisualattention).Itwasdemonstratedthatattentioncanmod-
ifythesensitivityofneuronstocontrast/salienceinsuchawaythatthemagnitudes
ofneuronalresponsestolow-contraststimuliareincreased(ReynoldsandDesimone,
2003).Spatialinteractionsbetweentheattentionalfocusandvisualstimulishowcom-
plexmodulationsoftheneuronalresponses.Onecomponentofthismodulationwas
identifiedasresponsegradientaroundtheattendedtarget.Thegradientdecreaseswith
increasingthedistancebetweentheattentionalfocusandastimuli.(Connoretal.,
1997).

4.2.3Visualattention

Focusingourmindontheobservationofoneobjectinacrowdedvisualscenecanmake
usnearlyblindtochangesthatdonotconcernthatobject.Thisstrikingphenomenon
istermed’changeblindness’(O’Reganetal.,1999)andisverylikelyresultingfrom
’visualselectiveattention’(Itti,2002).Visualattentionprobablyallowsthevisual
systemtoblankoutlargeamountsofincomingsensoryinput.Itwasalsoshown
thatvisualselectiveattentionisprerequisitedforshapeperception(Rocketal.,1992;
RockandGutman,1981).Forreviewsregardingvisualattentionsee(Itti,2002;Treue,

114

Chapter4:SelectiveVisualAttentioninV4/V1

Figure4.8:ShapetuningfunctionofanexampleneuroninV4forthefeaturescurvature
andangularposition,modeledbyGaussianshapedtuningfunctionsforeachstimulus
dimension.Thecolorscalerepresentsthenormalizedpredictedresponse.(Picturewas
takenfrom(PasupathyandConnor,2002),withpermissionfromMacmillanPublishers
Ltd/NaturePublishingGroup)

2001;DesimoneandDuncan,1995;ReynoldsandChelazzi,2004;vanRullenandKoch,
2005).Twodifferenttypesofattentionarediscussed:bottom-up(’stimulus-driven’)
andtop-down(’goal-directed’,’task-dependent’)attention(Connor,2004;Yantis,1998;
EgethandYantis,1997).
Bottom-upattentionseemstobealargelyunconsciousprocessandisconsideredto
bedrivenbyspecificvisualfeaturesoftheperceivedscene(’image-based’).Promi-
nentexamplesforbottom-upattentionaretypesofvisualsearchexperiments(Wolfe,
1998;TreismanandGelade,1980)whereatargethastobelocalizedthatishidden
amongotherdistractingelements.Dependingonthefeaturesofthetargetandthe
distractors,eitherthetarget’pops-out’fromthebackgroundofdistractorsoritcan
becomenecessarytocheckallelements,onebyone,tofindthetarget.Thepop-out
effectisbroughtintoassociationwithafastparallelsearchduringwhichourattention
isdracompariswnononttoostehearchtarget.-tasksInwhterehepcaseop-outofserialeffectsoscanning,ccur,thewhicfohciussnoformallyattentionslowlieseroinn
oneorasmallnumberofelementsfromthewholevisualscene.Theshiftofattention
fromoneelementtoanotheristhoughttobedirectedbyhighercognitivelevels.Asa
consequence,suchshiftsofattentionarecalledtop-downattention.Pop-outandserial
searcharethetwoextremecasesofanoftenmorecomplicatebehaviour(Itti,2002).
Oneapproachtounderstandbottom-upattentionisthe’featureintegrationtheory’
(Ttion,reisdmanirectionandofmGelade,ovemen1980)t,wandhderesisparitimpley)arefeaturesdetected(likeepre-.g.cattenolor,tivinelytiensniaty,omassivrieneta-ly
parallelwayovertheentirevisualfieldandrepresentedbytopographicalmaps(’early
representation’(KochandUllman,1984)).Thisprocessissupposedtotakeplacein
theearlyvisualprocessingareas(e.g.primaryvisualcortex).Inanextstep,these
featurescanthenbelinkedintomorecomplexobjectrepresentations.Beforefurther
processingbegins,mechanismsusingattentionareappliedtofilteroutmostofthe

4.2Thevisualsystem

115

non-attendedrepresentationsthroughtheso-calledattentionalbottleneck,(Itti,2002).
Severalcomputationalmodelsforexplainingbottom-upattentionhavebeenproposed
(see(IttiandKoch,2001)forareview).Manyofthesemodelshaveincommon
thattheyuse’saliencemaps’.Thesesaliencemapsarescalartopographicalmaps
thatcombineearlyrepresentationsfordifferentfeaturessuchthatthesaliencemap
representstheoveralldifferenceinfeaturespaceatonelocationincomparisonwith
itssurroundinglocations.Howthesesaliencemapsarecomputeddifferfrommodel
tomodel.Typicallysaliencemapsareusedtofindthemaximumofallsalienciesand
thentodirectattentiontothatposition.

Figure4.9:Illustrationoftwopossibilitiestoofmodifytuningfunctionsthroughat-
tention.Theredlinerepresentsatuningfunctionwithoutattention.Thebluelines
showsamultiplicativegainthroughattention,asitwasfoundby(McAdamsand
Maunsell,1999a)fororientationtuning,andthegreenlineillustratesanincreaseof
firingratesbyashiftofthetuningfunction,asdescribedin(Reynoldsetal.,2000)for
contrast-sensitiveneurons.

Top-downattentionseemstoactdifferentlythanbottom-upattention(foradetailed
reviewsee(Treue,2001).Thefollowingpartofthissectionisbasedonthisarticle.).
Itcanbedirectedbycuesfromhighercognitivelevels(likeverbalcues)(Itti,2002).
ExperimentsrevealedthatalreadyvisualinformationprocessinginV1isinfluencedby
sucheffects,(Motter,1993;ItoandGilbert,1999;Gilbertetal.,2000).Furthermore,
modulationsthroughattentionwerefoundintheventralanddorsalpathway(Treue
andTrujillo,1999;McAdamsandMaunsell,2000).Themodulationstartsrightat
thebeginningofthepathwaysandtheeffectsareincreasingalongthepathwaysin
directiontohigherhierarchicallevels(Tootelletal.,1998;Treue,2001).Forthedorsal
pathway,experimentsfoundthattheattentionfocus,whendirectedintothereceptive
fieldofneurons,canaltertheresponsepropertiesofneurons.Fortheventralpathway,
experimentsfoundonlyindicationsforasimilarbehaviourofcells.Thefindingsfor
theventralpathwayneedaverification(Treue,2001).

116

Chapter4:SelectiveVisualAttentioninV4/V1

Experimentssuggestanattentionalsystemwhichinteractswithreceptivefieldsthat
haveanoverlapwiththeso-called’spotlightofattention’.Inexperimentswithtwo
stimuliinsideofonereceptivefield(oneattendedandonenon-attended)(Reynolds
etal.,1999;MoranandDesimone,1985),itwasfoundthatthespotlightofattention
canactonspatialscalessmallerthanthecorrespondingreceptivefields.Theinfluence
ofattentiononneuronalresponsesseemstobeacombinationofanenhancingeffect
forattendedstimuliandaninhibitoryeffectonthenon-attendedstimuli(’push-pull’)
(Pinsketal.,2004;Treue,2001).Thissuggestsamodificationinthetuningproperties
ofaneuronsbyattention.Thiscanbeinterpretedintwoways:Asattentionalmodi-
ficationofthewholetuningfunction(’gainmodulation’(TreueandTrujillo,1999))or
asspecificimprovementofattendedstimuliwithasimultaneouspenaltyfortheunat-
tendedstimuli(biasedcompetition(LeeandSeung,1999)).Furthermore,top-down
attentioncanactinanon-spatial,feature-basedfashionwhichallowstoimproveex-
pectedandbehaviorallyrelevantvisualfeatures(Chelazzietal.,1993;Chelazzietal.,
1998).Fig.4.9showstwotypesofattentionalmodification(multiplicativegain(McAdams
andMaunsell,1999a)andgainthroughshifting(Reynoldsetal.,2000))totuning
functions,whichwerebothfoundinexperiments.

Thischapterwillcontinuewiththepresentationofresultsfromadataanalysisregard-
ingattentionaleffectsontheneuronalcorrelates,generatedbycomplexshapestimuli
2007a).al.,etund(Roterm

4.3ExperimentalSetting,PreparationsandMeth-
sod

4.3.1Theexperimentalsetting

ThedatafollosetswingaretheanalysisresultisofbasedexpoendrimenatatsmpeasuerformedredfbromytwKatjaoTamacaqueylorimntonkheeys.groupThesofe
AndreasKreiter(InstituteforTheoreticalBrainResearch,UniversityofBremen).The
experimentsweredesignedtoinvestigatetheeffectofselectiveattentiononneural
activitypatternsincortex,whileprocessingvisualinformation.Bothanimals,monkey
FandmonkeyM,weretrainedtoperformasocalled’delayed-match-to-sample’task
(seeFig.4.10).

Formakingtheuseofselectiveattentionforthistasknecessary,twodifferentstimuli
(shapes)arepresentedsimultaneouslyonacomputerscreen.Duringthebeginningof
thetrial(1550msoftheinitialstimuluspresentationperiod)theleftorrightshapeis
cuedbyagreencolouring.Thecoloursignalisestheanimalwhichsideofthescreen
canbeignored.Thegreencolourfadesoutwithin600msafterstimulusonset.Subse-

4.3ExperimentalSetting,PreparationsandMethods

117

Figure4.10:Schematicillustrationoftheshape-trackingtask.Twosequencesofshapes
werepresentedintheleftandrighthemifieldofacomputerscreen.(seeanexample
intheupperleftrectangle).Whilethemonkeyfixatestoadotinthecentreofthe
screen,itsattentionwasdirectedtooneoftheinitialshapespresentedduringperiod
T1–inthisexample,totheshapeontherighthandside.Thiswasdonebyshowing
oneshapeingreencolourduringtheperiod(inT1)whichismarkedwiththeshaded
segmentoftheline.Thetaskfortheanimalwastosignalthere-occurrenceofthe
initialshapeintheattendedhemifieldduringoneofthefollowingperiodsT2-T5.
Here,acorrectresponsewouldbeduringorafterpresentationofshapes(T5).(Drawn
byKatjaTaylor)

quently,thememorizedstimulushastobecomparedtodifferentteststimuliforming
atostimusignalizelusthissequence.eventInbycarseeleasingthatathelevmer.emoDruringisedthestimwulusholereaptrial,peatrhes,themonkaeynimahaslhtaos
isfixatestoppaedwfixationithapnoinet,rrorwahicndhiswillpnotositionedberewbetwarded.eenthetwoshapes,otherwisethetrial
Astimulussequenceiscomposedoftwotofivestimuli(behaviourallyrelevantshape
peandriodd.isOtractornepsehrioapde).ofAfterpresenthetationinitialwassfhapolloe,wtedhebsytimauli900weremsvisdelaibleyforphase,a500wherems
onlythefixationpointremainedonthecomputerscreen.Thedistractorshapeswere
wraerendomlyselectedfselectedromafrosmaubsetsetofofsixtenofdtifferenhesettenshasphaes.pes.Thebehaviourallyrelevantshapes
ThechronicallyimplantedepiduralelectrodearrayscoveredpartsofareaV4andV1
the(seecFiuedg.stim4.11),uluswithwas36sehownlectrotodestheforvisualmonkheyMemifieldandco37veredelectrobytdesheforelectromonkdeeyFarra.y,Whenthe

118

Chapter4:SelectiveVisualAttentioninV4/V1

Figure4.11:Mapoftheroughlocationsofthesubduralelectrodes(opencircles)in
relationtoareasV1andV4ofMonkeyF.Theregionswereestimatedbyretinotopy.
(DrawnbyKatjaTaylor)

wassituationshownistotermedthev’isualattendedhemifieldcondition’without(a-tChe).Theelectrootherdearracase,y,thewhenctheonditioncuedwsillhapbee
referredtoas’non-attendedcondition’(n-C).Ithastobenotedthatalwaysoneofthe
shapeswasinthefocusofselectivevisualattention(a-Candn-Csymbolisesonlythe
factwhethertheelectrodearray’sees’theattendedshape).
Averagebehaviouralperformanceofmonkeysduringrecordingsessionswasestimated
Difromallsregardingbutthefixationlongesetrrors,trialstheinwmonkhicheysapresponserformedewouldcorrecthaveforbeen83,1%always(monkeycorrect.M)
and73,4%(monkeyF)ofthetrials.Correctresponsesoccurred467ms(M)and418ms
(F)aftertargetstimulusonset(medianvalues).Errorsweredistributedroughlysimilar
overdifferentinitialfigures.
Formoredetailsofthebehaviouraltraining,visualstimulations,surgicalpreparations
andrecordingssee(Tayloretal.,2005;Rotermundetal.,2007a).

4.3ExperimentalSetting,PreparationsandMethods

119

Figure4.12:Exampleforatypicaltime-frequencyplotonthesametimeaxisasin
Fig.4.10,displayingthetrial-averaged,normalisedpowerspectraldensityA(t,f0)in
theattendedconditionforanelectrodeoverV1.

Figure4.13:Thisfigureshows,analogtoFig.4.12,thetimecourseofthetrailaveraged,
normalisedpowerspectraldensityA(t,f0)intheattendedconditionforanelectrode
overareaV4.

4.3.2DataPreprocessing

proPresenducestingloacavlfiisualeld-spotimtenulitiainlrtheespvonisualseshfortemifieldheconelectrodestralateralptoositionedtheovimplanerttheedarravisualy
areasV4andV1.Thesefield-potentialswerehigh-passedfilteredwithadigitalfilter
(fordetailssee(Tayloretal.,2005;Rotermundetal.,2007a)).Trialscontaining
1984)artefactswaswcereralculatedejected.forFurthermore,minimizingscurrepatialntsmsourceearing(and(Nunezsink)edtaensitl.,y1997):(CSD)A(ssGuemingvins,
thattheelectricalfieldEisproportionaltoagradientofascalarfunctionΦ
Φ,=−E∇wecanusethetworelevantMaxwellequationsforanelectro-staticalfield
4πρ=E∇•0=E∇×

120

Chapter4:SelectiveVisualAttentioninV4/V1

with∇=(∂x∂,∂y∂,∂z∂).ThisgivesusthePoissonianequation
24πρ=−Φ∇withchargecarriersinthecorrespondingspatialregionandtheLaplacianequation
20=Φ∇forspatialregionswithoutanychargecarriers(Jackson,1993).Thisallowsusto
approximateρ(x,y)fromtherecordeddate.Foreachmillisecondofdatathesecond
spatialderivativeofthefieldpotentialswascomputedwiththeLaplacianoperator
(∇2)(Perrinetal.,1987),usingGaussianradialbasisfunctions(RBF)forinterpolating
thedata,whichwasrecordedatdiscreteelectrodeposition,intoacontinuoussurface
(MoodyandDarken,1989).Attheendofthecalculation,foreachelectrodetheCSD
yieldsthesignalsvj(t)withjdenotingtrialnumber.
Basedonvj(t),thesignalswerewavelet-transformedintotimeandfrequencydepen-
dendamplitudesandphases.ForthiscalculationaconvolutionwithcomplexMorlet
waveletsw(t,f0)(Kronlandt-Martinetetal.,1987)wasperformed.Usingthismethod,
waveletpowercoefficientswereobtainedthroughtheequation
+∞2
−aj(t,f0)=w(τ,f0)vj(t−τ)dτ.(4.1)
∞Forthefollowingdataanalysis,onlytrialswheretheanimalsmadenofixationerrors
wereused.Thespacingoffrequencybandswaslogarithmicbetween5and200Hz,
chosenasf0(k)=Ωk−1f0(1)fork=1,...,17frequencybandsstartingatf0(1)=
4.84Hz.Forasufficientlytightcoverageoffrequencyspace,Ωwassetto1.2593.Ifwe
denotethepowerofasinusoidalwavewithfrequencyf0(k)atitsfrequencyf0(k)with
P(f0(k))thenthepowerofthiswavewilldecreaseto≈0.57P(f0(k))atthenearest
neighbouringfrequenciesf0(k±1).Forthefollowingfrequenciesf0(k±2)thepower
willdropto≈0.059P(f0(k)).Thisensuresthattherelevantpartoffrequencyaxisis
coveredtightly.
InFig.4.12andFig.4.13typicaltimefrequencyplotsareshownforoneelectrode
aboveV4andoneelectrodeaboveV1.Forbothfiguresthenormalisedmeanspectra
A(t,f0)=<aj(t,f0)>j−n(f0)(4.2)
)f(n0werecomputedusingnormalisationcoefficientsquantifyingthebackgroundactivity
n(f0),whichwasobtainedfrom
t21n(f0)=t2−t1t1<aj(t,f0)>jdt
witht1=300msandt2=350ms.Thetime-frequencyplotsshowthattheincrease
ofpowerthroughthestimulivaryoverthefrequencybandsandismostpronounced
between40and100Hz.

4.3ExperimentalSetting,PreparationsandMethods

4.3.3DiscriminatingStimuliwithSVMs

121

Discriminatingdifferentdynamicalstatesofthebrainoritssub-areasusingneural
signalshasbeenperformedinvariousexperimentalsettings,differentspeciesandwith
differenttypesofrecordedneuralsignals,rangingfromsingleunitstudiestoLFP/
EEGrecordings.Seesection2.2.4foraselectionofapplicablemethodsforsuchclassi-
ficationtasks.Oneprominentmethodisthesupportvectormachine(seesection2.2.4,
(Sch¨olkopfetal.,2000;Sch¨olkopfandSmola,2001)).Inthisdataanalysisthewidely
usedlibsvmsoftwarepackage(ChangandLin,2001)wasapplied.Itprovidesconve-
nientdatapre-processingroutinesandautomaticallysearchesthroughtheparameter
spaceforgoodSVMsettings.RadialbasisfunctionswereusedaskernelsfortheSVM
ssifier.claForclassifyingthepresentedshapesonthebaseofthemeasureddata,represented
bythewaveletpowercoefficients,thedatasetsweredividedintotwosubsetsofap-
proximatelyequalsize.Thetrialswerealternatelyassignedtoonetrainingsubsetand
toonetestsubsetinaninterleavedfashion.Theclassesweredefinedbytheshapes
sj(k)presentedtothemonkeysinselectedintervalsk∈{T1,T2,T3,...}ofthestimulus
sequence(seeFig.4.10)displayedonthe’recorded’sideofthevisualfieldduringtrial
j.Onlythesixclassesforthebehaviourallyrelevantshapess∈{1,...,6}wereused
astargetsfortheanalysis.Theremainingtrialswiththeotherfourdistractor-only
shapeswereignored.Inthefollowing,wherenecessary,theindexswillbeusedto
distinguishvariables,whichwerecomputedusingonlywaveletcoefficientsfromtrialsj
inwhichtheshapeshowninintervalk=T1wass,i.e.sj(T1)=s.Likewise,datafrom
trialswhereattentionwasdirectedtothevisualhemifieldrepresentedintherecorded
brainregionwillbedistinguishedwithasuperscriptA,whileusingasuperscriptN
otherwise.Fromallcoefficientsaj(t,f0)obtainedwithinaperiodTforthecentrefrequencyf0,
asubsetofa’sequallyspacedintimewereselectedandusedforcomputingaveraged
coefficientsa¯j(f0).Thespacingwasadjustedtoapproximatelytwicetheperiod1/f0
whichissufficienttocapturethetypicalrateofchangeinwavelet-analysedsignals.
Averagingledtoalargedecreaseincomputationalcomplexityforthetrainingofthe
SVMs.Dataanalyseswiththeoriginal,fullsetofcoefficientswerealsodoneand
yieldedinnosubstantialdifferenceinclassificationperformance,thusonlyaveraged
coefficientswereusedforthepresentedresults.
TheSVMsweretrainedonthetrainingsets,andtheirclassificationperformancewas
evaluatedonthetestsets.TheresultingperformancesPweremeasuredasthetotal
percentageofshapesclassifiedcorrectlybytheSVMinthetestsets.Thechance
levelPchancewascomputedastheratiooftheoccurrencesofthemostfrequently
presentedpatterninthetrainingsettothetotalnumberoftrialsinthisset.An
increase(decrease)inperformancePabovechancelevelwasconsideredtobesignificant
assoonastheprobabilitytoobtainanequalorhigher(equalorlower)performance
bydrawingfromabinomialdistributionaroundPbinom=Pchancewassmallerthan

122Chapter4:SelectiveVisualAttentioninV4/V1
p=0.02,respectively.AdifferenceinperformancePA−PNwasconsideredsignificant
assoonastheprobabilitytoobtainPAandPNfromtwobinomialexperimentswas
smallerthanpforanyputativeunderlyingprobabilityPbinom.

sultseR4.4

TheclassificationperformanceoftheSVMs,basedonsingletrialclassification,were
usedasanestimatefortheamountof(usable)informationaboutthepresentedstimuli
(classesofpresentedshapes)orabouttheattentionalstateofthestimuli(whetherthe
attendedconditionorthenon-attendedconditionwasused)containedintherecorded
data.Forselectingthenecessarydataforeachsetofdataanalyses,therelevanttime
intervalwasadaptedtotheregardingscientificquestion.

4.4.1Discriminatingshapes

Theanalysisstartedwiththeinitialshape.Inthissegmentofeachtrial,theanimalhas
Westartedouranalysiswithclassifyingtomemorizethetargetwhichithastocompare
totheteststimulirecognisedduringtherestofthetrial.Thewaveletcoefficientswere
selectedfromthetimewindow650-2200msaftertrialstart,andaveragedovertime.
Usingthesecoefficientsfromallelectrodesoftheimplantedelectrodearrayallowedto
identify93.1%oftheinitalstimulicorrectlyformonkeyFand84.9%formonkeyMon
thecorrespondingtestdatasets.Theseperformanceswereevaluatedusingthetrials
fromtheattendedcondition.Thechancelevelforclassifingthesixdifferentinitial
shapeclasseswere18.1%formonkeyFand18.5%formonkeyM,respectively.
InFig.4.14andFig.4.15theclassificationperformanceofsignalsfromindividual
electrodesareshown.Thefiguresrevealthattwoclustersofelectrodescontributethe
mainexplanatorypower.OneclusterwaslocatedaboveareaV4andtheotherone
aboveareaV1.UsingthemostdiscriminativesetoffourelectrodesfromtheV4cluster
(markedbyyellowcrosses,formonkeyFinFig.4.14aandformonkeyMinFig.4.15a)
resultedinaclassificationperformanceof64.2%formonkeyFand54.0%formonkey
M.AsimilaranalysiswasdonewiththreeelectrodesfromtheV1cluster(markedby
greencrosses,formonkeyFinFig.4.14aandformonkeyMinFig.4.15a).Thesignals
ofthiselectrodesetfromareaV1providedaclassificationrateof85.6%(monkeyF)
and79.4%(monkeyM),respectively.
TheclassificationperformanceofthesignalsfromsingleelectrodesfromtheV4cluster
reachedupto42.2%(monkeyF)and35.4%(monkeyM).Theelectrodesfromthe
V1clusterwiththemostexplainatorypowerreached67.8%(monkeyF)and49.2%
(monkeyM).Again,thisperformancevalueswereachievedbyusingthetrialsfromthe
ondition.cattended

esultsR4.4

123

Fig.4.16showstheclassificationperformancesfortheindividualelectrodesandtrials
intheattended-conditionfortheendperiod,wheretheanimalhastorecognizethe
reocurrenceofthememorizedshapeandhasthereleasethelever.Fig.4.16ashows
theperformanceofmonkeyFandFig.4.16bofmonkeyM.Ithastobenotedthat
thetimewindowwasalotsmallerthanintheexamplesfortheinitialperiodwitha
1550mstimewindow.Fortheendperiodthetimewindowwasbeginningwithstimulus
onsetofthelastshapeandhadalengthof400ms.TheclusteraboveareaV4revealed
aclassificationperformanceof49.1%formonkeyFand41.7%formonkeyM,while
thechancelevelwas18.0%(monkeyF)and18.6%(monkeyM).Theelectrodecluster
aboveV1allowedtoidentify80.5%(monkeyF)and67.3%(monkeyM)oftheshapes
.correctly

Takentogether,theresultsdemonstratethatfieldpotentialsrecordedatthesurfaceof
theduraarehighlyspecificfortheindividualstimuliprocessedinthecorticalcolumns
des.lectroetheunderneath

124

Chapter4:SelectiveVisualAttentioninV4/V1

fromFigure4t=650.14:ms(a)toClassit=2200ficationms,pseeFig.erformance4.10)PoforftMheonkinitialeyFsinhapethes(aT1)ttended(datacanalyondition.sed
PTheissphownerformanceindepelevelndenceiscontolour-cohepodedsitionaofccordingtheetolectrothedesbarinshothewnarratoyt(hesmallrightcofircles).the
array.Forthegreycolouredsquares,classificationperformancedidnotdiffersignifi-
cantly(p=0.02)fromthechancelevelof18%(indicatedbytheblackhorizontallinein
twtheocrolouregionsbar).correspClassiondingficationtopareasVerformance1andreacV4,rhesesppeakectivvelyalues(seeofFig.67.8%4.11).andG42.4%reyainr-
roofwselectromarkdestheinmVain4sVelected4-electrofordetheshowingfurtherthehighestcomputationalpearformance.nalysisaTherecommarkedbinationswith
yellowcrosses(V1,greencrosses).(b)Differenceinclassificationperformancebetween
attendedandnon-attendedstimuli,samedisplayasin(a).Thegreysquaresindicate
zero,electroordeswwhereheretheeitherpetherformancedifferencesunderinapttenetionrformancewasdnotseviatedignificannottlydsignificanifferentlytffromrom
chancelevel(p=0.02).

4.4

Results

125

Figure4.15:(a)ClassificationperformancePoftheinitialshapes(T1)(dataanalysed
fromt=650mstot=2200ms)forMonkeyMintheattendedcondition.Pisshownin
dependenceonthepositionoftheelectrodesinthearray(smallcircles).(b)Difference
inclassificationperformancebetweenattendedandnon-attendedstimuli,samedisplay
asin(a).(seeFig.4.14foramoredetaileddescription.)

126

Chapter4:SelectiveVisualAttentioninV4/V1

timeFigure4windo.16:wbClassieginningficationwithpstimerformanceulusonsetPofofthethel’end’astsshhapapee(anddataaanalydurationsedoffrom400a
depms)eforndencemonkoneytheF(pa)ositionandmofonktheeyeMlectro(b)desininthethearraattendedy(csmallcondition.ircles).Pisshownin

esultsR4.4

127

4.4.2Improvementofclassificationperformancesthroughat-
iontten

Uptonow,resultswerereportedonlyfortrialsfromtheattendedcondition.Inthe
following,thefocuswilllieonthedifferences,betweenclassificationratesobtained
fromtrialsoftheattendedandthenon-attendedcondition.Duringtheinitialperiod
(650-2200ms)theclassificationratefortheselectedsetoffourelectrodesfromtheV4
clusterwassignificantlyimprovedfortrialsfromtheattendedconditionincomparison
totheothercondition.FormonkeyF,theperformanceincreasedfrom55.5%to64.2%
(p<3E−06,binomialtest)andmonkeyMtheperformanceimprovedfrom41.6%to
54.0%(p<5E−04)throughattention.ExaminingtheelectrodefromtheV4cluster
displayingthemostexplanatorypowerunderattentionrevealsaperformanceincrease
from35.4%to42.2%(p<3E−04,monkeyF)andfrom29.5%to35.4%(p<0.08,
monkeyM).InFig.4.14bandFig.4.15btheabsolutedifferencesinclassification
performanceforallsingleelectrodesareshown.FormonkeyFsignificantdifferences
clusteraroundthehighlydiscriminativeregioninV4.Afewscatteredelectrodesalso
reachsignificantdifferences,butonlyforverylowclassificationratesclosetochance
level.FormonkeyM,thedifferencesinclassificationperformancebetweenthetrials
fromtheattendedandthenon-attendedconditionreachedvaluesupto6.5%andthe
electrodesfromtheV4clustershowedasimilartendencyasobservedinmonkeyF.
DuetothelowertotalnumberoftrialsrecordedfrommonkeyM,noneofthesingle
electrodesshowedasignificantdifference(seeFig.4.15b).Furthermore,inthedata
frombothanimalsnosignificantdifferenceswereobservedforelectrodeslocatedover
V1.

Sinceitwasnecessaryfortheanimalstoattendtoallstimulipresentedduringastim-
ulussequence,anattention-dependentinteractionbetweenclassificationperformance
andtheconditionofattentioncanalsobeexpectedbeyondtheinitialperiod.Asimilar
effectlikeintheinitialperiodwasindeedfoundduringallteststimuluspresentations.
Forthefinaltestperiod(endperiod),whichisassociatedwiththereoccurenceofthe
memorizedshapeandthecorrectbehaviouralresponse,classificationperformance(the
fourelectrodesetsfromtheV4clusters)measuredina400mstimewindow(starting
withstimulusonset)rosethroughattentionfrom40.6%to49.1%(p<2E−04,monkey
F)andfrom23.1to41.7%(p<7E−06,monkeyM).Chancelevelswereformonkey
F18.0%(a-C)/18.3%(n-C)andformonkeyM18.6%(a-C)/19.3%(n-c).

InFig.4.17andFig.4.18,thestabilityofstimulusspecificinformationandits
timeattenctoion-ursedepofeclndedassieficationnhancemenpertoformancveretimeformisonkeyFdemonstrated.(seeFig.4The.17)fiandguresmshoonkweytMhe
(seestimulusFig.4on-.18).orIonpffsetconarticular,tainformucthehmoreattendedicnformationonditionontherestimisuluslittleidenitityndicationthantthathe
pestaticrsistenpetriolydswhilebetwstimeenulithem.arepThisresenshoted.wstThehatcsurvtimesforulus-spmonkecificeyMactivitareymorepatternsnoisyoccurdue
tothefewertrialsperformedduringtheexperiment.

128

Chapter4:SelectiveVisualAttentioninV4/V1

Thisleadstothequestionwhetherthespecificsignalcharacteristics,whichsupport
identificationofpresentedshapebytheresultingrecordedneuralactivities,aresimilar
overthewholetrialorwhethertheychangeovertime.Thisquestionwillbeexam-
inedbytrainingSVMsondatafromonestimuluspresentationperiodandthenuse
theseSVMsontestdatafromanotherperiod.Inthecasethattheclassificationper-
formanceisstableunderthisinterchange,itcanbeassumedthattherelevantsignal
characteristicsremainunchanged.
Usingthisidea,oneSVMwastrainedwiththedatafromthefirst400msafterstimulus
onsetintheinitialperiod.AsecondSVMwastrainedonthedatafromthepre-terminal
period(theperiodprecedingtheendperiod,usingthefirst400msafterstimulusonset
oftheteststimulus).BothSVMswereappliedontestdatasetsfromtheinitialand
thepre-terminalperiod.TheresultsareshowninFig.4.19.Theperformancevalues
showthatsuccessfulclassificationisalsopossiblefortestdatatakenfromtrialperiods
farapartintime.Ingeneral,theperformancefordifferenttrainingandclassification
periodsiscomparablebutsomewhatsmallerascomparedtotheperformanceachieved
withtestandtrainingdatafromthesameperiod.Thisindicatesthatthecharacteristics
ofthesignalssupportingstimulusidentificationarestableovertime.
Anotherquestioniswhetherthisattention-dependentenhancementofdiscriminabil-
ityincorticalstatesisbehaviourallyrelevant.Ifsuchacorrelationexists,itcanbe
expectedthatfailuresofsuchenhancementsmayresultinbehaviouralerrors.Thishy-
pothesiswastestedbyevaluatingtheclassificationrateintrialswhichwereterminated
byabehaviouralerror.
Fortrialsinwhichmonkeysrespondedtoawrongteststimulusorfailedtorespond
totheteststimulusmatchingthesample,classificationperformanceinthestimulus
periodimmediatelyprecedingtheerroneousresponsefellsignificantlyunderthelevel
achievedincorrecttrials.InmonkeyF,classificationperformancefortheelectrode
combinationinV4wasreducedsignificantlyfrom49.1%to36.2%(p<1E−05),which
isevenlessthanthe40.6%observedforcorrecttrialsinwhichnoattentionwaspaid
tothestimulus.SimilarlyinmonkeyM,classificationperformancefellfrom41.7%
to29.0%(p<0.04).Nosignificantdifferenceinclassifyingnon-attendedshapeswas
foundbetweenthetrialswithcorrectandwrongresponses(monkeyF).Performance
fornon-attendedstimuliintrialswithacorrectresponseinmonkeyMwas23.1%and
againtherewasnosignificantdifferencetoperformanceinerrortrials.
Asimilarreductionoftheclassificationperformanceinerrortrialswasfoundalso
forthetemporallymuchearlierinitialperiod.Hereperformancereducedfrom64.2%
to51.9%(p<5E−06,monkeyF)andfrom54.0%to46.1%(p<0.18,monkeyM).
Thesefindingsindicateacloserelationshipbetweenattention-dependentenhancements
ofthediscriminabilityofthecorticalstatesassociatedwithdifferentstimuliandthe
behaviouralperformanceinthedelayed-match-to-sample-task.

esultsR4.4

129

Figure4.17:(a)Timecourseofclassificationperformancefortheselectedsetofelec-
trodesaboveV4(cf.Fig.4.14a,yellowcrosses),shownfortheattended(reddotted
line)andforthenon-attendedcondition(bluedottedline).Dataforthepowerco-
efficientsinafrequencyrangebetween5and200Hzwastakenfromatimeinterval
starting200msbefore,andending200msafterthetimesmarkedwiththeredand
bluecrosses,respectively.Theblackcirclesindicateasignificantdifferencebetween
forthetphecorresperformancesondinginbothcondition.conditionsIn(a),(p=0.the02),SVM’swhileweresolidtrainedlinesdtoepictctlassifyhechtheanceinitiallevel
shapes(T1)presentedtothemonkeysduringtheperiodT1shadedinlightgrey.Time
tismeasuredrelativetotrialonset.In(b)and(c),theSVM’sweretrainedtoclas-
(ssifytimuthelussecodisnd-playlastpaerionddsthearelastagainshaspehaded(targinet)lighdtisplagreyyed).inTitmehetissequence,measurespredrectivelativelye
totheonsetofthesecond-lastshapein(b),andrelativetotheonsetofthetarget
shapein(c).

130

Chapter

:4

eelectivS

isualV

iontttenA

ni

4/V1V

setFigureofe4lectro.18:TdesimeabovcourseeV4of(seeclassiFig.fication4.15,pyellowerformancecrosses),forsMhoonkwneyforMtheusingtheattendedselected(red
dottedline)andforthenon-attendedcondition(bluedottedline).(seeFig.4.17fora
escription.)ddetailed

4.4

esultsR

131

Figure4.19:Similarityofstimulus-specificactivitypatternssupportingclassification
alongtrials.Thetableshowsclassificationperformancefordatafromtheperiodfor
whichtheSVMwastrained,incomparisontoclassificationperformanceondatafrom
adifferentperiod(shadedingrey)obtainedfromtheselectedV4electrodecombination
(a-C,monkeyF).CorrespondingvaluesformonkeyM(a-C)areshowninbrackets.

132

Chapter4:SelectiveVisualAttentioninV4/V1

4.4.3Stimulus-specificsignalsandcoding

Allpresentedresultsarebasedonallofthe17waveletcoefficientsasinputforthe
SVMs.Inthefollowing,itwillbediscussedhowtheexplanatorypowerisdistributed
overthe17waveletcoefficients.Fortacklingthisquestion,wetesthowclassifica-
tionperformancechangeswhenselectingdifferentsubsetsofthesecoefficientsasinput
datafortheSVMs.Fig.4.20aandFig.4.21ashowtheaccuracyofclassificationin
dependencyoftheselectedintervalsofthefrequencyspectrum,usingdatafromthe
selectedelectrodecombinationoverareaV4,duringpresentationoftheinitialstimulus.
Theentriesinbothfiguresrepresenttheclassificationrateachievedwiththeindicated
numberofwaveletcoefficients(seetheverticalaxis)fromacontinuousfrequencyband
interval.Theupperfrequencyofthesefrequencybandsisindicatedonthehorizontal
axis.Thefiguresshowthatmostexplanatorypowerfordiscriminatingtheshapesis
locatedinthefrequencyrangeabove40Hz.FormonkeyF,Fig.4.20ashowsthatmax-
imallyachievedclassificationratesof64.6%canbeapproximatedbyonlysixofthe
17waveletcoefficients.Thecorrespondingsixwaveletcoefficientswereselectedfrom
thefrequencyintervalbetween38Hzand122Hz.Furthermore,aclassificationper-
formanceof59%isstillpossibleusingonlythreecoefficients(withcentrefrequencies
at61Hz,76Hz,and96Hz).Similardistributionsofclassificationratesovercertain
intervalsinthefrequencyspectrum,canbefoundinbothanimalsfortheattended
aswellasforthenon-attendedcondition.Theattention-dependentenhancement(see
Fig.4.20bandFig.4.21b)islargestwithinasimilarfrequencyrange(38Hz-122Hz).

Anotherquestioniswhethermostoftheexplanatorypoweriscontainedinthespec-
traldistributionofthewaveletcoefficientsorinthesignalenergy.Usingthedatafrom
theV4electrodewiththehighestclassificationrate,thesignalenergywasremoved
mofromnkeyeacFh,clatrialbssifyingyodividingnthertheemasiningelectedfeawtavureseletpreducedowerpcoerforefficienmatsnceboyvertheirchasnceum.levelIn
onaverageby3.1%(26.2%,monkeyM).Forcomparison,classificationwasalsoper-
formedonthesignalenergy,whichwascalculatedbysummingtheselectedwavelet
powercoefficientsofeachtrial.Classifyingonthesignalenergyreducedperformance
overchancelevelonaveragebyatleast45.8%(monkeyF)and12.9%(monkeyM),
respenergyectiv.Telyh,ustwhenheselectingdiscernabilittheyomfofisteldpinforotenmatialstivecausedfrequencybyrandifferengetinstimtermsulioisfsigbasednal
onstimulus-dependentdifferencesofspectralactivitypatternsandoverallenergyinthe
gamma-band.InmonkeyF,informationinthespectralpatternsisevenpredominant
overinformationinsignalenergy.

esultsR4.4

133

Figure4.20:(a)ClassificationperformancePforMonkeyF,usingdifferentsubsetsof
thepowercoefficientsfromtheselectedV4electrodecombinationobtainedduringthe
initialperiodT1(650msto2200msaftertrialonset)underattention.Eachsquare
showsincolour-codetheSVMperformanceonacombinationofsuccessivefrequency
bands,whosetotalnumberisindicatedbytheindexontheverticalaxis.Theupper
frequencybandisindicatedbythehorizontalaxis.Forexample,theperformancevalue
markedbythewhitecirclewasobtainedusingdatafromsixfrequencybandsstarting
atfrequency38Hz,andendingwith122Hz(symbolisedbythewhiterectangle).
Theperformanceshownmarkedbythegreycrosswasobtainedusingdatafromonly
3frequencybandsat61Hz,76Hz,and96Hz(symbolisedbythegreyrectangle).
(b)Percentageofincreaseinclassificationperformanceunderattention,forthesame
combinationsoffrequencybandsasin(a).

134

Chapter4:SelectiveVisualAttentioninV4/V1

Figure4.21:(a)ClassificationperformancePforMonkeyMandthecorresponding
attentionalgain,usingdifferentsubsetsofthepowercoefficientsfromtheselectedset
ofV4electrodes.(seeFig.4.20foradetaileddescriptionfortherepresentationused)

135esultsR4.44.4.4Attentioninducedstimulus-specificsignalschanges

Theprecedingdata-analysisrevealedthatattentionimprovesthediscriminabilityof
thestimulisignificantly.Thequestionishowthesignalcharacteristicsoftheneural
responsesbecomemoredistinctundertheinfluenceofselectiveattention.
Theperfectdiscriminationoftwostimuliclassesrequiresthatthecorrespondingsignals
filldifferentregionsindataspace.Errorsmadeduringclassificationcanberelatedto
datasamplesthatfellintoregionsindataspacewhichareoccupiedbymorethanone
stimuliclass.Thenumberoferrorsmadeisrelatedtotherelativeoverlapbetween
theseregions(Fig.4.22a).Forreducingthenumberoferrors,itisnecessarytoremove
theoverlapbetweentheseregions.Onewaytoreducetheoverlapistoshrinkthe
regionsindiameter(Fig.4.22b).Asecondoneistoincreasethedistancebetweenthe
centresoftheregions(Fig.4.22c).
Thementionedshrinkingeffectcanbeproducedbyincreasingthesignal-to-noise-ratio
(SNR).Anincreaseofdistancebetweenthecentersoftheregionscanbeaccomplished
byaclass-specificmultiplicativescalingwithconstantSNR.Ithastobenotedthatthe
combinationofbotheffectsandalsoincludingmorecomplicatedstatisticalchangesin
thedata,couldbeusedtofurtherreducethenumberofclassificationerrors.Forthis
datasetsitturnsoutthatitsufficestoquantifythesetwobasiceffectstoexplainmost
oftheeffectsofattention.
Analysingthedistributionsoftheaveragedwaveletpowercoefficientsa¯j(f0),onlysmall
changesintheSNRη
)f(μ0η(f0)=σ(f0)

andinthemeanvaluesμ
μ(f0)=<a¯j(f0)>
werefound.σisdefinedbythestandarddeviation
σ(f0)=<¯aj(f0)−μ(f0)>.
Fig.4.23(monkeyF)andFig.4.24(monkeyM)showchangesintheSNRsηandthe
scaFig.ling4o.23aftheandmFeaig.n4wa.24aveletpdemonsowertratecoefficienclearlytstμhatthethroughchaangesttenitnSion.NRThearesonlycatter-plotsminor,
attenwhiletionthe(mseeeanFcig.oe4fficien.23btsfandorFtheig.d4ifferen.24b).tshapTheesdareifferenscaledtialmuscalingchisstrongervisibleinthroughthe
differencesbetweenthecurvesattributedtodifferentshapes.Theinsetsquantifythe
gaininclassificationperformanceunderattention,demonstratingthatitisnotthe
smutrengthltipleofscfrequencyalingbforands,aswinglehichisfrequencycausingband,thebutimprothevcementoncertedinpeffecterformance.ofchangesAgain,in
datafrombothmonkeysshowsqualitativelythesamebehaviour.

136

Chapter4:SelectiveVisualAttentioninV4/V1

FortheSNRs,anaveragedchangeof<ηA/ηN−1>=4±7%(monkeyF)and
−1±9%(monkeyM)werecalculated.Forthemeanvaluesanabsoluteaveragechange
of<|μA/μN−1|>=14.9±10.4%(monkeyF)and3.3±4.2%(monkeyM)werefound
(A:attendedcondition,N:non-attendedcondition,averages<...>aretakenoverall
frequencybandsandstimuliclasses).
Asitwasoutlinedbefore,bothchangesmayberesponsiblefortheenhancedperfor-
manceunderattention.Theaveragevaluesalonedonotallowtoclarifytowhich
extentthesmallchangesinthemeanSNRmightexplainthefullattentionalgain.
Furthermore,itisnotclearwhether,bythedifferentialscalingofthemeans,theclass
regionsarereallyshiftedawayfromeachotherthantowardseachother.
ForquantifyingtheinfluenceoftheSNRsandmeanvaluesontheclassificationperfor-
mance,twotestswereperformedonthedatafromthenon-attentioncondition:Inthe
firstone,theSNR’swerechangedsuchthattheymatchtheSNR’softhedatafrom
theattendedcondition,whileholdingthemeanvaluesofthedatasetconstant.This
transformationwasperformedusingtheequation
Na(j,sI),qA(f0)=μsN(f0)+aNj,s(f0)−μsN(f0)ηηsA((ff0)).(4.3)
0sASVMwasthentrainedandtestedonthis’quasi’-attendeddatasettoquantifyhow
theseparationoftheshapeclassesimprovedthroughthistransformation.Inthesecond
test,theSNR’softhedatasetwerekeptconstant.Thistimethedatawastransformed
suchthatthemeanvaluesarethesamelikethemeanvaluesofthedatasetfromthe
ondition.cattendedμsN(f0)
a(j,sII),qA(f0)=μsA(f0)+aNj,s(f0)−μsN(f0)μsA(f0)(4.4)
Again,aSVMwastrainedonthisnew’quasi’-attendedtrainingdatasetandthe
classificationperformancewasevaluatedonthenew’quasi’-attendedtestdataset.
Asimilar,’inverse’testwasperformedontheattendeddatasetusingthetransforma-
tionAa(j,sI),qN(f0)=μsA(f0)+aAj,s(f0)−μsA(f0)ηηsN((ff0))(4.5)
0sforchangingtheSNRswhileretainingthemeanwaveletcoefficients,and
Na(j,sII),qN(f0)=μsN(f0)+aAj,s(f0)−μsA(f0)μμsA((ff0)),(4.6)
0sforchangingthemeanwaveletcoefficientswhileretainingtheSNRs.Thesetwo’quasi’-
non-attendeddatasetswerethenclassificatedwithSVMsfordeterminingthedegrada-
tionofclassificationperformance.Improvementsandlossinperformancewerefinally
comparedtotherealdifferencesinclassificationperformanceontheoriginaldata,and

esultsR4.4

137

expressedinpercentagesoftheseoriginaldifferencesbeingexplainedbythetwoscaling
cedures.pro

Fig.electrode4.25comandbFig.ination4.26insdephoewthendencyoresultsftheusingselectedthesefrequencyscalingiprontervceduresal.forStartingtheV4at
40Hzthehigherfrequenciesshowthestrongesteffectofmodulation.Scalingthe
SNRsexplanatoryproducespowoenrlyofatmheinordataincreasanalyseseinclausingssificathetiosnpcalingerformaproncedureces.InismomucnkheywM,eakether,
inshowFig.ing4the.25bylimitingdownseffectscalingofttheheatrialttentionstatistics.datasFig.etin4.26tothetconfirmswotheresult’quasi’-non-attendeddisplayed
data(monkesets.yM)Inofthesummaryoriginal,scalingincreasetheinSNRperformanceexplainsonlyunder1.6%attent(monkion.eyUnderF)atndhe28.7%other
waspostransformation,sibletoexwhereplainthe108.4%SNRsa(monkrekeeyptF)conasndtant50.2%andthe(monkmeaneyMva).luesThewreesreultscaled,clearlyit
indicatesthatattentionalgaininperformanceisonlytoaminorextentcausedby
cofhangesfrequencyinScNR,ompobutneniststoralenderingargeextenthetneuralexplaianedctivitbyyforshape-differenspecifictshapedifferensmoretialdistinctscaling
fromeachother.Thisfindingrevealsanewmechanismofattentionactingoncoherent
neuronalactivityinareaV4.

138

Chapter4:SelectiveVisualAttentioninV4/V1

Figurespanned4by.22:theExvaamplesriablesfaor(f1)classandiafication(f2)pwhichroblemscouldinatrepresenwott-dimenshewiavonaleletcodataesfficienpacets
fortwodifferentfrequencybands.Theregionsindicatedbytheblueellipsoidsin(a)
symbolizedatafromtwoclasses’AandB.Thesetwoclasseswouldcorrespondto
ensemblesofdatapointsobtainedbytherepeatedpresentationoftwoshapes.Whena
newdatapointintheshadedregionisobserved(greencross),anyclassifiertrainedon
thepreviouslyobserveddataislikelytomakeanerrorbecausethedatamaybelongto
sizeeitherofofthethesthadedworclasses.egionwTheheretotalthesenumbclasseserofoveerrorsrlap.(thb)usIfcorrespattentionondswtoouldtherdecreaseelative
theSNR,asindicatedbytheclassboundariesshrinkingaroundtheircentres(red
ellipsoids),thesameobservationcouldnowunambiguouslybeattributedtoclassA
thusreducingtheclassificationerrors(shadedregion).(c)Ifinsteadattentionshifts
theproblemregionandcenreducetres(thearronwsum),berthisofcherrorsange(scanhadedlikrewiseegion),diseamvenwbiguatehentthehecSNRlassiremainsfication
.tconstan

esultsR4.4

139

Figure4.23:(a)Signal-to-noiseratios(SNR)ofwaveletcoefficientsintheattended
condition,ηsA(f0),plottedagainsttheSNRinthenon-attendedcondition,ηsN(f0),for
monkeyF.TheSNRswerecomputedforeachfrequencybandf0andshapesduring
theinitialpresentationperiodT1(650msto2200ms).Thedatacanbefittedwitha
linearregression(dashedline),resultinginacoefficientof1.017±0.021foritsslope.
(b)ScalingfactorsorratiosμsA/μsN(f0)ofthemeanwaveletpowercoefficientsforthe
attendedandnon-attendedconditions,independenceonthepatternclass(shape)s
andthefrequencybandf0.Thecoefficientswerecalculatedfromthedataobtained
duringtheinitialperiodfromthemainelectrodeinV4,forMonkeyF.Ascalingfactor
of1indicatesnochangeinthemeanpowercoefficients(dashedblackline).Theinset
showstherelativechangeinclassificationperformancethroughattentionforeachof
thesixdifferentshapess.

140

Chapter4:SelectiveVisualAttentioninV4/V1

Figure4.24:(a)Signal-to-noiseratios(SNR)ofwaveletcoefficientsintheattended
condition,ηsA(f0),plottedagainsttheSNRinthenon-attendedcondition,ηsN(f0),for
monkeyM.Ananaloganalysis,likeexplainedinFig.4.23,revealsacoefficienAtforNthe
ofsloptheeofmaeanlineawarvregeletrpessioowenrcowithefficien1.031ts±for0.027the.a(b)ttendedScalingandfactorsnon-orattendedratioscμs/μonditions,s(f0)
independenceonthepatternclass(shape)sandthefrequencybandf0,forMonkey
M(formoredetailsseeFig.4.23).

esultsR4.4

141

Figure4.25:Explanatorypowerofhypotheticalscalingofwaveletcoefficientsunder
attention.TheupperrowquantifiestheeffectofscalingtheSNRsofthenon-attended
dataasdescribedinequationEq.(4.3),shownformonkeyF(a)andformonkeyM(b).
ThelowerrowquantifiestheeffectofscalingthemeanvaluesasdescribedinEq.(4.4),
shownformonkeyF(c)andmonkeyM(d).Theperformanceisplottedfordifferent
frequencyrangesinthesameschematicsasusedinFig.4.20andFig.4.21.Thecolour
codeindicateshowmuchofthepercentageofthegaininperformanceunderattention
isexplainedbytherespectivescalingprocedure.Valuesabove100%andbelow0%
areclampedtothisrangebeforebeingcolour-coded(fortheoriginalvalue,seethe
numbersdisplayedineachcolouredsquare).Frequencyrangesinwhichtherewasno
significantincreaseinperformanceintheoriginaldataaredisplayedingray.

142

Chapter4:SelectiveVisualAttentioninV4/V1

attenFiguretion.4.26:IncontrastExplanatorytopFig.ow4er.25,ofhtheypscalingotheticalwastscalinghisoftimewaveappliedletcoteotfficienhetsattendedunder
dataset,usingscalingcoefficientsasinEq.(4.5)andEq.(4.6).Consequently,theplots
showconditionhowcmanuchboeftheexplaineddecreasebyincpehangesrformanceinthebetwSNReen(upptheneron-roaw)ttendedandbaynddifferenattendedtial
changesinthemeanwaveletcoefficients(lowerrow),respectively.Samepresentation
.25.4Fig.inas

esultsR4.4

4.4.5AttentioneffectsinV1

143

InV1wefoundnosignificanteffectsofenhancingclassificationperformancethrough
attention.Thisrisesthequestionwhywewerenotabletofindsucheffects.Thus,we
madeaROCanalysis(seesection2.2.4)onthewaveletcoefficients,inadditiontothe
scalingprocedures.Specifically,foranytwodifferentshapesandeachspecificfrequency
band,weobtaintwodistributionsofwaveletcoefficients.AstandardROCanalysis
canthenestimatehowwellthesetwodistributionscanbedistinguishedfromeach
other,i.e.withwhichprobabilityonecancorrectlyclassifyanobservationascaused
bytheoneortheothershape,givenasuitablydecisioncriterion.Thisanalysiswas
performedforeachcombinationofthesixshapeclassesandforeachfrequencyband,
intheattendedaswellasinthenon-attendedcondition.Theresultsaresummarized
inFig.4.27(monkeyF)andFig.4.28(monkeyM).ItturnsoutthatinV1many
combinationsoffrequenciesexist,whichallowforanalmostperfectdiscrimination
betweentwoshapesalreadyinthenon-attendedcondition(integraloverROCcurve
between0.9and1),whiletherearenosuchcombinationsoffrequenciesinV4.In
fact,forV4almostallcombinationsoffrequenciesandshapesremainbelow0.8.This
givesamodulationbyattentionthenecessaryroom’forincreasingthediscriminability
betweentheshapes.Thisdependencecanalsobeseeninthemeanattentionalgain
displayedinthesamefigure,whichishighestformediumandlowvaluesoftheROC
integral.Takentogether,thesefiguresdemonstratewhyinV1attentiondoesnothave
asignificanteffectondiscriminationperformance,asrevealedbytheSVMsinFig.
4.14bandFig.4.15b.

144

Chapter4:SelectiveVisualAttentioninV4/V1

Figure4.27:Summaryofanalysisofthereceiver-operator-characteristicsforanytwo
distributionsofwaveletpowercoefficients.TheintegralsovertheROCcurvesquantify
howwellthewaveletcoefficientsforonefrequencybandallowedtodistinguishbetween
twoshapesinthenon-attendedcondition,takingvaluesbetween50%and100%.In
(a)and(b)theseintegralsweresortedintofiveperformanceclassesequallyspaced,the
histogram(b)quantifiesthedistributionoftheoccurencesofthesevaluesformonkey
F.Thecorrespondingmeangaininperformanceunderattentionisquantifiedforeach
oftheperformanceclassesin(a).Thewhitestarsdenotemeangains,whicharesignif-
icantlydifferentfrom0,whilethenumberofthestarsindicatesthesignificancelevel
intermsofstandarddeviations(threestarsforthreeandmorestandarddeviations).
DatawastakenfromtheelectrodeswiththehighestperformancesinV1(blue)and
(red).V4

4.4

esultsR

gureiF

4.28:

distributions

Summary

of

letevaw

of

p

analysis

erwo

co

of

the

ropr-eeceiv

ts,efficien

orf

haracteristicscerator-

eyMonk

M

for(

etailsd

see

for

g.iF

yna

145

owt

4.27).

146Chapter4:SelectiveVisualAttentioninV4/V1
4.4.6Modellingstimulus-specificsignals

Power Coeff.

RealModel

050[Hz]100150200
Figure4.29:Theredcurveshowsthepowercoefficientsofthemeasureddataforthe
17frequencybandsbeforetheinitialstimulusperiod,wherethescreenwasstillfree
fromshapestimuli.Thesepowercoefficientswereaveragedoverthewholetraining
dataset(MonkeyF).Thebluecurvedisplaysthecorrespondingresultsfromasimple
computationalmodel.Inthismodelapopulationof100neuronscreatedbinomial
spike-trainswithmeanrateof10Hz.Thesespike-trainswereaveragedandthen
foldedwithanartificialpostsynapticpotential(PSP(t)=exp(−t/τ),withτ=20ms)
tomimicthecreationofLFP-likesignals.Fordisplayreasons,theareaundereach
curvewasnormalisedto1.

Oneapproachtounderstandmoreaboutunderlyingbiologicalmechanismistobuild
acomputationalmodelandcomparethemodel’soutputtorealmeasureddata.Such
amodelistypicallybasedonseveralassumptions.Inthefollowing,theelectrodeof
monkeyFwiththehighestclassificationperformancewillbeused.Thegoalistomodel
themeanpowerspectraforthedifferentshapesandbothattentionalconditions.
Assumingthatthestimulus-specificinformation,storedinthewaveletcoefficients,
canbedescribedbymeanpowercoefficientsandthatthevariationsovertrialsare
justnoise,mayinthiscaseallowtosearchforasimpleunderlyingmodel.Asecond
assumptionforthisanalysiswillbethatspike-trainscanbetransformedintoLFP-
likesingals,justbyconvolutingtheaveragedspiketrainsfromapopulationofneurons
withpost-synapticpotentials(PSPs).ThesePSP’saremodeledbyanexponentially
functiondecreasing

PSP(t)=e−τt.

esultsR4.4

147

Theseassumptionsareverystrongsimplificationsoftherealmechanismswhichcreated
theepi-durallocalfieldpotentialsinthefirstplace.
However,performingthistransformationonspiketrainsgeneratedbybinomialrandom
processesrepresentingapopulationofneurons,producesapowercharacteristicsthat
showssimilaritiestorecordeddatafromtheperiodbeforetheinitialstimuliwere
shown.Fig.4.29showsthecomparisonbetweentherealdataandthissimplemodel.
Theoutputofthemodel(excepttheabsolutescale,whichwasremoved)issimilarfor
alargerangeofmeanratesusedinthebinomialprocess.Themodelapproximates
roughlytherealdatabutwasnotabletoreproducetherecordeddataperfectly.This
maybeduetothefactthatthecomputationalmodelwasindetailtoosimple.
Inanextstep,thegenerationofthespike-trainsbythepopulationofneuronsinthe
modelwasreplacedbyarandomprocess.Thisrandomprocessusesbasedongamma
distributionsforgeneratingthetime-intervalbetweentwospikes.Thistypeofdistribu-
tioncanbetunedbytwoparameters.Inalargenumberofsimulations,wesearchedfor
thebestfittingparametersetswhichcanexplainthepowerwaveletcoefficientcurves
forthesixdifferentshapesunderthetwoattentionalconditionsfortheinitialperiod.
TheresultscanbefoundinFig.4.30,usingthegammadistributionsshowninFig.
4.31.Asacontrolhowstronglytheresultsareinfluencedbythespike-traingenerating
process,anothersimulationwithapopulationofuncoupledleakyintegrate-and-fire
(IaF)neuronswasalsoperformed.FortheIaFneurons,thestrengthoftheinputand
theamplitudeoftheadditivenoiseonthemembranepotentialwereusedforfittingthe
modeltothestimulus-specificcharacteristicsoftherealdata.AllIaFneuronsreceived
exactlythesameexternalinput.Thenoiseprocesscreatedpositive/negativevalues,
drawnfromauniformdistributionforeachneuronandtimestep.Fig.4.32showsthe
results.Incomparison,bothapproacheswereabletoapproximatetheshapeofthecurves.
Theyalsoshowthatitisaproblemtoinferreliableinformationabouttheunderlying
neuronalsystemonlyonthebasisofthemeanpowerwaveletcoefficientcurves.Itisalso
notclearwhichcharacteristicsoftherealcurvesaretheimportantones.Inaddition,
applyingSVMsonthedatageneratedbythemodelsshowedextremely(andmuch
too)goodclassificationperformances.Comparingrealdataanddatafromthemodel,
revealedthatthevariancesofbothdatasetsdifferstrongly.Thiscanbecompensated
byapplyingexternalnoise(e.g.multiplicativenoise)tothecalculatedpowerwavelet
coefficientsbutthisseemsnottoallowtogetnewinsights.
Takentogether,thisanalysisshouldremindusthatevenifamodelapproximatesthe
datawellitmayshowonlyweakconnectionstotherealbiologicalprocess.Inthisex-
amplewehavetwodifferentmodelswhichcanreproducethemeasuredpowerspectra
withnearlythesamequality.Forabetterunderstandingofthecorrelationsbetween
perceivedstimuliandtheresultingneuronalactivitypatterns,datafromadditionaland
morespecialisedexperimentsarenecessary.Anotherinterestingquestioniswhetherit
ispossibletoreproducedthechangesmadebyattentioninaframeworkwithabio-
logicalplausiblenetworkwheretheattentionalstateismodeledbyaglobalparameter

148

(e.g.

het

lelev

fo

syronicithync

ni

het

Chapter

:4

ork).wnet

eelectivS

isualV

iontttenA

ni

4/V1V

esultsR4.4

149

Figure4.30:LikeinFig.4.29theredcurvesshowthemeanpowercoefficientscalcu-
latedfromthewavelet-transformedrecordeddata(trainingdatasetfromMonkeyF,
initialperiod)forallsixdifferentshapes(fromlefttoright).Theupperrowshows
thedatafortheattendedconditionwhilethelowerrowshowsthedataforthenon-
attendedcondition.Thebluecurvesarethecorrespondingmagnitudescalculatedfrom
asimplemodelwherethespike-trainsofapopulationof100neuronswereaveraged
andthenconvolutedwithPSP(t)=exp(−t/τ)(withτ=20ms),beforethepower
coefficientswerecalculated.Eachofthesespike-trainswasgeneratedusingagamma
distributionformodelingtheInter-Spike-Interval(ISI)distribution.SeeFig.4.31for
theISIdistributionswhichwereusedforcalculatingthebluecurvesinthisfigure.Like
inFig.4.29,thecurvesarenormalised.

150

Chapter

:4

eelectivS

isualV

tttenAion

ni

4/V1V

Figure4.31:Inter-Spike-IntervaldistributionsusedforsimulatingFig.4.30.The
distributionsaregammadistributionsp(t|K,L)=tK−1LKexp(−t·L)/Γ(k)wherethe
K’saretakenfrom[7.56,15.25]andtheL’saretakenfrom[0.37,0.83].Theupperset
ofcurvesarefortheattendedconditionandtheothersetrepresentsthenon-attended
condition.

esultsR4.4

151

Figure4.32:Theredcurvesshowtherecordedpowercoefficientsforthesixshapes
underbothconditionsintheinitialperiod.Thebluecurvesshowthecorresponding
weresultsreofcalculated.amodel.FortSeehisFig.figure,4.30theforspikmoreetrainsdetailsofhothewctheredomputationalcurvesmaonddelbluewerecurvgen-es
oferatedusingbyaapgammaopulationofdistribution.100Theuncoupledstrengthleakyofininputintegrate-and-firetotheIaF(IaF)neuronsneurons,(allninsteadeurons
receivedthesameinput)andthenoiseamplitude(drawnfromauniformdistribution
whicfittinghethequallymodelstodistributedthepodifferensitivteshapandesnandegativevconditions.alues)wereusedasparametersfor

152

Chapter4:SelectiveVisualAttentioninV4/V1

4.4.7DiscriminatingtheAttentionalCondition

Sinceouranalysesrevealedthatthedifferentattentionalconditionschangethestimulus-
representation,itisaninterestingquestionwhetheritispossibletoclassifytheatten-
tionalconditiononthebasisofaveragedpowerwaveletcoefficients.Incaseofhigh
classificationperformancesitwouldallowtouseselectivevisualattentione.g.asreli-
ableBCIsignalforcontrollingaspeller.

InFig.4.33theclassificationrateofthistwo-class-discriminationproblemforall
levsingleelforelectrothisdesoestimationfthelieselectroaderoundarra50%,yairetwshoaswpno(ssibleinitialtopreaceriohd).apeWhilerformancethecuphanceto
clas73%si(MonkficationeyrFates)/from78%the(MonkeelectroyMdes)wabithovseingleV1areelectronotdesfarfoveromrtchehanceV4levregion.el.UsTiheng
FtheacndominFig.binations4.15offVor4eMonklectroeydesM)(acdepictedlassibyficationyellopwcerformancerossesinofFig.80.3%4.14forforMmonkonkeeyy
Mand79.4%forMonkeyFwasachieved(initialperiod).

TforhethefiguressetsofFig.V44.34electroanddesFig.and4.35bosthhowmonktheeys.timeThesecourspeoficturesclassrievealficationtwopinsigherformancets:

1.Itispossibletoclassifytheattentionalconditionduringtheinitialandtheter-
minalperiodwithnearlythesameprecision.

2.Usnearlyingasthedhighataasswelectedithafromlargeatime400mswtindoimeww(650indo-w,2200thecmslassiafterficationtrialronsateect).anbe

Quantifyingthecontributionsfromdifferentfrequencybands,Fig.4.36showsthatfor
bothmonkeystheinformationabouttheattentionalconditionisstoreddifferentlyin
areaV4.Forreachingaclassificationrateof81.5%monkeyMrequiresonlythepower
coefficientsfromthefrequencybands61Hzand76Hz.ForMonkeyF,thepower
coefficientsofthefrequencybandrangefrom24Hzto122Hzarerequiredtoreacha
classificationperformanceof79%.

4.4

esultsR

153

Figure4.33:ClassificationperformancePfortheattentionalcondition(recordeddata
usedfromt=650mstot=2200ms,seeFig.4.10)forMonkeyF(a)andMonkey
Mof(sib).Tgnificancehepresen(p=0.02)tationisandmsimilararkitngotheclassioneuficationsedpinFig.erformance4.14,thatincludingdidnottheldifferevel
sigclasnsifyificaingntlythefrotwmotpheosscihablenceclaslevseeslwliesithatgrey50%coforlouMredonkseyquaFres.andThe51%cforhanceMlonkeveelyMfo.r
T78%heeforlectroMonkdeseyovMe.rrTheegionconV4stributionshowaclasfromsitheficationelectroratedesupabtoove73%V1forareiMnconkeyomparisFoandn
small.ratherV4to

154

Chapter4:SelectiveVisualAttentioninV4/V1

Figure4.34:Timecourseofclassificationperformancefortheselectedsetofelectrodes
aboveV4(cf.Fig.4.14a,yellowcrosses)forMonkeyF.Theconditionofattention
wHzaswcelaressificataketnedfbromyaSVMs.rangePsowetartingrcoe200fficienmstsbeinfore,aandfrequencyendingrang200ebmsetwaftereen5theandtimes200
themarkpedwitherformancetherandedthecrosses.chanceTheblevlacelk(circlesp=0.02).iTndicatehecahancesignificanleveltidsdifferenceepictedbebtywteenhe
shapsolidered(target)line.iIsna(a)nalysed.thepTimeerformancein(b)iforsmT1iseasuredshorwn.elativIne(tob),ttheheponseterioodftofhetheltargetast
e.psha

4.4

esultsR

Figure4.35:TimecourseofclassificationperformanceforMonkeyM(seeFig.
fordataafortdetailedhedesclassificationcription).proThecedureconditionwastaokfenafttenromttionhewsetasofclasesilectroficateddesbayboveSVMsV4.
Fig.4.15,yellowcrosses).

155

.344The(see

156

Chapter4:SelectiveVisualAttentioninV4/V1

Figure4.36:ClassificationperformancePforMonkeyF(a)andMonkeyM(b),us-
ingdifferentsubsetsofpowercoefficientsfromtheselectedV4electrodecombination
duringtheinitialperiodT1(650msto2200msafterthetrialonset).Thetaskof
theclassificationwastodecidewhetherthewasrecordedintheattentionalornon-
attentionalcondition.(seeFig.4.20foradetaileddescriptionofthistypeoffigure).
MonkeyMreachedaperformancePupto82.3%.Usingonlythecoefficientsfor76
Hzand61Hzallowsaclassificationperformanceof81.5%.MonkeyFreachesonlya
classificationperformanceof80.1%andthedistributionoverthefrequencyisdifferent.
E.g.forachievingaclassificationrateof79%,thefrequencybandsfrom24Hzupto
193Hzarenecessary.

esultsR4.4

4.4.8AttentiononMorphingShapes

157

Inaddition,thedatafromasecondshape-trackingtaskwastoinvestigatethediscrim-
inationofthetwoattentionalconditions.Fig.4.37illustratesthemodifiedtaskwas
performedindetail.Itdiffersfromtheprevioustaskinpresentingsetsofcontinuously
morphingshapesinstedofpresentingsequencesofstaticshapeswithblankperiods
betimetweencourstweoofsetsclasosifsficationhapes.pInFig.erformance4.38is(shomonkwneyforF)theandsingleFig.4electro.39(desmonk.eyBothM)mon-the
keysshowgoodclassificationratesforsignalsformaroundareaV4,whiledatafrom
theareafiV1guresyieldsFig.4only.40loawndcFig.lassification4.41,wrhicates.hshoTwhisthetrend,classiregardingficationpV1,iserformancesupportedfortbhey
incombinedcomparisonelectrototdesheovV4erV1.electrodeAgain,combonlyinatailoon.wUsingclassificationdatapfromaverformanceailableiselectroreached,des
fortheSVMyieldsaclassificationperformancebeyond90%.TheV4electrodecom-
bination,usedinthepreviousanalyses,showarelativebadperformanceformonkey
Mwhencomparedtotheperformancebasedonallelectrodes.Selectinganalterna-
tivalloewssetanofeincreaselectrodesincnearVlassification4,basepdonerformtheancecloseinformationtotofhepFig.e4.38rformanceandwFig.henu4.39sing
des.lectroeall

UsingthealternativesetofV4electrodes,Fig.4.42displayshowtheexplanatory
powerisdistributedoverdifferentfrequencybands.MonkeyMshowedaclassification
performanceupto91.3%andmonkeyFaclassificationperformanceupto87.2%.
Reacfrequencyhingbins90.8%,122Hz,with96theHz,adatand76fromHz.monkUsingeyM,onlyw96asHzeveinnpcombossibleinationwithwiththe76threeHz,
aclassificationperformanceof89.8%wasstillpossible.Providingthecoefficientsfor
96HzasinputfortheSVMsgivesaclassificationrateof88%,whileusingonlythe
ledataadtoforac76Hzlassiisficeationnoughpetrogetformanc87%.eofFor86.8%.monkeyFthefrequencybins122Hz-38Hz

158

Chapter4:SelectiveVisualAttentioninV4/V1

Figure4.37:Schematicillustrationofamodifiedversionoftheshape-trackingtask.
ThedifferencetothetaskshowninFig.4.10is:Insteadofseparatingthepresentation
oftwostaticshapesbyperiodsofemptyscreens,nowthetwoshapesaremorphed
continuouslyoverthetrial.Forexample,between1950msand3350msaftertrial
startshapeS1iscontinuouslymorphedintoshapeS2.Formoredetailsonthistask,
see(Tayloretal.,2005).

4.4

esultsR

159

Figure4.38:Timecourseofclassificationperformanceindiscriminatingthetwoatten-
tionalconditions,inthemodifiedshapetrackingtask(monkeyF).Dataforcalculating
thepowercoefficientswastakenfromthetimeintervalsmarkedinred,whichare
displayedabovetheclassificationperformancemaps.Theblackcurvesschematically
displaytothetwomorphcyclesbetween1950msand4750msaftertrailonset(seeFig.
4.37).Theredintervalhasalwaysalengthof400msandstartsinthefirstsub-picture
(upperleft)at1750ms.Theintervalwasshiftedby200msbetweensubsequentper-
formancemaps.Forthegreycolouredelectrodepositions,classificationperformance
didnotdiffersignificantly(p=0.02)fromchancelevel55%.Forallotherelectrodes,
theperformanceiscolour-codedinpercentageofclassificationperformanceaccording
thebarshowninthelowerrightcorner.

160

Chapter

:4

eelectivS

isualV

tttenAion

ni

4/V1V

Figure4.39:Timecourseofclassificationperformanceindiscriminatingthetwoatten-
tionalconditionsinthemodifiedshapetrackingtask(monkeyM).SeeFig.4.38fora
moredetaileddistribution.Forthisdataset,thechancelevelwas≈51%.

esultsR4.4

161

Figure4.40:Timecourseofclassificationperformancefordiscriminatingthetwoat-
oftentelectroionalcdes.onditionsTheredformcurvonkereyFepresenintsthemtheodclaifiedssificashaptioe-nptrackerfoingrmantask,ceffoorrthedifferen’old’tsV4ets
electrodecombination.Thegreencurveshowsdatafromacombinationofelectrodesin
themsproafterximittrialyofonseV4,t).seTlehceteddarkonbtheluebascurviseofexpFig.ress4es.38t(hewithclasstheidficationatafrromatef3750ora-c4150om-
inthebinationeoflectrothreedearraV1yemlectroap(lodes.werTherighpt).ositioThenuofsedtheelectrorecruiteddesareelectromarkdesedawithredepictedcrosses.
lighThetcblueolourcurvofetherepresencrossestsisthethepsameerformanceforthewhencorrespallelectroondingdescurvofes.theInelectroaddition,dearrathey
arequanused.tifiestThehebodashedrder.Bcyeloanwlinethisbshoowsrder,thethechpeancelevrformanceselandarethenotdashedsignificanmtagentadifferenlinet
fromthechancelevel(p=0.02).Theblackdashedlineschematicallyvisualizesthe
datamorphcyrecordedclesofintheanshinaptervesal(sbeeeFig.ginning4.37).200Formsbeacehforepoinandtineapndinge200rformancemscafter,urvewtheas
SVMs.hetforused

162

Chapter

:4

eelectivS

isualV

iontttenA

ni

4/V1V

Figure4.41:Timecourseofclassificationperformanceindiscriminatingthetwoat-
tentionalconditions,formonkeyManddifferentcombinationsofelectrodesinthe
modifiedshapetrackingtask.SeeFig.4.40foramoredetaileddistribution.The
alternatesetofV4electrodeswasselectedaccordingtoFig.4.39(usingthedatafrom
3350-3750msaftertrialonset).

R4.4esults

163

Figuredifferent4s.42:ubsetsClasosfitheficationpoweprcoeerformancefficientsPffromorthemonkeyalternativF(a)eVand4emlectroonkeydeMcom(b),businationing
(seeFig.4.40andFig.4.41)obtainedduringthemodifiedshape-trackingtask.The
FortargetmonkofeytheFcthedlassificationatafromwas3750to-4150discriminatems(aftetherttrialwoonsedifferent)wtaasuttensetdaionalndcformonditions.onkey
Mtimethewipondowewrscwoeeresfficienetlectedsfromonbas3350iso-fF3750ig.4ms.41a(afterndFig.trial4ons.39.et)Thweeuresesdetimelected.intTervhesalse
showedthehighestclassificationperformanceandwereselectedforthisreason.

164

Chapter4:SelectiveVisualAttentioninV4/V1

4.5SummaryandDiscussion

Thepresentedanalysis(Rotermundetal.,2007a)revealedthreemaininsights:

1.tainedActivityinthepatternslocalcfieldreatedpotenduringtialsthemproeasuredcessingoverofareaVdifferen4atlloswhapsestimurprisinglyuli,wcon-ell
todistinguishtheunderlyingshapeclassesaswellastheattentionalcondition.
2.neuralSelectiveaactivitttenytionpatternssubstanintciallyomparisonenhantcesocstimulionditionsdepewherendentthedifferencesstimuliofwasthesenot
attended.3.Behaviouralfailuresarerelatedtoreductionsinclassificationperformances.

theMostfofrequencytheicompnformationonentswhicinhthealloγ-wbtandodaisbovecriminate40Hz.Fdifferentsurthermore,hapestwheasstimfoundulus-in
sptrial.ecificTchehattenaracteristicstiondoepftheendentsignalisenhancemensimilartofduringstimulusdifferentstimdiscriminabilitulationypceanriodnotsinbea
astimexplainedulus-bsypaecificsimpledifferenincreasetialofscalingtheofSNR,thefbutrequencyturnsoutcomptobeonents.mostThisstronglyscalingrelatedresultsto
inanenhancedseparationbetweenthecharacteristicfrequencypatternsfordifferent
uli.stimTheenhanceddiscriminabilityunderconditionsofattentioncouldinprinciplebetraced
backtotwodifferentchangesinthesignals.First,thereisasmallbutstatistically
significantimprovementofthesignal-to-noiseratio.Thisfindingisinlinewithastudy
byMcAdamsandMaunsell(McAdamsandMaunsell,1999b)describinganattention-
dependentimprovementoftheSNRforspikecountdatafromareaV4.Intheirdata,
theriseoftheSNRresultedfromtheattentiondependentincreaseofstimulusresponses
approtogetherximatelywithalonlyessasthanthepsropquareorrtionalootoftincreaseheorespftheonse.Tstandardogetherdeviationwiththewehichnhancedrises
stimabsoluteulus-indepdifferenceendenbtetwgaineenfactor,thertheespionsesncreasedfordSNRifferentsresultedtimuliinbaneingimprovamplifiededorienbyta-a
tiondiscriminability.Incontrast,increasesoftheSNRinthepresentstudyexplained
onlyTheamamjorinorpartpartofofthetheeenffecttireisbasedenhancemenonantoattenfsthapion-edependendiscriminabilittincreaseyunderofattendifferencestion.
betweenresponsestodifferentstimuli,whichallowforimprovedstimulusdiscriminabil-
ityeventhoughSNRsstayalmostunchanged.Sinceremovinganydifferencesinsignal
power/strengthfromallstimulusresponseshasonlyminoreffectsonclassificationac-
curacyanditsattentiondependentenhancement,theeffectofattentiondoesnotrely
onastimulus-independentgaininresponsestrength.Theresultsratherindicatethat
nalsattentinioncdifferenhangestwatheyssfporectraldifferenctompsotimusitionli.andNotsonlypatialweredistributionthesechofangesthedifferenneuraltfsig-or
ofdifferenthertespstimectivuli,enbuteuralinaadditionctivitytpheiratterns(directioninwspatialasasucndhastofrequencyincreasecompodistinctivsition)e(nesssee

4.5SummaryandDiscussion

165

Fig.4.22forillustration).Arbitrarydirectionsofchangeswouldnotnecessarilyhave
causedanimprovementofstimulusdiscriminability.Thusthemajorpartoftheeffect
isthereforenotexplainedbyauniform,stimulus-independenteffectasingainmodels
forsinglecellfiringratedata,butbymorecomplexchangesinthecompositionand
distributionofneuralactivity.

Theunderlyingneuronalmechanismsarenotknown.Inliteraturetwocandidatesof
putativeneuronalmechanismscanbefoundwhichcouldcontributetoanexplanation:

•FMTeature-sp(Martinez-TecificrchujilloangesandinaTttenreue,tional2004),mowhicdulationhcouldhaveplabyeenasobseignificanrvedtrinoleareain
scalingneuralsignalsdifferentiallydependingonstimulusshape.
•Recruitingadditionalneuronalpopulationstoencodeastimulusbydynamically
ch2006)angingcouldtheiralsorrendereceptivenfieldeuralspropignalsertiesford(Connorifferentsethapal.,esm1997;oredWisotinctmelsdorffrometeacal.,h
other.

Whatdothesefindingsimplywithrespecttocorticalstimulusprocessinganditsde-
pendenceonattention?Classificationinthepresentstudydependsonthepatternof
frequenciesinthelocalfieldpotentialcausedbyneuralprocessingofdifferentstim-
uli.Whilefieldpotentialsclearlylackthespecificityofinformationcontainedinthe
activityofthecontributingsingleneurons,theyreflectsynchronisedpartsofneuronal
activitypatterns(Elul,1972;Nunez,1995;Robbeetal.,2006).Synchronousactivityis
knowntobeparticularlyeffectiveindrivingotherneurons(BrunoandSakmann,2006;
Usreyetal.,2000;SegevandRall,1998;AzouzandGray,2003)andhasthereforebeen
implicatedinstructuringeffectiveconnectivity(Aertsenetal.,1989)anddefiningina
transientandflexiblemannerneuronalassemblies(SingerandGray,1995;Abeles,1991;
Aertsenetal.,1986;KreiterandSinger,1996;vonderMalsburg,1985).Previousstud-
iesalreadydemonstratedthatattentionenhancesspecificallysuchoscillatoryactivity
inthevisualsystem(Tayloretal.,2005;Friesetal.,2001).Inaddition,theresults
showthatwithattentionthestructuralorganizationoffieldpotentialssystematically
changes,indicatinganattention-dependentchangeofcompositionanddynamicstate
inthenetworkofsynchronizedneuronsprocessingthestimulusinareaV4.Themore
distinctpatternsofneuralactivityassociatedwithprocessingofanattendedascom-
paredtonon-attendedstimulisuggestamoredifferentiatedandspecificcomposition
andstateofsynchronousneuronalassembliesiftheyprocessshapeunderconditions
ofattention.Suchindicationsforenhancedmodesofprocessingofattendedstimuli
arewellinlinewithpsychophysicalfindingsdemonstratingimprovedprocessingofat-
tendedstimuliandtheparticularimportanceofattentionforshapeperception(Rock
etal.,1992;RockandGutman,1981).

Furtherevidenceforthebehaviouralrelevanceoftheattention-dependentchangesof
corticalprocessingreflectedbytheobservedchangesinthepatternoffieldpotentials
comesfromthereduceddiscriminabilityofsignalsprecedingbehaviouralerrors.In

166

Chapter4:SelectiveVisualAttentioninV4/V1

fact,thefindingsimplythatonecouldpredicttheoccurrenceofbehaviouralerrorsfrom
suchlessdistinctsignals.Insummary,thepresentdataprovideevidencethatselective
attentionimprovesprocessingofattendedstimulibyenhancingthedistinctivenessand
discriminabilityofcorticalnetworkstatesinvolvedintherepresentationandprocessing
ofindividualstimulialreadyincorticalareaV4.
Whileattentioncausedaclearimprovementofthestimulusclassificationachievedfor
fieldpotentialsfromareaV4,nosignificanteffectwasfoundforareaV1.Alikely
reasonforthislackofeffectistheveryhighclassificationperformanceobservedforthe
V1recordingsites,evenwithoutattention.Itisbasedonthelargesizeofthestimuliin
comparisontothesmallsizeoftheRFsforlocalfieldpotentialsinV1(Eckhornetal.,
1993).Thisresultsinmassive,shape-dependentdifferencesinthecoverageofthese
RFsbydifferentstimuli.Consequentlystrongdifferencesoftheoverallactivation
ofthecorticalcolumnsunderneathaV1electrodeallowdistinguishingstimulivery
reliably,eveniftheyarenotattended.Thishighstimulusdiscriminabilitywasin
additionconfirmedbyaROC-analysistestinghowwellpairsoftwostimulicanbe
distinguishedbyasinglefrequencycomponentfromoneelectrode.Performingthis
analysisforallpossiblecombinationsofstimuluspairsandfrequencycomponentsforV1
revealedthatalmost30%(monkeyF)or17%(monkeyM)ofcombinationspermitted
analready90-100%correctdifferentiationbetweentwostimuli.Thepresenceofthese
simple,almostperfectlydiscriminativesignalsinthenon-attendedconditionstrongly
reducesthepossibilitytoobserveinV1anyfurtherattention-dependentclassification
enhancementsbasedonmorecomplexindicatorsofattentiondependentchangesin
corticalnetworkstates.
Incontrast,inV4therewasnocombinationwhichallowedforsuchahighclassi-
ficationperformance.Thisleavesroomtoobservesubstantialattention-dependent
improvements.Thustheresultsdonotexcludethepossibilitythatsimilarchangesas
inV4couldalsobeobservedforV1,ifstimuliwouldbeassimilarwithrespecttothe
smallV1RFsastheywerewithrespecttothelargeRFsofV4.
Thehighstimulusdiscriminabilityachievedwithlocalfieldpotentialsrecordedfrom
thesurfaceoftheduraisalsoofinterestinthecontextofBrainComputerInterfaces
(BCI).Inthisstudytheelectrodearrayswerecarriedoveryearsandrecordingswere
pooledfromrecordingsessionsoverseveralweeks.WhileBCIsbasedonsingle-or
multi-unitrecordings(Wessbergetal.,408;Tayloretal.,2002)typicallyneedan
initialcalibrationforeachsession,therecordingsforthepresentstudyaresufficiently
stabletoallowfordemandingstimulusdiscriminationswiththesameclassifierover
months.Thisisevenmoreremarkablesincethestimuliwerenotconstructedtobe
easilydistinguishable,buttorequireconsiderableeffortbythemonkeysforsuccessful
discrimination.Thefindingsthereforesuggestthatthecomparativelysimpleshapesof
lettersandmanyothersymbolscouldbedetectedwithhighreliabilityinthespatial
distributionoffieldpotentialsrecordedfromvisualcortex.
AnotherinterestingaspectforBCIapplicationsisthehighclassificationperformancein
discriminatingthetwoattentionalconditions.Withthedatafromthemodifiedshape-

4.5SummaryandDiscussion

167

trackingtaskitwaspossibletoidentifytheattentionalconditionwithaprecisionofup
to93.7%usingthedatafroma400mstimewindow.Forfurtherresearchprojectsit
willbeinterestingtolearnmoreaboutthespatialaspectsofselectivevisualattention.
Inthepresentedexperimentsattentionwasonlyappliedtotheleftorrightpartofa
computerscreen.However,animportantquestioninthefieldofBCIsisthepossibility
topartitionthevisualfieldevenfurtherintomoreandsmallerregionswhileretaininga
similarclassificationperformanceunderthesenewconditions.Furthermoretheanalysis
revealedthatthemodeofpresentation(stationaryshapesvs.morphingshapes)can
significantlyenhancetheclassificationratefortheattentionalcondition.Thisfinding
shouldguidethedesignofBCIsusingvisualselectiveattention.

168

Chapter

:4

eelectivS

Visual

iontttenA

ni

4/V1V

Chapter5

StabilizingDecodingAgainst
Non-stationaries

Motiv5.1tiona

Inchapter5.2.3webrieflydiscussedtheideaofbraincomputerinterfacesandfunc-
tionalneuro-prostheses,aswellasdifferentwaysofacquiringinformationfromactivity
patternsinthebrain(actionpotentials,LFP’s,EEG’s)forcontrollingexternalde-
vices.Manyexperimentshavesuccessfullydemonstratedthatexternaldevicescanin
principlebecontrolledbybrainsignals(Andersenetal.,2004a;Schwartz,2004).One
applicationiscontrollingprostheticdevicesforrestoringlostbodyfunctions.Espe-
ciallyforvoluntaryuseofneuro-prothesesinnormallifeoutsidealaboratory,itis
importanttoprovidelong-termstabilityofboth,theinformationacquisitionprocess
andthesuccessivetranslationofmeasureddataintocontrolsignals.Asantagonists
totherequirementofstability,differentprocessesinduceon-goingchangesintheob-
servedsignalcharacteristicsandinthephysicalpropertiesoftheprosthesis.These
includerelativemovementoftherecordingarraywithrespecttothebrain,neuron
deathandgrowthprocessesattherecordingsite,chemicalchangesatthecontactsur-
facebetweentheelectrodetipandtheneuronaltissue,oradaptationoftheneuronal
responsepropertiestothetask.Takentogether,non-stationaritiesimplythatover
sometimethesituationwilldiffermoreandmorefromtheonetowhichdecodingal-
gorithmsweretrainedwhentheprosthesiswasinstalled.Accordingly,errorsbetween
actualanddesiredmovementsmightincreaseandrendertheprosthesisuselessafter
sometime.Forcounteractingon-goingchanges,itisnecessarytoadaptthealgorithms
forreconstructingtherelevantinformationfromthemeasuredactivitypatterns(see
.2).2section

Astandardsolutionforthisproblemistore-traintheprosthesisbyrequiringthe
subjecttoperformawell-definedtaskatregularintervals(Hinterbergeretal.,2004;
Tilleryetal.,2003).Inthesetasksthedesiredmovementisknown,theperformancecan

169

170

Chapter5:StabilizingDecodingAgainstNon-stationaries

beaccessedbyanexternalobserver,andsubsequentlybeusedtoadaptthemapping
betweenbrainactivityandthecontrolsignalforexecutingtheestimated,intended
action.Thedrawbacksofthisschemeareobvious:Everydayuseoftheprosthesismust
beinterruptedbyretrainingsessions,whicharelikelytobeconductedundersupervision
inahospitalorspecializedlaboratory.Betweentrainingepochs,theperformanceofthe
prostheticdevicewillstilldecreasegradually,anditisparticularlyunlikelythatitcan
recoverfromabruptchangesoftherecordingsinducede.g.byextrememovementsof
thehead.Ifonlythebrainwouldberequiredtocompensateforthesenon-stationarities,
itislikelythatthenetworkcreatingtherecordedactivitybecomesstressedbyrequiring
ittooperateatthebrinkof,orbeyonditsfunctionalcapabilities.

Forvoluntarymovements,theerrorbetweendesiredandactualmovementisunknown
toanexternalobserver.InthischapterIwillpresentanewmethod(Rotermundetal.,
2006a)ofusinganextraerrorsignalmeasureddirectlyfromthebrainforadaptingthe
estimationprocessinanon-linefashion,andthusprotectingitagainstnon-stationaries.
Thisideaprovidesanefficientalternativetocurrentmethodsofre-learningprocedures
andhasseveralpotentialadvantagesforadisabledperson:althoughbeingtechnically
demanding,itcaneliminatetheneedforperformingtedioustrainingtasksinaclinical
environment,andbothadaptationandeveninitialcalibrationcanbedoneduringthe
everydayuseoftheprosthesis.Insummary,thisscenariomaydramaticallyincrease
independence,lifequality,and–mostimportantly–motivationofthepatient.It
hastobenotedthatthisscenariocanalreadybeusedforactualprosthesesbutit
mainlyaimsatneuro-prostheticdeviceswithhighqualityforreconstructingintended
movementswhichcanbeexpectedforthenearfuture.

Therequiredadditionalerrorsignalhastorepresenttheuser’saffectiveevaluationof
thecurrentneuro-prostheticsperformance.Severalpossiblecandidatesforsignalsrep-
resentingaffectiveevaluationsofperformancearealreadyknown(Ridderinkhofetal.,
2004b;Musallametal.,2004),andithasbeendemonstratedthatsucherrorsignals
caninprincipleserveforreinforcementlearning(SuttonandBarto,1998).Through
numericalsimulation,usingrealisticassumptionsabouttheoriginandqualityofneu-
ralsignals,itwillbeshownthateveniftheerrorsignalhasalowinformationcontent
andshowshighnoiselevels,likeitistypicallyencounteredinrecordingbrainactiv-
ities,itispossibletocounteracttheeffectofavarietyofnon-stationaries.Evenin
thecasewhentheerrorsignalitselfiseffectedby(aslightlyreducedclassofpossible)
non-stationaries,theprocedurecanbestillapplied.

5.2NeuronalandComputationalBackground

5.2.1Motorsystemandmovementsofarms

Thissectionwillstartwithaveryshortoverviewoftheneuro-anatomicalstructures.
Thenselectedrulesofarmmovementwillbediscussed,followedbyadiscussionabout

5.2NeuronalandComputationalBackground

171

Figure5.1:Schematicwiringdiagrambetweenspinalcord,subcortical,andcortical
motorareas.(Figureadaptedfrom(Johansen-Berg,2001))

correlationsbetweenneuronalactivitiesandmovementsaswellaspropertiesofneu-
ronalactivitypatternsrepresentingerrorsofmovements.Thissectionwillclosewith
aglanceoncurrentresearchinneuronalprosthetics.

oMcortextor

Thehumanmotorsystem(RizzolattiandWolpert,2005)andcorrespondingbrainareas
(RolandandZilles,1996;ChouinardandPaus,2006)showadegreeofcomplexitysim-
ilartothevisualsystem(seeFig.5.1forasimplifiedillustrationand(Johansen-Berg,
2001)formoredetails).Theprimarymotorcortex(M1,Brodmannarea4)receivesits
maininputfromtheparietalcortexandalsogetsinputfrome.g.thepremotorcortex
andthethalamus(Passingham,1993).Intracorticalstimulationsinthemotorcortex
oftengeneratereactionsingroupsofmusclesandindividualmusclescanoftenbead-
dressedfrommorethanonepositioninthemotorcortex,whichindicatesthatregions
ofthemotorcortexratherrepresentgroupsthanindividualmuscles(Johansen-Berg,
2001;Donoghueetal.,1992).CurrentstudiesinM1indicatethatthiscorticalarea
maybecomposedofseveralmodules.Eachofthesemodulesseemstobecontrolling
onephysicalsystemlikee.g.ahand,anarm,oraleg.Furthermore,itisgenerally

172

Chapter5:StabilizingDecodingAgainstNon-stationaries

believedthatthesemodulesareadaptivetotheactualconditionsofthesystem.In
total,theprimarymotorcortexisimportantforgeneratingmotoractionbutitis
alsoinvolvedinlearningnewactions.Furthermore,connectionsbetweentheneuronal
activityofM1andothercognitivevariables(SanesandDonoghue,2000)exit.
Themotorsystemconsitsofmoreareasthantheprimarymotorcortex,likee.g.the
posteriorparietalcortex,thepremotorcortex,thecerebellum,thebasalganglia,and
ord.cpinalstheTheneuronsinposteriorparietalcortexshowcorrelationsbetweentheirdischarging
behaviourandspecificmotoractslikee.g.grasping.Inaddition,itwasfoundthat
actionsseemtobeorganizedin’chains’ofelementarymotorcommands.Alsoseveral
neuronswerefoundthatshowmirrorproperties,whichmeansthattheynotonlyfire
duringperforminganaction,butalsoreactwhileonlyobservingaspecificmotoraction
(RizzolattiandWolpert,2005).
Thepremotorcortex(PMC,partsofBrodmannarea6)(Fulton,1935;Woolseyetal.,
1952)isbroughtintoassociationwithsensoryguidanceofmovementandpreparationof
motoractions(Wise,1985)whilethesupplementarymotorarea(SMA,partsofBrod-
mannarea6)(PenfieldandRasmussen,1950)seemstobeconcernedwithplanning,
executionandmemorizationofwholesequencesofmotoractions(TanjiandShima,
1994).Otherfunctionallyimportantpartsofthemotorsystemarethecerebellum,thebasal
ganglia,andthespinalcord.Ifthecerebellumisdamagedthendysfunctionsinco-
ordinatedmovementscanoccuraswellasdisturbancesinbalance(vertigo),jerking
involuntarymovements,lossofpowerandtoneinthelimbs,andabnormalposture
andstaggeringgait.Italsoplaysaroleinmotorlearning.Adamageofthebasalgan-
gliadoesnottypicallygenerateaparalysisorataxiabutleadstoabnormalmovement
patterns,likee.g.involuntarymovement,povertyofmovement,andalteredmuscle
tone(Johansen-Berg,2001).Furthermore,basalgangliaplayaroleinlearningaction
sequences(RizzolattiandWolpert,2005).
Thespinalcordisnotjustasimpleconnectionbetweenbrainandnervesprojecting
tothemuscles.Experimentswherethespinalcordwasstimulateddirectly,suggested
thatitgeneratesmotorprimitives(patternsofmuscleactivations)itself(Wolpertand
Ghahramani,2000;Treschetal.,1999;Giszteretal.,1993).
Insummary,themotorsystemisverycomplexandstillunderresearch.Alsothe
questionhowthemotorsystemcanhandlealterationsofthesystem’sdynamicsisnot
answered.Forexample,ifwetransportanobjectwithourhandthenthephysical
propertiesofthearmarechangedduetoextraload.Thepropertiesofthehandcan
alsobechangedbye.g.growthandfatigue.Itseemsthat,dependingonthesources
ofthesechanges,differentadaptationstrategiesareapplied.Itmakesadifferenceif
thechangeiscausedbychangesofintrinsicpropertiesorduetoexternalinfluences
(LacknerandDiZio,2000;Miall,2002;RizzolattiandWolpert,2005).

5.2NeuronalandComputationalBackground

Somerulesofmovinganarm

173

Ifapersonreachesforanobjectthenthehandcanfollow,onitsway,anearlyarbi-
trarytrajectorywithanearlyarbitraryvelocityprofile.Onemovementofthehand
canbecomposedofahugenumberofpossiblepostureconfigurations.Eachposture
configurationcanbegeneratedbydifferentcombinationsofmuscleactivations,and
thenecessarymuscletensionscanbecreatedbyvariousneuronalactivities.Taken
togetherallthesedegreesoffreedom,itseemsthatitisextremelyunlikelythattwo
personswouldusesimilarmovementstrategiesforreachingthesametarget.However,
itwasfoundthatasetofrulesformovementsisappliedandthattheserulesareeven
commontodifferentspecies(Schaal,2002;FlashandSejnowski,2001).Theseregular-
itiesweremainlyinvestigatedinisolatedarmorhandmovementstudies.Ifarmand
handaremovedatthesametime,thenbothmovementsinfluenceeachother,making
thesituationmorecomplex.Areviewregardingthistopic,whichIwillfollowinthis
section,canbefoundin(Schaal,2002).

Onerecurringtypeofmovementcharacteristicswasfoundforpoint-to-pointreaching
movementsinhumansandotherprimates.Thetrajectoriesarecomposedofapproxi-
matelystraightmovementsintothetargetdirectioncombinedwithanapproximately
symmetricbell-shapedvelocityprofile(BullockandGrossberg,1988).Deviationsfrom
theshapeofthevelocityprofiledependonmovementspeed.Differentmodelswere
proposedforexplainingthesevariations:e.g.asresultofperceptualdistortionanddy-
namicalpropertiesoffeedbackloopsinmotorplanning(BullockandGrossberg,1988).
Othermodelsexplainthefindingsbyusingdifferentoptimizationcriteria.Theye.g.
includethedynamicsofthearm(Kawato,1999;Unoetal.,1989)ordifferentnoiselev-
elsfordifferentvelocitiesgeneratedbythestochasticpropertiesofneuronalactivities
(HarrisandWolpert,1998).

Anotherapproachistocomposemovementsfromso-calledmovementprimitives,also
knownasunitsofaction,basisbehaviors,orgestures,see(Mussa-IvaldiandBizzi,2000;
FlashandSejnowski,2001;FlashandHochner,2005;ThoroughmanandShadmehr,
2000;SternadandSchaal,1999).Forexample,amovementcouldbeconstructed
fromanumberofstaticstraight-linedmovementswithsymmetricbell-shapedvelocity
profiles,oramovementcouldbeacombinationofdynamicmovementprimitives(e.g.
pointorlimitcycleattractors).Itisstillnotclearifmovementprimitivesareused
forthegenerationofmovements,butfromatheoreticalpointofviewtheexistenceof
suchbasicunitsofactionswouldreducethecomplexityofmotorlearningproblems
(SternadandSchaal,1999).

Formovementsrestrainedtotwodimensions(inplanarspacewithCartesiancoor-
dinates)itwasfound(Lacquanitietal.,1983;VivianiandFlash,1995;Moranand
Schwartz,1999a)thattheangularvelocitya(t)followspowerlawwithexponent2/3,
a(t)=k·c(t)32,
wherec(t)isthecurvatureofthetrajectoryandkisaproportionalityconstant.The

174

Chapter5:StabilizingDecodingAgainstNon-stationaries

originofthislawisstillunderdebate.Itissuspectedthatitreflectsanimportant
haveprinciplebeenoffoundmomeniftthergenerationestrainmeninttothetwcenodtralimensionsnervousislosystem.osenedV(SchiolationsaalandofSthisternad,law
2001).

Arobustphenomenoninhumanarmandhandmovement,evenusedasabehavioral
beGrosncsbehmarkrg,for1988)testoringspemoeddels-ac,cuisracyFitts’tradeLaw-off.(MottetItquaanndtifieBostotshema,requ2001;iredBullotimeckTandfor
rapidlyreachingatargetindependencyoftherequiredaccuracy

T=a+blog22·A,
Wwithaandbasfittingconstants,Abeingthedistancebetweenstartandendpoint
ofthemovement,andWdescribingthetargetwidth,whichdefinesthenecessary
precisionofreaching.ThephysiologicalcausesfortheFitts’Lawarestillunknown.

Itispossibletousetheserulesasaprioriknowledgeforestimationalgorithms,usedfor
controllingprostheticarmdevices.Thisallowstoincreasetheprecisionofpredicting
thenextpossiblepositionsoftheprostheticdevice.

5.2NeuronalandComputationalBackground

5.2.2Errorsignalsinthebrain

175

Behavioralactionscanleadtoerrorsifsomethingwentwrong.Thecentralnervous
systemreliesoninformationabouterrors,whichareusefulforadaptingandoptimizing
itsrepresentationsandfunctionsas,e.g.theadaptationofthemotorsystemtochanges
inphysicalpropertiesofthelimbs(LacknerandDiZio,2000;Miall,2002;Rizzolatti
andWolpert,2005).Laterinthischapter,wewilldevelopaframeworkthatuses
informationaboutperceivederrors,resultingfromactionsperformedwithprosthetic
prosdevices.theses.TheTehisrrorsisgnalignalshastowillbebeausedcquiredforfmromodtheifyingbrain’stheconneuronaltrolsystemactivitoyf.nSeuro-ince
thisisnewtoneuronalprotheses,itisnotclearwherethiserrorsignalcanbeacquired
from.relatedIsntignalsheinfollothewibngrainanisovgivervein.ewTheaboutfocnuswilleuronallieconorrelateserrorsrignalsegardingcreatederrorsbytandhe
movementofarms.Thisoverviewwillstartwithabriefdescriptionofrewardsignals
alsofoundbeinusefulthemforiammalianmprovbingrain.thepTheerfonrmaeuronalnceocfaorrelatesprostheticofsdomeevice.oftheserewardsmay

Rewardsinthebrain

Rewardsandpunishments,asakindofnegativereward,areimportantforbehavioural
learning.Rewardscanactasbehaviouralgoalsifinformationaboutrequiredactions
forobtainingrewardscanbededucedbythesubject(DickinsonandBalleine,1994).
Forareviewonrewardandneuronalcodingsee(Schultz,2004),whichIwilluseasa
basisforthispartofthisintroduction.

Interestingly,ithasbeenfoundthatrewardswhichhavebeenfullypredictedwillnot
conpredictedtributeatndolreceivearningedrew(Kamin,ardsare1969;oftenSchmoreultzeimptal.,ortant1997)(RescorlaandtahatndeWrrorsagner,betw1972;een
PtheearcemandagnitudeHall,ofan1980).rewaIrdnmugameltipiledtheoryby,ttheherelevprobabilitanceyofofaanreewxpardectedisdeprewaendingrd.Theon
longertheestimatedtimeuntiltherewardisexpectedtobeachieved,theloweristhe
vaandluetofhethetermrew’utilitard(y’Hoisetusedal.,1999).instead,Howwhicevher,aolsoftenincludes’reward’e.g.istnotheagoopreferencesddescofriptionthe
subject(Schultz,2004).

Experimentsrevealedalargenumberofsitesinthebrainthatareinvolvedinrepresen-
thetationslongoftrermewpardsanderformancerelatedofquanneuronaltities.Sprotheses.omeoftInhesethemafyollobewingslistuitablesevforeraletypnhancingesof
Drewepaerdndingsignalsonathendsubtheject’scorresppreferencesondingtobrainapregionsarticulartwillypbeeofenreward,umerated:neuronsintheor-
bitofrontalandstriatalneuronsareactivated.Furthermore,structuresinthestriatum,
orbitofrontalcortex,dorsolateralprefrontalcortex,anteriorcingulatecortex,perirhi-
nalparscortex,compactasupeofriorsubstancolliculus,tianigraparsshorwedeticulatadioffferencessubstanintiatheirnnigra,euronalandactivitdopaminergicywhen

176

Chapter5:StabilizingDecodingAgainstNon-stationaries

atrialwasorwasnotrewarded.Neuronsinstriatum,dorsolateralprefrontalcortex,
orbitofrontalcortex,parietalcortex,posteriorcingulatecortexanddopaminergicpars
nitudecompactaoforewfsardubstant(liquid).ianigraOreverbitofronaledtalcorrelaandstionstriatalbetweenneuronstheirraeactctivittoyasomendtythepemsoag-f
conditionedstimuliwhiletheiractivityhasinthiscaseacorrelationtothepredicted
reward.Furthermore,theneuronsoforbitofrontalcortexaswellastheamygdalaand
striatumincludingthenucleusaccumbensaresensitivetoreceptionofdifferentliquid
andfoodrewards(Schultz,2004).Theactivitiesofprefrontalcortex(Matsumotoetal.,
2003;Kobayashietal.,2002),intraparietalarea(PlattandGlimcher,1999),cingulate
motorarea(ShimaandTanji,1998),frontalandsupplementaryeyefieldsandpremotor
cortexdopaminergicarealsoneuronsinfluencedarebyproexpcessingectedtherewarddifference(RoescbheatwndeenOlson,expected2003),rewaandrdandmidbrainre-
ceivedreward(Nakaharaetal.,2004).Prefrontalcortex(Watanabeetal.,2002a),
midbraindopamin-containingneurons(Satohetal.,2003),ventralstriatum(Shidara
etal.,1998),andforvoluntarymovements(TremblayandSchultz,1999)orbitofrontal
neuronsreceivingaarerewinardadditionmocdulatesodingthemotivrespaonsetionalofaspneuronsectsoifnasuprewaeriorrd.cTheolliculusprobabilitandinytohef
parsreticulataofsubstantianigra,whiletheactivityofsomeparietalneuronscanbet-
oftertheberdewescribard.edSomeintermsdopamineoftheproneuronsductsbhoetwwaeendecatheyomftheiragnitudeactivitandytinhetheiprobabilitntervayl
betweenthestimulusthatadvertisesarewardandtherewarditself,whileneuronsin
midbrain(Fiorilloetal.,2003)andposteriorcingulatecortexseemtocodefortherisk
and/ortheuncertaintyinreceivingreward.Otherneuronsindorsolateralprefrontal
cortex,anteriorcingulatecortex,posteriorcingulatecortex,andfrontaleyefieldsrep-
resenteventswheretheanimalmadeabehaviouralerrorthatcausedamissingreward.
Itseemsthattheseneuronalrepresentationsofreward-relatedvariablesareusedby
thebraintodirectotherprocesseslikee.g.armandeyemovements(Schultz,2004).

oringmonitConflict

Notallactionsresultdirectlyinareward.Thisisonereasonwhytheabsenceofa
rewardafteraperformedactioncannotalwaysbeusedasanindicatorforabehavioural
error.However,itisstillimportantforananimaltomonitoritsownperformance.This
oalloccur.wstheAhaypnimalothesistorteacthatawithddressescounthisteractivmoneimtoringeasuresofinsconflictituationssituationswhereistheproblemsso-
called’conflict-monitoring’hypothesis(Botvinicketal.,2004;Botvinicketal.,2001;
Rusregionshwoinrththeetal.,brainm2004;onitorYeungoetccurringal.,2004b).errorsandThecideaonflictsbehind(NikiathisndhWypaotanabthesise,is1979),that
andthatthisinformationinturnisusedforadaptationprocessesinthecognitive
controlsystemsuchthatconflictscanbesolvedorcircumventedinthefuture.Studies
suggestthatanteriorcingulatecortex(ACC)(Ridderinkhofetal.,2004a;Ridderinkhof
etal.,2004b)isimportantforthedetectionofconflictsituationsandseemstobeused
forrepresentingatleastthreedifferenttypesofconflicts(Botvinicketal.,2004):First,
situationsthatrequireoverridingarunningactionwithabetterresponse.Thiscreates

5.2NeuronalandComputationalBackground

177

aancumonflictberbofetpweenossiblethenactionsewandisatvheailableobsoletebutbethehasviour.ituationisSecond,thereunderdetermined,arecaseswhicwhereh
oresultsccurs.inTahisproblemcanoforccure.g.selectingduringthebestsimpleoption.motorAtconflictasksbetwhereweensevtheeralpossibleequallygoactionsod
responsesareavailable(Frithetal.,1991).Thethirdclassofconflictsencompasses
sisithattuationsACCthatratherareinevncoolvdedesinthetheeffort’commisforasichonievofingerrorsa’.goalAnthanaalternativconflictehyspothesituationis
(Waltonetal.,2003).
Anobservableresultofconflictmonitoringseemstobetheincreaseinreactiontimes
afteGararvamnaketingal.,errors2002).(BotvHiere,nicktheetaml.,agnitude2001;Geofhringerroreatndal.,activ1993;ationGeofhringACetCaal.,re1995;con-
nected(Kernsetal.,2004).Otherstudies(PicardandStrick,1996;Paus,2001)refer
tothestrongconnectivitybetweentheACCandmotorstructureswhichsuggestthat
ACCinfluencesresponseselection.AnactivationofACCwasalsofoundwhensub-
jectsHolroygotdandfeedbacCokles,base2002).donIttheirhasbdeciseenionsrev(egamaledblingthattasktranss)ien(NieutwactivenhitiesuisetofAal.,CC2004;are
relatedtothedetectionoferrors.InEEGrecordings,eventrelatedpotentials(ERP)
areobserved,whichpeakbetween80and130msafterrespondingtoataskwithan
error(Rushworthetal.,2004;Falkensteinetal.,2000;Gehringetal.,1993;Krigolson
andHolroyd,2006).SimilarresponsesofACCtoerrorswerefoundinstudiesusing
etfMRIal.,(C2003).arterEextaal.,mples1998;aretasMenonkswethereal.,errors2001)andandreespolectrophnsecysonflictsiologicalarerassoecordingsciated(Itoto
fast2004b).correctionmComputationalovementsmo(odvelsedrridingescribingactions)thecduringonnectionerroneousbetwaeenctionsconflit(Yeungmonitoringetal.,
theoryanddetailsofERP(Yeungetal.,2004b)havebeenpublished.Analternative
study(vanVeenetal.,2004)showeddisagreementsbetweentheroleoftheACCin
beconflict-notedthatmonitoringERPhycanpoalsothesisbeaindtheirnfluencedexpbyerimentheotalbservaresults.tionFofuanrthermore,error(viatnhSaschtoie
2004).al.,te

Errorsinarmmovements

Forusingerrorsignalsinneuro-prostheticdevices,quantificationofthedeviationbe-
tweenintendedactionsandrealizedactionsisnecessary.Dependingonthetypeof
neuro-prostheticdevice(e.g.artificiallimbsoracommunicationdevicelikeaspeller)
thepossibleactionsaredifferent.Thisgeneratesanextradifficultyinfindingputative
sitesinthebrainwhichrepresentthecorrespondingerrorsbecausefordifferenttypes
ofprosthesestheerrorsignalmayberepresentedatdifferentpositionsinthebrain.
Wewillnowfocusourdiscussionofregardingsucherrorsignalsonprosthesesforarms.
Possibleactionsforanartificialarmaree.g.executionofreachingmovements.Atthe
endofanerroneousmovementwecanquantifytheerrorofsuchanactionforexample
bytheeuclideandistanceinexternalCartesiancoordinates.Actuallyitisnotpossible

178

Chapter5:StabilizingDecodingAgainstNon-stationaries

tosendbackdirectfeedback,abouttheactualconfigurationofaprostheticdevice,into
thecentralnervoussystem.Thustheerrorhastobeperceivedviathevisualsystem.
Sinceweneedboth,theintendedpositionandthevisuallyobservedactualposition,
candidatesforsuchanerrorsignalareneuronsinareaswhichcodecorrelationsbetween
plannedmovementsandobservationsofmovement.Putativebrainareasincludethe
superiorcolliculus(Stuphornetal.,2000),dorsalpre-motorarea(Boussaoudetal.,
1998;Musallametal.,2004;LeeandvanDonkelaar,2006;Fujiietal.,2000),ventral
pre-motorarea(Mushiakeetal.,1997),parietalreachregions(Batistaetal.,1999;
Musallametal.,2004),orparietalcortex(CulhamandValyear,2006).Anadvantage
ofthesesignalsourcescouldbetheirproximitytothemotorsystem:onecanexpect
thattheinfluenceofothervariablesonthefiringratelikee.g.thegeneralemotional
stateofthesubject,remainsmall.Inaddition,astrongactivationofcerebellumand
motorcortexcorrelatedtoexecutionerrorswasfound(Diedrichsenetal.,2005;Chen
2006).al.,te

Despitethisbodyofaccumulatedinformation,manyquestionsarestillunanswered,
especiallywheretofindsitesintheprimatebrainwhichrepresenterrorsofactions
andthecorrespondingneuronalcodingoferrorsignals.Furthermore,itisunclear
howtheseneuralcorrelatesoferrorswillchangewhenanartificialdevicereplacesthe
originalbodypart.

5.2NeuronalandComputationalBackground

5.2.3Braincomputerinterfaces

179

Abrain-computerinterface(BCI)isasystemthatidentifiestheuser’sintentionsfrom
suitablebrainsignals,see(LebedevandNicolelis,2006;Lebedevetal.,2005;Andersen
et2001;al.,Cu2004a;rranScahndwartz,Stokes,2004;2003).WolpaTwheetexal.,tracted2002;Wolpainformationwetal.,canb2000;eusedKueblertoceontatroll.,
BCIscomputerbypasstheapplicationsnormalorpathwelectromecaysbhetwanicaleenbraindevicesand(e.g.muwscles,heelcallohairswingorhandicappprostheses).ed
usersacquiringtocontrolinformationexternaabloutinfothermadtionesiredproactioncessingfromdevices.thebrainExperimenencompasstaltecnon-hniquesinvasivfoer
methodsandinvasivemethods.

Non-invasivemethods

Non-invasivemethodsbaseondataacquisitiontechnologywhichdoesnotrequireto
opentheskull.AprominentexampleistherecordingofEEGsignals(electroen-
cephalogram)(WolpawandMcFarland,2004a;Fabianietal.,2004),whichrecords
electricactivitygeneratedbythebrainfromthescalp.DataacquisitionviafMRI
(Weiskopfetal.,2004;Yooetal.,2004)fallsintothesamecategory,butduetosize
andcostsforfMRIsystems,itisnotapplicableforthedailyuse.Alsothetemporal
resolutionofafMRIscanislowandliesintheorderofmagnitudeofasecond.
OnemainadvantageofEEGrecordingsisthatitcanbetestedonhealthysubjects
becauseitdoesnotrequiresurgery.Theriskofusinganinvasivemethod,including
necessarysurgeryatheadandbrain,istypically(withtheactualstateoftechnology)
tooriskyformostusersofBCI.AnotheradvantageisthatEEG-basedBCIsareafford-
able,portableandeasytoinstall.ThemaindisadvantageofEEGliesinthelimited
informationbandwidthofEEGrecordings,duetoacoarsespatialresolutionanda
smallamplitudeofhigh-frequencysignals.Theselow-passfilteringpropertiesinspace
andtimeareaconsequenceofthepropertiesoffluid,tissueandskullbetweenthebrain
andthesensorsofanEEGsystem.
InEEGsignals,differenttypesofneuronalcorrelateswerefoundthatcorrespondto
differentfunctionsofthecentralnervoussystem.FourinterestingcorrelatesforBCI
are:applications

•Signalsgeneratedbythemotorsystem
Itgerwmasovshoemenwntsthat(Xutheetarl.,epresen2004;tAsationstolfiofetal.,motora2005))ctionsandinalstheotheEEGi(e.g.maginationslikefin-of
motoractions(WolpawandMcFarland,2004a;PfurtschellerandNeuper,2001)
canwithbelimubseadtocmputationsontrolcshowomputeralowerpapplicateions.rformanceItthhanastohbealtheynotedsubjthatects.pPeatienrfor-ts
mancedecreaseswiththetimeintervalsincethelimbwaslost(Blankertzetal.,
2006).

180

Chapter5:StabilizingDecodingAgainstNon-stationaries

300P•AnothercharacteristicofEEG,thatcanbeusedforsingletrialanalysisinthe
contextofBCIapplications,isthesocalled’P300’(or’P3’)(FarwellandDonchin,
1988;Paulusetal.,2002;Spenceretal.,2001;Heetal.,2001;Meinickeetal.,
2003;Suttonetal.,1965).TheP300isapositivecomponentoftheevent-related
potential(ERP)whichcanbedetectedapproximately300msafterstimulus
onset.Itshowscorrelationstosoundandlightstimuli.Thesecorrelationsdepend
onthesubject’sdegreeofuncertainty,itsattentionalstate,thefrequencyof
occurrenceandthenoveltyofthestimulus.ThistypeofBCIsignalisusedfor
communicationBCIslikespellersbutisnotveryhelpfulforreconstructionsof
intendedmovementsforprostheses.
•Steady-statevisuallyevokedpotentials
Ifflickeringvisualstimuliwithfrequenciesofapproximately8-15Hz(e.g.peri-
odicfull-fieldflashes)arepresented,EEGrecordingsrevealoscillatoryneuronal
activitieswithsimilarperiodicityinthevisualcortex.Thisphenomenoniscalled
Steady-StateVisuallyEvokedPotential(SSVEP)(M¨ulleretal.,1985)anditcan
bemodulatedbyvisualattention(Vannietal.,1999).
•Multifocalvisuallyevokedpotential(mfVEP)
Insteadofstimulatingtheretinawithafull-fieldflashandthenmeasuringthe
EEG-response,itisalsopossibletostimulatedifferentsegmentsoftheretina
independentlyfromeachother.Thismethodusesthecorrelationsbetweenthese
differentregionsofthevisualfieldandtheircontributiontothewholeEEG
signal.ThistechniqueistermedmultifocalVEP(Nobreetal.,2000;Teder-
Salejarvietal.,1999)andshowsacorrelationtothepositionofthefocusof
attention(Seipleetal.,2002).

Invasivemethods

WhileexclusivelyEEGonisausednimals(e.gpredominanmacaquetlyformonkheyumans,s)(LebinvaedevsiveandmethodNicolelis,sare2006;testedCarmenaalmost
etal.,2003;Musallametal.,2004;Schwartz,2004;Andersenetal.,2004a).However,
itwasrecentlyshown(Hochbergetal.,2006)thatthesemethodscanbeadaptedto
humanusers.ApatientwithanimplantedBCIwasshowntobeabletocontrolhis
TVanddrawsimplepictureswithhiscorticalsignals.Itwasestimatedthatitwill
still(Lebeneeddevand10-20yeNicolelisars,before2006).sucIthisinvunclearasiveBhoCIwlwillongbitewillusabletakeforunctiltlinicalheexpapplicationsectations
inscience-fictionliterature(likee.g.(Shirow,1991))canbemetandartificialbody
partswillfullyreplacelostones.
Sendinginformationintotheotherdirection,fromexternalsensorsintothecentral
nervoussystem,reachedclinicalapplication.Insomecasesitispossibletoplace
so-called’cochlearimplants’intotheinnerearsofdeafpatients.Asetofelectrodes
stimulatestheacousticnervedirectlybyemittingelectricimpulsessuchthattheuser

5.2NeuronalandComputationalBackground

181

Figure5.2:Prototypeofawirelessimplantbasedonourideas(PawelzikandRoter-
mund,2005)buildbytheFraunhoferIMS(Duisburg).Thisimplantreceivesenergy
viaelectromagneticwaves,measurestheelectricpotential,andsendsthisinformation
toareceiveroutsidetheskullviaelectromagneticwaves.PhotowastakenbyHarald
Rehling.

getsimplanantsandimpresvissiuonalofcorticalitsapcousrostticheseensvfallaironmenlsoitntot(GraheydencategoryandoCf’slarkens,ory2005).iRnformationetinal
transmittedintotheCNS’interfaces.Bothtypesofprosthesesarenotreadyyetfor
bJaevaingheriusetableal.,in2006)clinicalfunctionsapplicationsasa.byThepassofideaofrdegenerationetinaliofmplanthetsr(etina,Zrenner,likee2002;.g.
corticalretinitisppigmenrosthesestosa,aimbyatfeedingdirectlyavstimisualsulatingignalareasdirectlyintinhetothevisualopticcortex.nerve.TheVisualfirst
testsonhumanpatientsstartedrecently(Dagnelie,2006).

Anotherquestionishowanexternalartificialdeviceisintegratedintothe’bodyrep-
resentation’.Doesthebraininterpretaneuro-prostheticarmjustasatoolordoesit
reallyreplacethelostlimb(Irikietal.,1996;Maruishietal.,2004;Gurfinkeletal.,
it1991;isunclearMaravitawhatetal.,happ2003)?ensifTshisensoryquestionfeedbaiscknotfromanswtheeredlimcb(ompletelye.g.toucyet.hoErsppeciallyosition,
information)isgiventothebrain.Firstexperimentswithmonkeysperformingareach-
ingtaskwhilestimulatingprimarysomatosensorycortexbyelectricalimpulsesinorder
toBCI(induceFitzstaskimmonscuesetshoal.,wed2005).thatsuchafeedbackcanimprovetheperformanceofan

Theadvantageofinvasivemethodsliesinthehighspatialresolutionoveralarge
frequencyrangeresultinginalargeinformationbandwidth.Hightransferratesare
typicallyverydesirablebutinvasivemethods,likenotedearlier,havethedisadvan-
tagethattheyrequiretoopentheskullandtoplaceelectrodesontopof,orinsidethe
brain.Anotherdrawbackofinvasiverecordingsisthatcablesconnectingtheimplanted

182

Chapter5:StabilizingDecodingAgainstNon-stationaries

electrodestoexternaldeviceshavetoleavetheskullthroughanopening,whichisa
pWoetenproptialsosedourceanofideaitonfectionseliminateandathepoptenrobletialmhwazardithtohefgcablesettingbystucuksingwithtwirelesshecables.infor-
conmationtributionsandefonergycusonstransmissioncalability(Pinaswelizezik(asandsmallRotermaspund,ossible)2005)and(seeontheFig.n5um.2).berOurof
measurementsites(asmanyaspossible).Weexpectthatifweuseanaloginformation
processingandinformationcodingthenthiswillenableustobuildverysmallimplants
canwithbeaulosedwninumthebersofamercompegion.onenButts.nTothesonlymallerthestizeheisimpimplantortansatre,fortthehemorequestionimplanhowts
formansyendingimplantthescmanbeeasurediinstalled.nformationAlsotheoutaspoftectheofskulltheavarestrictsilablethenuinformationmberofbimplanandwidthts.
Ourapproachdealswiththisproblembyusingadifferentcarrierfrequencyforeach
ofparalleltheiandmplantsarealtime.ndthSusinceallotwshistotypetransmitofwirelesstheinformationinformationfrommantransmissionyimplanisatskieyn
technologyinthefield,severalothergroupsalsodevelopwirelessinvasivemeasure-
mentsystemsforneuronalactivities,e.g.(Mohsenietal.,2005;Knuttietal.,1979;
M2001;orizioIrazetoqal.,ui-Pa2005;storCehtienal.,and2005).Jaw,2005;Irazoqui-Pastoretal.,2003;Marteletal.,

Neuro-prostheticarms

Inthefollowing,Iwanttodiscussinmoredetailneuro-prostheticdevicesforarms.For
reconstructingintendedmovementsfromneuronalactivities,itishelpfultounderstand
howinformationaboutmovementsisexpressedinneuronalactivitiesofmotorcortex
cells.Afterintroducingthemaintheoriesregardingthistopic,Iwillcontinuepresenting
sometechniquestoextractinformationabouttheintendedmovementfrommeasured
data.

NeuronalcodingofmovementsFormovementsintwodimensions,differentstud-
iesinvestigatedtheresponsebehaviorofneuronsfromthemotorsystem(mainlymotor
cortex).Typicalparadigmsaree.g.center-outtasks.Therethesubjecthastomoveits
handfromastartmarker(attheorigin)toatarget.Thesetargetsarecircularlyaligned
aroundthestartmarker.MoranandSchwartz(MoranandSchwartz,1999b)suggested
thatthecorticalactivityf(t)independencyof(theabsolutevalueof)velocityvand
directionϑofamovingfingerevolvesas
f(v(t−τ),ϑ(t−τ))=v(t)(b0+b1sin(ϑ(t))+b2cos(ϑ(t))),(5.1)
withb0,b1,andb2asconstants,andτasthetypicaldelaybetweentheneuronal
activityofamovementandperformingthemovementitself.
In(Paninskietal.,2004)itwasproposedtodescribetheneuronalresponseforhand
movementsinfirstorderby
f(v(t),ϑ(t))=b0+b1v(t)cos(ϑ(t)−ϑPD),(5.2)

(5.2)

5.2NeuronalandComputationalBackground

183

thewherenϑeuronPDisshothews’itsspreferredtrongestdirection’response.oftheb0nandeuron.b1ϑarePDbothdenotesctonshetants.directionAsiwmilarhere
conclusioncanbefoundin(Georgopoulosetal.,1986;Georgopoulosetal.,1982)as
wellasitsextensionto3Dmovements(Georgopoulosetal.,1988).Inthispublication
theyarefocusedonthecorrelationbetweenthedirectionϑandtheneuronalactivity.
Themighitbenfluenceanovoftheversimplificationelocityisbecauseignored.aItsymmetricseemstahatndsbimoimpledaltcosine-uningshapedfunctionsfunctionswere
alsofoundinthiscontext(AmirikianandGeorgopoulos,2000).
Ithastobenotedthatifloadisappliedtoanarmthentuningchanges(Kalaska
etal.,1990).Experimentswithprimatesworkingagainst3Dforce-fields(Tairaetal.,
1996)suggestthattheconnectionbetweenneuronalactivitiesandaforce(withFas
magnitudeandϑFasdirectionoftheforce)canbedescribedby
f(F,ϑF)=b0+b1cos(ϑF−ϑF,PD)+b2F+b3,(5.3)
withϑF,PDasthepreferreddirectionsoftheneuronsforforcesandb0,b1,b2andb3
asconstants.
Itisimportanttounderstandthattherealsituationismorecomplexthandescribed
bytheseequations.Theneuronsaretypicallynottunedexclusivelytooneparam-
eter.AsheandGeorgopoulos(AsheandGeorgopoulos,1994)revealedthatmostof
thetestedneuronshaveastrongcorrelationbetweendirectionandmagnitudeofthe
lorespcityonse,andliktheedactualescribedpositionearlier.onIntheaddition,neuron’sthererespeonsexistsapropweakerties.erinfluenceAccelerationoftheavlsoe-
affectstheneuronalactivitiesofthosecells,butitsinfluenceislessintense.These
resterucltsmorrelationsotivatedwTithondoroveuronal(Tordoroespv,onses2000)aretojustpropaosetsecondaryhattheseeffect’high-levfromel’correlationsparame-
between’low-level’musclescommandsandtheirneuronalrepresentation.
Informationaboutarmmovementscanalsobeextractedfromepicorticalfieldpoten-
ettialsal.,(2005;MehringScetherbal.,erger2004;etal.,Ball2005).etal.,The2004)LFPsandcanloccalontainfieldpoteninformationtialsthat(LFP)is(Riccomple-kert
mentarytotheinformationcarriedbysimultaneouslymeasuredspiketrains(Mehring
2003).al.,te

DecodingofmovementsfromneuronalsignalsAftermeasuringneuronalac-
tivities,theuser’sintendedactionhastobeextractedfromthatdata.Thiscanbea
complexproblem.Often,thefirststepisto’clean’therecordeddataanddividethe
data,asbestaspossible,intoindependentinformationchannels.Thiscanbedoneby
e.g.applyingthecurrentsourcedensitymethodonEEGorLFPdataforreducing
spatialperturbations(MitzdorfandSinger,1979;Mitzdorf,1985)orbyperforming
’spike-sorting’(Lewicki,1998;Kreiteretal.,1989)foridentifyingandassigningthe
correspondingneuronidentifierstospikesrecordedfromextracellularelectrodes.The
ideabehindthisprocedureistoimprovepropertiesofthedataforenhancingtheex-
tractionoftherelevantfeatures.Thelaststepistoextractintendedactionsfromthe

184

Chapter5:StabilizingDecodingAgainstNon-stationaries

informativefeaturesonthedatausingsuitableestimationalgorithm(e.g.thesupport
vectormachine(CortesandVapnik,1995),seesection2.2.4).Theestimatormustbe
trainedonatraining-datasetbeforeitcanbeused.Whenusingsupportvectorma-
chines,thelearningprocesssegmentsdataspace,assigningdifferentpossibleactions
todifferentsegments.Afterlearning,theSVMcanbeusedtoselecttheintended
actionfromthefeatureswhilerecordingthedata.Theselectionofthepre-processing
steps,featureextractionmethods,andestimationalgorithmsdependshighlyonthe
statisticalpropertiesofthedataandhowtherequiredinformationishiddeninthe
recordedsignals.Thesepropertiesofthedatamaynotbestableovertimewhichcan
adaptationoftheestimationalgorithmnecessary.
Apopularmethodistoreconstructmovementsfromneuronalpopulationactivities
assumingcosine-tunedneurons(Georgopoulosetal.,1986;Georgopoulosetal.,1988).
ForthismethodoneassumesthatonemeasurestherateffromNdifferentneurons
andthatthetuningfunctionisgiven,foreachoftheseneurons,by
fi(ϑ)=b0,i+b1,icos(ϑ−ϑPD,i).
Thepopulationvector(DayanandAbbott,2001)isdefinedby
Nx^=fi−b0,i·(cos(ϑPD,i),sin(ϑPD,i)).
i=1b1,i
Withperfectcosinetuningwithuniformlydistributedpreferreddirections,thepop-
ulationvectorfordecodingdirectionisequivalenttomaximumlikelihoodestimation
(SerruyaandDonoghue,2004).Differencestothistypeoftuningcauseanon-optimal
decoding.Inparticular,testsshowedthattheuseofpopulationvectorsforrecon-
structingabsolutevaluesofvelocitymayleadtounbeautifulresults.Itisoftenbetter
tousetheOLE(SalinasandAbbott,1994),butthenumberofalternativeestimators
ishuge(SerruyaandDonoghue,2004;Andersenetal.,2004a).
Inaddition,astrategytoimprovetheperformanceofon-goingreconstructionsofin-
tendedmovementsistousepriorinformationabouttheexpectedlimbdynamicsor
aboutthetypicalrangesofthevariablestobeestimated.Forexample,itisreasonable
toassumethatarmmovementsarecontinuousinspace.Forincorporatingthisinfor-
mationintothesimulationitispossibletousee.g.sequentialMonte-Carlomethods
(oralsocalled’particlefilters’)(Arulampalametal.,2002)whichareanextensionof
Kalmanfilters(WelchandBishop,2004;Blacketal.,2003).Ortoincludeinformation
aboutthenextpossibleactionby(hidden)Markovmodels(Rabiner,1989).

5.3Themodelforthesimulations

Fordemonstratingthecapabilitiesofthenewadaptationprocedure,asituationis
simulated,wherea2Droboticarmhastobecontrolledbyanadaptivedecodingof

5.3Themodelforthesimulations

visual fieldmovementerror

executedintendedmovementmovement

roboticarm

= multielectrode or EEGrecording siteshandicappedvpersonpersonal computererror betweenEintended andexecuted movementKestimator forEverrorvintendedadaptationmovementalgorithmkPestimator formovement

v

185

ecuteFigurear5.3:eachScmovhematicemenvt.iewInotfhethebrainmoodel.ftAhehsubjandicappecte(lighdtpersongreen),isithentvendingelocitytovex-of
therecoinrdedtendedasasmopikveemencoutntisveectoncordedk.inkisthefedspikinestoofanneuronsalgorithmfrommrunningotoronacortexpbersonaleing
^incomputertendedm(yovelloemenw,t.righv^t)isutilizingsubsequenthetlypusedarameterstoconPtotrolamakeroboanticesarmtimation(yellov^w,ofleft)the
moveexecutingmentisthepemorceivveemendbt.ytheThersubjesultingect(tomismatcwhomhtheEobetwriginallyeeninintendedtendedamndovemenexecutedtis
known),andencodedbyneuronsinerror-monitoringbrainregions.Thiserrorsignal
Kissim(fromuwhicltaneouslyhanerecordedstimatebE^yofastheecondoriginalelectroederrorcarraanybdeeliveringcomputed).atotalAnspikeadaptationcount
aMalgorithmonte-Csiarlomilarprotorcedureforeinforcemenchtanginglearningthepscarametershemes(P^SuttonofathendmotionBarto,es1998)timator,emploseekys-
ingsignificantoimprotlyfveromtheapenormalrformanceprostheticoftheapplicprosthesisationbbyecauseminimizingofitsuseE.oTfahisniscnthemeernalediffersrror
source(additionalcomponentsandsignalsrequiredaredrawninred).

186

Chapter5:StabilizingDecodingAgainstNon-stationaries

brainsignals(seeFig.5.3).Itshouldbenotedthattheon-lineadaptationstrategy
isnotlimitedtothisspecialkindofdevice.Itcanalsobeexpectedtobeappliable
prostheseswheretheuserofthedevicecanevaluateitsperformanceanddelivera
suitableerrorsignalaboutthisevaluation.Insetupswherewecanapplytheproposed
on-lineadaptationschema,thesystemtypicallyconsistsoftwoparts,namelyone
’internal’,andone’external’part.Theinternalpartreferstotwoneuronalpopulations.
Onepopulationissupposedtoevokeactivitypatternsthatcorrelatewiththeintended
action.E.g.forarmmovements,wecanfindsuchneuronalpopulationsintheposterior
parietalreachregionorthedorsalpremotorcortex(Andersenetal.,2004a).Thesecond
populationofnervecells(fromanerror-monitoringbrainregion(Ridderinkhofetal.,
2004b))issupposedtodeliveranerrorsignalthatiscorrelatedtotheperformance
ormismatchbetweentheintendedandperceivedactionsoftheexternaldevice.The
activitypatternsofbothneuronalpopulationsarerecordedanddeliveredasinputto
art.pexternal’’theTheexternalpartofthemodelisthecontrollingsystemfortheprostheticdevice
andthedeviceitself.Itcomprisesthreealgorithmsforprocessingtheincomingdata.
Oneestimationalgorithmdecodestheintendedmovementsignal.Asecondestimator
interpretstheerrorsignal.Thethirdcomponentistheon-lineadaptationalgorithm.
Theestimatedintendedmovementisusedtocontroltheexternaldevice.Inparallel,
theerrorsignalisusedtoadaptthedecodingalgorithmoftheintendedmovement
gnal.is

5.3.1NeuralEncodingofIntendedMovement

Forthesimulationweuse,asthevariableofinterest,amovementvelocityvectorin2
ectorvhisTdimensions.v={vcos(ϕ),vsin(ϕ)}(5.4)
withdirectionϕandlengthv(rangingfrom0tovmax)isrepresentingtheintended
movementoftheroboticarm.Insection5.2.1and5.2.3wediscussedseveralpossible
sourcesforneuronalactivitycorrelatedwithintendedmovementsignals.Agoodsource
forsuchsignals(Schwartz,2004;Andersenetal.,2004a;Andersenetal.,2004b)are
neuronsintheposteriorparietalreachregionorthedorsalpremotorcortexwhich
haveshowntodisplayasubstantialvelocitytuningsuitabletobeexploitedinneural
prosthesis.Wewillapproximatetheshapeoftheneurons’tuningtothedirection
ofmovementbycosinefunctions(Schwartzetal.,2001).Dataabouttheintended
movementisvirtuallyrecordedfromNvneurons.Regardingtheabsolutevalueofthe
velocitywewilluseanapproximationbyalinearfunction(MoranandSchwartz,1999b;
Paninskietal.,2004;Blacketal.,2003).
Themeanfiringratefiofneuronicanthenbewrittenas
dmofi(v)=fioff+fi1+vcos(ϕ−ϕi).(5.5)
v2xma

(5.5)

5.3Themodelforthesimulations187
fioffdenotesthebaseline,andfimodthemaximummodulationofthefiringrate,re-
spectively.ϕiindicatesthepreferreddirectionoftheneuron.Inordertosimulatea
situationlikeitisencounteredinrealexperimentalsettings,werequireasubstantial
setofneuronstobenottunedatall.Furthermore,weassumethatalargefractionof
theNvneuronsarenearlysilent(thechoiceofthecorrespondingparameterswillbe
discussedlater).ThefiringoftheneuronsisdescribedbyaPoissonianprocess,which
isagoodapproximationofthein-vivofiringpropertiesofcorticalneurons(Shadlen
andNewsome,1998).Itfollowsthattheprobabilityptoobservekispikesinatime
windowoflengthTisgivenby
p(ki|fi(v,ϕ),T)=1(fi(v,ϕ)T)kiexp(−fi(v,ϕ)T).(5.6)
!kiForsimplicity,timeinthesesimulationisslicedintointervalsofdurationT.For
eachinterval,avectorofspikecountsk={k1,...,kNv}isdrawnfromthePoissonian
distributionandusedfortheestimationoftheintendedmovement.Therealparameter
vectordescribingneuronalencodingP={f1off,...,foffNv,f1mod,...,fNmovd,ϕ1,...,ϕNv}is
unknowntotheestimationalgorithm.Thetaskoftheadaptationprocedureistofind
parametersP^whichareascloseaspossibletotherealvaluesP.

5.3.2EstimationofIntendedMovement

Insection2.2.3andA.2.2wediscussedhowtoderivetheoptimallinearBayesianesti-
matorfortheminimummeansquarederrorbetweenestimatedvelocityv^andintended
velocityvfortuningfunctionswithlinearspeedmodulationandcosine-tuningforthe
directionofvelocityunderPoissonnoise.Giventhespikecountvectork={k1,...,kNv}
andthereal(unknown)setofparametersPforthevelocitycodingtuningcurves,the
estimatorisgivenbytheexpression
v^(k)=kj−Tj+fjoffDj,(5.7)
Nvfmod
2jwiththevectorcoefficientsDjdefinedby
NvDj=fimodvmax{cos(ϕi),sin(ϕi)}
imod−1
×Tfimodfjmodcos(ϕi−ϕj)+8δi,jfi+fioff(5.8)
22i,jSincethetuningpropertiesPoftheneuronsareunknown,wehavetousetheadapted,
approximateparametersetP^insteadfortheestimationwithEqs.(5.7),(5.8).Inthe
typicalcasewherebothparametersetsdonotmatch,wecanuseoneparticularlyuseful
featureofthelinearvelocityestimatorforcalculatingtheerrormadethroughusingthe

188

Chapter5:StabilizingDecodingAgainstNon-stationaries

wrongparameterset.IfweknowPandP^,thenitispossibletocomputeananalytical
solutionforthemeansquarederrorinclosedform,giventhatcodingtakesplacewith
parametersPwhiledecodingusestheparametersP^,
χ(P^|P)2=v2max−2NLi,xD^i,x+Li,yD^i,y
2i+Qi,j+M^iM^j−2MiM^jD^i,xD^j,x+D^i,yD^j,y.(5.9)
NN
jiUsingtheequationsfromtheappendixC.1,L,QandMarecomputedwiththe
realparametersPofthecodingsystem,whileiD^ii,jandM^iiarecalculatedwiththe
parametersetP^usedintheadaptationprocess.χ(P^|P)2canbeusedasato2olfor
thetestingsmallestadaptationmeansquaredalgorithmserrorduringwhichtcheiranbdeveeachievlopmenedt.Iundernatheddition,neuronalχ(P|P)noisegivbyes
anyconceivableadaptationprocess(seeFig.5.5).

5.3.3NeuralEncodingofPerceivedError

Theon-lineadaptationprocedurerequiresameasureableneuronalrepresentationof
theerrorbetweenintendedandperformedactionsoftheprothesis.Theneuralbasis
oferrorrepresentation,error-monitoringanderrorencodingisstillasubjectofintense
research(seeforexample(Diedrichsenetal.,2005;Ridderinkhofetal.,2004b;Schultz,
2004;Schultz,1998;vanVeenetal.,2004;HolroydandColes,2002;Yeungetal.,
2004a;Nakaharaetal.,2004;vanSchieetal.,2004;Musallametal.,2004)).In
section5.2.2webrieflydiscusseddifferenttypesofrewardanderrorrepresentationsin
theCNS.Actually,itisnotclearwhichregionsinthebraincanactassuitablesignal
sources.Thereforeitisessentialtospecifythekeypropertiesofanerrorsignalrequired
foradaptingtheparametersofneuralprostheses.Thiswillallowexperimentaliststo
searchspecificallyforasignalwiththeseproperties.
Theon-lineadaptationprocessisexclusivelybasedonabinarydecision:Asuitable
errorsignalshouldreliablyindicatewhetherapotentialnewsetofparametersforthe
estimatorofthemovement(orofanyotheraction)resultsinabetterperformance
(asmeasurede.g.bythedifferencebetweentheintendedvelocityandtheperformed
velocity,thedifferenceoftheintendedpositionandtherealposition,orthemismatch
betweenintendedandexecutedtrajectory).Theerrorsignalshouldherebyreflectthe
performanceaveragedoverarepresentativesetofsingleactions.Fortheadaptation
processanaveragedperformanceisnecessarybecausetheerrorvaluehastorepresent
thequalityoftheestimatorforallpossiblemovements.Usingonlytheerrorvalue
ofonesinglemovementfortheadaptionprocesscanleadtoanoptimisationofthe
decodingforthemostfrequentmovementswithsimultaneousdisintegrationofthe
qualityforlessfrequentactions.Eitherthesignalitselfisalreadyanaveragedquantity
whenbeingrecorded,ortheaveragingprocessmustbecarriedoutafterrecordingina
pre-processingstepbythecontroller.

5.3Themodelforthesimulations

189

Sincetheputativeerrorsignalisrecordedfromaneurophysiologicalsignalsourceitmay
alsosufferfromnon-stationarities.Foracorrectbinarydecisionduringadaptation,
itiscrucialthatthiserrorsignaldependsmonotonicallyontheperformanceofthe
prosthesis.Inanyotherrespect,thesignalisallowedtobenon-stationaryonatime
scaledecisionlargeristmade.hantIhetistimealsoscpaleoermittednwhicthhatthethepeerrorrformancesignalisisavdeelaragedyedatondthetheabctuallyinary
executedlimbmovement,aslongasthedelayissmallerthantheaveragingtime
interval.However,itisnotallowedthatmonotonicityreversesitssignovertime,
becausethiswouldthenleadtoerrormaximization.
Foerrorrthebetweenconcreteintendedimplemenandtation,executedletuvseloacityssumevectoanr.errorFsurthermoignalcre,odletingustheasssumequareda
error.linearItdepeshouldndencybebetwnotedeentthehatfitringheseassrateofumptionserror-monitoringarenotessenneuronstialforandthethessuccessquaredof
theadaptionprocess.
Indetail,thesimulationisbasedonthefollowingequations:Thedifferencebetween
theintendedmovementvanditsestimatev^isquantifiedbythesquarederrorE(v,v^)
givenby
E(v,v^)=(vx−^vx)2+(vy−^vy)2.(5.10)
Wewillassumethatrecordingsinanerror-monitoringbrainregionaremadefromNE
neuronswithsimilartuningfunctionsincreasingmonotonouslywithincreasingerror
E.Inparticular,letusassumethatthepopulationfiringratefEisgivenbyalinear
functionoftheerrorE,
fE(E)=Foff+FmodE,(5.11)
withoffsetFoffandmaximumfiringratemodulationFmod.Asforthemotorcortical
neurons,thepopulationfiringratefEisobservedasastochasticspikecountKdrawn
fromaPoissoniandistributionaccordingtoEq.(5.6).

dA5.3.4aptation

Duringadaptation,theparametersP^ofthemovementestimatorareoptimizedwith
respecttoalossfunction.Theultimategoalofanadaptationprocessistoreacha
globalminimumofthelossfunctionwithrespecttotheadaptedparameterset,which
inthissimulationisthecaseifP^=P.Ifthissetofparameterswasfoundthenthe
prosthesiswouldoperatewithmaximalperformance.
Intheory,awell-definedlossfunctionincorporatesthestatisticsoftheintendedmove-
ments,anerrormetrics(afunctionthatquantifiesthedifferencebetweenthereal
movementandtheestimatedmovementinascalarvalue)andconstraintsonthesys-
tem.Thisinformationcanthenbeusedtofindtheoptimalestimator.Inreality,the
lossfunctionmaybeunknowntoexternalobserversandthereforealsototheadap-
tationalgorithm.Thatisnotaproblemaslongastheperformanceoftheprosthesis

190

Chapter5:StabilizingDecodingAgainstNon-stationaries

Figure5.4:Schematicdiagramofthesimulations(fordetails,seethetext).An’inner
loop’executedeachtimeintervalTdrawsanintendedmovementvelocityfromadis-
andtribution,performstcomputesheesthetimationscorrespoftheondingvelospikciteycandountesorrorftshevignalselo.citTyheandeadaptationrrorneurons,algo-
rithmevaluatesdifferentparametersetsconstructedbyaMonte-Carloprocedure,by
averagingtheerrorsignaloverasuccessionofperiodsT.Hereby,itrealizesastochastic
gradientdescentontheerrorsignalE.

5.3Themodelforthesimulations

191

isrepresentedmonotonicallythroughtherecordederrorsignalandthereforeallowsto
findbettersetsofparameterswhileminimizingthevalueoftheerrorsignal(forthe
on-lineadaptationprocessitwasassumedthatalowerperformanceresultsinahigher
neuronalactivityoftheerror-codingneurons).Byperformingonlyabinarydecision,
wemaysacrificeadaptationspeedbutgainrobustnessagainstnon-stationaritiesinthe
ing.docerrorTherealizationoftheadaptationalgorithmembracestheon-goingestimationofv
andtheobservationofthecorrespondingerrorsignalK(thenumberofspikesfromthe
populationoferrorcodingneurons)ineachtimeintervalT,bychangingtheestimation
parametersP^onalargertimescale.Foranoverviewofthecompleteprocedure,see
thediagraminFig.5.4.
Theon-goingestimationofv,whichusestheequationsintroducedintheprevious
subsectionsandthemeasurementoftheerror-relatedneuronalactivityisexecutedin
aso-called’innerloop’.Withinthisloop,atfirstanintendedmovementvelocityvis
drawnfromauniformdistributiononaunitdisc(vmax=1).Theresulting,randomly
drawnspikecountvectorkfromthemodel’smotorcorticalareaisthenusedtoestimate
thevelocityv^withthecurrentparametersetP^.v^controlsthemovementoftherobotic
arm,leadingtoanerrorEwhichisencodedinapopulationspikecountKfromthe
modelerror-monitoringbrainregion.Thisloopisthenrepeated.
Theactualadaptationalgorithm(’outerloop’)thenproceedsasfollows:

0^1.moPriorvetmenotthevelocityadaptation,istinitializedhewithparameterrandomsetvPalues.fortheestimationofintended
0^^2.loTheopcisurrenexecutedtuparameterntilaspetre-dPefinedusedninumtheberiofnnerslpikooespiNsssethastobPeen,arecondrdedtheifronnerm
theerrorn0eurons.ThenecessarytimeforreceivingtheNsspikesisstoredina
variablet.
3.P^A0,nwewhoseparametersizeisvscaledectorinP^disependencerandomlyondratw0nf(forromadetails,smallseeneighthebaopprhooendixdaroundC.2).
^^4.TheagaincNurrenstspikesparameterhavebseenetPisrecordedsettofromP,athenderrortheinnnereurons.loopTisheelengthxecutedofunthetil
necessarytimeintervalforaccumulatingtheseNsspikesisstoredinavariable
.t5.Ift0>t,thenewparametersetP^isdiscardedandthealgorithmgoesbackto
steptwo.Ift0≤t,parametersetP^0willbereplacedbyP^,t0byt,andthe
algorithmwillproceedwithstepthree.

Inregulartimeintervals,re-evaluationofthecurrentlyusedparametersetP^0through
step2isnecessary.Thisisimportant,becauset0canbecomecorruptedduetonon-
stationarities,stochasticneurons,andsamplingnoise.

192

Chapter5:StabilizingDecodingAgainstNon-stationaries

ChoosingNsissubjecttoatradeoffbetweenprecisionandspeed.WithlowNs,adapta-
tionspeedwillbehigh,buttheperformanceisonlyaveragedoveranon-representative
setWithoffewhighmoNv,emenperfotsrmawhicnhcemisayevleaalduatotedthemaorecceptaanceccuratelyofabutlessofewpertimaaldparaaptationmeterstepsset.
socanccursbemwhenadeithenapotenfixedtialamoungaintofofantime.aSdaptionsaturationtepisoonftthehesaqualitmeyoorderftheofmparametersagnitude
asthenoiselevelontheevaluationoftheperformance.0Thisproblemcanbealleviated
byincreasingNs(e.g.dynamicallyindependenceont).
BesidesusingaMonte-Carlosampling,thereareseveralotherputativeadaptation
strategiesinreinforcementlearningproblems(foranoverview,see(SuttonandBarto,
is1998)),notknolikewne.g.ganalyticallyradient.-basNedumericalmethocds.Hoomputationwever,ofinathisNse-dimensttingitheonalobjgradienectivetfrequunctionires
stoequtheenntialoise,evthisaluationproofcedurethelotcakalesvaavriationeryolfongthetlosimesandfunctionthusinallrendersNgradiendimensions.t-basedDue
methogeneraldsamorepplicabilitvy,ulnerableaMontote-Carlonon-stalgoritationaritieshmw.asFcorhosenthesewrhiceashoisns,maorendrbobustecauseoagainstfits
non-stationaritiesandperformsafasteradaptation,butonlyintoarandomdirection.

5.3.5ChoiceofParameters

Recentprogressinmulti-electroderecordingtechniquesallowstorecordfromupto
64largelyindependentchannelsinthemotorcortexofamacaquemonkey(Schwartz,
2004).However,typicallyonlyafractionofallchannelswillcontainsignalswhichare
strongenoughandreliableenoughforuseinaneuralprosthesis(Mehringetal.,2003).
Wedecidedtochooseparameterswhichestablishasortofworst-casescenario,inorder
totestforthecomputationallimitsoftheadaptationandestimationalgorithms.
Forthesimulation,letusassumethatitispossibletorecordfromNv=64neurons.
Thetuningpropertiesoftheseneuronsaredrawnfromaprobabilitydistributionwhich
characterizesthetypicalvariationswheninsertingarealmultielectrodearrayintoa
brain:withprobabilityp=0.5,neuronsarechosentobenearlysilent(firingrates
below3Hz).Withprobabilityp=0.25,neuronsarebarelytuned,butfirewithrandom
frequenciesbetween0and50Hz(MoranandSchwartz,1999a),independentlyonthe
movementvelocity.Notethattakentogether,theseneuronsmakeupapercentageof
75%ofallneurons,andthattheyareofnousefordecodingtheangleandabsolute
velocityofthemovement.Withprobabilityp=0.25,neuronswillhavearandom
firingrateoffsetfioffandarandommodulationfimod,choosensuchthatfimod+fioffdoes
notexceed50Hz.Directiontuningpreferenceϕiisassignedrandomly,whichyieldsa
veryinhomogeneouscoverageofthemovementanglesbetween0and2π.Wefurther
setvmax=1andT=1sinthesimulations.
Whenrecordingfromerror-monitoringbrainregions,itmaybenecessarytorestrict
recordingtoonlyoneorafewchannels.Thereasonforthislimitationisthatmanyof

5.4ResultsfromtheSimulations

193

theseareasarelocateddeepbelowthebrain’ssurface(Ridderinkhofetal.,2004b).In
thisdeeperregions,recordingismoredifficult.Moreover,theliteratureonthesesignals
revealsthatwecanassumeasaworstcasethattypical,maximumfiringratesofthe
correspondingneuronsoftendonotexceed15Hz,withaspontaneousbaselinefiring
rateofabout5Hz(forexample,reward-relatedneuralactivity(TremblayandSchultz,
am2000)).eanmUsingaximumthesepfiringarametersrateof,a10simHzwasultaneoussimrulated,ecordingthusfromleadingNE=to5themneuronsaximwumith
populationfiringrateofFmod+Foff=NE·15Hz=75HzwithabaselineofFoff=25
Hz.Tralakpentrosthesisogether,mtustheworkrparameter’seliably:cshoiceynapticreflectsandtheneuronalharshcnoise,onditionsneuronsunderwwhichichhdaisplaneu-y
noresponseorhavenotuningproperties,andlargeoffsetsinthefiringratedueto
spontaneousactivity.

5.4ResultsfromtheSimulations

Themainmotivationforthischapterwastoshowthatneuralprosthesescansuc-
cessfullybeadaptedwithasuitable,internalerrorsignal,counteractingstrongnon-
stationaritiesinneuralcodingandsignalrecording.Consequently,ascenarioinwhich
acompletechangeofthetuningpropertiesofthemotorcorticalneuronstookplace
(Fig.5.5)wassimulatedfirst.Intheshownexample,beforet=0theadaptationof
P^hadreachedastationarystate,andthedecodingofasampleintendedmovement
trajectorywasalmostperfect(seeleftmostinsetinFig.5.5).Att=0,thetuningof
theneuronswaschangedcompletely,correspondingtoanentirere-initializationofthe
parametersPwithnew,randomvaluesdrawnfromthetypicaldistributionoftuning
values.ThepredictionerrorEinstantlyincreasedtovalueswhereasuccessfuldecoding
oftheintendedmovementwasimpossible(seecenterinsetinFig.5.5).However,with
ahalf-lifeperiodofabout90minutes,theMonte-Carloreinforcementlearningalgo-
rithmsucceededinre-adaptingP^andindecreasingE.Reconstructionofanintended
movementwasagainpossibleafteronlyacoupleofhoursofadapting^P(rightmost
5.5).Fig.ininsetInFig.5.6,atypicaldistributionoftuningcurvesforthe64neuronsdescribedbyP
isshown.Onlyfewneuronsaresubstantiallytuned,andareonthataccountuseful
indecodingtheintendedmovement.InFig.5.7,thecorrespondingestimatedtuning
curveswithadaptedparametersP^areshown.From64signalsavailable,theadaptation
algorithmhaschosenonly8substantiallytunedneuronsformovementestimation.In
comparison,Fig.5.7showsthattheoriginaltuningcurvesoftheselectedneurons,
depictedasthebluebars,donotdiffersignificantlyfromtheestimatedones.Inboth
figures(Figs.5.6,5.7)onlytuningfor8directionsisshown,likeitisobservedina
typical2D’center-out’reachingtaskwith8targets(Mehringetal.,2003).

194

Chapter5:StabilizingDecodingAgainstNon-stationaries

plete,Figurer5.5:adicalSimchangeulationindemonsneuronaltratingtuning:thattheredadaptationcurveisshopowsssitbleheevmeanenaftererrora<Ecom->
inestimatingintendedvelocity,averagedover1000trials(chancelevelis≈0.9)with
on-goingadaptationofestimationparametersduringthewholesimulationperiod.At
t=0thedirectionpreferences,spontaneousfiringrates,andthemaximumfiringfre-
quenciesofall64’recorded’neuronswithcosine-shapedtuningcurvesandlinearspeed
modulationarecompletelyre-initializedwithrandomvalues.Fort=0,tuningprop-
ethertiepsareerformanceheldconspriortantt.otAftehecrhsangeomehhasoursbeenofrestored.re-adaptingThetheesgreentimator’scurvepdisplaarametersysthe,
minimalmeanerrorachievableifthevelocityestimationwouldhavebeenmadewith
therealtuningparameters(whichareunknowntoanexternalobserver).Thethree
insetsvisualizetheprosthetic’sperformancepriortothetuningchange(left),immedi-
batelyeenaftersuccessfulthecifhangethee(censtimater),tedandtra24jectoryhours(inared)fterthecloscelyhangeappro(righximat).testheAdaptationintendedhas
blue).in(jectorytra

5.4ResultsfromtheSimulations

40f [Hz]20040f [Hz]20040f [Hz]20040f [Hz]20040f [Hz]20040f [Hz]20040f [Hz]20040f [Hz]2002ϕ022ϕ022ϕ022ϕ022ϕ022ϕ022ϕ022ϕ02

195

muFiguremve5lo.6:citDyvmaxirectionalandtvaryuningingofangletherϕes,fpoornsesi=to1,...anin,64tendedmotormnoveurons.ementTwheithmheighaxi-ts
oftheconditions.blueNbarsotethatindicateintthishemeanexample,nummbanerynofeuronsspikesbarelyrecordedrespinondtheatcall.orresponding

196

Chapter5:StabilizingDecodingAgainstNon-stationaries

40f [Hz]20040f [Hz]20040f [Hz]20040f [Hz]20040f [Hz]20040f [Hz]20040f [Hz]20040f [Hz]2002ϕ022ϕ022ϕ022ϕ022ϕ022ϕ022ϕ022ϕ02

Figure5.7:DirectionaltuningestimatedbytheMonte-Carloadaptationalgorithm.
Recordingsitesexcludedbecauseoflackinvariationofthespikecountaremarked
withredcrosses.Theheightoftheredbarsdenotestheexpectedfiringfrequencyat
thecorrespondingsite,ifamovementoforientationϕwithvelocityvmaxisintendedby
thesubject.Thebluebarsshowtherealtuningcurvesatthecorrespondingrecording
sites.

5.4ResultsfromtheSimulations

197

Figure5.8:Errorindecodingintendedvelocityundernon-stationaritiesintherecord-
ingStartingfromatthet=m0,otoronaveneurons.rageeveryBefore30tmin=0utes,toneuningrpropecordingertiessitewcehreangedhelditsconstantuningt.
sites.properties.Blue,mAteantheeerrorndofof500thestrialsimwulationith,atuningdaptationhasofcesthangedimationpcompletelyarametersforallduring64
attheftimeulltsim=0.ulationForperiocomparisd.Ron,ed,themeanminimerrorumofmean500errortrialsχw(ithP|Pa)2acdaptationhievasblewitcusinghedotheff
’correct’parametersPwiththeoptimallinearestimatorisshownasthegreenline.

198

Chapter5:StabilizingDecodingAgainstNon-stationaries

Furthermorewesimulatedascenariowhererecordingfromtheneuronsinmotorcortex
issubjectedtoanon-goingchangeintuning(Lebedevetal.,2005;Andersenetal.,
2004a).InthesimulationdepictedinFig.5.8,startingattimet=0,tuningproperties
oftheneuronschangerandomlyaccordingtoaPoissoniandistribution:onaverage,
every30minutesofsimulatedtimeonerecordedsitechangeditstuningproperties
completely.Thenewtuningfunctionatthecorrespondingsitewasrandomlydrawn
fromthesamedistributionfromwhichthetuningcurvesweredrawninitially.Fora
reinforcementlearningalgorithmthissituationistypicallymoredifficulttohandle:if
thechangeoftheencodingsystemisfasterthanthealgorithmisabletofindbetter
parametervaluesforthevelocityestimation,theresultingerrorwillincreasesubstan-
tially,insteadofdecreasing.Intheexample,however,thenon-stationaritiesinfluence
adaptationonlymarginally,andtheestimationerrorremainslow.Ifnoerrorsignal
wouldhavebeenused,asisthetypicalcaseincurrentprostheticapplications,decoding
ofanintendedvelocitywouldhavebeenimpossibleafteronlyafewhoursofsimulation
time.Inallthepresentedsimulations,theintendedvelocitiesarerestrictedtoamaximal
lengthofvmax.Thisrestrictionwasnotenforcedfortheestimatedvelocities,thusthe
errorofanestimationdonewithabadestimatorisnotbound.

SummaryandConclusion5.5

Inthecontextofneuralprosthetics,onecandistinguishbetweenthreeapproachesof
choosingandoptimizingtheparametersofanestimator:

arningeL•Learningstartswithaninitial,possiblyguessedorrandomlyselectedsetofpa-
rameters.Foroptimizingtheparameters,ataskwithwell-definedgoalisper-
taskformedisusedrepetoatedly.optimizeThethemismatcparametershbetwofeenthethispgoalrosthesis.andtAfterhetheoutcomesubjoectfthehas
exdoneample:sufficiencto-adaptationtraining(Ttrials,aylortheetpal.,arameters2002)).oftheprosthesisremainfixed(for
•ReRe--LLearnearning,ingwhichisalsotermed’OfflineAdaptation’,employsessentiallythe
sameproceduresaslearning.Theonlydifferenceisthataftersometimeinterval
ofusingtheprosthesis,thelearningprocedureisrepeated.Thisapproachcan
Hocopweevewithr,betnon-weenthestationaritiesspecialintrainingdatasessions,acquisitiontheandpinerformancetheneuralofctheodpingrosthesisitself.
degradescontinuously.
•(On-line)Adaptation
Incontrasttotheothermethods,(On-line)Adaptationisdoneduringthenormal

5.5ConclusionandSummary

199

useoftheprosthesis,anditdoesnotrequireataskwithpre-definedgoal.Asa
consequence,theactualperformanceoftheprosthesismustbeevaluatedunder
istheonlycponditionossibletifhatanthegoalindirect,ofaanndactiontask-iisndepunkenondenwnt.Fsignalorthissourcereasonfortheadaptationactual
performanceisavailable.Ifsuchasignalcanberecorded,itoffersthepossibility
toperformafreelychosentaskwhilesimultaneouslyoptimizingtheestimation
parameters.

Themostusefulsignalforthisexamplewoulddirectlyencodethedifferencebetween
thetargetlocationorspeedofanintendedandactuallyexecutedmovement.Fora
motorprosthesis,candidatesforsuchanerrorsignalcouldbeneuronsinornearbyareas
whichcodethecorrelationsofmotorcorticalplansofmovementandobservationsof
movement(putativeareasincludethesuperiorcolliculus(Stuphornetal.,2000),dorsal
pre-motorarea(Boussaoudetal.,1998;Musallametal.,2004;Fujiietal.,2000),
ventralpre-motorarea(Mushiakeetal.,1997)orparietalreachregions(Batistaetal.,
1999;Musallametal.,2004)).Anadvantageofthesetypesofsignalscouldbetheir
proximitytothemotorsystem:onecanexpectthattheinfluenceofothervariablesas
e.g.thegeneralemotionalstateofthesubjectonthefiringratesremainssmall.But
eveniftheerrorsignaldoesnotexclusivelyrepresenttheperformanceoftheprosthesis,
therequiredbinarydecisionintheadaptationprocesscanstillbemadebecauseall
influencesontheerror(likeforexamplethroughotherinternalmentalstates),which
areactingonalargertimescalethantheaveraginganddecisionprocess,willbefiltered
out.

However,thesebrainareasmaynotbeaccessibleforBrain-ComputerInterfacesusing
electrophysiologicalrecordingsbecauseoftheirpositiondeepinsidethebrain.Other
typesofsignalsrelatetotheperceiveddifferencebetweentheintentionofasubject,
andtheoutcomeofoneofitsactions(reward).Conveniently,suchasignalwouldalso
beusefulforothertypesofneuralprostheseswhichbarelyinvolvethemotorsystem.
Butmostbrainsignalsencodingrewardaredifferentialinnature:theydependheavily
ontheexpectedrewardwhichyieldsabaselinefortheactivitiesinthecorresponding
neurons(Ridderinkhofetal.,2004a;Kobayashietal.,2002;ShimaandTanji,1998;
PlattandGlimcher,1999;Itoetal.,2003;Shidaraetal.,1998).Dependingonthis
baseline,firingratesarepositivelyornegativelymodulated,iftherewardishigheror
lowerthanexpected,respectively(Tobleretal.,2005;Schultzetal.,1997;Matsumoto
etal.,2003;Fiorilloetal.,2003).Inprinciple,adifferentialsignalposesnoproblemfor
thealgorithm,butthesuccessofusingsuchanerrorsignaldependsonthedynamics
ofitsbaseline.Asexplainedearlier(seeSection5.3.3),thebaselinehastoremain
fairlyconstantwhilecalculatingtherepresentativeaveragederrorfortheoldandthe
newparametersetforthecomparison.Afurthercomplicationmaybethatreward
signalsarestronglyinfluencedbytheemotionalstateandmotivationofthesubjects
(Satohetal.,2003;Watanabeetal.,2002b;RoeschandOlson,2003;Tremblayand
Schultz,1999)orthecontext(TremblayandSchultz,1999),andmaythereforeprove
inappropriateforareliablesourceoferrorcoding.Theerrorsignalneedstodepend
monotonicallyontheperceivedmismatch,whichiscertainlynotthecaseforneurons

200

Chapter5:StabilizingDecodingAgainstNon-stationaries

whoseresponsesincreaseindependentlyofwhethertherewardislargerorsmallerthan
ticipated.anAntionEEGsourcefortheBrain-ComputererrorvInaluebterfaceycusingouldevserveneast-relatedanon-brainpelectrophotentialsysiological(Schalkietnforma-al.,
2000;Gehringetal.,1993;vanSchieetal.,2004)toextractsuchasignal.Alter-
2002)nativelyto,thegenerateuserancanerrorbestrainedignalw(hicWohlpaiswatransndmittedMcFatrland,hrough2004b;EEGBlankrecordingertztoettheal.,
adaptationsystemoftheprosthesis.
thefThroughuturetphedescriberformanceedoftheadaptationnsceuro-prostheticheme,fdailuresevicecanduringeffectivvolunelybtarye,unsedtoon-instructedincrease
weuse.akTheassuoriginsmptionsandwpereropmeadertiesaofndserrorignalssignalswitharelonwotyetinformationknowncontpreciselyent,wthereususeonlyd.
Thesimulationsdemonstratethatsuchhypotheticalerrorsignalssufficeforon-line-
adaptation,preservingandenhancingtheperformanceofaprosthesissubjectedtonon-
consideredstationarities.forSaevrealeralpimprostheticortantaissuespplication:havetobekeptinmindwhenthisapproachis

•Monotonicityisacrucialpropertyoftheerrorsignal,andachangeofitssign
isnotpermitted.Butsinceanerrorsignalwillberecordedfromapopulation
ofneuronsitisstillveryprobablethatthewholepopulationretainsthecorrect
signoftheoriginaldependency,evenifsingleneuronsoccasionallyviolatethe
ondition.cymonotonicit•Reinforcementlearningisdifficultandveryslowifmanyparametershavetobe
signalsadapted.fromThaus,mugreatltitudeeffortofavmustailablebeputrecordingintoansites.Itisappropriatenotthesbelectionestofstrategythe
touseasmanysignalsaspossible.Insteadthemostpredictivesignalsshouldbe
used.•Usingpriorknowledgemaysubstantiallyincreaseadaptationperformance.In-
corporatingthisknowledgeintomoresophisticatedreinforcementapproaches(as
e.g.in(XieandSeung,2004a;Murataetal.,2002))mayacceleratelearningand
mayleadtolowererrors.
•Onthecontrarytousingthesemoresophisticatedapproaches,theadaptation
algorithmwhichwasproposediscapabletoadaptthedecodingparameterset
evenwithoutexplicitinformationaboutthegeneralshapeoftheneuron’stuning
curves.However,itisimportanttoreducethenumberofparametersasfaras
possibletoavoida’curseofdimensionality’.
•Oneputativeproblemmaybecompetitionbetweentheadaptationprocessesin
thebrainandtheon-lineadaptationofthecontrolsystem.Itisunclearhow
thesetwoprocesseswillinteract.Thebehaviourofthisinteractionwillstrongly
dependonthetwotimescalesofadaptation.

5.5ConclusionandSummary

201

Ithastobenotedthatbeforetestingthisapproachinanexperimentalenvironment,

itisnecessarytoidentifyreliableneuralsourcesforanerrorsignalwiththerequired

properties.Onlythenitwouldbepossibletoapplyourproceduretoarealprosthetic

device.Atthesametime,thismoredetailedinformationabout

error

ignalss

could

eb

used

to

evimpro

het

design

of

algorithms.

het

attributes

of

hsuc

202

Chapter

:5

tabilizingS

ecoD

ding

Against

tationariesson-N

6Chapter

CandSummaryonclusion

Indatainteractingshortwithtime.ourdThisynamicenincomingvironmenstreamtofrequirestoinformationprocessishucomgebinedamounwtsithofintsensoryernal
states(e.g.memoriesorintentions)andresultsinactions.Thefundamentalmech-
anismsinformationbehindisthisstoredfastin,andinformationtransmittedprowcessingithsareequencesstillnotofactionunderstopoodten.tEialsvenishostillw
underheavydebate.Thisthesisprovidesnovelideastoaccomplishfastinformation
processing,tounderstandadaptivecodingstrategies,andtoperformunsupervisedon-
linelearningofnon-stationaryrepresentations.
Initsfirst,genuinelytheoreticalpart(chapter3-InformationProcessingSpikeby
Spike)thisthesisdevelopsanewconceptinthefieldoffastinformationprocessing
withsingleactionpotentials.Theframeworkisbasedonstochasticgenerativemodels
usingPoissonianspiketrainsasinput.Itiscapableofrealizingarbitraryinput-output
functions,updatinganinternalrepresentationwitheachincomingspike,forperform-
ingcomputationsasfastaspossible.
Leavingthosepurelytheoreticalconsiderationsbehind,thesecondpartofthisthesis
(chapter4-SelectiveVisualAttentioninV4/V1)investigatesprinciplesofadaptive
evneuralokedcobydingselectivinerealvisualdata,afottenctusingion,moonthedifiesqiuestionnformationhowproanincessingternalinthecorticalbrain.state,In
collaborationwithmonkeyneuro-physiologistswestudiedtheinfluenceofattention
onthediscriminabilityofvisualstimulithroughtheirneuronalcorrelatesrecordedas
epiduralfieldpotentials.
Thefinalpartinthisthesis(chapter5-StabilizingDecodingAgainstNon-Stationaries)
takesustowardsamedicalapplicationforextractinginternalbrainstatesfromneu-
ronalactivities.Forcontrollingprostheticdeviceswithbrainsignals,reliablealgo-
rithmsdesignedforwhicehastimatingllowstotheintstabiliseendedtheactionsestimaoftorapofaersonareneuro-prosthesisrequired.Aagainstmethoddwisrup-as
tionsfromnon-stationaritiesinthecharacteristicsofcodingtheintendedactions,and
fromchangesintheirrepresentationsinthemeasuredneuronalcorrelates.

InthefollowingIwillsummarizeanddiscusstheresultsfromthesethreeparts:

203

204

InformationProcessingSpikebySpike

Chapter6:SummaryandConclusion

Forfastinformationprocessing,theSpike-by-Spikeframeworkpresentedinthefirst
part3isapromisingalternativetoapproachesusingrank-ordercoding(Thorpeetal.,
2001).Incontrast,thisframeworkisveryrobustevenunderstrongnoise.Temporal
therinformation,elativendumbeescribingrofsthepiktesimingemittedbetbwyeenaptowospulationpikes,iofsineuronsgnoredisucompletelysedforiandnforma-only
tiontransmissionandinformationprocessing.Nevertheless,simulationsshowedthat
alreadythisnoisyinput,togetherwithacleveralgorithm,allowstocomputegiven
ettaskal.,s2005;rapidlyErnsandtaetal.,ccurately2004;P(Ernsawteetlzikal.,etal.,2007b;2004;PaPwawelzikelziketetal.,al.,2006a;2003).Rotermund

TheinformationisprocessedwithintheframeworkofagenerativemodelforPoisso-
nianspiketrains.Inthisapproachprobabilitiesforelicitingspikescanbeinterpreted
asneuronalactivitiesandsynapticweightsarerelatedtoconditionalprobabilitiesto
observespikesgivenaputativecause.ThestructureoftheSpike-by-Spikenetwork
allowstoextendtheapproachtoincludee.g.statisticalexpectations,task-relevantin-
formation,andattentionalmodulations.Twotypesoflearningstrategiesfortraining
theweightsweredeveloped:Abatchlearningrule,whichfirstcollectstheinformation
aboutmanyinputpatternsandthenupdatestheweights,andanon-linelearningrule
thatisformallyequivalenttoaHebbian-likelearningrulewithweightnormalization.
Theon-lineruleupdatestheweightsaftereachreceivedinputspike,butwithanup-
datestrengthbeingsomeordersofmagnitudessmallerthantheupdateoftheinternal
tation.represen

ThenewalgorithmwasabletolearnandcomputeBooleanfunctions.Thissuggests
tothatlearnthetnoetiwognorerkininputprinciplebits,canwhichwimplemenerentotrelevarbitraryantforrcomputations.ealizingaItdwasesiredalsoinput-able
output-relation,duringtrainingandthentoperformthecomputationwiththe(for
thishierarcBoolhicaleannfetwunctioorkn)withsmamorellestpothanssibleonelnetaywerork.thatItwcanasbeeveunsedpotossiblecomputetocoBonstructoleana
functions.Inadditiontodemonstratingtheself-consistencyofthebasicframework,
usingmaticallymore.Uthannfortunatelyonela,yiertisalloawsctuallytonotreduceclearthehowtotaltonlumearnbertheseofhmiddenulti-lanyoedresSpikdra-e-
boby-Spikjectiveenfetwunctionorksfortconsistenheintly.Ttermediatehisislayepartiallyrs.Onethecidea,onsequencethatstillofhasnottoknobewtingested,an
isreducetotapplyheawakdimensionalite-sleep-yaoftlgorithmhelearning(seesectionproblemAb.4.2).yusingAnotherSpike-by-approacSpikhecvouldersionsbeotof
BoTheseoleanpprimitivrimitivesesaret(likheneec.g.ombinedAND,toNANDimplemen,XtOR,theanddesiredOR)Boasoleanfpredefinedunction,molikedules.in
anymoderndigitalcomputer.

Spike-by-Spikenetworkswerealsotrainedtorecognisehandwrittendigitsfromthe
USPSdatabase.Withroughly500hiddennodes(andthecorrespondingsetofweights),
theSpike-by-Spikeinformationprocessingalgorithmwascapabletoexceedtherecog-

205

nitionrateofanearestneighbourclassifier(whoseperformanceprovidesagoodbench-
markforclassificationproblems)whichusesmorethan7000patterns.Alsothecom-
putationwasdonenearlyasrapidaswiththebenchmarkalgorithm.Inaddition,
therecognitionperformanceturnedouttoberelativelyrobustagainstpixelnoiseand
occlusionsinthedigitimages.

TheinfluenceSpikte-bhaty-aSpikneweaspikelgorithmwillhashaveaonfreetheaparameterctualinternalforthuningiddenrthesepresentrengthtation.ofIthen
theexample,wherethehandwrittendigitshadtoberecognised,itwasfoundthat
precisioncanbeincreasedifthisfreeparameterwasadaptedtothenumberofreceived
inputneuralspikmeces.hInanismsanlikeuro-ee.g.biologicalsynapticcondtextepression.thisscalingInanofacoudditionalldmasybetudybeitrewaolisuldedbbey
interestingtoinvestigatetheimprovements,whichcanbeobtainedbyusingbiological
plausibleadaptationprocessfor.
Furthermore,fordifferentvaluesoftheappearanceoftheweightschanges.Using
annaturaliorthogonalmagesabassiisfnputunctionforsettrainingforpixSpikele-bydata.-SpikTehunset,winorksthelimitresultedofforveryssmallmallin,
thenaturalimagesarerepresentedbylinearsuperpositionsofpixels.Largerleadto
spatiallyextendedweightstructuresandmoresparseactivationdistributionsoverthe
hiddennodes.Withtheextremevalueof=1(maximalsparsenessonthedistribution
ofhiddenactivities)awinner-takes-allnetworkisrealised,andonlyoneoftheweights
is-vusedaluesforbeteweenxplainingthesethetwoexinput.tremeThesibetuationhaviourhasofsttillhetonetbeworkinthoroughlydependencyanalysedoftandhe
ized.rotegcathevImplemenisualtsingystem,arwoealisticuldpallowre-protoccessingompareforthetheweighnaturaltsigeneratedmages,blikyetheitisSpikke-nobwny-Spikfrome
netUsingworkamworeithrmealisticeasuredprre-proeceptivecessingfieldwopropuldletertiesusefromxpecttheabmetterammalianmatchvbeisualtwseenystem.the
learnedweightsfordifferent-valuesandthemeasurements.
oftheAnotherintinernalrterpretationepresentofationtheand-valueregulateissthattheitinlengthtroofducesthea’timememory’scaleinforthethenetvwoaliditrk.y
moreSmallerfeaturesinduceorbaasislongerfunctions,memoryacthanhievinglargerhigher.aThisccuracyallowsinforthesrmallepresentotationincludeof
theinput.However,buildingthismoredetailedrepresentationsneedsalsomoreinput
spikesandthusmoretime.Thiscreatesatrade-offbetweenspeedandaccuracy.Speed
andaccuracycanalsobeimprovedbyincreasingthenumberofhiddenneurons.In
hythispoconthestisext,(Thorpcomparingeetal.,thes2001)peedwoofuldSpikbee-biny-teresSpikteing.netTwohisrkswillwithbeaThorpsube’sjectrankinfutureorder
.harceres

Anwhetherimpitortanctanbequestionimplemenfortmoeddelsbyexplainingbiologicallyrealisticinformationmpeans.roTcessingheshortinathenswCerNSforis
thisSomemodelproblemsis:Itisrelatednotstoure,thisbutquestioncouldbaereponon-ssible.localinteractionsbetweenthehidden

206

Chapter6:SummaryandConclusion

nodesintroducedbythedenominatorinthedynamicsofthehiddenneurons.The
requiredinformationcomprisinganalogvaluesofweightsandhiddenactivity,hasto
beexchangedbetweenallhiddennodes.Thisdatamustbetransmittedviaspikes.
Oneworkaroundistorepresentthelocalinformationofonehiddennodebythepopu-
lationactivityofmanyneuronsandusethespikecountasanestimatefortheproducts
betweenthehiddenvaluesandthecorrespondingweights.Simulationsshowedthat
thisispossible,butinformationprocessingisthennotinstantaneousandslowsdown
considerably,becausesomenumberofspikeshastobecollectedtoreliablyrepresent
alues.vnalogatheThenextconcernregardingbiologicalplausibilityisraisedbythenecessaryoperations
tobeperformedbythenetwork:multiplicationsanddivisivenormalisationofthehid-
dennodedynamics.Multiplicativeinteractionsanddivisivenormalisationmediated
byinhibitoryinteractionshavebeenobservedinrealneurons.Butitisnotclearif
these’implementations’arepurerealisationsoftherequiredcalculationprimitives,or
iftheyarecombinationsofe.g.additiveandsubtractiveinteractions.
Onefinalopenquestiontobementionedishowthehiddenvariablecouldbestoredin
arealbiologicalsystem.Thisquantityisnotallowedtochangebetweentwospikes,so
themembranepotentialofaneuronisnotagoodcandidate.Oneworkinghypothesis
isthatthehiddenvariablemaybeimplementedasonepartofamulti-compartmental
neuronmodel.Thisideawastestedinasimulationwherethespike-by-spikemodelwas
successfullycombinedwithaleakyintegrate-and-firemodel.Asecondapproachwould
betoallowfordecayinghiddenactivities.Preliminarytests(notshown)revealedthat
thisapproximationhasthepotentialfordeliveringapartialsolution.
However,forreallyfindingsatisfyinganswerstoalltheseremainingquestionsconcern-
ingthebiologicalplausibilityofthemodel,moreresearchhastobedone.

Allpresentedresultswereproducedforstaticinputdistributions.Anotherapproach
solvingpartiallythisproblemassumesthattheobservedinputwasgeneratedbyonly
onecause.ThemodelwasproposedbyDeneve(Deneve,2007a;Deneve,2007b)and
rithm,describesonethendeuroneynamicsstimatesofthethisexternalhiddenwMorldarkoavsamodhiddenel.AnMoarkn-olinevmodlearningel.Inthisalgorithm,algo-
allowingtotraintheparametersofthemodel(theweightsoftheconnectionsandthe
time-constants)forthissystemisavailable(MongilloandDeneve,2007).Acompar-
isonbetweenthismodelandthepresentedmodelwasdonebyErnstetal.(Ernst
etal.,2007a).Othermorecomplexmodelsweredeveloped(BeckandPouget,2007;
Rao,tions.F2004),orfwurtherhoseresearcimplemenhtactivities,ationinitnwilleuronalbeinmodterestingelsreqtouirevextenderysthetrongSpike-bapproy-xSpikima-e
modeltodynamicinputdistributions.ThiswouldallowtoapplytheSpike-by-Spike
networktomovies(e.g.forcompressingmovies)andcontrollingtasks(e.g.balancing
aninvertedpendulum).

SelectiveVisualAttentioninV4/V1

207

Forabetterunderstandingoftheneuronalmechanismsofselectivevisualattention,
thelaboratoryofProf.AndreasKreiter(UniversityofBremen)trainedtwomonkeys
toperformshape-trackingtaskswhilefieldpotentialsfromvisualcortexwererecorded.
Weanalysedthisdatabyclassifyingthepresentedshapesandconditionsofattention
usingestimationalgorithmsfrommachinelearning(SupportVectorMachines)(Roter-
mundetal.,2007a;Rotermundetal.,2007c;Rotermundetal.,2007b;Pawelziketal.,
2006d;Rotermundetal.,2006b;Rotermundaetal.,2005).Withthesemethods,we
gainedvaluableinsightintoinformation-carryingandbehaviourallyrelevantaspects
ofthemeasureddata,andidentifiedtheinfluenceofselectivevisualattentiononthe
patterns.yactivitneuronal

Herebyitwaspossibletoreachaclassificationperformanceofupto93%correctusing
thedatafromallelectrodesandofupto64.6%usingonlythedatafromacombination
ofelectrodesaboveareaV4.

represenAnalysingtationtheeconxptainerimenre-toaldataccurringshowactivitedythatthepatternsnteuronalhatacllowtorrelatesoidenofttifyhetheshapcor-e’s
respondingshapesorconditionsofattentionwithhighprecision.Afteraperiodof
training,theseclassificationscanbemadeinreal-time.Thisallowstousethesetypes
ysofesrevelectro-phealedysithatologicalitissposignalssibleas(withdatasaourceprecisforionBofrainupCotomputer93.7%)Intodterfacesecide.Thewithinanal-a
400mstimewindowwhetherashapeintheleftorrightvisualhemifieldwasattended
ornot.Thiscouldallowtoconstructaspellingdevicewhereeach400msabinary
decisiontowardsanintendedwordorlettercanbemade.Itispossibletousesucha
deviceascommunicationneuro-prosthesis,allowinghandicappedpersons(e.g.totally
locked-inpatients)tocommunicatewiththeirenvironment.Thisputativeapplication
risesdirectlyfurtherquestionsaboutusefulorusablepropertiesofselectivevisualat-
tentionandtheirneuronalcorrelates,e.g.:Howfastcanselectivevisualattentionbe
whichtransferredwasinfromtendedoneiobnfluencedjecttobyanother?thespatialHowisdensitpyorerformancethenuinmberinferringofobthejectsobbjeingect
t?presenWhilethefirstquestionisrelevantforfindingouthowmanydecisionstepscanbe
madewithinatimeinterval,thesecondquestionisinterestingfordetermininghow
onemanyhastodecisionsfind(outbits)intocanhboewmmanadeywithindifferentoneaoftttendablehosesteps.partsFotheravnswisualesringpacethelcanatterbe
segmented.Unfortunately,thesequestionscannotbeansweredwiththeexistingdata
andthusrequirenewexperimentsaddressingtheseotheraspectsofselectivevisual
ion.tattenAthemcainronditionesultooffathisttentthesisioniismprothat,vesinthepadditionerformancetobeingofawellclassifyingthediscriminableunderlyingstate,
shapes.Thisresultmotivatedustoinvestigatethechangesinthecorrespondingac-
tivitypatternsinducedbyattention,whichmakethepowercoefficientsfromdifferent
shapesmoredistinct.

Itwaspossibletoquantifythecontributionsoftwodifferentenhancingeffects.Itis

208

Chapter6:SummaryandConclusion

knownthatattentioncanimprovethesignal-to-noiseratio(SNR)ofspikecountdata
fromareaV4(throughamultiplicativeandstimulus-independentgainfororientation-
selectiveactivity)(McAdamsandMaunsell,1999b).Asignificantbutsmallimprove-
mentthroughanincreasedSNRwasalsofoundforourdata.Butthemajorenhance-
mentwasfoundtobecausedbyanattention-dependent,intricateincreaseofdifferences
betweentheresponsestodifferentstimuli.Thismechanismmakesthepowercoefficient
vectorsofdifferentshapesmoredistinctbymovingthecorrespondingdatacloudsin
thedataspaceoftheclassesmoreapartwhilemainlyconservingtheSNR.
Theunderlyingneuronalmechanismsforthisattention-dependentshape-selectivemod-
ificationarenotknown(adetaileddiscussionofputativemechanismscanbefoundin
chapter4).Forstudyingtheproblemfromatheoreticalpointofview,theideaarose
thatthisquestioncouldbestudiedwithinaframeworkwhereneuronalnetworkswith
aglobalcontrolparameter(representingtheconditionofattention)mayproducea
similarbehaviourintheir’neuronalcorrelates’.Thetargetwastoreproducethechar-
acteristicsofthecurves,describingthemeanpowercoefficientsindependencyofthe
frequencies,forthedatameasuredfromtheelectrodesoverareasV4.Inafirststep,
twotypesofmodelswereconstructed.Bothsystemswereabletomimictheseaveraged
powerwaveletspectra.Wesoughtforasimpleconnectionbetweentheparametersof
themodelswithpropertiesoftheshapes.Itwasnotpossibletofindsuchasimple
linkage.However,evenifwewouldhavefoundasimplecorrelationbetweentheprop-
ertiesoftheshapesandtheparametersofthemodel,itwouldnotbeforsurethatthis
connectionhassomemeaningorcantellussomethingabouttheunderlyingneuronal
isms.nahmecAsafutureproject,itwillbeinterestingtotakeonestepbackandfirsttrytoanswer
thisquestions:Isitpossibletoconstructabiologicalplausibleneuronalnetworkwith
aglobalcontrolparameter(whiche.g.switchesthestrengthoftheconnectionsbe-
tweentwostates)thatmapsitsinputmoredistinctlyontoitsoutputspacewhenthe
controlparameterissettoitsvaluecorrespondingtothestateofattention?Whatre-
strictionsontheinputandoutputspacearecompatibleforobtaininganenhancement
effect?Whatlimitationsontheimprovementareimposedifthecontrolmechanismis
constrainedtoe.g.amultiplicativegainoranoffsetaddedtotheweights?
Onecaveatofthemeasuredepi-duralfieldpotentialsisthattheyaresignalsthat
lostalotofinformationaboutmorelocalizedneuronalprocessesbecauseofspatially
averagingovertheactivitiesoflargeneuronalpopulations.Thuswecanonlyidentify
thatoscillatoryactivityintheγ-bandabove40Hzisimportant,butwecannotexclude
thatevenmoreinformationmaybecontainedinasynchronousactivitynotcaptured
bythisrecordingtechnique.Abetteralternativewouldbetoperformmorelocalized
intra-corticalrecordingsprovidingahigherspatialresolution.Thiswouldallowus
toconstrainthephenomenologytobereproducedbycomputationalmodelsinmore
detail,andthusincreasethechanceofidentifyingtheunderlyingneuronalmechanisms
studies.lationusiminAsreportedabove,theclassificationusingdatafromareaV4showedasignificant

209

enhancementinperformanceunderattention.ThedatafromareaV1didnotshowsuch
aneffect.AROCanalysisrevealedthattheclassificationperformanceofdiscriminating
oneclassoutoftwoclassesisalreadyveryhighinthenon-attendedcondition.These
highclassificationratesleavenoroomtoobserveanattention-inducedimprovement,
whileforareaV4thisenhancementispossible.Newexperimentsrevealedthatif
thetwosimultaneouslypresentedshapesaremovedmorecloselytogether(withstill
spatiallyseparatedrepresentationinV1butlargelyoverlappingrepresentationsinV4),
theactivityinV1synchronizeswiththeactivityinareaV4(Smiyukhaetal.,2006).
Howtheseinter-arealsynchronizationphenomenainfluencethediscriminabilityofthe
underlyingshapeshasstilltobeinvestigated.

StabilizingDecodingAgainstNon-Stationaries

Realizingthattheownbodylostpossibilitiesofinteractingwiththeenvironmentwould
beabigproblemforallofus,causingtheunderstandabledesiretocircumventthese
handicaps.Overtheyears,alargeresearchindustryforfindingsolutionsforthis
problemhasemerged,whichallowstocombinethesearchforabetterunderstanding
oftheCNSwithhelpingdisabledpersonstore-gainautonomyorqualityoflife.

deveActuallylopmenath.ugePnumrominenbertofdexamplesifferentatreypesretinaofimplanfunctionaltsandneuronalmotorcprosthesesortexparerostheses.under
Today,onlycochleaimplants(forre-gainingalimitedsenseofhearingbyreplacing
theinputtothehearingnervethroughartificiallyproducedelectricalimpulses)and
pacemakerforthebrain(fortreatinge.g.Parkinsondiseaseanddystoniabyelectrical
stimulations)havesuccessinmedicalapplications.Allothersystemsarenotready
.tye

Fromalltheavailableapproachesofcreatinganinterfaceforinterchanginginformation
betweentheCNSandexternaldevices,Ifocusedmyresearchontheaspectof’reading
thoughactivitiests’.thatHerecanthebegoalconistotrolledidentvifyolunreotarilyccubrringythepatternsuserionftthehesystem.correlatesoUfnsingeuronalthis
proinformationceduresarealloswsimilartotoconttrolhoseew.g.ecusedomputforaers.nalysingThetheaeppliedffectsiofanformationttention(chextractionapter
4)orforthesearchforoptimalcodingstrategies,givenasetofconstraints(Bethge
etetal.,al.,2002b;2003b;BBethgeethgeeettaal.,l.,2001).2003a;Bethgeetal.,2002a;Bethgeetal.,2002c;Bethge

Mostacquisitionresearcorhhersowatorewoextractrkingasonmucthehquestinformationionhowcontotentaincreasespossiblethebfromandwidththerofecordeddata
data.Ideviatedfromthesegoalsandtriedtounderstandhowitwouldbefeasibleto
byprotectnon-theistationariesnformationlikee.g.extractionmovpingerformelectroancedesasorclonghangesasposinsibletheragainsegardingtdegenerationneuronal
informationprocessingnetworks.Thisquestionwillincreaseitsimportancewhenthese
BrainComputerInterfacetechnologies,whicharecurrentlyunderdevelopment,are

210

Chapter6:SummaryandConclusion

transferredtolong-termmedicalapplications(PawelzikandRotermund,2005).

Mysolutionofcounteractingthesedegenerativeprocessesisbasedonanextraerror
signalextractedfromtheneuronalactivitiesoftheCNS,whichdescribestheper-
ceivedactualperformanceoftheprostheticdevice(Rotermundetal.,2006a;Pawelzik
etal.,2006c).Usingthisperformancemeasure,itispossibletoapplyareinforcement
typeofon-lineadaptationstrategythatallowstocompensatedifferenttypesofnon-
stationaries.Theerrorsignalisusedforguidingastochasticsearchintheparameter
spaceoftheestimationalgorithmtowardsanoptimalsetofparameters(thestatewith
theminimalerror).Thisadaptationprocedurehasalwaystokeeptrackoftheon-
goingchangesinducedbythenon-stationariesappliedtothe’real’parameters,used
forcodingtheinformationabouttheintendedactionintothecorrespondingcorrelates
ofneuronalactivities.

Fortestingtheidea,simulationsweremadewhereapopulationofmotorneuronscon-
trolledaroboticarmintwodimensions.Asecondpopulationofneuronsrepresented
theerrorbetweentheintendedmovementsoftheroboticarmandtheexecutedmove-
ments.Theenvisionedon-lineadaptationschemawasappliedtothissetting.The
presentedsimulationsshowedthatitispossibletoreversethedegenerationofper-
formancecausedbyacompleteresetofallparameters(likewecanexpectthrougha
movementofawholeelectrodearray)orrandomchangesinasubsetofparameters(if
thechangesarenotoccurringtofrequent).

Despitetheclearevidencefromthesimulationsthattheideaworks,transferringitto
arealexperimentalsetupisabsolutelynottrivial.Themajorproblemliesinthelack
ofasuitableerrorsignalsource.Asdiscussedinchapter5ingreatdetail,forarm
movementssomepotentialareasinthebrainareknown(Diedrichsenetal.,2005).But
evenfortheseregions,thecorrelatesoferrorvaluesintheneuronalactivityarenot
thoroughlyresearchedyet.Thus,findingsuitableareasinthebrainhastobethenext
step.Thenecessarypropertiesofthesignalareweak:Ithastodependmonotonically
ontheperceivedmismatchbetweentheintendedactionandtheexecutedaction.It
shouldbeasindependentaspossiblefromotherinfluences.Evennon-stationarieson
theerrorsignalareallowediftheireffectsareslowerthanthenon-stationariesdegen-
eratingtheperformanceoftheestimator.However,evenafterasuitablesourcefor
suchanerrorsignalhasbeenfound,recordingtheneuronalactivitybymulti-electrode
recordingsremainsasaproblem,whichhasstilltobesolved.
Dependingonnewknowledgeaboutthecodingofintendedactionsandtheircor-
respondingerrorsignals,theestimatorandtheon-lineadaptationalgorithmcanbe
optimised.Forthisdevelopmentitisalwayshelpfultoremembertwoaspects:Itis
importanttokeepthesystemaslowdimensionalaspossiblebecauseofthe’curseofdi-
mensionality’,andtheon-lineadaptationschemashoulduseaslessinformationaspos-
sibleforanupdatestep,otherwisethereaction-timetochangesduetonon-stationaries
wouldbeincreased.Inadditionitmaybehelpfultoincludetheadaptationprocesses
oftheCNSintotheon-lineadaptationschema.
Thus,itisimportanttogatherthroughexperimentsmoreinformationbeforeanext

steponthealgorithmicsidetowardarealmedicalapplicationcanbedone.

Takentogether,thisthesispresentedthreenewcontributions:

211

•Atheoreticalmethodofprocessinginformationspikebyspikeinafastand
efficiengeneratingtfPashion.oissonianThisspikstudyetrains,alsoforshopweedrformingthatitisfastandsufficienetfficientotuseinformationneurons,
cessing.pro

•Athatnewrendersmechanism,informationproducedaboutthroughdifferenstelvectivisualesvtimisualuli,attenrepresention,tedwaisnrγev-bealedand
oscillatoryactivityofneuronalpopulations,moredistinctforanexternalobserver
canandaplterrobablythenforteuronalheabrainctivitiytself.pItatternalsoinsahocwedomplexthatminannerternalandstatesitofthedemonstratedbrain
thatthepoweroftheγ-bandcontainssignificantinformationaboutvisually
perceivedshapes.

•Amethodforneuronal-prosthesescapableofprotectingestimatorsofintended
actions(likearmmovements)againstnon-stationaries,forthecostofanextra
errorsignaldescribingthemismatchbetweentheintendedandexecutedaction.

212

Chapter

:6

Summary

and

iononclusC

ndixeAppA

groundkBacAdditional

A.1Modelingofneurons

A.1.1HodgkinandHuxleymodel

FigureA.1:Schematicactionpotential.Ifthemembranepotentialpassesthethresh-
old,thecreationofanactionpotentialisinitiated.(Figurewasadaptedfrom
wikipedia.org)

Actionpotentialsthataremeasuredinrealexperimentsshowrelativecomplexfunc-
tionalcharacteristics(seeFig.2.1ortheschematicillustrationinFig.A.1).The

213

214

ChapterA:AdditionalBackground

moFiguredel.(FigureA.2:Iwllusastrationadaptedoffroman(DayaexemplarynandsoAbbolutiontt,of2001))theHodgkinandHuxley

voltage-clampexperimentsonthesquidgiantaxon(e.g.(ColeandCurtis,1939))
inscanmpiredimicHodthegkinexpanderimentalHuxleyd(ata.HoAndgkinactionandpoHuxleytent,ialis1952)thetoresudevlteoflopanaexmochdelangethatof
ionsbetweentheinsideandtheoutsideofthenervecell.Thedrivingforceforthese
exccell.hFoangesroinestaypeofdifferenceions,btethewpeenotenthetialccanoncenbetrationsquanotifiedfibonsytoheutsideNernstandinsideequationofthe
kbTconcentration(outside,ion-type)
E=zion-typeqlogconcentration(inside,ion-type)(A.1)
chwhereargeTofisthetheiton.empIeon-raturepumpsininKelvin,themkBemthebraneBareoltzmanncreatingconstanthesetandzdifferencesion-typeinqionthe
concentrations.Duringgeneratinganactionpotentialselectiveionchannelsregulate
theexchangethroughthemembrane.
tenThetialHoVd,wgkinhichaHuxleyccounmtsodelisdifferentbasedioniconacurrendifferentstialequationforthemembranepo-
CdtdV=Iext−IK−INa−Ileak.(A.2)
Iextrepresentsanexternalelectriccurrent,e.g.anelectricalcurrentinjectedintothe
neuron.IKandINaquantifythein-andoutflowofK-andNa-ionsintooroutofthe
cellthroughionchannels.Furthermore,IdescribesaleakofchargecarriersandC
denotesthecapacitanceofthemembrane.leakInthismodel,themembranepotentialis
onlydescribedbyasinglevariableV.Thesemodelsaretermedsinglecompartment
dels.mopFoorgtassiumeneratingionsasthewtellempasoralthedlyeakncamicsurrenoftaarenamoctiondeledpotenbytial,theftheolloflowingwofsoequationsdiumand
IK=gk(V,t)·(V−EK)
INa=gNa(V,t)·(V−ENa)
Ileak=gleak·(V−Eleak).

A.1Modelingofneurons215
EK,ENaandEleakarecalledequilibriumpotentials.gk(V,t)andgNa(V,t)represent
thedynamicsoftheconductanceoftheionchannels,whiletheconductancegleakis
constant.Usingconductancesformodelingtheionicflow,showsthattheHodgkin-
Huxleymodelisaconductance-basedmodel.Thedynamicsoftheconductancesofthe
sodiumandpotassiumchannelsaresimplifiedandapproximatedby
gk(V,t)=g¯K·n4
gNa(V,t)=g¯Na·m3·h.
g¯Kandg¯Naareconstantsandthefollowingdifferentialequationsdescribethetemporal
evolutionofn,mandh
dndt=αn(V)·(1−n)−βn(V)·n
dm=αm(V)·(1−m)−βm(V)·m
dtdtdh=αh(V)·(1−h)−βh(V)·h
whereα(V)andβ(V)areexponentialfunctions(DayanandAbbott,2001;Johnston
andWu,1997).OnetraceoftheHodgkin-HuxleymodelisshowninFig.A.2.

A.1.2McCullochandPittsneurons
wi,1

wi,2

iwi,3FigureA.3:SchematicsofaMcCullochandPittsneuron.Theoutputsfromother
neuronsjaremultipliedbyweightswi,jandthensummed.Thisvalueisthencompared
withathresholdϑi.Theoutputoftheneuroiis1ifthethresholdistransgressedand
otherwise.0

MoanddelsStevlikeense,.g.1971)thefoHocusdgkinonaexndplainingHuxleymtheodselhaporetheofCactiononnorpoStentevtensialsmoanddelt(omConnorimic
thedynamicsofa’real’neuronalnetwork.Butitisalsointerestingtouseonlythe
informationaboutthetimingofactionpotentialsforcomputations.Oneofthesimplest
neuronmodelsofthistypeistheMcCullochandPittsmodel(McCullochandPitts,
1943).Itcalculatesabinaryresponserfromaweightedsumofinputsfromother
neurons.Ineverytimestep,this’artificial’neuronsumstheweightedinputsrjwi,jand

216

ChapterA:AdditionalBackground

comparestheresulttoathreshold.Ifthetotalinputisabovethreshold,theneuronal
unitdelivers1(firing),otherwiseitdelivers0(notfiring).Thisbehaviorisdescribed
by

r(it+1)=Θj

wri,j(jtwithϑiasthresholdforneuroniand
Θ=1ifx≥0
otherwise0

)−ϑi

(A.3)beingtheHeavysidefunctionΘintroducinganon-linearity.Theweightsareallowed
tohavepositiveornegativevalues.Ifwi,jiszero,thenneuronjhasnoconnectionto
neuroni(Hertzetal.,1991).

A.2Propabilitybasedestimators
A.2Propabilitybasedestimators

A.2.1Minimummeansquarederrorestimator

217

parabolictuningfunctionfasymp(x)isoptimalmax(hereshownwithfmin=0).Forsmall
FigureA.4:(a)Intheasymptoticlimit(f·T→∞),itcanbeshownthatthe
fonmafx·maTx·itTcan(andbeafordvfanmin=tageous0),totheuseoptimaltuningthresholdfunctionsϑlikeliesdbisplaetwyeeendin2(b).andD1.epending
23

Inthefollowingexample(Bethgeetal.,2003a),itwasassumedthatoneneuronhasto
rateencodehasastombeuchtransmittedinformationbyaaschposannelsiblewabithoutPoisassocnnalaroiseintoitsEq.(2.1)rate.andFtuherthermore,informationthe
hastobedecodedbyaMMSEestimator.

FigureA.5:Comparisonoftheminimummeansquarederrorforthethestepfunc-
tion(dashedline)andtheparabolictuningfunction(solidline).Theχ2-axishasa
ale.cslogarithmic

Ifitisallowedtouseinfinitelongtimeintervals(T→∞)fortransmittingtheinfor-
mation,thentheoptimaltuningfunctionforsuchaneuroncanbedeterminedinthe

218

ChapterA:AdditionalBackground

asymptoticlimitbytheFisherinformation(foradiscussionsee(Bethgeetal.,2002c)).
Asresultweobtainanoptimaltuningfunctionwithparabolicshape(seeFig.A.4a)
fasymp(x)=(fmax−fmin)x+fmin2,(A.4)
wherefminrepresentstheminimumfiringrate(alsocalledbaseline)andfmaxdenotes
themaximumfiringrateoftheneuron.TheMMSEoftheasymptoticallyoptimal
tuningfunctionisgivenby
2asymp11∞1Γ0,2fmaxT(k+1)
χ[f]=3−2(√fmaxT)3k=0k!Γ0,fmaxT(k+21).(A.5)
InthecasewherefmaxTisfinite,itisnotguaranteedthatthetuningcurvewiththe
parabolicshapeisstilloptimal.ThisisespeciallytrueforthecasefmaxT→0,where
thePoissondistributionconvergestoaBernoullidistribution
p(k|fT)=(fT)k(1−fT)1−kfork∈{0,1}.
Forthiscase,itcanbeshownthattheoptimaltuningfunctioncanbedescribedby
fbinary(x)=fmin+(fmax−fmin)Θ(x−ϑfmin(fmaxT)),(A.6)
whereΘ(z)istheHeavisidefunction(Fig.A.4,right)
Θ(z)=01ifotherwisez>0.(A.7)
Forfmin=0,itcananalyticallybecalculatedthattheoptimalthresholdϑfmin(fmaxT)∈
[21,32]isafunctionoffmaxTgivenby
3−√8e−fmaxT+1
ϑ0(fmaxT)=1−4(1−e−fmaxT)(A.8)
Furthermore,wecancalculatethecorrespondingMMSE(fordetailssee(Bethgeetal.,
2003a)):2binary13ϑ02(fmaxT)
χ[f]=121−[(1−ϑ0(fmaxT))(1−e−fmaxT)]−1−1.(A.9)
NowweknowtheoptimaltuningfunctionsforverysmallandinfinitelargefmaxT,(see
Fig.A.5foracomparisonbetweenbothtuningfunctions)butthequestionremains
whattypeoftuningfunctionsareoptimalbetweenthetwoextremecasesandinwhich
rangeoffmaxTwecanusethesolutionsfortheextremcases.
Oneideafortacklingthisquestionistoconstructanewtuningfunctioncomposedout
ofbothtuningfunctions
⎨⎧fmin,0<x<α
fα,ramβp(x)=((√fmax−√fmin)xβ−−αα+√fmin)2,α<x<β.(A.10)
⎩fmax,β<x<1

A.2Propabilitybasedestimators

219

fα,ramβFigureptuningA.6:OfunctionptimalindepparametersendencyforotfhefmastxepT.tuningfunction(dashedline)andthe

Eq.(A.10)includes,withα=β,thestepfunctionandturnsintotheoptimalparabolic
simshapuelaftioorns,theitawaspsymptoticossiblelimit,tocawlculaithteαo=p0timalandαβand=1β.vaUluesingsfortextensivhefeα,ramβnpumericaltuning
function,function.inTFheig.Aresults.6.Ianretheshown,regionoftogetherfmaxTwith≈3thetheofptimalramptfunctionhresholdstforartsthetostepdiffer
thefromfamilythesoftepftrampuningwherefunctionα=ϑ0and−weandxceedsβt=heϑ0p+ew,wrformanceewereaofbtlheetosstephowanfunction.alyticaFllory
theexistenceofaphasetransition(Bethgeetal.,2003b).

FigureA.7:Thecriticalramp,fγcmaxT,wherethephasetransitionoccurs,isshownasa
functionofγforthefα,βtuningfunction.

Anotherinterestingresultinthiscontextis,thatwewereabletoanalysenumerically
anextendedversionofthisrestrainedframptuningfunction
⎧⎨fmi√n√√,0<x<α
fmax,β<x<1
fα,ramβp,γ(x)=⎩((fmax−fmin)βx−−αα+fmin)γ,α<x<β.(A.11)

220

ChapterA:AdditionalBackground

InFig.A.7,thedependencycofthefcmaxT,wherethephasetransitionoccurs,fromγ
isshown.TheminimumoffmaxT≈3canbefoundforγ=1.9.
Ananalysisoftheseandsomeothertuningfunctionsaswellasadetaildiscussion,
includingthephasetransition,canbefoundin(Bethgeetal.,2003a;Bethgeetal.,
2003b;Bethgeetal.,2002a).

A.2.2Linearminimummeansquarederrorestimator

Inchapter5thelinearminimummeansquarederrorestimatorisamajorpartofthe
presentedsimulations.Inthatsimulations,regardingthepossibilitiesofstabilising
aindepneeuro-prosndentlytheticfromdeacevhiceother,againstandnton-shattatheirtionariesspik,eswearewdillraaswnsumefromthatPoissonianneuronsfidis-re
p(k|tributionsx,P)aswithp(kmean|TΛv(ax,luesP))Λwhic=h{λ1due,...to,λtheN}.iWndepithetndencehesecconditions,onditionfweactorisescaninwriteto
N1kp(k|TΛ(x,P))=i=1ki!(Tλi(x,P))iexp(−Tλi(x,P)).(A.12)
UsingEq.(A.12)togetherwiththefirstandsecondmomentumofthePoissondistri-
butionp(k|x,P)ki=Tλi(x)
kp(k|x,P)ki2=Tλi(x)+T2λi(x)2,
ktheexpressionsforgiandQi,jsimplifyto
gi(x)=Tλi(x)(A.13)
Qi,j=ρ(x)T2λi(x)λj(x)+Tλi(x)δi,j.(A.14)
xIntuningthestudyfunctionabforoutntheeuronsneuro-fromtheprostheticmotorconcortex.trollingThesesystem,mowdelenwilleuronsselectwillλi(encoP)deas
intheirneuronalresponseinformationaboutvelocitiesformovements.Tobemore
withprecise,cosine-eachsofhapethedtsimuningulatedfortvehelocity-directioncodingϕ(Geneuronsorgopwilloulosbeemotal.,deledb1982;yaFuetfunctional.,
1997),andwithalineartuningfortheabsolutevelocityv,
dmoλi(P)=fi(v)=fioff+fi21+vvcos(ϕ−ϕi).(A.15)
xmaabInovthisetsphresholdecialcafise,mod,thetheparoffsameteter(spvonectortaneousP)comprisesfiringratesthefmioff,aximandumthedmeanirectionsfiringrateswith

A.2Propabilitybasedestimators

221

btheehshoigwnhestthatresptohenseOLEϕi.fAorftertheavleloongcitayisnalydticalefinedcbyalculation(seeC.1fordetails)itcan

with

Dj

==×

v^(k)=ki−Tfi+fioff
Nmod
2i

DiNLi(Qij−MjMi)−1
iNfimodvmax{cos(ϕi),sin(ϕi)}
iTfimodfjmodcos(ϕi−ϕj)+8δi,jfi+fioff.
mod−1
22i,j

(A.16)(A.17)

222

A.3Recurrentnetworks

ChapterA:AdditionalBackground

FigureA.8:Exampleofarecurrentnetwork.Theshadedcyclesrepresenthiddenunits.
theTheloowpeerncunits).yclecanInbgeoeneral,utputitisnneuronsot(e.g.necessarythethatupperarunits)ecurrentandnetwinputorkconneuronstains(e.g.all
threetypesofneurons(input,outputandhidden).

Incomparisonwithfeedwardnetworks,recurrentnetworkshave(Hertzetal.,1991;
Pearlmutter,1990)aloopstructure.Forthisclassofnetworks,itisallowedthat
neuronscanhaveconnectionstoallotherneuronsinthenetwork,evenaconnection
tothemselves.Fig.A.8showsanexampleofarecurrentnetworks.
Thenumberofdifferenttypesofrecurrentnetworkmodelsandtrainingmethodsis
large.Itisbeyondthescopeofthisthesistogivearepresentativeoverviewoverthis
kindofmodels.ForthisreasonIwillonlyintroduceHopfieldnetworksasanexample
forimplementingassociativememorythroughanattractordynamics,Boltzmannma-
chinesandliquidstatemachines.Theliquidstatemachineisanexampleforrecurrent
networkscapableofprocessingtimeseriesasinput.

A.3.1Hopfieldnetworks

TheHopfieldnetwork(Hopfield,1982;Hertzetal.,1991)canbeseenasamember
ofthefamilyofrecurrentnetworks.Hopfieldnetworksareexamplesforassociative
memorysystemsandprovideanimplementationoftheideaofstoringinformation
indynamicalattractors.Givenanincompleteinputpatternfromasetoftrained
patterns,thenetworkmaycompletetheinputpatternusingthestoredinformation.
Thisinformationaboutlearnedpatternsisstoredintheconnectionweightsandcanbe
retrievedbythedynamicsofupdatingneuronalactivities.TheneuronsareMcCulloch-

A.3Recurrentnetworks

223

Pittsunits(seesectionA.1.2).Theinput-outputrelationisgivenby
Si(t+1)=Θwi,jSj(t)−ϑi(A.18)
jwhereΘ(.)istheHeavysidefunction(Eq.(A.3))withthresholdϑiforunitiandwi,j
aretheweightsbetweenneuroniandj.TheHeavysidefunctionissometimesreplaced
bythesgn(.)function,whichresultsintheoutputstates{−1,+1}insteadof{0,+1}.
Anintroduction,whichIuseasasourceforthisdiscussion,canbefoundin(Hertz
1991).al.,teTheupdateoftheunitsinaHopfieldnetworkcanbedoneinasynchronousorasyn-
chronousfashion.Withthesynchronousupdatestrategy,allunitsareupdatedin
parallelatthesametimestep.Intheasynchronousupdate,onesingleunitisselected
randomlyandupdated.Alternatively,anupdatestepcanbeperformedonarandom
subsetfromtheneurons.Theasynchronousupdatemodeissaidtobemorebiologi-
callyrealisticbecausetheothertypeneedsakindofglobaltimeframeforcoordinating
ate.dputheForsymmetricalsetsofweightmatriceswi,j=wi,jitispossibletodefineacostfunction
(Lyapunovfunction).Theretrievablepatternscanbeunderstoodaslocalminimaof
thecostfunction,whichisgivenby
1E=−2wi,jSiSj.(A.19)
i,jSettingasubsetofunitstoinitialvaluesidenticaltoonetrainedpatternandallowing
thesystemtoevolveaccordingtoitsdynamics,itmaysettleinoneofthoselocal
minima(attractors).Dependingonthepropertiesofthesetofmemorizedpatterns,
similarityofthepatterns,andstoragecapacityoftheHopfieldnetwork,theresultscan
differfromthetrainedpatternsbecausetheemergenceofadditionalunwantedlocal
minima.Assumingthattheenergyofthesystemisminimizedbytheconfigurationwherethe
overlapbetweenthetrainingpatternsandtheactualstateoftheneuronsismaximized,
itispossibletoderivea(Hebb-like(Hebb,1949))learningrulefortheweights:
wi,j=N1OiμOjμ,
μwithNasthenumberofneuronsinthenetworkandOiμastheinputvaluesofpattern
μforneuroni.Oftentheweightswi,iaresettozerobecausethiscanhelptoreduce
spuriousstateswhichdonotcorrespondtoanyofthelearnedpatterns.
WecanextendtheHopfieldmodelbyreplacingthedeterministicstatesoftheunitsri
bystochasticunits.Nowtheactualstatescanbesampledusingtheprobabilities
1pA,i=1+e−2β(jwi,jSj−ϑi)(A.20)

224

forstateA,e.g.0and

ChapterA:AdditionalBackground

1pB,i=2β(jwi,jSj−ϑi)
e+1

(A.21)

forstateB,e.g.1.
βiscalled(inversepseudo-)temperature,whichgivesahinttothesimilarityofthe
isIsingonermoeasondelufsedorinthepstatisticalopularitymofechanicsHopfieldtotmohisdelstypbeofecauseitHopfieldallowsmotodel.intercThishangelink
computationaltoolsbetweentwoscientificdisciplines.Inthefollowing,wewillcontinue
withBoltzmannmachines,whicharerelatedtoHopfieldnetworks.

ineshmacBoltzmannA.3.2

InaHopfieldnetworkonlytheproperties<ri>and<ri·rj>canbechosenfreely
(<.>denotestheexpectationvalue)Higherordercorrelationscannotbetrained
whicindephesndenolvetly.thisBoltzmannproblem.macInhainesHaopfieldreannetwextensionorkallofstoneuronschasticcanHbeopfieldobservneted.worksIn
Boltzmannmachinenetworkssomeneuronsmaybehiddenandsomeneuronsmaybe
dedicatedtoinputandoutputtasks(Hertzetal.,1991).
Thestateipnput-orobabilitiesutputErelationqs.(A.20)fortheand(unitsA.21)Eare,q.(A.18),fortHhecostopfieldnetfunctionworksandEq.(A.19)Bandoltzmannthe
inmachines,Boltzmannthesmame.achines.ThedifferenceLearningliescaninbledearningoneeb.g.ecausebyomfvisibleinimizingandthehKiddenullbacunitsk-
Leiblerdivergencewithgradientdescentmethod.TheKullback-Leiblerdivergence
iscalculatedbetweentheintendedandactual,overthehiddenunitsmarginalized,
Boltzmanndistributions.WestartwithEq.(A.19)andsplitthevisibleandhidden
componentsofthenetwork:
H(v,h)=−1vTVv+1hTWh+vTJh.
22vandhdenotethevisibleandhiddenunitsinvectorialnotationwhilevTandhT
aretheirtransposedvectors.Vareweightsbetweenthevisiblenodes,Wbetween
thehiddenunitsandJbetweenthehiddenandvisibleneurons.Theprobabilistic
parametersofthesystemaredescribedbytheBoltzmanndistribution
p(v,h)=1Ze−βH(v,h).
TheKullback-Leiblerdivergence(seesection2.2.2)isgivenby
DKL(p0(v)pEQ(v))=p(h|v)p˜0(v)loghp(h|v)p˜0(v).
vhhpEQ(h,v)

A.3Recurrentnetworks

225

p0(v,h)=p(h|v)p˜0(v)isthedatadistribution,whilep˜0(v)istheempiricaldata
distributionwhichcanbeobservedatthevisibleunits.pEQ(h,v)istheequilibrium
distributionofallunitsafteriteratingthesystemforalongtime.Asresultofthe
gradientdescentmethodweobtainthefollowingupdaterules
ΔW=<hhT>0−<hhT>EQ
ΔV=<vvT>0−<vvT>EQ
ΔJ=<vhT>0−<vhT>EQ.(A.22)
va<.lue>s0anisd<.>calculatedEQiswithevaluatedsettingwthehilevallisibleunitsunitscantoevtheolveinfreelytended.iFornputtheandavoeragingutput
processthenetworkneedstobeinequilibrium.Takentogether,thislearningruleis
slowandthenumericalevaluationoftheaveragecorrelationscanbeinaccurateevenfor
smallβbecauseofstrongfluctuations.Otherlearningmethodshavebeendeveloped
toconcomptrastivenseadteivthesergenceeplroblemsearning,eand.g.rmeaneinforcemenfieldtmlethoearningds,r(seeecurrensectiontbac2k.4.4)(W-propagation,elling
andHinton,2002;Hertzetal.,1991).

A.3.3Liquidstatemachine

Anotherconceptofanalysingoftimeseriesistheliquidstatemachine(Maassetal.,
2002;Natschl¨ageretal.,2002).Theideabehindthisframeworkisbasedonamedium
forstoringthereceivedinputasperturbationsorechos(likerainfallingonaquietsea
causingripplesonthewater)thatdecayovertimebutondifferenttimescales.The
desiredinformationcanbereadoutbynetworkssimilartoperceptronsobservingthe
statesofthemedium.
Moreformally,theinputisdenotedbyu(t).ForaliquidstatemachineMtheinternal
statesaredenotedbyxM(t)andbasedonu(s)withs≤t.Theinternalstateis
ybtedlculacaxM(t)=(LMu)(t),
whereLMisthesocalled’liquidfilter’,whichmapstheseriesofinputsontointernal
states.Forextractingthedesiredinformationy(t)fromxM(t),memoryfreereadout
functionsfM(.)canbeused
y(t)=fM(xM(t)).
Thesereadoutmapsaretypicallytask-specificandforextractingdifferentproperties
ofthesignal,differentmapsareappliedtothesameliquidstate.Aliquidstatemachine
canbebuilte.g.outofrandomlyandrecurrentlyconnectedintegrate-and-fireneurons
withnon-linearsynapses.Itwassuspectedthatliquidstatemachinescanhave’uni-
versalpowerofcomputationswithfadingmemoryonfunctionsoftime’(Maassetal.,
2002).

226

A.4Generativemodels

ChapterA:AdditionalBackground

Generativemodelsaretypicallyusedformodelingprobabilitydistributionsofobserv-
ableobservedata.ddata.TheseExamplesgenerativforesucmohdelsmomadelsycareonhtainiddenhiddenMarkopvmoarametersdelsafndorHelmholtzdescribing
machines,whichwewilldiscussinsomemoredetail.

A.4.1HiddenMarkovmodel

FigureA.9:IllustrationofaMarkovmodelwiththreestates.Thetransitionprobabil-
itiesbetweenthestatesaredenotedbytheai,j’s.

HiddenMarkovmodels(HMM)areverypopularandoftenusedintechnicalapplica-
tions.Agoodreviewofthesemodelscanbefoundin(Rabiner,1989),whichIwill
useasbasisforthisoverview.HMMsarebasedonMarkovmodels(Fig.A.9).The
probabilityai,jforchoosingthenextstateSjdependsonlyontheactualstateSi,
ai,j=p(qt+1=Si|qt=Si).

A.4Generativemodels

227

FigureA.10:AhiddenMarkovprocess,usingthetransitionprobabilitiesai,j,gener-
atesasequenceofhiddenstatesqt,fromwhichobservationsdtaredrawnfromthe
.brobabilitiespconditional

S={S1,···,SN}representstheavailablestatesofthemodel,whileqtdenotesthe
realisationattimet.InHMMstheactualstateqtishidden.Thestatechanges
randomlyineachtimestepaccordingthetransitionprobabilitiesa.Inaddition,an
obsFig.ervA.10ablesymillustratesboldttishedrawholewnfporroeaccess.htimeThepsteprobabilitoutoyfdaistributionprobabilityfdordraistributionwingtheb.
observablesymbolsisdefinedby
bi(k)=p(dt=vk|qt=Si).
Patoeacssiblehstate.observaTheblersymealisationbolsaoreftVhe={observv1,···able,vMpro}.cessNotatalltimesymtboislsmarepresenybetedavabyilabledt.
Byintroducingtheinitialstatedistributionπi,theHMMisnowfullyspecified,
πi=p(q1=Si).
Allinformationaboutthemodelcanbesummarisedby
λ=({ai,j},{bi(k)},{πi}).

Typicallythreedifferenttypesofcomputationalproblemsareconsideredinthecontext
HMMs:fo

•toevaluatetheprobabilitythatasequenceofobservationswasgeneratedbya
hiddenMarkovmodel
•tofindtheoptimalsequenceofhiddenstatesthatexplainsasequenceofobser-
vations
•tolearnahiddenMarkovmodelfromdata
Forsolvingtheseproblems,itishelpfultoreducethecomputationalcostsbyintro-
ducingthesocalledforwardandbackwardvariables.Thesevariablesshowstructural

228ChapterA:AdditionalBackground
similaritiestotheforwardandbackward-propagatedcomponentsusedfortrainingfeed-
forwardnetworksbyusingthebackpropagation-learningrule.Theforwardvariables
ybefineddareNαt+1(j)=αt(i)ai,jbj(dt+1)
1=ifor1≤t≤T−1,startingwith
α1(j)=πjbj(d1).
Thisforwardvariablepropagatesforwardintime,incontrasttothebackwardvariable
βwhichisinitializedattheendofthesequence,βT(i)=1,andthenevaluated
backwardlyvia
Nβt(i)=ai,jbj(dt+1)βt+1(j)
1=ifor1≤t≤T−1.
Thefirstofthethreecomputationalproblemsistoevaluatetheprobabilitythata
sequenceofobservationswasgeneratedbyahiddenMarkovmodelwiththeparameters
λ.Forthiswehavetocalculate
p({d1,···,dT}|λ)=p({d1,···,dT}|{q1,···,qT},λ)·p({q1,···,qT}|λ)
q1,···,qT
withTp({d1,···,dT}|{q1,···,qT},λ)=p(dt|qt,λ)
1=t=bq1(d1)·bq2(d2)···bqT(dT)
andp({q1,···,qT}|λ)=πq1·aq1,q2···aq(T−1),qT.
Usingtheforwardvariables,theequationcanbewrittenas
Np({d1,···,dT}|λ)=αT(i).
1=iThesecondcomputationalproblemistofindtheoptimalsequenceofhiddenstatesthat
explainsasequenceofobservations,givenamodelλ.Asusual,’optimality’depends
ontheappliedcriteria.Onepossiblechoiceistoalwaysselectthemostprobablestate
q^tineachtimestep.Thiscanbecalculatedbyusing
γt(i)=p(qt=Si|{d1,···,dT},λ),

A.4Generativemodels
beingexpressedbytheforwardandbackwardvariablesas
αt(i)·βt(i)
γt(i)=jN=1αt(j)·βt(j).
Theindividuallymostlikelystatescanbedeterminedby
q^t=argmaxγt(i).
i

229

Anothercriteria,thatcanbeusedinthiscontext,isthemostlikelypaththroughthe
statesp({q1,···,qT}|{d1,···,dT},λ).ThiscanberealisedbytheViterbi-algorithm.
Thisalgorithmusesauxiliaryvariablesδt(j)andΨt(j),definedby
δt(j)=maxδ(t−1)(i)ai,ji=1,···,Nbj(dt)(A.23)
andΨt(j)=argmiaxδ(t−1)(i)ai,ji=1,···,N.(A.24)
Eq.(A.23)andEq.(A.24)areusedfor2≤t≤T.Theinitialvaluesaregivenby
δ1(j)=πjbj(d1)
andΨ1(j)=0.
Usingthisinformationwecandefineabacktrackingprocedure,beginningwith
q^T=argmiax({δT(i)}i=1,···,N)
andcontinuingby
q^t=Ψ(t+1)(^q(t+1))
untiltheestimateofthewholesequenceiscomplete.
ThelastclassofproblemsinHMMsislearning,whereitisnecessarytofindthe
’bfindingest’theseparametersparameteretλ=(sets{ati,jhe},{bi(’Baum-Wk)},{eπlci}h)mwithethod’respect(Baumtoetp(a{l.,d1,···1970),dcanT}|beλ)u.Fsed.or
Alternatively,gradientdescentmethodsandotherEMalgorithms(seesection2.4.3)
canbeapplied.ThetaskshowssimilaritiestolearninginBayesianbeliefnetworksas
.4.2.2sectionindiscussedOnewaytoextendHiddenMarkovmodelsarethesocalled’Hiddensemi-Markov’
mosequencesdels(Murphinsteady,of2002).singleThissymbmoolswdificationhilebeaingllowinsatomohiddendelsttate.hegenerationofsymbol

230

A.4.2Helmholtzmachines

ChapterA:AdditionalBackground

FigureA.11:Exampleofathree-layerHelmholtzmachine.(Illustrationadaptedfrom
(Dayanetal.,1995))

TrainingthenetworkforaBoltzmannmachineisacomplexproblemandsometimesnot
practical.Ifintra-layerconnectionsarenotnecessaryforperformingtheaimedinfor-
mationprocessing,thenitispossibletoswitchtoasimilarnetworkstructure,termed
Helmholtzmachine.Forthistypeofneuronalnetworkafeasibletrainingalgorithmis
ailable.va

TheHelmholtzmachine(Dayanetal.,1995;DayanandHinton,1996)isahierarchi-
calallonwseuronalself-supneetwrvisedorkwithlearning.binaryThisstocunitshasticareunitsconnectedincludingbytawostrainingetsofweighalgorithmts.Cthaton-
nectionconnectionsetssθetsinφtop-realisdoewnadirectionrecognitionimmplemenodelta(sgeeFig.enerativeA.11).modelUsingwhilethisthebnetwottom-orkaups
givengenerativienformation,modelcanwhilebeintherterpretedecognitionasrmodeleconstructingcanbetheintinputerpretedonasthebasismappingofthethe
inputonto’representations’(activitiesofneuronsinthehiddenlayer).

Nointra-layerconnectionsareallowed(otherwisethismachinewouldbeaBoltzmann
machine).Ifthenetworkisusedwithweightsfromthegenerativemodelortherecog-
nitionmodel,informationflowsinonlyonedirection.Thismeansthatthefollowing
layerintheinformationprocessinghierarchydoesnotprojectitsresultsbackwards.
Theconnectionscanbypassoneorseverallayers.

TheHelmholtzmachineisusedintwodifferentversions:ThedeterministicHelmholtz

A.4Generativemodels

231

stomacchhasticine(DayanHelmholtzetal.,mac1995),hine.uFsoringthesmeantochvaasticluesvforersion,theraecognitionlearningpromodcedureel,acndalledthe
’wtionakmoe-sleep’deandathelgorithmrepresencanbetationused.Ininferredthefwromakethephase,inputtheisnetusedworktoiupsudatesedithenrweighecogni-ts
ofthegenerativemodel.Duringthesecond,’sleep’phase,thenetworkproduces(fan-
tasizes)representationsandinputs,withoutexternalinput.Thisinformationisused
toupdatetheweightsusedbytherecognitionmodel.Theupdaterule(local’delta
rule’)issimilartoEq.(A.22).
Fweorigshtomseetseθxa.UmplessingittheisgpoenerativssibleetomousedelEpartMoofnaloHg(p(elmholtzDatam|acθ))hine,forwecfindingancalculatesuitable
thelogprobabilityofgeneratinganexample’Data’,whichisgivenby
log(p(Data|θ))=logp(Data,α|θ)
α=Pα(Data)log(p(Data,α|θ))−Pα(Data)log(Pα(Data))
αα(A.25)canwithbePαc(Dataomplex)=andp(maαy|beData,notθ).decompIngosedeneral,intothepropoductssteriorofpsimplerrobabilitdyPα(istributions.Data)
Thiscanmakethecomputationalevaluationdifficult.DayanandHinton(Dayanand
Hinton,1996)suggesttouseQα(Data)instead,whichisanarbitraryprobability
distributionthatneednottodependonθ.UsingthisdistributioninEq.(A.25)yields
log(p(Data|θ))=Qα(Data)log(p(Data,α|θ))−Qα(Data)log(Qα(Data))
αα+DKL(Q(Data)P(Data))
=−F(Data;θ,Q)+DKL(Q(Data)P(Data))(A.26)

withQα(Data)
DKL(Q(Data)P(Data))=Qα(Data)logPα(Data).
αEq.(A.26)isespeciallyinterestingbecauseitwasshown(NealandHinton,1999)that
F(Data;θ,Q)isminimised,whereQα(Data)equalsPα(Data)andDKL(Q(Data)P(Data))=
.0FortheHelmholtzmachinemanyextensionswereproposed,see(DayanandHinton,
1996)foradetaileddiscussion.

232

Chapter

:A

dditionalA

kgroundacB

BndixeApp

Informationprocessingspikeby
espik

B.1PatternPre-Processing

Beforeusingpatternsasinputforthespike-by-spikealgorithm,itisnecessarytoconvert
therawinputdataintosuitableprobabilitydistributions.

Toavoidnegativefiringrates,therawimageorbitpatternsBμwithNcomponents
eachwerepre-processedtoformthepositiveinputpatternsBμ+finallyappliedtothe
network.Inafirststep,thebμ,s,s=1,...,Nwerenormalizedbysubtractingthe
Nindividualmeanvalue<bμ,s>=1/Ns=1bμ,s,
b˜μ,s=bμ,s−<bμ,s>.(B.1)

(B.1)

Then,theNcomponentswereduplicatedandassignedpairwisetotheevenanduneven
inputnodepairs,yieldingS=2Nnon-negativecomponentsbμ,s+accordingtothe
snexpressio

˜˜bμ,2s−1+=+bμ,sforbμ,s>0
otherwise0˜bμ,2s+=0forbμ,s>0.
−b˜μ,sotherwise

(B.2)(B.3)

Thispre-processingismotivatedbythepropertiesofearlyvisualprocessinginthe
sembrain:blesthesplittinganalystheismofean-visvaualluestimcorrectedulibyon-inputainndtooff-cellsnegativeinandtheploateralsitivevaluesgeniculatere-
nucleus(LGN).

233

234ChapterB:Informationprocessingspikebyspike
B.2TrainingProcedures

Forthetrainingprocedure(seeFig.3.2),theSinputnodesaresplitintotwosetsof
sizesSp(indexedbys=1,...,Sp)andSc(indexedbys=Sp+1,...,M=Sp+Sc).
intheCorresptrainingondinglyset,tchereomprisesalsoaexistnon-twnoegativsetseofcinputonditionalpatternpBtr+robabilitiestogetherp(s|iwith).Eitsacchitorrectem
μclassificationcμtr.ThepatternBμtr+isappliedtothefirstsetofSpinputnodes,while
trtrPtheactternorrectandcclassificationlassificationciμnputsactivaaretesinthecadditionμ-thwnoeighdeotedftbheyasecondfactorsetλ.oIftcinputontrolsnodes.the
strengthofthe’input’and’output’argumentsthroughthenumberofspikesandthus
balancesthecombinationoftwostreamsofinformationduringtrainingthenetwork.
Usingtheseassignments,thefinaltrainingscenesVμtraregivenbytheexpressions
Spvμ,trs=λbμ,trsbμ,trsfors∈[1,Sp](B.4)
1=sandvμ,trs=(1−λ)δs−Sp,cμtrfors∈[Sp+1,Sp+Sc].(B.5)
trh(iDuring)andp(training,s|i)arespikuepsdareateddrawithwnrtheandomlySbS-onlinefromorVμSbS-batcaccordinghatolgorithms.Eq.(3.2),whileboth

B.3ClassificationandComputationProcedures

Fortheclassificationrun,inputscenesVμtsarecomposedsolelyofthepre-processed
patternsBμts+viavμ,tss=bμ,tss/sSp=1bμ,tss.(seeFig.3.3).Thefirstsetofconditional
probabilitiesp(s|i)fromthelearningprocedurewiths=1,...,Spisre-normalized,
yieldingthepatternweights
Spp∗(s|i)=p(s|i)/p(s|i).(B.6)
1=sForeachpresentedpatternorsceneμ,nowonlytheinternalstateshμ(i)butnot
theweightsp(s|i)areupdatedusingEq.(3.20).TheremainingScweightvectorsare
normalized,yieldingtheclassificationweights
^p(c|i)=Scp(c+Sp|i)forc=1,...,Sc.(B.7)
c=1p(c+Sp|i)
Fromh(i)andp^(c|i),quantitiesqμ(c)=iH=1^p(c|i)hμ(i)aretscomputed.Theseqμ(c)
denotetheprobabilitiesforeachoftheMtstestpatternsVμtobelongtotheclassc.
Fromtheqμ(c),apredictionfortheclassification^cμtiscomputedandupdatedwithin
eachtimestept,
c^μt=argmaxcqμt(c).(B.8)

B.4DetailsandParametersfortheComputationofBooleanFunctions235

ThemeanclassificationerroretoverallMtstestpatternsisfinallycomputedas
Mtste=1/Mtsδcμts,c^μt.(B.9)
1=μ

B.4DetailsandParametersfortheComputation
ofBooleanFunctions

Fromallpossible232Booleanfunctionsof5inputand1outputbits,Fo=100Boolean
functionswerechosenrandomlyfortheSbSnetwork(withon-linelearning).Bysplit-
tingtheinputsargumentsintoon-offchannelsasdescribedinsectionB.1,wewill
thereforehaveS=Sp+Sc=12inputnodes.
DuringSbSwithon-linelearning,the32patternwerepresentedsequentiallywitha
numberof4620spikesperpatternandwithλ=5/6.Thisprocedurewasrepeated
Z=20times.Beforeeachlearningstep,andforeachdifferentpatternμ,theh’s
wereresettoaflatpriorofhμ0=1/H.Beforethefirstlearningstep,thepz(s|i)were
uniformlyinitializedwithp0(s|i)=1/S.Afterlearning,theh’swereagainresettohμ0,
andeachinputpatternwaspresentedforT=1000spikesfortestingtheclassification
performance.Learningconstantswerechosenas=1000/1001andγ=0.01/1.01,
otherwise.statedunless

B.5DetailsandParametersfortheClassification
DigitsHandwrittenof

coThentaidnsataMbtrase=of7291handwrittentrainingdandigitsMftsro=mt2007hetestUnitedpaStatestterns,PeaochstalonecServiceon(sistingUSPS)of
N=16×16greyscalevalues(pixels)rangingfrom−1(black)to1(white).According
tothedatapreprocessingdescribedbefore,thenetworkthencomprisesSp=512
patterninputchannels,andSc=10classificationinputchannelsforthetraining.
UsingtheSbSalgorithmwithbatchlearning,onelearningstepincludedtheparallel
andpresenZ=tation20ofsimilaralltotrainingthepropatternscedureweithmploλye=dw0.5,iththe=Bo0.1,Tolean=f4620unctions.,Δ=Du4620ring,
classification,eachdigitpatternwaspresentedforT=10000spikes.
Inordertotestfortherobustnessofthealgorithms,therecognitionwasmademore
difficultintwodifferentways.Thefirstchallengewasintroducedbyanocclusionofa
vadigitriablepattern,numbperixelofvroawluessorofwcolumnsholero.wsSandtartingcolumnsfromthewerecensettertocanolumnsintorermediaterowsofvaluethe

236

ChapterB:Informationprocessingspikebyspike

of0.Forexample,averticalocclusionof6columnsonapatternofsize16×16was
orealizedcclusionbyof4roaffectingwswathesreapixellizedvabyluesofsettingcaolumnsllpixel6tova11luesinofallrorowwss71toto1016of.Aallhocrizoolumnsntal
1to16toavalueof0.Thesecondchallengewasintroducedbysuperimposingrandom
Bμnoiserndownetrehedradigitwnfrompattern.aInduniformetail,foredistributionachdigitbettowbeene−1noisified,and1.Brandomypmixeleanvsaofluesa
parameterη∈[0,1],theneworiginalpatternwaslinearlycombinedwiththenoisepattern
toanewinputpatternbμ,saccordingtotherule

bμ,nesw=(1−η)bμ,s+ηbμ,rnsd.

(B.10)

Theparameterηregulatestheamountofnoiseontheoriginalpattern.η=0represents
thenoise-freeoriginalpattern,andη=1denotesthecasewhentheoriginalpattern
hasbeenfullyreplacedbythenoisepattern.

CndixeApp

Stabilizingdecodingagainst
non-stationaries

C.1Theestimatorforthevelocity

Thesimulationsshowninchapter5useoptimallinearBayesianestimatorsforinferring
veloestimatorscitiesofcanmobveedmenerivtsedfromfortspikuning-e-fcoununctits.onsInwtithhecfolloosinewing,tuningweforwilltheseehodirectionwthisof
Ftheumovrthermore,ementaaPndolinearissonianrtuningandomforptrohecesslengthisaofssumedthevforectordrawingdescribingthectheorrespmovemenondingt.
ts.uncoe-spikForthecomputationoftheoptimallinearestimatoraccordingtoEq.(2.33)

Dj=(Li−MiZ)R−1
i

i,j,R={Qi,j−MMi}j=i,j,...1,N,Mi=xρ(x)gi(x),Li=xρ(x)gi(x)x,
Z=xρ(x)x,Qi,j=xρ(x)p(k|x,P))kikj,
kandgi(x)=p(k|x,P)ki,
k

weassumethatthevelocityvisdefinedonadisk-shapedsetwithradiusvmax(maxi-
mumvelocity).Forsimplicity,wefurtherassumethatalldirectionsandabsolutevalues
forthevelocitiesareuniformlydistributedonthisset.ItfollowsdirectlythatZ=0.
ThethreeremainingquantitiesLi,MiandQi,jfromEq.(2.33),fordeterminingMj,

237

238ChapterC:Stabilizingdecodingagainstnon-stationaries
Dj,andZwhichcomposetheestimatorEq.(2.32)
Nx^(k)=(kj−Mj)Dj+Z,
1=jarecalculatedanalyticallyusingthefollowingequations
12πvmax2
Li=πvmax20dϕ0dvv{cos(ϕ),sin(ϕ)}
×fiT1+vcos(ϕ−ϕi)+fioffT
mod
v2xma=Tfimodvmax{cos(ϕi),sin(ϕi)}(C.1)
812πvmaxfimodTvoff
Mi=πvmax20dϕ0dvv21+vmaxcos(ϕ−ϕi)+fiT
dmo=Tfi+fioff(C.2)
2Qij=12dϕdvvfiT1+vcos(ϕ−ϕi)+fioffT
2πvmaxmod
dmoπvmax002vmax
×fjT1+vcos(ϕ−ϕj)+fjoffT+δi,jMi
v2xmaT2fmodfmod
=168i2+fioffj2+fjoff+fimodfjmodcos(ϕi−ϕj)
dmo+Tδi,jfi+fioff(C.3)
2InsertingtheseexpressionsintothedefinitionforDjyields
NDj=Li(Qij−MjMi)−1
iN=fimodvmax{cos(ϕi),sin(ϕi)}
imod−1
×Tfimodfjmodcos(ϕi−ϕj)+8δi,jfi+fioff(C.4)
22i,jwhichcanbedirectlyinsertedintotheestimatorequation(2.32)reading
Nmod
v^(k)=ki−Tfi+fioffDi.(C.5)
2iThisequationsassumethatweknowtherealparameterofthetuningfunctionsP.But
Parenotknown.Thusmakesitnecessarytousethelearnedapproximateparameter
setP^forthecomputationofv^(k).

.3)(C

C.2Parameteradaptation
C.2Parameteradaptation

239

TherealparametersPofthesystemthatencodestheinformationabouttheintended
movementintospiketrainsareunknown.Thismakesitnecessarytofindsuitable
approximationsofPinstead.Errorsmadebythereconstructionoftheintendedmove-
mentareusedasameasurehowgoodtheapproximationoftheparametersis.The
preferreddirectionsϕiareapproximatedbyarandomwalkwhichisguidedbythesize
oftheerrorsignal.Othersubsetsofparameters(fimodandfioff)canbeinferredfrom
themeanactivitiesandvarianceoftheactivitiesoftheneuronsitself.Inthefollowing,
thewholeprocedureisdescribedindetail.
AnewparametersetP^wascomputedinthreedifferentsteps.

1.Tbyheavmereaagning<kiov>ertandhevlastarianceTavgν=iof900thesecofiringndsrateofeachneuronwascomputed
t<ki>(t)=1/Tavgt=t−Tavg+1ki(t)(C.6)
tνi(t)=(ki(t)−<ki>(t))2.(C.7)
t=t−Tavg+1
Anestimateoffimodandfioffwasthenobtainedbytheexpressions
f^imod=4νi(t)−<ki>(t)(C.8)
f^ioff=<ki>(t)−f^imod.(C.9)
ItthusisimponehortanastottoselectnoteaTthatthesewhicqhiuansntotitiestoomsaymall.alsosufferfromsamplingnoise,
gav2.Onlyneuronswhichrespondwell,orshowedasubstantialvariationintheirfiring
byratesaignoringrechosenactivitiesforthefromestimationneuronswproithf^imocedure.d<fTThis,withcriterionthemowasdulationimplementhresh-ted
oldpdistributionarameteirntrofTsetducedtoin10theHz.mSaintelectingextbothedy,toneuningtcurvypicallyesfromobtainsthebretwandomeen
Nv=6...15neuronswhosesignalswillbeusedintheestimationprocess.
3.Anewestimateforthepreferreddirectionsϕiwasthenobtainedbyarandom
angleshiftonthecurrentϕi’s,
ϕ^i=^ϕi0+2πφηi,(C.10)
andwheresηitandarddenotesadeviationrandom1.nTumheberscalingdrawnvafromriableanφormalhastdhepurpistributionoseofwithmdecreasingean0
theadaptationspeedwithdecreasingmovementerror,andviceversa.Itwas

240

ChapterC:Stabilizingdecodingagainstnon-stationaries

φ=0.1<^E>.<^E>denotestheestimateoftheperceivederroraveraged
adjustedbyhand,andsubsequentlyfoundtoyieldgoodresultsifchosentobe
overeitherthesecondorfourthstepoftheadaptationalgorithm(whichever
camelast).Thenumberofspikesoverwhichthenewlyconstructedestimator
P^={f^1off,...,f^Noffv,f^1mod,...,f^Nmovd,ϕ^1,...,ϕ^Nv}istobetestedwasthensetto
Ns=1000/<^E>.

sectionInEq.(2.29).2.32

^Γ(2+K,Fmin,Fmax)Fmin
f(K)=(Fmax−Fmin)Γ(1+K,Fmin,Fmax)−(Fmax−Fmin)

showshowanoptimalBayesianestimatorforthistypeoferrorsignal(withlineartun-
ofing)errorloovksalueslike.inIttheshouldintervbeal[noted0,1].tAhatnyEq.(2.29)deviationwfasromdterivhisedforassumedavuniformaluedistributiondistribution
leadstoabiasintheestimation.

DndixeApp

sourcesinformationAdditional

Inadditiontotheliteraturewhichwascitedinthetext,Iused(Bethge,2003;Schulzke,
2006;Hoyer,2002;MacKay,2003;DayanandAbbott,2001;Goebeletal.,2003;Hubel,
1989;Johansen-Berg,2001)andthe’AntragaufFinanzierungdesSonderforschungs-
bereiches5172005-2007’oftheUniversityofBremenandCarlvonOssietzkyUniversity
ofOldenburg.Thesesourceswereusede.g.asguidelinesfordecidingwhichtopicscould
beinterestingformyintroductionintothefieldofcomputationalneuroscienceaswell
ashowthepresentationcanbestructured.

Furthermore,Iusedvariousarticlesfromwikipedia.org,especiallyfromthefieldof
neuro-scienceandinformation-theory,forgettingamorebroaderimpressionofthe
fields.sub-

Anothertypeofadditionalsourcesforwholetextsegmentsandimageswerepublica-
tionswhichIpublishedwithotherscientists(seemypublicationlistforamoredetailed
list).

241

242

apterhC

:D

dditionalA

information

urcesos

yBibliograph

Abeles,M.(1991).Corticonics.CambridgeUniversityPress.

Aertsen,A.,Gerstein,G.,Habib,M.,andPalm,G.(1989).Dynamicsofneuronalfiring
correlation:modulationofeffectiveconnectivity.J.Neurophysiol.,61:900–917.

Aertsen,A.,Gerstein,G.,andJohannesma,P.(1986).Braintheory.pages7–24.

Aldrich,J.(1997).R.A.Fisherandthemakingofmaximumlikelihood1912–1922.
Statist.Sci.,12(3):162–176.

Amirikian,cells.NeB.urandosciGReseorgop.,oulos36(1):73–79.,A.(2000).Directionaltuningprofilesofmotorcortical

Andersen,neuralR.,prosthetics.Burdick,J.,TRENMussallam,DSinCoS.,Pegnitivesaran,SciencB.,ae,nd8Cham,(11):486–493.J.(2004a).Cognitive

Andersen,brain-Rmac.,hineinMussallam,terface.S.,CurrandentPesaran,OpinionB.in(Neur2004b).obiology,Selecting14:720–726.thesignalsfora

Arulampalam,M.,Maskell,S.,Gordon,N.,andClapp,T.(2002).Atutorialonpar-
ticlefiltersforonlinenonlinear/non-gaussianbayesiantracking.SignalProcessing,
50(2):174–188.

Ashe,J.andGeorgopoulos,A.(1994).Movementparametersandneuralactivityin
motorcortexandarea5.CerebralCortex,4:590–600.

Astolfi,L.,Cincotti,F.,Babiloni,C.,Carducci,F.,Basilisco,A.,Rossini,P.,Salinari,
S.,Mattia,D.,Cerutti,S.,Dayan,D.,Ni,L.D.L.Y.,He,B.,andBabiloni,F.
(2005).Estimationofthecorticalconnectivitybyhigh-resolutioneegandstruc-
turalequationmodeling:simulationsandapplicationtofingertappingdata.IEEE
TransBiomedEng,52(5):757–768.

Azouz,R.andGray,C.(2000).Dynamicspikethresholdrevealsamechanismfor
synapticcoincidencedetectionincorticalneuronsinvivo.PNAS,97(14):8110–
8115.

Azouz,conR.troliandnvGisualray,cC.ortical(2003).neuronsAindaptivviveoc.Neuroincidenceon,d37:513–523.etectionanddynamicgain

243

244

APHYRBIBLIOG

Ball,T.,Nawrot,M.,Schulze-Bonhage,A.,Aertsen,A.,andMehring,C.(2004).
Towardsabrain-machineinterfacebasedonepicorticalfieldpotentials.Biomed.
Eng.,49(Suppl.2):756–759.
Barlow,H.(1960).Thecodingofsensorymessages,pages331–360.CambridgeUni-
versityPress.
Batista,A.,Buneo,C.,Snyder,L.,andAndersen,R.(1999).Reachplansineye-
centeredcoordinates.Science,285:257–260.
Baum,L.,Petrie,T.,Soules,G.,andWeiss,N.(1970).Amaximizationtechnique
occurringinthestatisticalanalysisofprobabilisticfunctionsofmarkovchains.
Ann.Math.Statist.,41(1):164–171.
Beck,J.M.andPouget,A.(2007).Exactinferencesinaneuralimplementationofa
hiddenmarkovmodel.NeuralComputation,19:1344–1361.
Bell,A.andSejnowski,T.(1997).The’independentcomponents’ofnaturalscenesare
edgefilters.VisionResearch,37(23):3327–3338.
Bernardo,J.(1979).Expectedinformationasexpectedutility.AnnalsofStatistics,
7(3):686–690.Berry,M.,Warland,D.,andMeister,M.(1997).Thestructureandprecisionofretinal
spiketrains.Proc.Natl.Acad.Sci.,99:5411–5416.
Bertsekas,D.(1995).Non-linearprogramming.AthenaScientific,Belmont,MA.
Bethge,M.(2003).CodesandGoalsofNeuronalRepresentations.PhDthesis,Uni-
versityofBremen.
Bethge,M.,Rotermund,D.,andPawelzik,K.(2001).Optimalshort-termpopulation
coding:Whenfisherinformationfails.Proceedingsofthe28thG¨ottingenNeuro-
biologyConference.
Bethge,M.,Rotermund,D.,andPawelzik,K.(2002a).Binarytuningisoptimalfor
neuralratecodingwithhightemporalresolution.AdvancesinNeuralInformation
ProcessingSystems,15.
Bethge,M.,Rotermund,D.,andPawelzik,K.(2002b).Optimalneuralpopula-
tioncodingandfisherinformation.VerhandlungenderDeutschenPhysikalischen
GesellschaftDPG(VI)37,1,V.
Bethge,M.,Rotermund,D.,andPawelzik,K.(2002c).Optimalshort-termpopulation
coding:whenfisherinformationfails.NeuralComput.,14(10):2317–2351.
Bethge,M.,Rotermund,D.,andPawelzik,K.(2003a).Optimalneuralratecodingleads
tobimodalfiringratedistributions.Network:Comput.NeuralSyst.,14:303–319.

APHYRBIBLIOG

245

Bethge,M.,Rotermund,D.,andPawelzik,K.(2003b).Asecondorderphasetransition
inneuralratecoding:Binaryencodingisoptimalforrapidsignaltransmission.
Phys.Rev.Lett.,90(8):088104–1.
Bilmes,J.(1997).Agentletutorialontheemalgorithmanditsapplicationtoparame-
terestimationforgaussianmixtureandhiddenmarkovmodels.Technicalreport,
UniversityofBerkeley.
Binder,J.,Koller,D.,Russell,S.,andKanazawa,K.(1997).Adaptiveprobabilistic
networkswithhiddenvariables.MachineLearning,29(2-3):213–244.
Black,M.,Bienenstock,E.,Donoghue,J.,Serruya,M.,Wu,W.,andGao,Y.(2003).
Connectingbrainswithmachines:Theneuralcontrolof2Dcursormovement.In
1stInternationalIEEE/EMBSConferenceonNeuralEngineering,pages580–583.
Blahut,R.(1987).Principlesandpracticeofinformationtheory.
Blankertz,B.,Curio,G.,andMueller,K.-R.(2002).ClassifyingsingletrialEEG:To-
wardsbraincomputerinterfacing.AdvancesinNeuralInf.Proc.Systems,14:157–
164.Blankertz,B.,Dornhege,G.,Krauledat,M.,M¨uller,K.,Kunzmann,V.,Losch,F.,and
Curio,G.(2006).Theberlinbrain-computerinterface:Eeg-basedcommunication
withoutsubjecttraining.IEEETransNeuralSystRehabilEng.,14(2):147–152.
Bliss,T.andCollingridge,G.(1993).Asynapticmodelofmemory:long-termpoten-
tiationinthehippocampus.Nature,361:31–39.
Bonhoeffer,T.andGrinvald,A.(1991).Iso-orientationdomainsincatvisualcortex
arearrangedinpinwheel-likepatterns.Nature,353:429–431.
Botvinick,M.,Braver,T.,Barch,D.,Carter,C.,andCohen,J.(2001).Conflict
monitoringandcognitivecontrol.PsycholRev.,108(3):624–652.
Botvinick,M.,Cohen,J.,andCarter,C.(2004).Conflictmonitoringandanterior
cingulatecortex:anupdate.TrendsCognSci.,8(12):539–546.
Boussaoud,D.,Jouffrais,C.,andBremmer,F.(1998).Eyepositioneffectsontheneu-
ronalactivityofdorsalpremotorcortexinthemacaquemonkey.J.Neurophysiol.,
80:1132–1150.Britten,K.,Shadlen,M.,Newsome,W.,andMovshon,J.(1993).Responsesofneurons
inmacaquemttostochasticmotionsignals.VisualNeuroscience,10:1157–1169.
Brodmann,K.(1909).VergleichendeLokalisationslehrederGrosshirnrindeinihren
PrinzipiendargestelltaufGrunddesZellenbaues.
Bromley,J.andS¨ackinger,E.(1991).Neural-networkandk-nearest-neighborclassi-
fiers.Technicalreport,Tech.Rep.11359-910819-16TM,AT&T.

246

APHYRBIBLIOG

Bruno,R.andSakmann,B.(2006).Cortexisdrivenbyweakbutsynchronouslyactive
thalamocorticalsynapses.Science,312:1622–1627.
Bryson,A.andHo,Y.(1969).Appliedoptimalcontrol:optimization,estimation,and
control.
Bullock,D.andGrossberg,S.(1988).Neuraldynamicsofplannedarmmovements:
emergentinvariantsandspeed-accuracypropertiesduringtrajectoryformation.
95(1):49–90.,ev.RPsycholBuntine,W.(1994).Operationsforlearningwithgraphicalmodels.JournalofArtificial
IntelligenceResearch,2:159–225.
Buracas,G.,Zador,A.,DeWeese,M.,andAlbrigh,T.(1998).Efficientdiscrimination
oftemporalpatternsbymotion-sensitiveneuronsinprimatevisualcortex.Neuron,
20(5):959–969.Burkitt,A.(2006a).Areviewoftheintegrate-and-fireneuronmodel:I.homogeneous
synapticinput.BiologicalCybernetics,95(1):1–19.
Burkitt,A.(2006b).Areviewoftheintegrate-and-fireneuronmodel:Ii.inhomoge-
neoussynapticinputandnetworkproperties.BiologicalCybernetics,95(2):97–112.
Burns,B.andWebb,A.(1976).Thespontaneousactivityofneuronsinthecat’s
cerebralcortex.Neuron,20:959–969.
Callaway,E.(1998).Localcircuitsinprimaryvisualcortexofthemacaquemonkey.
AnnualReviewofNeuroscience,21:47–74.
Carandini,M.,Demb,J.,Mante,V.,Tolhurst,D.,Dan,Y.,Olshausen,B.,Gallant,J.,
andRust,N.(2005).Doweknowwhattheearlyvisualsystemdoes?Journalof
25(46):10577–10597.,eosciencNeurCarandini,M.andHeeger,D.(1994).Summationanddivisionbyneuronsinprimate
visualcortex.Science,264:1333–1336.
Carmena,J.,Lebedev,M.,Crist,R.,O’Doherty,J.,Santucci,D.,Dimitrov,D.,Patil,
P.,Henriguez,C.,andNicolelis,M.(2003).Learningtocontrolabrain-machine
interfaceforreachingandgraspingbyprimates.PublicLibraryofScienceBiology,
1(2):193–208.Carter,C.,Braver,T.,Barch,D.,Botvinick,M.,Noll,D.,andCohen,J.(1998).An-
teriorcingulatecortex,errordetection,andtheonlinemonitoringofperformance.
280(5364):747–749.,eSciencCasagrande,V.(1999).Themysteryofthevisualsystemkpathway.TheJournalof
517(3):630.,gyPhysioloChance,F.S.,Abbott,L.F.,andReyes,A.D.(2002).Gainmodulationfromback-
groundsynapticinput.Neuron,35:773–782.

APHYRBIBLIOG

247

Chang,C.andLin,C.(2001).Libsvm:alibraryforsupportvectormachines.
Chelazzi,L.,Duncan,J.,Miller,E.,andDesimone,R.(1998).Responsesofneurons
ininferiortemporalcortexduringmemory-guidedvisualsearch.JNeurophysiol,
80:2918–2940.Chelazzi,L.,Miller,E.,Duncan,J.,andDesimone,R.(1993).Aneuralbasisforvisual
searchininferiortemporalcortex.Nature,363:345–347.
Chen,H.,Hua,S.,Smith,M.,Lenz,F.,andShadmehr,R.(2006).Effectsofhuman
cerebellarthalamusdisruptiononadaptivecontrolofreaching.CerebralCortex,
16(10):1462–1473.Chien,C.andJaw,F.(2005).Miniaturetelemetrysystemfortherecordingofaction
andfieldpotentials.JNeurosciMethods.,147(1):68–73.
Chouinard,P.andPaus,T.(2006).Theprimarymotorandpremotorareasofthe
humancerebralcortex.TheNeuroscientist,12(2):143–152.
Cole,K.andCurtis,H.(1939).Electricimpedanceofthesquidgiantaxonduring
activity.TheJournalofGeneralPhysiology,22:649–670.
Connolly,M.andvanEssen,D.(1984).Therepresentationofthevisualfieldinparvi-
cellularandmagnocellularlayersofthelateralgeniculatenucleusinthemacaque
monkey.TheJournalofComparativeNeurology,226(4):544–564.
Connor,C.(2004).Visualattention:Bottom-upversustop-down.CurrentBiology,
14:R850–R852.Connor,C.,Preddie,D.,Gallant,J.,andvanEssen,D.(1997).Spatialattentioneffects
inmacaqueareav4.JournalofNeuroscience,17(9):3201–3214.
Connor,J.andStevens,C.(1971).Inwardanddelayedoutwardmembranecurrentsin
isolatedneuralsomataundervoltageclamp.JPhysiol.,213(1):1–19.
Cooper,G.(1990).Thecomputationalcomplexityofprobabilisticinferenceusing
bayesianbeliefnetworks.ArtificialIntelligence,42:393–405.
Cortes,C.andVapnik,V.(1995).Support-vectornetworks.MachineLearning,
20(3):273–297.Crair,M.,Ruthazer,E.,Gillespie,D.,andStryker,M.(1997).Relationshipbetweenthe
oculardominanceandorientationmapsinvisualcortexofmonocularlydeprived
cats.Neuron,19:307–318.
Cramer,H.(1946).Acontributiontothetheoryofstatisticalestimation.Skandinavisk
19:85–94.,tuarietidskriftkACulham,J.andValyear,K.(2006).Humanparietalcortexinaction.CurrOpin
Neurobiol.,16(2):205–212.

248

APHYRBIBLIOG

Curran,E.andStokes,M.(2003).Learningtocontrolbrainactivity:Areviewof
theproductionandcontrolofeegcomponentsfordrivingbraincomputerinterface
(bci)systems.BrainandCognition,51(3):326–336.
Dagnelie,G.(2006).Visualprosthetics2006:assessmentandexpectations.ExpertRev
MedDevices,3(3):315–325.
Dayan,P.andAbbott,L.(2001).ComputationalandMathematicalModelingofNeural
.emsstSyDayan,P.andHinton,G.(1996).Varietiesofhelmholtzmachine.NeuralNetworks,
9(8):1385–1403.Dayan,P.andHinton,G.(1997).Usingexpectation-maximizationforreinforcement
learning.NeuralComputation,9:271–278.
Dayan,P.,Hinton,G.,Neal,R.,andZemel,R.(1995).Thehelmholtzmachine.Neural
Comput.,7(5):889–904.
Dempster,A.,Laird,N.,andRubin,D.(1977).Maximumlikelihoodfromincomplete
dataviatheemalgorithm.JournaloftheRoyalStatisticalSociety,B39:1–38.
Deneve,S.(2005).Bayesianinferenceinspikingneurons.InSaul,L.K.,Weiss,Y.,
andBottou,L.,editors,AdvancesinNeuralInformationProcessingSystems17,
pages353–360.MITPress,Cambridge,MA.
Deneve,S.(2007a).Bayesianspikingneuronsi:Inference.notpublishedyet.
Deneve,S.(2007b).Bayesianspikingneuronsii:Learning.notpublishedyet.
Deneve,S.,Latham,P.E.,andPouget,A.(1999).Readingpopulationcodes:aneural
implementationofidealobservers.NatureNeuroscience,2(8):740–745.
Deneve,S.,Latham,P.E.,andPouget,A.(2001).Efficientcomputationandcue
integrationwithnoisypopulationcodes.NatureNeuroscience,4(8):826–831.
Desimone,R.andDuncan,J.(1995).Neuralmechanismsofselectivevisualattention.
Annu.Rev.Neurosci.,18:193–222.
Desimone,R.andSchein,S.(1987).Visualpropertiesofneuronsinareav4ofthe
macaque:sensitivitytostimulusform.JournalofNeurophysiology,57(3):835–
868.Devroye,L.(1986).Non-UniformRandomNumberGeneration.
Dickinson,A.andBalleine,B.(1994).Motivationalcontrolofgoal-directedaction.
Learn.behav.,22(1):1–18.
Diedrichsen,J.,Hashambhoy,Y.L.,Rane,T.,andShadmehr,R.(2005).Neural
correlatesofreacherrors.JournalofNeuroscience,25(43):9919–9931.

APHYRBIBLIOG

249

Dobbins,A.,Jeo,R.,Fiser,J.,andAllman,J.(1998).Distancemodulationofneural
activityinthevisualcortex.Science,281(5376):552–555.
Donoghue,J.,Leibovic,S.,andSanes,J.(1992).Organizationoftheforelimbareain
squirrelmonkeymotorcortex:representationofdigit,wrist,andelbowmuscles.
ExperimentalBrainResearch,89(1):1–19.
Eckhorn,R.,Frien,A.,Bauer,R.,Woelbern,T.,andKehr,H.(1993).Highfrequency
(60-90hz)oscillationsinprimaryvisualcortexofawakemonkey.Neuroreport,
4:243–246.Eckhorn,R.andPoeppel,B.(1974).Rigorousandextendedapplicationofinforma-
tiontheorytotheafferentvisualsystemofthecat.i.basicconcepts.Biological
Cybernetics,16(4):191–200.
Edwards,F.,Konnerth,A.,Sakmann,B.,andBusch,C.(1990).Quantalanalysisof
inhibitorysynaptictransmissioninthedentategyrusofrathippocampalslices:A
patchclampstudy.J.Physiol.,430(1):213–249.
Egeth,H.andYantis,S.(1997).Visualattention:Control,representation,andtime
course.AnnualReviewofPsychology,28:269–297.
Elul,R.(1972).Thegenesisoftheeeg.InternationalReviewofNeurobiology,15:227–
272.Ernst,M.O.andBanks,M.S.(2002).Humansintegratevisualandhapticinformation
inastatisticallyoptimalfashion.Nature,415:429–433.
Ernst,U.,Gutkin,B.,Pawelzik,K.,andDeneve,S.(2007a).Dynamicsofprobabilistic
neuronalcomputation.notpublishedyet.
Ernst,U.,Rotermund,D.,andPawelzik,K.(2004).Analgorithmforfastpattern
recognitionwithrandomspikes.26thDAGMSymposiumProceedings,pages399–
406.Ernst,U.,Rotermund,D.,andPawelzik,K.(2007b).Efficientcomputationbasedon
stochasticspikes.NeuralComputation.
Fabiani,G.,McFarland,D.,Wolpaw,J.,andPfurtscheller,G.(2004).Ieeeconversion
ofeegactivityintocursormovementbyabrain-computerinterface(bci).Trans
NeuralSystRehabilEng,12(3):331–338.
Falkenstein,M.,Hoormann,J.,Christ,S.,andHohnsbein,J.(2000).Erpcomponents
onreactionerrorsandtheirfunctionalsignificance:atutorial.BiolPsychol.,
51(2-3):87–107.Farwell,L.andDonchin,E.(1988).Talkingoffthetopofyourhead:towardamental
prosthesisutilizingevent-relatedbrainpotentials.ElectroencephalogrClinNeuro-
70(6):510–523.,physiol.

250

APHYRBIBLIOG

Fernandez,E.,Ferrandez,J.,Ammermller,J.,andNormann,R.(2000).Population
codinginspiketrainsofsimultaneouslyrecordedretinalganglioncells.BrainRes.,
887:222–229.Ferrera,V.andMaunsell,J.(2005).Motionprocessinginmacaquev4.NatureNeuro-
:1125.8,esciencFiorillo,C.,Tobler,P.,andSchultz,W.(2003).Discretecodingofrewardprobability
anduncertaintybydopamineneurons.Science,299:1898–1902.
Fisher,R.(1922).Onthemathematicalfoundationsoftheoreticalstatistics.Philo-
sophicalTransactionsoftheRoyalSocietyA,222:309–368.
Fitzsimmons,N.,Drake,W.,Sandler,A.,Hanson,T.,Lebedev,M.,andNicolelis,M.
(2005).Long-termbehavioralimprovementsinareachingtaskcuedbymicrostim-
ulationoftheprimarysomatosensorycortex.SocietyforNeuroscienceAbstracts,
31:402.7.Flash,T.andHochner,B.(2005).Motorprimitivesinvertebratesandinvertebrates.
CurrOpinNeurobiol.,15(6):660–666.
Flash,T.andSejnowski,T.(2001).Computationalapproachestomotorcontrol.
CurrentOpinioninNeurobiology,11:655–662.
Fourcaud,N.andBrunel,N.(2002).Dynamicsofthefiringprobabilityofnoisy
integrate-and-fireneurons.NeuralComputation,14:2057–2110.
Friedman,N.(1998).TheBayesianstructuralEMalgorithm.InUAI,pages129–138.
Fries,P.,Nikolic,D.,andSinger,W.(2007).Thegammacycle.TINSspecialissue,
30(7):309–316.Fries,P.,Reynolds,J.,Rorie,A.,andDesimone,R.(2001).Modulationofoscillatory
neuronalsynchronizationbyselectivevisualattention.Science,291:1560–1563.
Frith,C.,Friston,K.,Liddle,P.,andFrackowiak,R.(1991).Willedactionandthe
prefrontalcortexinman:astudywithpet.ProcBiolSci.,244(1311):241–246.
Fu,Q.,Flament,D.,Coltz,J.,andEbner,T.(1997).Relationshipofcerebellarpurkinje
cellsimplespikedischargetomovementkinematicsinthemonkey.Journalof
78:478–491.,gyophysioloNeurFujii,N.,Mushiake,H.,andTanji,J.(2000).Rostrocaudaldistinctionofthedorsal
premotorareabasedonoculomotorinvolvement.J.Neurophysiol.,83:1764–1769.
Fulton,J.(1935).ANoteontheDefinitionofthemotorandpremotorAreas.
Gallant,J.,Connor,C.,Rakshit,S.,Lewis,J.,andEssen,D.V.(1996).Neural
responsestopolar,hyperbolic,andcartesiangratingsinareav4ofthemacaque
monkey.JNeurophysiol,76:2718–2739.

APHYRBIBLIOG

251

Garavan,H.,Ross,T.,Murphy,K.,Roche,R.,andStein,E.(2002).Dissociable
executivefunctionsinthedynamiccontrolofbehavior:inhibition,errordetection,
andcorrection.Neuroimage.,17(4):1820–1829.
Gehring,W.,Coles,M.,Meyer,D.,andDonchin,E.(1995).Abrainpotentialmanifes-
tationoferror-relatedprocessing.JournalofElectroencephalographyandClinical
Neurophysiology,Supplement44:287–296.
Gehring,W.J.,Goss,B.,Coles,M.G.H.,Meyer,D.E.,andDonchin,E.(1993).
Aneuralsystemforerrordetectionandcompensation.PsychologicalScience,
4:385–390.Geman,S.,Bienenstock,E.,andDoursat,R.(1991).Neuralnetworksandthe
bias/variancedilemma.NeuralComputation,4:1–58.
Geman,S.andGeman,D.(1984).Stochasticrelaxation,gibbsdistributions,and
thebayesianrestorationofimages.IEEETransactionsonPatternAnalysisand
MachineIntelligence,6:721–741.
Georgopoulos,A.,Kalaska,J.,Caminiti,R.,andMassey,J.(1982).Ontherelations
betweenthedirectionoftwo-dimensionalarmmovementandcelldischargein
primatemotorcortex.J.Neurosci.,2(11):1527–1537.
Georgopoulos,A.,Kettner,R.,andSchwartz,A.(1988).Primatemotorcortexand
freearmmovementstovisualtargetsinthree-dimensionalspace.ii.codingofthe
directionofmovementbyaneuronalpopulation.JNeurosci.,8(8):2928–2937.
Georgopoulos,A.,Schwartz,A.,andKettner,R.(1986).Neuronalpopulationcoding
ofmovementdirection.Science,233(4771):1416–1419.
Gerstner,W.andKistler,W.(2002a).SpikingNeuronModels.CambridgeUniversity
ess.PrGerstner,W.andKistler,W.(2002b).SpikingNeuronModels-SingleNeurons,Pop-
.Plasticityulations,Gevins,A.(1984).Analysisoftheelectromagneticsignalsofthehumanbrain:mile-
stones,obstacles,andgoals.IEEETransBiomedEng,31:833–850.
Ghose,G.andTs’O,D.(1997).Formprocessingmodulesinprimateareav4.J
77:2191–2196.,ophysiolNeurGilbert,C.,Ito,M.,Kapadia,M.,andWestheimer,G.(2000).Interactionsbetween
attention,contextandlearninginprimaryvisualcortex.VisionRes.,40(10–
12):1217–1226.Giszter,S.,Mussa-Ivaldi,F.,andBizzi,E.(1993).Convergentforcefieldsorganizedin
thefrog’sspinalcord.JournalofNeuroscience,13:467–491.

252

BIBLIOGAPHYR

Goebel,R.,Muckli,L.,andKim,D.-S.(2003).TheHumanNervousSystem,2nd
Edition-Chapter:TheVisualSystem.
Goodale,M.andMilner,A.(1992).Separatevisualpathwaysforperceptionandaction.
TrendsinNeurosciences,15:20–25.
Grayden,D.andClark,G.(2005).CochlearImplants:APracticalGuide,2ndEdition
–Chapter:Implantdesignanddevelopment.
Greschner,M.,Bongard,M.,Rujan,P.,andAmmermller,J.(2002).Retinalganglion
cellsynchronizationbyfixationaleyemovementsimprovesfeatureestimation.Na-
tureNeuroscience,5:341–347.
Gurfinkel,V.,Levick,Y.,andLebedev,M.(1991).Bodyschemeconceptandmotor
control.bodyschemeintheposturalautomatismsregulation.IntellectualPro-
cessesandTheirModelling,pages24–53.
H¨am¨al¨ainen,M.,Hari,R.,nadJ.Knuutila,R.I.,andLounasmaa,O.(1993).Magne-
toencephalographytheory,instrumentation,andapplicationstononinvasivestud-
iesoftheworkinghumanbrain.Rev.Mod.Phys.,65(2):413–497.
Hanazawa,A.andKomatsu,H.(2001).Influenceofthedirectionofelementallumi-
nancegradientsontheresponsesofv4cellstotexturedsurfaces.TheJournalof
21(12):4490–4497.,eosciencNeurHarris,C.andWolpert,D.(1998).Signal-dependentnoisedeterminesmotorplanning.
394:780–784.,eturNaHastings,W.(1970).Montecarlosamplingmethodsusingmarkovchainsandtheir
57(1):97–109.,rikaBiometapplications.He,B.,Lian,J.,Spencer,K.,Dien,J.,andDonchin,E.(2001).Acorticalpotential
imaginganalysisofthep300andnoveltyp3components.HumBrainMapp.,
12(2):120–130.Hebb,D.(1949).Theorganizationofbehavior:Aneuropsychologicaltheory.
Heckerman,D.(1995).Atutorialonlearningwithbayesiannetworks.Technicalreport,
MicrosoftResearch.
Heckerman,D.,Geiger,D.,andChickering,D.(1995).Learningbayesiannetworks:
Thecombinationofknowledgeandstatisticaldata.MachineLearning,20(3):197–
243.Heeger,D.(1992).Normalizationofcellresponsesincatstriatecortex.VisualNeuro-
:181–198.9,esciencHeeger,D.andRess,D.(2002).Whatdoesfmritellusaboutneuronalactivity?Nat
RevNeurosci,3:142–151.

APHYRBIBLIOG

253

Hegde,J.andvanEssen,D.(2005).Stimulusdependenceofdisparitycodinginprimate
visualareav4.J.Neurophysiol.,93:620–626.
Hertz,J.,Krogh,A.,andPalmer,R.(1991).Introductiontothetheoryofneuronal
computation.
Herzog,M.H.andFahle,M.(2002).Effectsofgroupingincontextualmodulation.
415:433–436.,eturNaHinkle,D.andConnor,C.(2001).Disparitytuninginmacaqueareav4.Neuroreport.,
12(2):365–369.Hinkle,D.andConnor,C.(2002).Three-dimensionalorientationtuninginmacaque
areav4.NatureNeuroscience,5:665–670.
Hinterberger,T.,Schmidt,S.,Neumann,N.,Mellinger,J.,Blankertz,B.,Curio,
G.,andBirbaumer,N.(2004).Brain-computercommunicationwithslowcor-
ticalpotentials:Methodologyandcriticalaspects.IEEETrans.Biomed.Eng.,
51(6):1011–1018.Hinton,G.,Osindero,S.,andTeh,Y.W.(2006).Afastlearningalgorithmfordeep
beliefnets.NeuralComput.,18(7):1527–1554.
Ho,M.,Mobini,S.,Chiang,T.,Bradshaw,C.,andSzabadi,E.(1999).Theoryand
methodinthequantitativeanalysisofimpulsivechoicebehaviour:implications
forpsychopharmacology.Psychopharmacology,146(4):362–372.
Hochberg,L.,Serruya,M.,Friehs,G.,Mukand,J.,Saleh,M.,Caplan,A.,Branner,
A.,Chen,D.,Penn,R.,andDonoghue,J.(2006).Neuronalensemblecontrolof
prostheticdevicesbyahumanwithtetraplegia.Nature,442:164–171.
Hodgkin,A.andHuxley,A.(1952).Aquantitativedescriptionofmembranecurrent
anditsapplicationtoconductionandexcitationinnerve.JournalofPhysiology,
117:500–544.Holroyd,C.andColes,M.(2002).Theneuralbasisofhumanerrorprocessing:Re-
inforcementlearning,dopamine,andtheerror-relatednegativity.Psychological
109:679–709.,eviewRHopfield,J.(1982).Neuralnetworksandphysicalsystemswithemergentcollective
computationalabilities.ProcNatlAcadSci,79(8):2554–2558.
Hosoya,T.,Baccus,S.A.,andMeister,M.(2005).Dynamicpredictivecodingbythe
retina.Nature,436:71–77.
Hoyer,P.(2002).ProbabilisticModelsofEarlyVision.PhDthesis,HelsinkiUniversity
ofTechnology.
Hoyer,P.O.(2004).Non-negativematrixfactorizationwithsparsenessconstraints.
JournalofMachineLearningResearch,5:1457–1469.

254

APHYRBIBLIOG

Hoyer,P.O.andHyv¨arinen,A.(2003).Interpretingneuralresponsevariabilityas
montecarlosamplingoftheposterior.InAdvancesinNeuralInformationPro-
cessingSystems15(2002),volume15,pages277–284.MITpress.
Hubel,D.(1989).AugeundGehirn–NeurobiologiedesSehens.
Hubel,D.andWiesel,T.(1968).Receptivefieldsandfunctionalarchitectureofmonkey
striatecortex.JPhysiol.,195(1):215–243.
Hubel,D.andWiesel,T.(1977).Functionalarchitectureofmacaquemonkeyvisual
cortex(ferrierlecture).Proc.R.Soc.,198:1–59.
Irazoqui-Pastor,P.,Mody,I.,andJudy,J.(2003).In-vivoeegrecordingusingawireless
implantableneuraltransceiver.NeuralEngineering,2003,1:622–625.
Irazoqui-Pastor,P.,Mody,I.,andJudy,J.(2005).Recordingbrainactivitywirelessly.
inductivepoweringinminiatureimplantableneuralrecordingdevices.IEEEEng
MedBiolMag.,24(6):48–54.
Iriki,A.,Tanaka,M.,andIwamura,Y.(1996).Codingofmodifiedbodyschemaduring
toolusebymacaquepostcentralneurones.Neuroreport.,7(14):2325–2330.
Ito,M.andGilbert,C.(1999).Attentionmodulatescontextualinfluencesinthepri-
maryvisualcortexofalertmonkeys.Neuron,22:593–604.
Ito,S.,Stuphorn,V.,Brown,J.,andSchall,J.(2003).Performancemonitoringbythe
anteriorcingulatecortexduringsaccadecountermanding.Science,302:120–122.
Itti,L.(2002).Thehandbookofbraintheoryandneuralnetworks-Chapter:Visual
.ttentionAItti,L.andKoch,C.(2001).Computationalmodelingofvisualattention.Nature
Reviews:Neuroscience,2(3):194–203.
Jackson,J.(1993).KlassischeElektrodynamik,2.Auflage.
Javaheri,M.,Hahn,D.,Lakhanpal,R.,Weiland,J.,andHumayun,M.(2006).Retinal
prosthesesfortheblind.AnnAcadMedSingapore,35(3):137–144.
Jeffery,G.(2001).Architectureoftheopticchiasmandthemechanismsthatsculpt
itsdevelopment.Physiol.Rev.,81:1393–1414.
Johansen-Berg,H.(2001).Reorganisationandmodulationofthehumansensorimotor
system:implicationsforrecoveryofmotorfunctionsafterstroke.PhDthesis,
Oxford.ofyersitUnivJohansson,R.andBirznieks,I.(2004).Firstspikesinensemblesofhumantactile
afferentscodecomplexspatialfingertipevents.NatureNeurosci.,7(2):170–177.
Johnston,D.andWu,S.-S.(1997).FoundationsofCellularNeurophysiology.

APHYRBIBLIOG

255

Jordan,M.(1999).LearninginGraphicalModels.
Kalaska,J.,Cohen,D.,Prud’homme,M.,andHyde,M.(1990).Parietalarea5neu-
ronalactivityencodesmovementkinematics,notmovementdynamics.Experi-
mentalBrainResearch,80(2):351–364.
Kamin,L.(1969).FundamentalissuesinassociativelearningChapter:Selectiveasso-
ciationandconditioning.
Kawato,M.(1999).Internalmodelsformotorcontrolandtrajectoryplanning.Current
OpinioninNeurobiology,9:718–727.
Kerns,J.,Cohen,J.,3rdMacDonald,A.,Cho,R.,Stenger,V.,andCarter,C.
(2004).Anteriorcingulateconflictmonitoringandadjustmentsincontrol.Science,
303(5660):1023–1026.Kistler,W.andvanHemmen,J.(2000).Modelingsynapticplasticityinconjunc-
tionwiththetimingofpre-andpostsynapticactionpotentials.NeuralComput.,
12:385–405.Knutti,J.,Wildi,E.,Marshall,J.,Allen,H.,andMeindl,J.(1979).Totallyimplantable
dimensiontelemetry.BiotelemPatientMonit.,6(3):133–146.
Kobatake,E.andTanaka,K.(1994).Neuronalselectivitiestocomplexobjectfeatures
intheventralvisualpathwayofthemacaquecerebralcortex.JNeurophysiol.,
71(3):856–867.Kobayashi,S.,Lauwereyns,J.,Koizumi,M.,andSakagami,M.(2002).Influenceof
rewardexpectationonvisuospatialprocessinginmacaquelateralprefrontalcortex.
87:1488–1498.,ophysiol.NeurJ.Koch,C.andUllman,S.(1984).Selectingoneamongthemany:Asimplenetwork
implementingshiftsinselectivevisualattention.Technicalreport,MITA.I.Memo
770.Koechlin,E.,Anton,J.L.,andBurnod,Y.(1999).Bayesianinterferenceinpopulations
ofcorticalneurons:amodelofmotionintegrationandsegmentationinareamt.
80:25–44.,b.CyBiol.K¨ording,K.P.andWolpert,D.M.(2004).Bayesianintegrationinsensorimotorlearn-
ing.Nature,427:244–247.
Kreiter,A.,Aertsen,A.,andGerstein,G.(1989).Alow-costsingle-boardsolution
forreal-time,unsupervisedwaveformclassificationofmultineuronrecordings.J
NeurosciMethods,30(1):59–69.
Kreiter,A.andSinger,W.(1996).Braintheory.pages201–227.
Krigolson,O.andHolroyd,C.(2006).Evidenceforhierarchicalerrorprocessinginthe
humanbrain.Neuroscience,137(1):13–17.

256

APHYRBIBLIOG

Kronlandt-Martinet,R.,Morlet,J.,andGrossmann,A.(1987).Analysisofsound
patternsthroughwavelettransforms.InternJPatternRecognitArtifIntell,1:273–
302.Kuebler,A.,Kotchoubey,B.,Kaiser,J.,Wolpaw,J.,andBirbaumer,N.(2001).Brain-
computercommunication:Unlockthelockedin.Psychol.Bull.,127:358–375.
Kuffler,S.(1953).Dischargepatternsandfunctionalorganizationofmammalianretina.
JournalofNeurophysiology,16(11):37–68.
Kullback,S.andLeibler,R.(1951).Oninformationandsufficiency.AnnalsofMath-
ematicalStatistics,22(1):79–86.
Lackner,J.andDiZio,P.(2000).Aspectsofbodyself-calibration.TrendsinCognitive
:279–288.4,eSciencLacquaniti,F.,Terzuolo,C.,andViviani,P.(1983).Thelawrelatingthekinematic
andfiguralaspectsofdrawingmovements.ActaPsychol,54(1-3):115–130.
Land,M.andFernald,R.(1992).Theevolutionofeyes.AnnualReviewofNeuro-
15:1–29.,esciencLand,M.andMcLeod,P.(2000).Fromeyemovementstoactions:howbatsmenhit
theball.NatNeurosci.,3(12):1340–1345.
Lanteri,H.,Roche,M.,andAime,C.(2002).Penalizedmaximumlikelihoodimage
restorationwithpositivityconstraints:multiplicativealgorithms.InverseProb-
18:1397–1419.,lemsLanteri,H.,Roche,M.,Cuevas,O.,andAime,C.(2001).Ageneralmethodtode-
visemaximum-likelihoodsignalrestorationmultiplicativealgorithmswithnon-
negativityconstraints.SignalProcessing,81:945–974.
Lapicque,L.(1907).Recherchesquantitativessurlexcitationelectriquedesnerfstraitee
commeunepolarization.JournalofPhysiologyandPathololgy,9:620–635.
Lebedev,M.,Carmena,J.,O’Doherty,J.,Zacksenhouse,M.,Henriquez,C.,Principe,
J.,andNicolelis,M.(2005).Corticalensembleadaptationtorepresentvelocity
ofanartificalactuatorcontrolledbyabrain-machineinterface.TheJournalof
25(19):4681–4693.,eosciencNeurLebedev,M.andNicolelis,M.(2006).Brain-machineinterfaces:past,presentand
future.TrendsNeurosci.,29(9):536–546.
Lee,D.D.andSeung,H.S.(1999).Learningthepartsofobjectsbynon-negative
matrixfactorization.Nature,401:788–791.
Lee,D.D.andSeung,H.S.(2000).Algorithmsfornon-negativematrixfactorization.In
Leen,T.K.,Dietterich,T.G.,andTresp,V.,editors,NIPS–AdvancesinNeural
InformationProcessingSystems,volume13,pages556–562.TheMITPress.

APHYRBIBLIOG

257

Lee,J.andvanDonkelaar,P.(2006).Thehumandorsalpremotorcortexgenerates
on-lineerrorcorrectionsduringsensorimotoradaptation.JNeurosci.,26(12):3330–
3334.Lee,T.andMumford,D.(2003).Hierarchicalbayesianinferenceinthevisualcortex.
J.Opt.Soc.Am.,20(7):1434–1448.
Lehmann,E.andCasella,G.(1999).Theoryofpointestimation.
Levy,W.andBaxter,R.(1996).Energy-efficientneuralcodes.Neur.Comp.,8:531–
543.Lewicki,M.(1998).Areviewofmethodsforspikesorting:thedetectionandclas-
sificationofneuralactionpotentials.Network:ComputationinNeuralSystems,
9(4):53–78.Linden,D.andConnor,J.(1995).Long-termsynapticdepression.Annu.Rev.Neu-
rosci.,18:319–357.
Livingstone,M.andHubel,D.(1987).Psychophysicalevidenceforseparatechannels
fortheperceptionofform,color,movement,anddepth.JournalofNeuroscience,
7:3416–3468.Livingstone,M.andHubel,D.(1988).Segregationofform,color,movement,and
depth:Anatomy,physiology,andperception.Science,240:740–749.
Logothetis,N.(1999).Vision:awindowonconsciousness.SciAm.,281(5):69–75.
Logothetis,N.,Pauls,J.,Augath,M.,Trinath,T.,andOeltermann,A.(2001).Neu-
rophysiologicalinvestigationofthebasisofthefmrisignal.Nature,412:150–157.
Lucy,L.(1974).Aniterativetechniquefortherectificationofobserveddistributions.
Astron.J.,74:745–754.
Maass,W.andBishop,C.(2000).PulsedNeuralNetworks.
Maass,W.,Natschlager,T.,andMarkram,H.(2002).Real-timecomputingwithout
stablestates:anewframeworkforneuralcomputationbasedonperturbations.
14(11):2531–2560.,Comput.alNeurMacKay,D.(1995).Probablenetworksandplausiblepredictions-areviewofpracti-
calbayesianmethodsforsupervisedneuralnetworks.Network:Computationin
NeuralSystems,6:469–505.
MacKay,D.(2003).InformationTheory,InferenceandLearningAlgorithms.
Madigan,D.,York,J.,andAllard,D.(1995).Bayesiangraphicalmodelsfordiscrete
data.InternationalStatisticalReview,63(2):215–232.
Mainen,Z.andSejnowski,T.(1995).Reliabilityofspiketiminginneocorticalneurons.
268(5216):1503–1506.,eScienc

258

APHYRBIBLIOG

Malpeli,J.andBaker,F.(1975).Therepresentationofthevisualfieldinthelateral
geniculatenucleusofmacacamulatta.J.Comp.Neurol.,161:569–594.
Mandon,S.andKreiter,A.K.(2005).Rapidcontourintegrationinmacaquemonkeys.
VisionResearch,45:291–300.
Maravita,A.,Spence,C.,andDriver,J.(2003).Multisensoryintegrationandthebody
schema:closetohandandwithinreach.CurrBiol.,13(13):R531–539.
Martel,S.,Hatsapoulos,N.,Hunter,I.,Donoghue,J.,Burgert,J.,Malasek,J.,Wise-
man,C.,andDyer,R.(2001).Developmentofawirelessbrainimplant:the
telemetricelectrodearraysystem(teas)project.EngineeringinMedicineand
BiologySociety,2001,4:3594–3597.
Martinez-Conde,S.,Macknik,S.,andHubel,D.(2000).Microsaccadiceyemovements
andfiringofsinglecellsinthestriatecortexofmacaquemonkeys.NatureNeuro-
:251–258.3,esciencMartinez-Trujillo,J.andTreue,S.(2004).Feature-basedattentionincreasestheselec-
tivityofpopulationresponsesinprimatevisualcortex.CurrBiol,14(9):744–751.
Maruishi,M.,Tanaka,Y.,Muranaka,H.,Tsuji,T.,Ozawa,Y.,Imaizumi,S.,Miy-
atani,M.,andKawahara,J.(2004).Brainactivationduringmanipulationof
themyoelectricprosthetichand:afunctionalmagneticresonanceimagingstudy.
Neuroimage,21(4):1604–1611.
Mason,S.andGraham,N.(2002).Areasbeneaththerelativeoperatingcharacteristics
(roc)andrelativeoperatinglevels(rol)curves:Statisticalsignificanceandinterpre-
tation.QuarterlyJournaloftheRoyalMeteorologicalSociety,128(584):2145–2166.
Matsumoto,K.,Suzuki,W.,andTanaka,K.(2003).Neuronalcorrelatesofgoal-based
motorselectioninthepreforntalcortex.Science,301:229–232.
McAdams,C.andMaunsell,J.(1999a).Effectsofattentiononorientation-tuningfunc-
tionsofsingleneuronsinmacaquecorticalareav4.TheJournalofNeuroscience,
19(1):431–441.McAdams,C.andMaunsell,J.(1999b).Effectsofattentiononthereliabilityof
individualneuronsinmonkeyvisualcortex.Neuron,23:765–773.
McAdams,C.andMaunsell,J.(2000).Attentiontobothspaceandfeaturemodulates
neuronalresponsesinmacaqueareav4.Neurophysiol.,1751-1755:83(3).
McCulloch,W.andPitts,W.(1943).Alogicalcalculusoftheideasimmanentinneural
nets.BulletinofMathematicalBiophysics,5:115–133.
Mehring,C.,Nawrot,M.,deOliveira,S.C.,Vaadia,E.,Schulze-Bonhage,A.,Aertsen,
A.,andBall,T.(2004).Comparinginformationaboutarmmovementdirection
insinglechannelsoflocalandepicorticalfieldpotentialsfrommonkeyandhuman
motorcortex.JPhysiol,98(4-6):498–506.

BIBLIOGAPHYR

259

Mehring,C.,Rickert,J.,Vaadia,E.,deOliveira,S.C.,Aertsen,A.,andRotter,S.
(2003).Inferenceofhandmovementsfromlocalfieldpotentialsinmonkeymotor
cortex.NatureNeuroscience,6:1253–1254.
Meinicke,P.,Kaper,M.,Hoppe,F.,Heumann,M.,andRitter,H.(2003).Improving
transferratesinbraincomputerinterfacing:Acasestudy.AdvancesinNeural
InformationProcessingSystems,15:1107–1114.
Menon,V.,Adleman,N.,White,C.,Glover,G.,andReiss,A.(2001).Error-related
brainactivationduringago/nogoresponseinhibitiontask.HumBrainMapp.,
12(3):131–143.Metropolis,N.,Rosenbluth,A.,Rosenbluth,M.,Teller,A.,andTeller,H.(1953).
Equationsofstatecalculationsbyfastcomputingmachines.JournalofChemical
21:1087–1091.,PhysicsMetropolis,N.andUlam,S.(1949).Themontecarlomethod.J.Amer.Statist.Assoc.,
44:335–341.Miall,R.(2002).Thehandbookofbraintheoryandneuralnetworks-Chapter:Motor
Control,BiologicalandTheoretical.
Minsky,M.andPapert,S.(1969).Perceptrons:anintroductiontocomputational
.ometrygeMishkin,M.,Ungerleider,L.,andMacko,K.(1983).Objectvisionandspatialvision:
Twocorticalpathways.TrendsinNeurosci.,6:414–417.
Mitzdorf,U.(1985).Currentsource-densitymethodandapplicationincatcerebralcor-
tex:investigationofevokedpotentialsandeegphenomena.PhysiolRev.,65(1):37–
100.Mitzdorf,U.andSinger,W.(1979).Excitatorysynapticensemblepropertiesinthevi-
sualcortexofthemacaquemonkey:acurrentsourcedensityanalysisofelectrically
evokedpotentials.JCompNeurol.,187(1):71–83.
Mohseni,P.,Najafi,K.,Eliades,S.,andWang,X.(2005).Wirelessmultichannel
biopotentialrecordingusinganintegratedfmtelemetrycircuit.NeuralSystems
andRehabilitationEngineering,IEEETransactions,13(3):263–271.
Mongillo,G.andDeneve,S.(2007).On-linelearningwithhiddenmarkovmodels.not
.etydpublisheMoody,J.andDarken,C.(1989).Fastlearninginnetworksoflocally-tunedprocessing
units.NeuralComput.,1:281–294.
Moran,D.andSchwartz,A.(1999a).Motorcorticalactivityduringdrawingmove-
ments:Populationrepresentationduringspiraltracing.J.Neurophysiol.,82:2693–
2704.

260

APHYRBIBLIOG

Moran,D.andSchwartz,A.(1999b).Motorcorticalrepresentationofspeedand
directionduringreaching.J.Neurophysiol.,82:2676–2692.

Moran,J.andDesimone,R.(1985).Selectiveattentiongatesvisualprocessinginthe
extrastriatecortex.Science,229:782–784.

Morizio,J.,Irazoqui,P.,Go,V.,andParmentier,J.(2005).Wirelessheadstagefor
neuralprosthetics.NeuralEngineering,2005,2nd:414–417.

Motter,B.(1993).Focalattentionproducesspatiallyselectiveprocessinginvisual
corticalareasv1,v2,andv4inthepresenceofcompetingstimuli.Journalof
70(3):909–919.,gyophysioloNeur

Mottet,D.andBootsma,R.(2001).Thedynamicsofrhythmicalaimingin2dtask
space:relationbetweengeometryandkinematicsunderexamination.HumMov
,Sci.20(3):213–241.

M¨uller,K.,Gopfert,E.,andHartwig,M.(1985).Vep-untersuchungenzurkodierung
dergeschwindigkeitbewegterstreifenmusterimkortexdesmenschen.EEG-EMG,
2(16):75–80.

Mumford,D.(2002).Patterntheory:themathematicsofperception.InTatsien,
L.,editor,InternationalCongressofMathematicians,Beijing,China,volumeIII,
pages401–422.WorldScientific.

Murata,N.,Kawanabe,M.,Ziehe,A.,Mueller,K.-R.,andAmari,S.(2002).On-line
learninginchangingenvironmentswithapplicationsinsupervisedandunsuper-
visedlearning.NeuralNetworks,15:743–760.

Murphy,K.(2002).Hiddensemi-markovmodels.Technicalreport,MITAILab.

Musallam,S.,Corneil,B.,Greger,B.,Scherberger,H.,andAndersen,R.(2004).Cog-
nitivecontrolsignalsforneuralprosthetics.Science,305:258–262.

Mushiake,H.,Tanatsugu,Y.,andTanji,J.(1997).Neuronalactivityintheventral
partofpremotorcortexduringtarget-reachmovementismodulatedbydirection
ofgaze.J.Neurophysiol.,78:567–571.

Mussa-Ivaldi,F.andBizzi,E.(2000).Motorlearningthroughthecombinationof
primitives.PhilosophicalTransactionsoftheRoyalSocietyB:BiologicalSciences,
355(1404):1755–1769.

Mysore,ariesS.,inVmogels,acaqRue.,v4.JRaiguel,NeurS.,andophysiolO,rban,G.95:1864–1880.(2006).Processingofkineticbound-

Nakahara,H.,Itoh,H.,Kawagoe,R.,Takikawa,Y.,andHikoska,O.(2004).Dopamine
neuronscanrepresentcontext-dependentpredictionerror.Neuron,41:269–280.

APHYRBIBLIOG

261

Natschl¨ager,T.,Maass,W.,andMarkram,H.(2002).Theliquidcomputer:Anovel
strategyforreal-timecomputingontimeseries.SpecialIssueonFoundationsof
InformationProcessingofTELEMATIK,8(1):39–43.
Neal,R.(1993).Probabilisticinferenceusingmarkovchainmontecarlomethods.
Technicalreport,UniversityofToronto.
Neal,R.andHinton,G.(1999).LearninginGraphicalModel–Chapter:Aviewof
theEMalgorithmthatjustifiesincremental,sparse,andothervariants.
Nieuwenhuis,S.,Yeung,N.,Holroyd,C.,Schurger,A.,andCohen,J.(2004).Sensi-
tivityofelectrophysiologicalactivityfrommedialfrontalcortextoutilitarianand
performancefeedback.CerebCortex,14(7):741–747.
Niki,H.andWatanabe,M.(1979).Prefrontalandcingulateunitactivityduringtiming
behaviorinthemonkey.BrainRes.,171(2):213–224.
Nobre,A.,Sebestyen,G.,andMiniussi,C.(2000).Thedynamicsofshiftingvisuospa-
tialattentionrevealedbyevent-relatedpotentials.Neuropsychologia.,38(7):964–
974.Norris,J.(1997).MarkovChains.
Nunez,P.(1995).Neocorticaldynamicsandhumaneegrhythms.OxfordUniversity
Press,NewYork,pages3–67.
Nunez,P.,Srinivasan,R.,Westdorp,A.,Wijesinghe,R.,Tucker,D.,Silberstein,R.,
andCadusch,P.(1997).Eegcoherencyi:statistics,referenceelectrode,volume
conduction,laplacians,corticalimaging,andinterpretationatmultiplescales.
ElectroencephalogrClinNeurophysiol,103:499–515.
Olshausen,B.andField,D.(1996).Emergenceofsimplecellsreceptivefieldproperties
bylearningasparsecodefornaturalimages.Nature,381:607–609.
O’Regan,J.,Rensink,R.,andClark,J.(1999).Change-blindnessasaresultof’mud-
splashes’.Nature,398:34.
Paninski,L.,Fellows,M.,Hatsopoulos,N.,andDonoghue,J.(2004).Spatiotemporal
tuningofmotorcorticalneuronsforhandpositionandvelocity.J.Neurophysiol,
91:515–532.Panzeri,S.,Treves,A.,Schultz,S.,andRolls,E.(1999).Ondecodingtheresponsesofa
populationofneuronsfromshorttimeepochs.NeuralComputation,11:1553–1577.
Parker,D.(1982).Learninglogic.Technicalreport,MITCenterforComputational
ResearchinEconomicsandMangementScience.
Passingham,R.(1993).Thefrontallobesandvoluntaryaction.

262

APHYRBIBLIOG

Pasupathy,A.andConnor,C.(1999).Responsestocontourfeaturesinmacaquearea
v4.JNeurophysiol,82:2490–2502.
Pasupathy,A.andConnor,C.(2001).Shaperepresentationinareav4:Position-specific
tuningforboundaryconformation.JournalofNeurophysiology,86:2505–2519.
Pasupathy,A.andConnor,C.(2002).Populationcodingofshapeinareav4.Nature
Neuroscience,5:1332–1338.
Pauli,W.(1933).HandbuchderPhysik.
Paulus,K.,Magnano,I.,Piras,M.,Solinas,M.,Solinas,G.,Sau,G.,andAiello,
I.(2002).Visualandauditoryevent-relatedpotentialsinsporadicamyotrophic
lateralsclerosis.ClinNeurophysiol.,113(6):853–861.
Paus,T.(2001).Primateanteriorcingulatecortex:wheremotorcontrol,driveand
cognitioninterface.NatRevNeurosci.,2(6):417–424.
Pawelzik,K.,Ernst,U.,andRotermund,D.(2004).Computingspike-by-spike.Dy-
namicPerceptionWorkshop,pages145–150.
Pawelzik,K.,Ernst,U.,andRotermund,D.(2006a).On-lineadaptationofneuro-
prostheseswithneuronalevaluationsignals.ProceedingsESANN’06,pages53–58.
Pawelzik,K.,Herden,B.,andStieler,W.(2006b).Interview:Dasistnichtderzugang
derphysik.TechnologyReview,1.
Pawelzik,K.andRotermund,D.(2005).filedpatent:PCTWO002005094669A1.
Pawelzik,K.,Rotermund,D.,andErnst,U.(2003).Buildingrepresentationsspikeby
spike.Proceedingsofthe29thG¨ottingenNeurobiologyConference.
Pawelzik,K.,Rotermund,D.,andErnst,U.(2006c).Towardson-lineadaptationof
neuro-prostheseswithneuronalevaluationsignals.NeuroscienceMeetingPlanner
13.13.,2006Pawelzik,K.,Rotermund,D.,Taylor,K.,Ernst,U.,andKreiter,A.(2006d).Attention
improvesobjectencodinginmonkeyareav4.FENS2006,9915010554.
Pearce,J.andHall,G.(1980).Amodelforpavlovianlearning:variationsintheeffec-
tivenessofconditionedbutnotofunconditionedstimuli.PsycholRev.,87(6):532–
552.Pearl,J.(1988).ProbabilisticReasoninginIntelligentSystems:NetworksofPlausible
.eencInferPearl,J.(2000).Causality:Models,reasoning,andinference.
Pearl,J.andRussell,S.(2003).TheHandbookofBrainTheoryandNeuralNetworks
–Chapter:BayesianNetworks.

APHYRBIBLIOG

263

Pearlmutter,B.(1990).Dynamicrecurrentneuralnetworks.Technicalreport.
Penfield,W.andRasmussen,T.(1950).Thecerebralcortexofman.
Perrin,F.,Bertrand,O.,andPernier,J.(1987).Scalpcurrentdensitymapping:value
andestimationfrompotentialdata.IEEETransBiomedEng,34:283–288.
Perrinet,L.,Samuelides,M.,andThorpe,S.(2004).Codingstaticnaturalimages
usingspikingeventtimes:Doneuronscooperate.IEEETrans.Neur.Networks,
15:1164–1175.Petersen,R.,Panzeri,S.,andDiamond,M.(2002).Populationcodinginsomatosensory
cortex.CurrOpinNeurobiol.,12(4):441–447.
Pfurtscheller,G.andNeuper,C.(2001).Motorimageryanddirectbrain-computer
communication.Neuralengineering:mergingengineeringandneuroscience,
89(7):1123–1134.Picard,N.andStrick,P.(1996).Motorareasofthemedialwall:areviewoftheir
locationandfunctionalactivation.CerebCortex.,6(3):342–353.
Pinsk,M.,Doniger,G.,andKastner,S.(2004).Push-pullmechanismofselective
attentioninhumanextrastriatecortex.JNeurophysiol,92:622–629.
Platt,M.andGlimcher,P.(1999).Neuralcorrelatesofdecisionvariablesinparietal
cortex.Nature,400:233–238.
Pollen,D.(1999).Ontheneuralcorrelatesofvisualperception.CerebralCortex,
9(1):4–19.Pouget,A.,Dayan,P.,andZemel,R.S.(2003).Inferenceandcomputationwith
populationcodes.AnnualReviewsNeuroscience,26:381–410.
Quiroga,R.Q.,Reddy,L.,Kreiman,G.,Koch,C.,andFried,I.(2005).Invariantvisual
representationbysingleneuronsinthehumanbrain.Nature,435:1102–1107.
Rabiner,L.R.(1989).Atutorialonhiddenmarkovmodelsandselectedapplications
inspeechrecognition.ProceedingsoftheIEEE,77(2):257–286.
Rao,C.(1946).Informationandtheaccuracyattainableintheestimationofstatistical
parameters.Bull.CalcuttaMath.Soc.,37:81–91.
Rao,R.P.(2004).Bayesiancomputationinrecurrentneuralcircuits.NeuralCompu-
16:1–38.,tationRescorla,R.andWagner,A.(1972).ClassicalconditioningII:Currentresearchand
theoryChapter:Atheoryofclassicalconditioning:variationsintheeffectiveness
ofreinforcementandnon-reinforcement.
Reynolds,J.andChelazzi,L.(2004).Attentionalmodulationofvisualprocessing.
AnnualReviewofNeuroscience,27:611–647.

264

APHYRBIBLIOG

Reynolds,J.,Chelazzi,L.,andDesimone,R.(1999).Competitivemechanismssubserve
attentioninmacaqueareasv2andv4.J.Neurosci.,19:1736–1753.
Reynolds,J.andDesimone,R.(2003).Interactingrolesofattentionandvisualsalience
inv4.Neuron,37(5):853–863.
Reynolds,J.,Pasternak,T.,andDesimone,R.(2000).Attentionincreasessensitivity
ofv4neurons.Neuron,26(3):703–714.
Richardson,W.(1972).Bayesian-basediterativemethodofimagerestoration.J.Opt.
Soc.Am.,62:55–59.
Rickert,J.,deOliveira,S.C.,Vaadia,E.,Aertsen,A.,Rotter,S.,andMehring,C.
(2005).Encodingofmovementdirectionindifferentfrequencyrangesofmotor
corticallocalfieldpotentials.JNeurosci.,25(39):8815–8824.
Ridderinkhof,K.,Ullsperger,M.,Crone,E.,andNieuwenhuis,S.(2004a).Theroleof
themedialfrontalcortexincognitivecontrol.Science,306:443–447.
Ridderinkhof,K.,vandenWildenberg,W.,Segalowitz,S.,andCarter,C.(2004b).
Neurocognitivemechanismsofcognitivecontrol:Theroleofprefrontalcortexin
actionselection,responseinhibition,performancemonitoring,andreward-based
learning.BrainandCognition,56:129–140.
Ringach,D.(2002).Spatialstructureandsymmetryofsimple-cellreceptivefieldsin
macaqueprimaryvisualcortex.JNeurophysiol,88:455–463.
Ripley,B.(1987).Stochasticsimulation.
Rizzolatti,G.andWolpert,D.(2005).Motorsystems.CurrOpinNeurobiol.,
15(6):623–625.Robbe,D.,Montgomery,S.,Thome,A.,Rueda-Orozco,P.,McNaughton,B.,and
Buzsaki,G.(2006).Cannabinoidsrevealimportanceofspiketimingcoordination
inhippocampalfunction.NatNeurosci,9:1526–1533.
Rock,I.andGutman,D.(1981).Theeffectofinattentiononformperception.journal
ofexperimentalpsychology.Humanperceptionandperformance,7:275–285.
Rock,I.,Linnett,C.,Grant,P.,andMack,A.(1992).Perceptionwithoutattention:
resultsofanewmethod.Cognit.Psychol.,24:502–534.
Roesch,M.andOlson,C.(2003).Impactofexpectedrewardonneuronalactivityin
prefrontalcortex,frontalandsupplementaryeyefieldsandpremotorcortex.J.
90:1766–1789.,ophysiol.NeurRoland,P.andZilles,K.(1996).Functionsandstructuresofthemotorcorticesin
humans.CurrOpinNeurobiol.,6(6):773–781.

APHYRBIBLIOG

265

Rosenblatt,F.(1958).Theperceptron:aprobabilisticmodelforinformationstorage
andorganizationinthebrain.PsychologicalReviews,65:386–408.
Rotermund,D.,Ernst,U.,andPawelzik,K.(2005).Processingnaturalimageswith
singlespikes.Proceedingsofthe6thMeetingoftheGermanNeuroscienceSoci-
ety/30thG¨ottingenNeurobiologyConference,445A.
Rotermund,D.,Ernst,U.,andPawelzik,K.(2006a).Towardson-lineadaptationof
neuro-prostheseswithneuronalevaluationsignals.BiologicalCybernetics,95:243–
257.Rotermund,D.,Taylor,K.,Ernst,U.,Kreiter,A.,andPawelzik,K.R.(2007a).At-
tentionimprovesobject-specificityofcorticalstates.submittedtoPLoSBiology.
Rotermund,D.,Taylor,K.,Ernst,U.,Pawelzik,K.,,andKreiter,A.(2007b).At-
tentionimprovesobjectrepresentationinmonkeyareav4.Proceedingsofthe7th
MeetingoftheGermanNeuroscienceSociety/31thG¨ottingenNeurobiologyCon-
ference,T32-1A.
Rotermund,D.,Taylor,K.,Ernst,U.,Pawelzik,K.,andKreiter,A.(2006b).Attention
improvesobjectdiscriminabilityinmonkeyareav4.CNS,S2.
Rotermund,D.,Taylor,K.,Ernst,U.,Pawelzik,K.,andKreiter,A.(2007c).Attention
inmonkeyareav4ismodulatedbytaskdemand.submittedtoNeuroscience.
Rotermunda,D.,Taylor,K.,Ernst,U.,Pawelzik,K.,andKreiter,A.(2005).Attention
improvesobjectencodinginmonkeyareav4.SocietyforNeuroscienceAbstracts,
31:591.6.Rumelhart,D.,Hinton,E.,andWilliams,R.(1986a).Learningrepresentationsby
back-propogatingerrors.Nature,323:533–536.
Rumelhart,D.,Hinton,G.,andWilliams,R.(1986b).ParallelDistributedProcessing,
Vol.1–Chapter:Learninginternalrepresentationsbyerrorpropagation.
Rushworth,M.,Walton,M.,Kennerley,S.,andBannerman,D.(2004).Actionsets
anddecisionsinthemedialfrontalcortex.TrendsCognSci.,8(9):410–417.
Sahu,S.andRoberts,G.(1999).Onconvergenceoftheemalgorithmandthegibbs
sampler.StatisticsandComputingarchive,9(1):55–64.
Sakmann,B.andNeher,E.(1984).Patchclamptechniquesforstudyingionicchannels
inexcitablemembranes.AnnualReviewofPhysiology,455–472:46.
Salinas,E.andAbbott,L.(1994).Vectorreconstructionfromfiringrates.J.Compu-
tationalNeurosci.,1:89–107.
Sanes,J.andDonoghue,J.(2000).Plasticityandprimarymotorcortex.AnnuRev
Neurosci.,23:393–415.

266

RBIBLIOGAPHY

Satoh,T.,Nakai,S.,andKimura,M.(2003).Correlatedcodingofmotivationandout-
comeofdecisionbydopamineneurons.TheJournalofNeuroscience,23(30):9913–
9923.Schaal,S.(2002).Thehandbookofbraintheoryandneuralnetworks-Chapter:Arm
andhandmovementcontrol.
Schaal,S.andSternad,D.(2001).Originsandviolationsofthe2/3powerlawin
rhythmic3dmovements.ExperimentalBrainResearch,136:60–72.
Schalk,G.,Wolpaw,J.,McFarland,D.,andPfurtscheller,G.(2000).Eeg-basedcom-
munication:presenceofanerrorpotential.ClinNeurophysiol,111(12):2138–2144.
Schein,S.andDesimone,R.(1990).Spectralpropertiesofv4neuronsinthemacaque.
JNeurosci.,3369–3389:10(10).
Scherberger,H.,Jarvis,M.,andAndersen,R.(2005).Corticallocalfieldpotential
encodesmovementintentionsintheposteriorparietalcortex.Neuron,46(2):347–
354.Sch¨olkopf,B.,Burges,C.,andVapnik,V.(1995).Extractingsupportdataforagiven
task.Proceedings,FirstInternationalConferenceonKnowledgeDiscovery&Data
Mining,1:252–257.
Sch¨olkopf,B.andSmola,A.(2001).LearningwithKernels-SupportVectorMachines,
Regularization,OptimizationandBeyond.
Sch¨olkopf,B.,Smola,A.,Williamson,R.,andBartelett,P.(2000).Newsupportvector
algorithms.NeuralComp,12:1207–1245.
Schultz,W.(1998).Predictiverewardsignalofdopamineneurons.J.Neurophysiol.,
80:1–27.Schultz,W.(2004).Neuralcodingofbasicrewardtermsofanimallearningtheory,game
theory,microeconomicsandbehaviouralecology.CurrentOpinioninNeurobiology,
14:139–147.Schultz,W.,Dayan,P.,andMontague,P.(1997).Aneuralsubstrateofpredictionand
reward.Science,275:1593–1599.
Schulzke,E.(2006).NeuronaleKodierungundDekodierungvonmultiplenunddy-
namischenReizen.PhDthesis,UniversityofBremen.
Schwartz,A.(2004).Corticalneuralprosthetics.Annu.Rev.Neurosci.,27:487–507.
Schwartz,A.,Taylor,D.,andTillery,S.H.(2001).Extractionalgorithmsforcortical
controlofarmprosthetics.CurrentOpinioninNeurobiology,11:701–707.
Segev,I.andRall,W.(1998).Excitabledendritesandspines:earliertheoretical
insightselucidaterecentdirectobservations.TrendsNeurosci.,21:453–460.

APHYRBIBLIOG

267

Seiple,W.,Clemens,C.,Greenstein,V.,Holopigian,K.,andZhang,X.(2002).
Thespatialdistributionofselectiveattentionassessedusingthemultifocalvisual
evokedpotential.VisionRes.,42(12):1513–1521.
Sejnowski,T.andTesauro,G.(1989).Thehebbruleforsynapticplasticity:algorithms
andimplementations.NeuralModelsofPlasticity,Chapter6:94–103.
Serruya,M.andDonoghue,J.(2004).Neuroprosthetics:TheoryandPractice–Chap-
ter:DesignPrinciplesofaNeuromotorProstheticDevice.
Seung,H.andSompolinsky,H.(1993).Simplemodelsforreadingneuronalpopulation
codes.ProceedingsoftheNationalAcademyofSciences,90:10749–10753.
Shadlen,M.andNewsome,W.(1998).Thevariabledischargeofcorticalneurons:
implicationsforconnectivity,computationandinformationcoding.J.Neurosci.,
18(10):3870–3896.Shafer,G.andShenoy,P.(1998).Localcomputationinhypertrees.Technicalreport,
UniversityofKansas.
Shannon,C.(1948).Amathematicaltheoryofcommunication.BellSystemTechnical
623–656.and27:379–423,JournalShidara,M.,Aigner,T.,andRichmond,B.(1998).Neuronalsignalsinthemonkey
ventralstriatumrelatedtoprogressthroughapredictableseriesoftrials.The
JournalofNeuroscience,18(7):2613–2625.
Shima,K.andTanji,J.(1998).Roleforcingulatemotorareacellsinvoluntarymove-
mentselectionbasedonreward.Science,282:1335–1338.
Shirow,M.(1991).GhostintheShell.
Silberberg,G.,Bethge,M.,Markram,H.,Pawelzik,K.,andTsodyks,M.(2004).
Dynamicsofpopulationratecodesinensemblesofneocorticalneurons.Journal
ofNeurophysiology,91:704–709.
Simons,D.andRensink,R.(2005).Changeblindness:past,present,andfuture.
TrendsCognSci.,9(1):16–20.
Singer,W.andGray,C.(1995).Visualfeatureintegrationandthetemporalcorrelation
hypothesis.Ann.Rev.Neurosci.,18:555–586.
Skottun,B.,Bradley,A.,Sclar,G.,Ohzawa,I.,andFreeman,R.(1987).Theeffectsof
contrastonvisualorientationandspatialfrequencydiscrimination:Acomparison
ofsinglecellsandbehaviour.J.Neurophysiol.,57:773–786.
Smith,J.(2001).Someobservationsontheconceptsofinformation-theoreticentropy
andrandomness.Entropy,3:1–11.

268

APHYRBIBLIOG

Smiyukha,Y.,Mandon,S.,Galashan,F.,Neitzel,S.,andKreiter,A.(2006).Attention-
dependentswitchingofinterarealsynchronizationbetweenv4neuronsanddiffer-
entsubpopulationsoftheirv1afferents.SocNeurosciAbstr,32:11.2.
Snowden,R.,Treue,S.,andAndersen,R.(1992).Theresponseofneuronsinareasv1
andmtofthealertrhesusmonkeytomovingrandomdotpatterns.Experimental
BrainResearch,88:389–400.
Spencer,K.,Dien,J.,andDonchin,E.(2001).Spatiotemporalanalysisofthelateerp
responsestodeviantstimuli.Psychophysiology.,38(2):343–358.
Steinmetz,P.,Roy,A.,Fitzgerald,P.,Hsiao,S.,Johnson,K.,andNiebur,E.(2000).
Attentionmodulatessynchronizedneuronalfiringinprimatesomatosensorycor-
tex.Nature,404:187–190.
Sternad,D.andSchaal,S.(1999).Segmentationofendpointtrajectoriesdoesnot
implysegmentedcontrol.ExperimentalBrainResearch,124(1):118–136.
Stuphorn,V.,Bauswein,E.,andHoffmann,K.-P.(2000).Neuronsintheprimate
superiorcolliculuscodingforarmmovementsingaze-relatedcoordinates.J.Neu-
83:1283–1299.,ophysiol.rSutton,R.andBarto,A.(1998).ReinforcementLearning:AnIntroduction.MIT
ess.PrSutton,S.,Braren,M.,Zublin,J.,andJohn,E.(1965).Evokedpotentialcorrelatesof
stimulusuncertainty.Science,150:1187–1188.
Taira,M.,Boline,J.,Smyrnis,N.,Georgopoulos,A.,andAshe,J.(1996).Onthe
relationsbetweensinglecellactivityinthemotorcortexandthedirectionand
magnitudeofthree-dimensionalstaticisometricforce.ExperimentalBrainRe-
109(2):367–376.,charseTanji,J.andShima,K.(1994).Roleforsupplementarymotorareacellsinplanning
severalmovementsahead.Nature,371:413–416.
Taylor,D.,Tillery,S.H.,andSchwartz,A.(2002).Directcorticalcontrolof3D
neuroprostheticdevices.Science,296:1829–1832.
Taylor,K.,Mandon,S.,Freiwald,W.,andKreiter,A.(2005).Coherentoscillatory
activityinmonkeyareav4predictssuccessfulallocationofattention.CerebCortex,
15:1424–1437.Teder-Salejarvi,W.,Munte,T.,Sperlich,F.,andHillyard,S.(1999).Intra-modaland
cross-modalspatialattentiontoauditoryandvisualstimuli.anevent-relatedbrain
potentialstudy.BrainResCognBrainRes.,8(3):327–343.
Thoroughman,K.andShadmehr,R.(2000).Learningofactionthroughadaptive
combinationofmotorprimitives.Nature,407:742–747.

RBIBLIOGAPHY

269

Thorpe,S.,Delorme,A.,andvanRullen,R.(2001).Spike-basedstrategiesforrapid
processing.NeuralNetworks,14:715–725.
Thorpe,S.J.,Fize,D.,andMarlot,C.(1996).Speedofprocessinginthehumanvisual
system.Nature,381:520–522.
Tillery,S.H.,Taylor,D.,andSchwartz,A.(2003).Trainingincorticalcontrolof
neuroprostheticdevicesimprovessignalextractionfromsmallneuronalensembles.
Rev.Neurosci.,14(1-2):107–119.
Tobler,P.,Fiorillo,C.,andSchultz,W.(2005).Adaptivecodingofrewardvalueby
dopamineneurons.Science,307:1642–1645.
Todorov,E.(2000).Directcorticalcontrolofmuscleactivationinvoluntaryarm
movements:amodel.NatNeurosci.,3(4):391–398.
Tolhurst,D.,Movshon,J.,andDean,A.(1983).Thestatisticalreliabilityofsignalsin
singleneuronsincatandmonkeystriatecortex.VisionResearch,23:775–785.
Tolias,A.,Keliris,G.,Smirnakis,S.,andLogothetis,N.(2005).Neuronsinmacaque
areav4acquiredirectionaltuningafteradaptationtomotionstimuli.Nature
Neuroscience,8(5):591–593.
Tomko,G.andCrapper,D.(1974).Neuronalvariability:non-stationaryresponsesto
identicalvisualstimuli.BrainRes.,79(3):405–18.
Tootell,R.,Hadjikhani,N.,Hall,E.,Marrett,S.,Vanduffel,W.,Vaughan,J.,and
Dale,A.(1998).Theretinotopyofvisualspatialattention.Neuron,21:1409–1422.
Treisman,A.andGelade,G.(1980).Afeature-integrationtheoryofattention.Cognit
,Psychol.12(1):97–136.Tremblay,L.andSchultz,W.(1999).Relativerewardpreferenceinprimateor-
bitofrontalcortex.Nature,398:704–708.
Tremblay,L.andSchultz,W.(2000).Reward-relatedneuronalactivityduringGo-Nogo
taskperformanceinprimateorbitofrontalcortex.J.Neurophysiol.,83:1864–1876.
Tresch,M.,Saltiel,P.,andBizzi,E.(1999).Theconstructionofmovementbythe
spinalcord.NatureNeuroscience,2:162–167.
Treue,S.(2001).Neuralcorrelatesofattentioninprimatevisualcortex.TRENDSin
24(5):295–300.,esosciencNeurTreue,S.andMaunsell,J.(1996).Attentionalmodulationofvisualmotionprocessing
incorticalareasmtandmst.Nature,382:539–541.
Treue,S.andTrujillo,J.M.(1999).Feature-basedattentioninfluencesmotionpro-
cessinggaininmacaquevisualcortex.Nature,399:575–579.

270

APHYRBIBLIOG

Tuckwell,H.(1979).Synaptictransmissioninamodelforstochasticneuralactivity.J
77(1):65–81.,Biol.orTheTuckwell,H.(1988).Introductiontotheoreticalneurobiology.
Uno,Y.,Kawato,M.,andSuzuki,R.(1989).Formationandcontrolofoptimaltra-
jectoryinhumanmultijointarmmovement.minimumtorque-changemodel.Biol
61(2):89–101.,rn.eCybUsrey,W.,Alonso,J.-M.,andReid,R.(2000).Synapticinteractionsbetweenthalamic
inputstosimplecellsincatvisualcortex.J.Neurosci.,20:561–567.
vanRullen,R.andKoch,C.(2005).HandbookofclinicalneurophysiologyChapter:
VisualAttentionandVisualAwareness.
vanRullen,R.andThorpe,S.J.(2001).Ratecodingversustemporalordercod-
ing:Whattheretinalganglioncelltellsthevisualcortex.NeuralComputation,
13:1255–1283.vanRullen,R.andThorpe,S.J.(2002).Surfingaspikewavedowntheventralstream.
VisionResearch,42:2593–2615.
vanSchie,H.,Mars,R.,Coles,M.,andBekkering,H.(2004).Modulationofactivityin
medialfrontalandmotorcorticesduringerrorobservation.NatureNeuroscience,
7:549–554.vanVeen,V.,Holroyd,C.,Cohen,J.,Stenger,V.,andCarter,S.(2004).Errorswithout
conflict:Implicationsforperformancemonitoringtheoriesofanteriorcingulate
cortex.BrainandCognition,56:267–276.
Vanni,S.,Portin,K.,Virsu,V.,andHari,R.(1999).Murhythmmodulationduring
changesofvisualpercepts.Neuroscience.,91(1):21–31.
Villringer,A.andChance,B.(1997).Non-invasiveopticalspectroscopyandimaging
ofhumanbrainfunction.TINS,20(10):435–442.
Viviani,P.andFlash,T.(1995).Minimum-jerk,two-thirdspowerlaw,andisochrony:
convergingapproachestomovementplanning.JExpPsycholHumPerceptPer-
21(1):32–53.,form.vonderMalsburg,C.(1985).Nervousstructureswithdynamicallinks.Ber.Bunsenges.
89:703–710.,Chem.Phys.vonHelmholtz,H.(1860).HandbuchderPhysiologischenOptik-BandII.
Walsh,B.(2002).Markovchainmontecarloandgibbssampling.Technicalreport,
LectureNotesforEEB596z.
Walton,M.,Bannerman,D.,Alterescu,K.,andRushworth,M.(2003).Functional
specializationwithinmedialfrontalcortexoftheanteriorcingulateforevaluating
effort-relateddecisions.JNeurosci.,23(16):6475–6479.

APHYRBIBLIOG

271

Warland,D.,Reinagel,P.,andMeister,M.(1997).Decodingvisualinformationfrom
apopulationofretinalganglioncells.J.Neurophysiol,78:2336–2350.
Warzecha,A.andEgelhaaf,M.(2000).Responselatencyofamotion-sensitiveneuron
intheflyvisualsystem:dependenceonstimulusparametersandphysiological
conditions.VisionRes.,40(21):2973–2983.
Watanabe,M.,Hikosaka,K.,Sakagami,M.,andShirakawa,S.-I.(2002a).Codingand
monitoringofmotivationalcontextintheprimateprefrontalcortex.TheJournal
ofNeuroscience,22(6):2391–2400.
Watanabe,M.,Tanaka,H.,Uka,T.,andFujita,I.(2002b).Disparity-selectiveneurons
inareav4ofmacaquemonkeys.TheJournalofNeurophysiology,87(4):1960–1973.
Weiskopf,N.,Mathiak,K.,Bock,S.,Scharnowski,F.,Veit,R.,Grodd,W.,Goebel,R.,
andBirbaumer,N.(2004).Principlesofabrain-computerinterface(bci)based
onreal-timefunctionalmagneticresonanceimaging(fmri).IEEETransBiomed
51(6):966–970.,Eng.Welch,G.andBishop,G.(2004).Anintroductiontothekalmanfilter.Technical
report,UniversityofNorthCarolinaatChapelHill.
Welling,M.andHinton,G.(2002).Anewlearningalgorithmformeanfieldboltzmann
machines.LectureNotesInComputerScience,2415:351–357.
Werbos,P.(1974).Beyondregression:newtoolsforpredictionandanalysisinthe
behavioralsciences.PhDthesis,HarvardUniversity.
Wessberg,J.,Stambaugh,C.,Kralik,J.,Beck,P.,Laubach,M.,Chapin,J.,Kim,J.,
Biggs,S.,Srinivasan,M.,andNicolelis,M.(408).Real-timepredictionofhand
trajectorybyensemblesofcorticalneuronsinprimates.Nature,2000:361–365.
Wilke,S.andEurich,C.(2002).Onthefunctionalroleofnoisecorrelationsinthe
nervoussystem.Neurocomputing,44–46:1023–1028.
Wilkinson,F.,James,T.,Wilson,H.,Gati,J.,Menon,R.,andGoodale,M.(2000).
Anfmristudyoftheselectiveactivationofhumanextrastriateformvisionareas
byradialandconcentricgratings.CurrentBiology,10(22):1455–1458.
Wilson,H.R.andCowan,J.D.(1972).Excitatoryandinhibitoryinteractionsin
localizedpopulationsofmodelneurons.Kybernetik,12:1–24.
Wise,S.(1985).Theprimatepremotorcortex:past,present,andpreparatory.Annu
RevNeurosci.,8:1–19.
Wolfe,J.(1998).Attention-chapter:Visualsearch.
Wolfe,J.andBennett,S.(1997).Preattentiveobjectfiles:shapelessbundlesofbasic
features.VisionRes.,37:25–43.

272

BIBLIOGAPHYR

Wolpaw,J.,Birbaumer,N.,Heetderks,W.,McFarland,D.,Peckhamand,P.,Schalk,
G.,Donchin,E.,Quatrano,L.,Robinson,C.,andVaughan,T.(2000).Brain-
computerinterfacetechnology:areviewofthefirstinternationalmeeting.IEEE
Trans.Rehab.Eng.,8(2):164–173.
Wolpaw,J.,Birbaumer,N.,McFarland,D.,Pfurtscheller,G.,andVaughan,T.(2002).
Brain-computerinterfacesforcommunicationandcontrol.Clin.Neurophys.,
113:767–791.Wolpaw,J.andMcFarland,D.(2004a).Controlofatwo-dimensionalmovementsig-
nalbyanoninvasivebrain-computerinterfaceinhumans.ProcNatlAcadSci,
101(51):17849–17854.Wolpaw,J.andMcFarland,D.(2004b).Controlofatwo-dimensionalmovementsig-
nalbyanoninvasivebrain-computerinterfaceinhumans.PNAS,101(51):17849–
17854.Wolpert,D.andGhahramani,Z.(2000).Computationalprinciplesofmovementneu-
roscience.NatureNeuroscience,3:1212–1217.
Womelsdorf,T.,Anton-Erxleben,K.,Pieper,F.,andTreue,S.(2006).Dynamicshifts
ofvisualreceptivefieldsincorticalareamtbyspatialattention.NatNeurosci,
9:1156–1160.Woolsey,C.,Settlage,P.,Meyer,D.,Sencer,W.,Hamuy,T.P.,andTravis,A.(1952).
Patternsoflocalizationinprecentralandsupplementarymotorareasandtheir
relationtotheconceptofapremotorarea.ResPublAssocResNervMentDis.,
30:238–264.W¨org¨otter,F.andPorr,B.(2005).Temporalsequencelearning,predictionandcontrol
-areviewofdifferentmodelsandtheirrelationtobiologicalmechanisms.Neural
17:245–319.,Comp.Xie,X.andSeung,H.(2004a).Learninginneuralnetworksbyreinforcementofirreg-
ularspiking.PhysicalReviewE,69:041909.
Xie,X.andSeung,H.S.(2004b).Learninginneuralnetworksbyreinforcementof
irregularspiking.PhysRevEStatNonlinSoftMatterPhys.,69(4Pt1):041909.
Xu,W.,Guan,C.,Siong,C.,Ranganatha,S.,Thulasidas,M.,andWu,J.(2004).High
accuracyclassificationofeegsignal.ProceedingsofthePatternRecognition,17th
InternationalConferenceon(ICPR’04),2:391–394.
Xu,X.,Ichida,J.,Allison,J.,Boyd,J.,Bonds,A.,andCasagrande,V.(2001).A
comparisonofkoniocellular,magnocellularandparvocellularreceptivefieldprop-
ertiesinthelateralgeniculatenucleusoftheowlmonkey(aotustrivirgatus).The
JournalofPhysiology,531(1):203–218.
Yantis,S.(1998).Attention-Chapter:Controlofvisualattention.

APHYRBIBLIOG

273

Yeung,N.,Botvinick,M.,andCohen,J.(2004a).Theneuralbasisoferrordetec-
tion:Conflictmonitoringandtheerror-relatednegativity.PsychologicalReview,
111(4):931–959.

Yeung,conflictN.,mCohen,onitoringJ.,andandBtheotvinick,error-relatedM.(2004b).negativitThey.neuralPsycholbasRisev.of,errord111(4):931–959.etection:

Yoo,S.,Fairneny,T.,Chen,N.,Choo,S.,Panych,L.,Park,H.,Lee,S.,andJolesz,
F.(2004).Brain-computerinterfaceusingfmri:spatialnavigationbythoughts.
Neuroreport.,15(10):1591–1595.

Young,T.(1802).Onthetheoryoflightandcolors.PhilosophicalTransactionsofthe
RoyalSociety,91:12–49.

Zeki,S.(1993).Avisionofthebrain.

Zhang,N.andPoole,D.(1994).Asimpleapproachtobayesiannetworkcomputations.
ProceedingsoftheTenthBiennialCanadianArtificialIntelligenceConference,AI-
94:171–178.

Zipser,K.,Lamme,V.,andSchiller,P.(1996).Contextualmodulationinprimary
visualcortex.TheJournalofNeuroscience,16(22):7376–7389.

Zrenner,E.(2002).Willretinalimplantsrestorevision?Science,295(5557):1022–1025.

274

APHYRBIBLIOG

Publications

sArticle

•D.improvesRotermobjeund,ctK.-speTacificitylor,yUof.A.cortiErnst,calstaA.K.tes,sKreiter,ubmittedandtKo.R.PPLoSawBelzik,iologyA(2007).ttention

•tiU.csErnst,pikes,DN.euralRotermCund,omputationandK.1P9aw(5),elzik,1313-1343Efficientc(2007).omputationbasedonstochas-

•D.Rotermund,U.Ernst,andK.Pawelzik,Towardson-lineadaptationofneuro-
prosthesiswithneuronalevaluationsignals,BiologicalCybernetics95(3),243-257
(2006).

•M.Bethge,D.Rotermund,andK.Pawelzik,Asecondorderphasetransition
inneuralratecoding:Binaryencodingisoptimalforrapidsignaltransmission,
Phys.Rev.Lett.90(8),088104-1(2003).

•M.Bethge,D.Rotermund,andK.Pawelzik,Optimalneuralratecodingleadsto
bimodalfiringratedistributions,Network:Comput.NeuralSyst.14,303-319
(2003).

•raM.tecoBethge,dingDwit.hRhighotermteund,mporandalrKe.Psolutawionelzik,,AdvBinaryancestinuningNeuralisoptimalInformationforneurPro-al
cessingSystems1515,S.Becker,S.Thrun,K.Obermayer(ed.),MITPress,
(2003).189-196

•M.Bethge,D.Rotermund,andK.Pawelzik,Optimalshort-termpopulation
coding:WhenFisherinformationfails,NeuralComputation14(10),2317-2351
(2002).

FtsatenPiled

•K.Pawelzik,andD.Rotermund,SystemundineinGewebevonLebewesenim-
plantierbareVorrichtungzurErfassungundBeinflussungvonelektrischerBio-
Aktivit¨at,in:DE102004014694A1,WO002005094669A1,(2004).

275

276

ublicationsP:DChapter

publicationsOther•D.Rotermund,K.Taylor,U.A.Ernst,K.R.Pawelzik,andA.K.Kreiter,Attention
inmonkeyareaV4ismodulatedbytaskdemand,in:submittedtoNeuroscience
(2007).•D.Rotermund,K.Taylor,U.A.Ernst,K.R.Pawelzik,andA.K.Kreiter,Atten-
tionimprovesobjectrepresentationinmonkeyareaV4,in:Proceedingsofthe
7thMeetingoftheGermanNeuroscienceSociety/31thG¨ottingenNeurobiology
Conference,T32-1A(2007).
•K.Pawelzik,U.Ernst,andD.Rotermund,On-lineadaptationofneuro-prostheses
withneuronalevaluationsignals,in:ProceedingsESANN’06,MichelVerleysen
(ed.),d-side,Evere,Belgium,53-58(2006).
•K.Pawelzik,D.Rotermund,K.Taylor,U.Ernst,andA.Kreiter,Attention
improvesobjectencodinginmonkeyareaV4,in:FENS2006,9915010554(2006).
•K.Pawelzik,D.Rotermund,andU.Ernst,towardson-lineadaptationofneuro-
prostheseswithneuronalevaluationsignals,in:NeuroscienceMeetingPlanner
(2006).13.132006,•D.Rotermund,K.Taylor,U.Ernst,K.Pawelzik,andA.Kreiter,Attention
improvesobjectdiscriminabilityinmonkeyareaV4,in:CNS,S2(2006).
•D.Rotermund,U.Ernst,andK.Pawelzik,Processingnaturalimageswithsingle
spikes,in:Proceedingsofthe6thMeetingoftheGermanNeuroscienceSoci-
ety/30thG¨ottingenNeurobiologyConference,445A(2005).
•D.Rotermund,K.Taylor,U.Ernst,K.Pawelzik,andAndreasKreiter,Atten-
tionimprovesobjectencodinginmonkeyareaV4,in:SocietyforNeuroscience
Abstracts31,591.6(2005).
•U.Ernst,D.Rotermund,andK.Pawelzik,Analgorithmforfastpatternrecogni-
tionwithrandomspikes,in:26thDAGMSymposiumProceedings,Rasmussen,
C.E.etal.(ed.),Springer-Verlag,399-406(2004).
•K.Pawelzik,U.Ernst,andD.Rotermund,Computingspike-by-spike,in:Dy-
namicPerceptionWorkshop,UweJ.Ilg,HeinrichH.B¨ulthoff,andHanspeterA.
Mallot(eds.),IOSPress,145-150(2004).
•K.Pawelzik,D.Rotermund,andU.Ernst,Buildingrepresentationsspikeby
spike,in:Proceedingsofthe29thG¨ottingenNeurobiologyConference,N.Elsner
andH.Zimmermann(eds.),GeorgThiemeVerlag,Stuttgart,1041(2003).
•M.Bethge,D.Rotermund,andK.Pawelzik,Optimalneuralpopulationcod-
ingandFisherinformation,in:VerhandlungenderDeutschenPhysikalischen
GesellschaftDPG(VI)37,1,V.H¨aselbarth(ed.),Physik-VerlagGmbH,509
(2002).

277

K.R.Pawelzik,U.A.Ernst,D.Trenner,andD.Rotermund,Buildingrepre-
sentationsspikebyspike,in:SocietyofNeuroscienceConference2002,Orlando,
(2002).557.12

M.Bethge,D.Rotermund,andK.Pawelzik,Optimalshort-termpopulationcod-
roing:biologWhenyCoFishernference,N.informationElsnerfailsan,din:G.ProW.Kceedingsreutzbofergthe(eds.),28thGG¨eorgottingenThiemeNeu-
Verlag,Stuttgart,250(2001).

278

Chapter

:D

ublicationsP

Acknowledgment/Danksagung

AmEndediesesWerkes,mitdemichmich¨uber5.5Jahrebesch¨aftigthabe,m¨ochteich
gerneeinigenMenschendanken,diemichbeimeinerArbeitunterst¨utztundbegleitet
n.ehab

Allenvoranm¨ochteichmichbeiKlausPawelzikbedanken,dassermirdieM¨oglichkeit
beiihmzupromovierengegebenundmichstetsunterst¨utzthat.Ohneihn,als¨uber-
langwsprudelndeneiliger.EundrhatnievmireeinersiegendemenormeQFruelleiheitneuerbeiImdeen,einerwF¨areorscmhungeineArbgelasseneitumundeinwavrbielesei
seineProblemenkostbiaremmernDzaturens¨aStelle.tzeAherauchngmela¨ochssenteichaht,beAndreasidenKDireiterskussiodanknenen,¨ubdasserediermichdarausan
hervorgegangenErgebnissedenFingerimmerwiederindieWundegelegtundtrotz
WeextremrkzuebengemgutacZehten.itplansichdieZeitgenommenhat,diesesdochrechtumfangreiche

Besondersinspirierendwarf¨urmichdieZusammenarbeitmitMatthiasBethgeund
UdoErnst.Esistwirklichschade,dasesindenletztenJahrennichtzueinerWeit-
erf¨uhrungZusammenarbdergeitwaremeinsamenimmerFhoocrschhgradigungmit’optimal’MatthiasundfB¨urethgemicghekimmerommensiehrst.lUehrreicnsereh.
MitUdoErnstistesimmerm¨oglichdiekompliziertestenProblemeanzugehenundoft
auchzul¨osen,obwohlunsdochdaseineoderandereAdventuregen¨otigthat,indem
entsprechendenL¨osungsbuchnachzuschauen.

Icdankhmen.¨ochOtehneaucheureUdoHilfehErnst¨attebundestimmKatjatTayniemandlorf¨vuredasrstandenKwasorrekturlesenichmitdermeinerAArbnsamm-eit
lungtionsdingenvonW¨waorternrundsagenistawbsolutollte.unvAgneserzicJhtanßensbar.HilfebeiVerwaltungs-undKoordina-

WestellungnnmanderseichigenenimmerDowktorarbiedereitdurcahbhaltenneuel¨undasst,zus¨bleibtatzliceshenicProhtjaekteusvodasnsdermanFevrtig-iele
denMitgliederPersoneninfd¨urerdieArbesch¨oneitsgruppAetmoskph¨ommenare,dieundingehenteressasniehtent.GIechspr¨m¨acohechteunddendiefolggutenen-
Ratschl¨agen(dieichoftignorierthaben,waswiederumdieBearbeitungszeitweiter
Scverhlw¨angertegler,hat)AndreasdankBen:euthner,ChristianDieterGEuricahuc,k,RolfBailuHenkSi,el,NadjaOnnoScB¨hinkohler,el,EricProf.hSchHuelmlzkeu,t
DavidEngelskirchen,DanielBartz,FelixPatzelt,MarkusRiegel,UlrichCherdron,Ste-

279

280

ChapterD:Acknowledgment/Danksagung

fanLiehr,RolandRothenstein,StefanWilke,KlausBowe,AxelEtzold,KlausFranke,
PitHankel,AndreasThiel,DennisTrenner,RonaldBormann,FrankEmmert-Streib
Mirmohammedhosseini.Aladinund

IcherationfreuemitmichSunitaschonMaufandon,dieWeAndreasiterf¨Kuhrungreiter,derKiatjantTaeressanylortenundundKerstinspannendenSchwabe.Koop-

Außerdemm¨ochteichDavidRotermundf¨urseinenunerm¨udlichenEinsatzbeiderIn-
stallationundWartungderArbeitsgruppen-Computerdanken,ohnediesezeitraubende
Hilfew¨aredieseDoktorarbeitnichtm¨oglichgewesen(istdochwahr!).

Ichhoffe,dassalledieLeute,dieichimmerwiedervertr¨ostenmusste,weilichmal
wiederkeineZeithatte,daichanderDoktorarbeitarbeitenmusste,mirirgendwann
erden.wrzeihenev

nicUndhtsoohnegl¨ucdasklicHhenZerumgekeitenaspdeenremitletztenMRicesthaelmeinesPluraVh¨atteerstandesichbvestimmerloren.tinvielender

zuOhnebeendendenBodereistandgarmzueinerpromoFvamilieieren.w¨Icarehems¨ocmhirtenicbhtesondersm¨oglichmeinergewMesen,uttermeindanken,Studiumdie
michaufmeinenlangenWegimmerunterst¨utzthat.Wirhabenunsgemeinsamdurch
schwereundunangenehmeZeitengek¨ampft.OhnedieUnterst¨utzungdurchmeinen
Großvater,WilhelmRotermund,w¨areichbestimmtverzweifelt.Undichm¨ochtean
meinenverstorbenenVatererinnern,ohnedessenaußergew¨ohnlicheUnterst¨utzungich
vermutlichniestudierth¨atte.

nslaufeLeb

14.03.1976

1986-1983

1988-1986

1992-1988

1994-1992

1995-1994

2004-1994

1996-1995

2002-1996

2007-2002

Name:DavidRotermund

GeboreninDelmenhorstalsSohnvonWilfriedRotermundund
Ursulaund.Roterm

BesuchderBernard-Rein-Schule,Delmehorst(Grundschule)

BesuchdesPestalozzi-Schulzentrums,Delmenhorst
tierungsstufe)(Orien

BesuchderRealschuleanderLilienstrasse,Delmenhorst
ule)h(Realsc

fachscAusbildunghulef¨urzumcAssistenhemisctenhb-tecerufe,hnischenBremenAssistentenanderBerufs-

FachhochschulreifeBiologie/ChemieanderFachoberschule,
Bremen

GewerbeimBereich:Erstellung,WartungundVerkaufvon
Hard/Software,DienstleistungenimBereichComputer,u.a.

StudiumderElektrotechnikanderHochschuleBremen(Vordiplom
inhnik)Elektrotec

StudiumderPhysikanderUniversit¨atBremen(DiplominPhysik)

DoktorandbeiProf.Dr.K.PawelzikanderUniver-
sit¨atBremen(MitarbeiterimBMBFProjektDeutsch-Israelische-
Projektkooperation(DIP)-ModelsandExperimentstowards
AdaptiveControlofMotorProstheses(METACOMP),Zentrum
f¨urKognitionswissenschaften(ZKW)undSonderforschungsbereich
Neurokognition(SFB517)derDFG)

281