cloud_upload
clear
en
131 Pages
English

Predicting sensory performance [Elektronische Ressource] : maps, optimality, feature extraction, and learning / Andreas B. Sichert

Free trial
Gain access to the library to view online
Learn more

Description

Predicting Sensory PerformanceMaps, Optimality, Feature Extraction, and LearningAndreas B. SichertcCover illustration: Photograph courtesyof Guido Westho Autor: Andreas B. SichertTitle: Predicting Sensory PerformanceSubtitle: Maps, Optimality, Feature Extraction, andLearningFunding has been provided by the Bernstein Centerfor Computational Neuroscience (BCCN) { Munich,the CoTeSys Exzellenz Cluster, and the Sloan-SwartzFoundation.Physik DepartmentTechnische Universität MünchenPredicting Sensory PerformanceMaps, Optimality, Feature Extraction, and LearningAndreas B. SichertVollst andiger Abdruck der von der Fakult at fur Physik der Technischen Universit at Munc henzur Erlangung des akademischen Grades eines Doktors der Naturwissenschaften genehmigtenDissertation.Vorsitzender: Univ.-Prof. Dr. Th. HugelPrufer der Dissertation: 1. Dr. J.L. van Hemmen2. Dr. R. NetzDie Dissertation wurde am 14.07.2010 bei der Technischen Universit at Munc hen eingereicht unddurch die Fakult at fur Physik am 20.09.2010 angenommen.Prefaceow, you are reading my PhD thesis! I mean, you are reading { which is an almost unbelievablyWcomplex process. It is, however, a good example of what makes neuroscience so fascinating.Let’s take a closer look at the above example of reading to understand what neuroscience is allabout.At rst glance, reading starts with some tiny black dots on a white sheet of paper which formgeometrical structures that we denote as letters.

Subjects

Documents
Knowledge
Physik

Informations

Published by
Published 01 January 2010
Reads 19
Language English
Document size 23 MB
From the same publisher
Evaluation of an active immunoprophylaxis against prion diseases [Elektronische Ressource] / Gunnar Kaiser-Schulz
Evaluation of an active immunoprophylaxis against prion diseases [Elektronische Ressource] / Gunnar Kaiser-Schulz
Spectrochemical analysis of solid samples by laser-induced plasma spectroscopy [Elektronische Ressource] / Igor Radivojevic
Spectrochemical analysis of solid samples by laser-induced plasma spectroscopy [Elektronische Ressource] / Igor Radivojevic
Stable and user controlled assistance of human motor function [Elektronische Ressource] / Heike Vallery
Stable and user controlled assistance of human motor function [Elektronische Ressource] / Heike Vallery
High level ab initio potential energy surfaces and vibrational spectroscopy [Elektronische Ressource] / Kiran Sankar Maiti
High level ab initio potential energy surfaces and vibrational spectroscopy [Elektronische Ressource] / Kiran Sankar Maiti
Reprocessing of a global GPS network [Elektronische Ressource] / Peter Steigenberger
Reprocessing of a global GPS network [Elektronische Ressource] / Peter Steigenberger
Spatial dynamics of pushbelt CVTs [Elektronische Ressource] / Thorsten Schindler
Spatial dynamics of pushbelt CVTs [Elektronische Ressource] / Thorsten Schindler

Predicting

Maps,

rySenso

yitOptimal,

eatureF

Andreas

rmanceerfoP

Extraction,

B.

Sichert

nda

rningLea

Photograph

c

courtesy

Coverillustration:Photographccourtesy
esthoffWGuidoofTitle:Autor:PredictingAndreasB.SicSensoryhertPerformance
Subtitle:Maps,Optimality,FeatureExtraction,
Learning

and

FundinghasbeenprovidedbytheBernsteinCenter
theforCoTeSysComputationalExzellenzNeuroscCluster,ienceand(BCCN)the–Sloan-SwMunich,artz
ion.taoundF

PredictingSensoryPerformance
Maps,Optimality,FeatureExtraction,andLearning

B.AndreasSichert

Vollst¨andigerAbdruckdervonderFakult¨atf¨urPhysikderTechnischenUniversit¨atM¨unchen
zurErlangungdesakademischenGradeseinesDoktorsderNaturwissenschaftengenehmigten
Dissertation.

Vorsitzender:Univ.-Prof.Dr.Th.Hugel
Pr¨uferderDissertation:1.Univ.-Prof.Dr.J.L.vanHemmen
NetzR.Dr.Univ.-Prof.2.

DieDissertationwurdeam14.07.2010beiderTechnischenUniversit¨atM¨uncheneingereichtund
durchdieFakult¨atf¨urPhysikam20.09.2010angenommen.

Preface

Wow,complexyouareprorecess.adingItmis,yhoPhDwevtheser,ais!goIodmean,eyxampleouareofwhatreading–makeswhichneurisosanciencealmostsounbelievfascinating.ably
Let’stakeacloserlookattheaboveexampleofreadingtounderstandwhatneuroscienceisall
out.abAtfirstglance,readingstartswithsometinyblackdotsonawhitesheetofpaperwhichform
geometricalstructuresthatwedenoteasletters.Wecanperceivethembecauseoureyesandthe
subsequentneuronalprocessingofthevisualsystemareabletotranslatepicturesintheexternal
worldintoneuronalinformationthatformsacorrespondingrepresentationwithinourbrain.The
visualsystem,however,doesnotsettleinformationprocessingbypresentingcloudsofblackdots.
Atgroupthisthepoint,ineuronalnformationrepreseprontationcessingofstartsthesetocloudsgetreallyofdotstosophisticated.mentalobjeCognitivcts–enetheuronalletters.circuitsThe
activitiesindividualetc.lettersToarefinallyrelategaindatoeacreasonablehothermetoaningform,wtheordsbrainthatusestherepresenundterreallyingobjectsgrammar,attributes,thatis
thespecificmethodologyoftheparticularlanguagetobuildsentencesoutofindividualwords.
andForsenanytencesonearewhoisonlynotalreadymeaningfulbimpressedecauseletwemeprelateointtheoutsetwocloudsfurtherofdotsissues.andFirst,thecorrespletters,wondingords,
abstractmentalobjectstopastsensoryexperience.Wecaninterprettheword“barking”because
oursomevisualtimesysago.temThisrelatesisduethetoblacthekfactdotsonthatthediffsheeterentofmopapdalitieseratarehandabtoletotheinsoundteractofafaithfudoglly,heardfor
instance,bymeansofcombiningthecorrectmemoriesofvisualandauditorysensoryevents.
Second,theanswertothequestionofwheresuchintelligentneuronalcircuitscomefromis
elusive.Mayevolutionbeananswer,viz.,aretheseneuronalstructuresencodedinourgenes?
No,notinthefirstplace.OtherwisetherewouldbegenesforBavarian,German,English,and
otherlanguages.Theabilitytoread,tointerpretabstractmethodologyandsymbols,islearned.
Moreover,thereisnoteacherthatgivesbabieslecturesintheirmothertongue.So,ourbrain
teachesitself,thatisitlearnspurelyfromsensoryexperiencetobuilditselfinawaytobebest
suitedforComparingprocessingthesesensoryabilities–expourerience.abilities–withthoseofman-madeandsocalled“cognitive”
Fromsystemsmyispersonalimpressivpe.erspInectivconethtrastatistowhnatureatmakweeshatheve,fieldhowofever,justneurosciencestartedsotoinfascinating:ventsuchDiscovsystems.ering
thenervoussystem,ahellofamachinery!

ofdiffTheerentotolsfordisciplinesdiscovisneringethecessarynervtoousenlighsystemtenthisarecomplexmanifold.topic.ItturnsByoutmeansthatofathisbroadthesisintewreaclotionok
atsituatedthenervatoustheinsystemtersectionthroughofthebiology,glassesofmedicine,theoreticandalphysicsneuroscience.employingTheoreticalmathematicalneurosciencetools.Inis
mashort,jorthisdrivingdissertationforceforfothecuseswholeon“sPystemredicting–includingSensoryPitsdeverformance”elopment.sinceForsensorythatpuexprperieose,nceweiswillthe
themderivetospecifimathematicalcsettings,viz.,descriptionsdiffoferentsensorysenses.systems,featureextractionandlearning,andapply

v

Preface

1ChapterThefirstchaptergivesanintroductiontofundamentalconceptsofsensoryprocessingsuchasthat
ofaneuronalmapwhichareneededtounderstandtherestofthework.Besides,thereferences
giveninthecompactintroductionshouldprovidethereaderwithagoodstartingpointforown
studies.

2ChapterInordertoexploittheinformationprovidedbysensorysystems,animalsreconstructthespatio-
temporalenvironmentfromtheirsensorysystems.Inthischapterwederiveauniversalframework
thatallowstocalculatethespecificlayoutoftheinvolvedneuronalnetworkbymeansofa
generaltransmissionmathematicalandrudimenprinciple,talknoviz.,wledgestocofhasticthedetectionoptimality.procThatess,is,thisgivenapproacaknohwnallophwstoysicalestimatesignal
thepossibleperformanceandtopredictneuronalpropertiesofbiologicalsensorysystems.

3ChapterInthischapterweapplyourframeworkderivedinChap.2totheauditorysystem.Thatis,
comingfromthegeneralmathematicalprincipleofoptimalitywededuceanarchetypemodeland
correspondingtemporalreceptivefieldsforauditorysignalenhancementandechosuppression.We
comparetheperformanceofboththeneuronalandthemathematicalimplementationofthemodel
tostate-of-the-artalgorithmsandbiologicalsolutions.

4ChapterThelateral-linesystemisauniquemechano-sensoryfacilityofaquaticanimalsthatenablesthem
notonlytolocalizeprey,predators,obstacles,andconspecificsbutalsotorecognizehydrodynamic
objectsaswedemonstrateinthischapter.Wepresentbothananalyticapproachthatshowsthe
effectofsubmergedmovingobjectshapeandanexplicitmodelexplaininghowaquaticanimalscan
distinguishdifferentshapes.Furthermore,wepredictnaturallimitationsofthissensorycapability.

5ChapterTwogroupsofsnakespossessaninfrareddetectionsystemtoperceiveaheat-imageoftheir
environment.Thischapterexploresthedifferentstagesoftheinfraredpathwaysuchastransmission,
detection,andneuronalen-anddecodinginordertoprovidefundamentalinsightstothissensory
system.Thethoroughphysicaldescriptionoftheinvolvedphenomenaaswellastheincorporation
ofanatomicaldataintoanexplicitmodelextendthepresentpictureofthissensorysystem.

6ChapterForneuronalprocessingofsensoryinformationuni-andmultimodalmapsplayakeyrole.The
preciseinteractionofsuchmapsisnotpresentatbirthbuthastobelearnedduringthedevelopment
ofananimalbymeansofsensoryexperience.Thischapterpresentstheconceptofintegrated
MultimodalTeachingthatcombinessupervisedspike-timing-dependentplasticityandmultimodal
integrationtoinherentlydevelopafaithfulinteractionofdifferentsensorymodalities.

vi

yThankou

Preface

ManypeoplehavesupportedmeduringmytimeatthePhysikDepartmentoftheTechnische
Universit¨atM¨unchen.Inthefirstplace,IthankLeovanHemmen,mythesissupervisor.Hehas
introducedmetothetopicoftheoreticalneuroscience,viz.,biophysics.Hisdedicationtoscientific
researchisveryimpressive,andthroughmanyinspiringdiscussionsIhavelearnedalotabout
science.

Writingthisthesiswouldnothavebeenpossiblewithoutthepleasantinteractionandcooperation
mawithysamyythankcolleaguesyoutoattheMoritzB“T35”¨urcck,hairStefanieandaB¨urcfewk,pReopleolandbeyUtz,ond.InAndreasthecVonollmatextyr,ofandthisthesis,ChristineI
Voßenwhohaveproofreadpartsofthethesisandhavegivenhelpfulcomments.

Asawhole,Iwasgiventheopportunitytomakemanyvaluableexperiencesduringthetimeof
mUnivyPhDersity.wIork;havinealsoaddition,meteIwxceptionasablealptoeoplejoinandthehaBavyeeriscmadehenewEliteAkfriends.ademieForandalloftothisvisitIamHarvvardery
grateful.

Finally,Ithankmyfriends,myparents,mybrother,andBabsi,whohavealwaysbeenthereto
me.ortsupp

2010Juneh,Munic

vii

Contents

PrefacetstenConInformationSensorycessingPro11.1Stepsofsensoryprocessing...............................
1.1.1Signalmapping..................................
1.1.2Sensoryencoding.................................
1.1.3Neuronaldecoding................................
1.2Neuronalmaps......................................
1.2.1Mapproperties..................................
1.2.2Collectionofcomputationalmaps........................
1.2.3Mapdevelopment................................
1.3Learninginsensorysystems...............................
1.3.1Localmechanisms................................
1.3.2Globalstructures.................................
ormationFMapOptimal22.1Startingpoint.......................................
2.2Mathematicalmodel...................................
2.2.1Definitionoftheproblem............................
2.2.2Optimalreconstruction.............................
2.2.3Matrixnotation.................................
2.3Relationtocommonmethods..............................
2.3.1Pseudo-inverse...................................
2.3.2Maximum-likelihoodapproach..........................
2.4Neuronalrealizationofthemodel............................
2.5Multimodalinteraction..................................
2.5.1Integration....................................
2.5.2Pooling......................................
2.5.3Development...................................
3Auditoryprocessing:Exemplifyingoptimality
3.1Introduction........................................
3.2Archetypemodel.....................................
3.2.1Optimality:Anarchitecturalprinciple.....................
3.2.2Receptivefields:Acomputationalbuildingblock...............
3.3Technicalprospect....................................
3.4Biologicalprospect....................................
3.5Discussion.........................................

ix

vix112234568991113131314151818181920232323242525262627293132

Contents

35erceptionPLateral-line44.1Introduction........................................35
4.2Analyticdescriptionofshape..............................36
4.2.1Mathematicalformulation............................37
4.2.2Effectsofshape..................................37
4.3Hydrodynamicobjectrecognition............................40
4.3.1Definitionofhydrodynamicobjects.......................40
4.3.2Objectrecognitionperformance.........................42
4.4Discussionandoutlook..................................46
5InfraredSystemofSnakes49
5.1Introduction........................................49
5.2Warmingup:Signalmapping..............................51
5.2.1Athermodynamicvista.............................51
5.2.2Mappingtheheat................................57
5.3Sensoryencoding.....................................59
5.4Neuronaldecoding....................................63
5.4.1Minimalmodel..................................63
5.4.2Theextendedmodel...............................67
5.4.3Featureextraction................................69
5.5DiscussionandOutlook.................................73
6IntegratedMultimodalTeaching75
6.1Introduction........................................75
6.2IntegratedMultimodalTeaching(iMT).........................76
6.2.1HowdounisensorymapsdetermineiMT?...................77
6.2.2HowdoiMTcharacteristicsinfluencemapadaptation?............79
6.2.3HowdoesiMTcalibratedifferentunimodalmaps?..............84
6.3Modelresultsandexperimentalfindings........................84
6.3.1Experimentsprovision-guidedmapformation.................85
6.3.2Experimentscontravision-guidedmapformation...............87
6.4DiscussionandOutlook.................................87
AAppendix:OptimalMapformation95
A.1Arecipeformakingmaps................................95
A.2Nonlinearitiesininformationprocessing........................95
A.3Remainingderivationstepsleadingto(2.16)......................96
A.4Gaussianblurredsignal.................................98
A.5Pseudoinverse.......................................99
BAppendix:Lateral-linePerception101
B.1Planepotentialflow...................................101
B.2Rotational-symmetricpotentialflow..........................101
103iMTendix:AppCC.1Optimalcombinationoftwomodalities.........................103
C.2Optimalcombinationofthreemodalities........................104
C.3ModelParameters....................................105

x

bDieeginntReisezuvondeinentausendF¨ußen.Meilen

1.ProcessingSensoryInformation

Laotse

Thisthroughchapterthepglassesrovidesofthisawosynopsisrk,weofapplythewideOccam’sarearazoofrbysensorymeansinfoofintrormationducingproandcessing.foLocussingokingon
inthethesessentialubsequentconceptschapters.usedinItthisstartsthesiswithenabthelingusquestiontoofderivewhatparedictionsstimulusoninsensoryexternalperfoworldrmanceis,
discussestheconceptualstepsinthesensorypathway,andpresentsabriefoverviewofneuronal
repsynapticresentationplasticitbyy,ameanstoolofformaps.understaFinallynding,wefocusdevelopmourentofattentionaoncommonmapfomulti-sensormationryandspaceintrothatduce
enablesbeingstoactfaithfullyintheexternalworld.

1.1Stepsofsensoryprocessing
SensoryprocessingcanbedividedintothreeconceptualstepsasdepictedinFig.1.1(A−C).First,
astimstimulusulusinisaphspecificysicallywaymappandedontranslatetodetectorthephysicalorgans.signalSecond,intotheneuronaldetectorinforgansormation,respviz.,ondtospikthees
ofseethee.g.corr[164esp,267onding],andprimaryforthesensorydifferentneurons;levelsforofinmotrodelingductionstoneuroscienceneurons,e.g.neuronal[119].Third,computationthe
provideinformationausefulcontainedrepresenwithintationtheofspikthee-trainsoriginalofstimtheulussensoryto,forsysteminstance,isproinitiatecessedmotorneuronallycommands.soasto

Figure1.1:Threeconceptualsteps(A−C)ofsensoryprocessing.Everystimulusintheexternal
worldemitssignals,inourexamplelight.(A)First,thesignalsaremappedontothesensoryorgans
wheretheyaffectthephysicalstateofspecialized(protein)structureswithinthe(photo)receptors.
(B)Second,thesensoryneuronsrespondonlytospecificstimuli,forinstancebluelight,bymeans
ofgeneratingcorrespondingspikes.(C)Third,theneuronalsystemdecodestheimbeddeddata
withinthespike-trainstorevealtheessentialinformationinausefulway.

1

1.ProcessingSensoryInformation

mappingSignal1.1.1Initially,anyobjectintheexternalworldrevealsitspresencebygeneratingdifferentsignals.A
runninganimalmay,forinstance,generateasoundandachangingvisualimageasitmoves,as
wellasvibrationsandaninfraredprofile.Thegenerationandtransmissionprocessfollowsspecific
physicallaws.Thisfact,assimpleasitmaylookatfirstglance,providesapowerfultoolfor
understandingsensoryprocessing:themathematicaldescriptionofsensorystimuliasphysical
tities.quanableobservInconcreteterms,givenanysignalthatvariesxasafunctionofspatialpositionxandtimet,it
ispossibletocalculatethetime-dependentinputi(t)tothereceptororgans.Thatis,thephysical
mappingofsignalssx(t)ontosensorydetectorscanbedescribedbyasetoftransferfunctionsh,
h:sx(t)→ix(t).(1.1)
Moreover,realobjectsintheexternalworldtypicallyemitabundleofsignalsthroughseveral
channelsbasedondifferentphysicalpathwaysatthesametime.Thatis,theypermanentlyleavea
multi-sensoryfingerprintthatcanbedetectedandprocessedbysensorysystemstoformacoherent
representationoftheexternalworldinspace-time.Thisissuewillleadustomultimodalintegration
andlaterontothequestion,howthispreciseinterplayofoursensesevolves.Weleavetheissue
ofmultimodalprocessingforthetimebeingandnowresumediscussingthefundamentalstepsof
sensoryprocessingwithinonemodality.

dingencorySenso1.1.2Inthesecondstepofsensoryprocessing,thesignalthatismappedontothesensorydetectors
istranslatedintoneuronalinformation–spikes.Thisprocesscanalsobedescribedbyasetof
oftransferthefudetector.nctionsk.ComingInthebacbktoeginningtheofexampletheproincess,Fig.the1.1,externallightinsignalthevisualaffectstherangeph–ysicalsimplifiedstate
–changesthestateofthewavelength-sensitivephotoreceptorsintheretina.Bymeansofthis
physicalprocess,spikegenerationisinduced.Theresultingspikesti(t)consequentlyencodethe
intensityofthecorrespondingspecificwavelength[263,267,281]ataparticulardetectorpositioni
,ttimeandk:ix(t)→ti(t).(1.2)
issueTwisothataspectssensoryofsensoryencodingencodingincludesareatofleastfurthertwoinstepsterestofinsignthealconfiltetextring.ofthisThephthesis.ysicalThechangefirst
ofstatewithinthereceptorisspecifictoaparticularstimuluscharacteristicandhencefiltersthe
signal.Inourexample,thephotoreceptorrespondsonlytobluelight;forreviewofthevisualsignal
cascadeInseeaddition,[263].theSo,prothecesswavofspikelength-speecgenerationificityalsselectsofiltersoutthelightofsignal.allWotherenowcolors,takecf.aclFig.oser1.1.look
atthisimportantissue.First,translatingsensorysignalsintoneuronalinformationcomparesto
ananalog-digital-conversionprocessbecauseaspikecodeisabinarycodeconstitutedofzeros
(nospikesatthispointintime)andones(spikeatthisparticulartime).Second,intheprocess
oflimitspiktheesamplinggenerationrate,atthee.g.levelofrefractiononetimesinglelimitsneuron,thespikbiopheysicalfrequencydyn[am267ics,of164].cellThisrescompartmenultsintsa
restrainedinformationconsamplingtent.Third,accuracy,thethatneuronalis,thetransmissionanalog-digital-conprocessvalsoersionalteprorscesstheimbprincipallyeddedreduceinformation.sthe
Synapses,forinstance,typicallyfeaturecomplexnon-lineardynamicsthatcontributetoaniterative
sequenceoffragmentation,selection,andamplificationofinformation,andthusfilterthesignal
[89].information.Fourth,thereNeuronsarecandifferenresptondkindstoofspikeneuronalratesco(“ratedingcothatding”)butadditionallyinothermaycasesmodifyneuronstheresporiginalond

2

1.ProcessingSensoryInformation

tofiringonset,changeofspikerates,skipping,bursting,orcomplextemporalpattern–justtolist
afew[18,37,247].Wesee,generatingandtransmittingneuronalinformationpresentsdefinitely
afilteringprocess.Thereisnodoubt,however,thattheneuronalfilteringprocessesneednot
generallybebugs.Theymay,onthecontrary,benecessaryfeaturesreducingthearrivingphysical
signalstotheessentialonesandthusmakingexternalsensoryinformationstreamscomputationally
feasible.Thesecondimportantaspectofsensoryencodinginthisthesisistopography.Somesensory
organsareorganizedinsuchawaythattheyareabletoreceiveandconservetopologicallyordered
information.Inourexample,theretinashowsatopologicalorganizationofthedetectorsand
lensesassuringawelldefinedtopologicalrelationbetweenpositionsintheexternalworldxand
thedetectorpositionsi.Translatingspatialinformationfromtheexternalworldintoneuronal
information,however,inducesatransferfromcontinuoustopologiestodiscretetopologies.This
spatialanalog-digital-conversionprocessatleastcausesuncertaintiesinthesensorypathway.
Topography,moreover,isnotonlyaspatialcharacteristic.Itisalsoimportantinthetemporal
domain.Withintheauditorysystemitiscrucialtoknowthetemporalsequenceofeventsinorder
to,forinstance,processinterauraltimedifferences.Withinthelimitationofaneuronalsystemour
brainisevenabletotranslateneuronallytemporaldataintospatialinformationtogainauseful
representationofexternalspace.TheJeffress’modelisafamousexampleofsuchatranslation
[132].cesspro

dingdecoNeuronal1.1.3InTheythemanextycomestagefofromsensorydifferentprosenscessing,orytheneuronsineuronalwhichmasystemyhasadditionallytodecohadeveathespspikatialore-trainstempti(oralt).
relation.Inmathematicalterms,theneuronalsystemprincipallyhastosolvetheneuronalinverse
andproblem.signalThatmapping)is,theandsystemkhas(sensorytoinvertdetectingthecomandpositionneuronalofencotransferding)tofunctionsrevealhthe(signaloriginalgeneratingsignal
s.Ingeneral,itishardlypossibletofindanexactsolution.Thesystem,however,cantrytofind
anoptimalinversefunctionltoreconstructsˆtheoriginalsignalfromtheneuronaldata
l:ti(t)→ˆsx(t).(1.3)
Additionally,inmanycasesthesensorysystemevensettlesforcomputingtheessentialstimulus
informationinsteadofreconstructingthecompletestimulus.Energycostsandthelimitationsof
neuronalprocessingcapabilitiesmaybereasonsforthat.
Thelimitationseachsensorysystemisfacedwithleadtoanotherglobalissueinsensory
processing.Asensorysystemcanonlyposeandanswerthefollowingquestion:Givenuncertainties,
sensorylimitations,respandonse?aForsensoryillustration,response–thewhatfiniteisntheumbermostofprdeteobablectorsissignalnotthaablettocouldsamplehaveconcausedtinuousthis
signalsorspacewithoutuncertainties.Thelimitingtemporalaccuracyofneuronalcomputationas
wellmatterasthewhatneuronalcausesthesenoiselevelsystemadditionallyrestrictions,increasetheproquestioncessingitselfreduceimpreciseness.snaturallyConsequentoaqtlyues,andtionnoof
conditionalprobability.So,itisnotsurprisingthateverywhereinthecourseofsensoryprocessingwe
findneuronalstructuresbehavinglikeestimators[7,59,61,67,69,113,120,183,220,237,238,239]
that,forinstance,canoptimallysolvetheneuronalinverseproblem[75,76,252].Itremains,
however,torevealhowtheseestimatorsworkonthelevelofneuronalcircuitryandtoquantify
erformance.ptheir

3

1.ProcessingSensoryInformation

Figure1.2:Threecategoriesofmapsbasedontheirtopography;adaptedfrom[11,154,208].
NelsonandBower[208]suggestthreemapgroups,i.e.continuousmaps(A),patchymaps(B),
andscatteredmaps(C).(A)Mapcorrespondstoacontinuousmappingofauditoryspaceinthe
optictectumofthebarnowlshowingaglobaltopography[154].(B)Thepatchymappingofvisual,
auditory,andmultisensorysensitiveareasinthehumansuperiortemporalsculusmultisensoryarea
showsatopographyonlywithinsmallanddistinctareasofthewholeorganization[11].(C)Inspite
ofthespatialorganizationinthelattertwoexamples,thescatteredmappingofolfactoryinputto
thepiriformcortexoftheratlacksanyspatialorganization.Dependingonthecomputationaltask,
thedifferentstructuresleadtodifferentcomputationaladvantagesasbrieflyaddressedinthemain
text.

aimInofallsummarythe,theresultingphysicsneuronalofexternaleffortissignaltoachievtransdueactionreasonabledictateandtheusefulsensoryrepresedatantationstructure.ofThethe
originalstimulusinformation,viz.,thesignal.Thecentralnervoussystem(CNS)canrepresentthe
thecomputedtopologicallyinformationorderedindifferenneuronaltwamapys.Inplaythisanthesisessenthetialprole.opulationThecomapdeand,accuracymoredeimpfinedortanastltoy,
howpreciseamapcanrepresentanexternalstimulusisagoodmeasurefortheperformanceofthe
correspondingsensorysystem.Sowewillconsidermapsinthenextparagraphinmoredetail.

mapsNeuronal1.2Amajorroleinsensoryprocessingisreservedforneuronalmaps1[116,161].Theconceptofamap
suggeststhatthereisalawfulrelationbetweenthesurfaceoftheneuronalsubstrateandarelevant
aspectoftheexternalspace.Tocutalongstoryshort,aneuronalmapistypicallyanarrayof
neuronsinwhichneighboringneuronsrespondto“similar”sensorystimuli.Thissimplification
holdstrueatleastforspatialsensorymapsonwhichwefocusinthecourseofthisthesis.
Asanexample,visualinputinthemammalianbrainisprocessedthroughmultiplecorticallayers
thatareorganizedaccordingtothetopographyoftheretinalinputcells(“retinotopicorganization”)
[281,143].Inthisway,neighboringneuronsrespondtovisualinputfromneighboringpointsin
space,andthusformaspatialmap.Spatialmapshavebeendiscoveredinvarioussensorysystems
inmanygroupsofvertebrates[10,40,62,109,123,149,151,192,259,264,295]andeveninsome
[130].ertebratesnon-v1InReihenfolge1879,kHelmholtzennen[gelern114]tremarkwird,inedder“DasssichdurcihrehdasEindr¨Enucketlangf¨darbieten,uhrendesdassdiesetastendenReihenfolgeFingersansichdenalsObunabh¨jectenangigdie
davderenonEerwlemeneist,tenobmanmanmitimmerdiesemwiederodervor-jenemoderrFinger¨uckw¨artstastet,indassderselsiebenfernerOrdnnichungteinedurceinl¨hlaufenaufigmbuss,estimmumtevonReiheeinemist,
zumRiemann’sanderenTermizunoklogieommen,,einealsokeineMannigfaltigklinienf¨eitzwormigeeiterReihe,Ordnung,sonderndasalleseinfl¨istacleichenhahtfteseinzuseheNebn.”eneinander,Thatis,oderHelmholtznach
alreadyrecognizedthataneuronalrepresentationofatwo-dimensionalsurfaceconstitutesatwo-dimensional
brain.theinmanifold

4

1.ProcessingSensoryInformation

Figure1.3:Thefiringprofileofaspatialmapencodesthelikelihoodoffindinganobjectatacertain
positioninsensoryspace.Focusingonthefiringrateofonlyasingleneuronsuchasneuroni
withfaithfulppreferrederceptiondirectioncanonlyxibandeachievignoringedifitstheneighactivitbyorsofprevtheentswholeafaithfulneuronalpmaperception(justofcomparereality.theA
“neighborhood”ofiwiththerest)istakenintoaccount.Consequently,eventhoughthenominal
valueofthefiringrateofneuroniisidenticalincaseAandB,therepresentedphysicalreality
differssignificantlyinbothcases.

Neuronalmapsarenotlimitedtospatialrepresentations.Prominentexamplesarefoundin
theauditorysystem.Therearemapsrepresentingfrequency,interauraltimedifference,interaural
amplitudedifference,andevenamplitudemodulationfrequencies[84,187,214,225,245,265].So,
weseethatmapsareneitherrestrictedtoparticularsensorymodalitiesnorparticulardomains
space.ashsucAtypicalwayofcategorizinginphysicsistosearchforcharacterizinglengthscales.Lookingat
maps,wecanfindtwotypesoflengthscales.Firstandobviouslywecanconsiderthedimensions
oftheneuronalarray.FollowingthepresentationofDehaeneandCohen[58],onemayhence
distinguishmapsbytheirsizeintomacromaps(afewcentimeters),mesomaps(uptocentimeters),
andmicromaps(millimeters).Second,besidethelengthscaleofmapdimensionswecanconsider
thescaleuptowhichacommontopologicmaporganizationcanbeidentified.NelsonandBower
[208]thereforesuggestthreecategories,continuousmap,patchymap,andscatteredmap;see
Fig.1.2.Thesecategoriespresumablyarisemainlyfromcomputationalefficiencyreasonsaswewill
seeinthenextsection.Strictlyspeaking,thelatter,thescatteredmap,isnotamapasitshows
.yographtopnoIntrinsicallytiedtotheconceptofaneuronalmapisthatofreceptivefields.Areceptivefieldof
asensoryneuronisdefinedastheregionofspaceinwhichthepresenceofastimulusaltersthe
activityoftheneuron.Sincereceptivefieldsofsensorymapsareeasilyaccessibleexperimentally
theyarewellstudied,andthecorrespondingdatashedlightontheunderlyingmapproperties.

ertiesroppMap1.2.1Aswehaveseenjustabove,mapsareusedinmanyareasinthebrain.Thecompellingquestion
[116]thereforeis:Whatisthefunctionaladvantageofneuronalmaps?Thatis,whychooseamap
structureforneuronalprocessing?Here,wefocusonneuronalmapswithaglobaltopography,viz.,
continuousmaps;cf.Fig.1.2.Fromthetheoreticalperspective,thefundamentalargumentisthat,
incontrasttoarbitrarypopulationcodingorclusteredneuronalarchitectures[51],neuronalmaps
ensureatopographicneuronalorganization.
Thisconceptualadvantageofmapsarisesfromcombiningbasictopologicalinformationwith
actualstimulus-evokedresponses.Thatis,incaseofsensorymaps,thepreferreddirectionofthe
andsensoryactivityorgansalloorwsforreceptorsinisterpretinglinkedthewithfiringtheirprofileactualofmapactivity.neuronsInotherwithwrespords,ecttomergingtheirtopneighographbors,y
withviz.,conpreferredtext-depdirectionendent.xiFisortheexample,sameinas(A)illustratedand(B),ininFig.the1.3,contextalthoughofitstheneighbactivitoringyofneuronsneuronitsi
informationMoreover,conthetentmapdiffersorganizationdrasticallyfromunderlyingcase(Athe)to(Bneuronal).processingallowsforanefficient

5

1.ProcessingSensoryInformation

therepresenlikelihotationodinofterprconetintationuouslyofvaaryingfiringinputpatternsignonalsaasstospatialchasticmap,thatquantitiesis,.thFeorlikelihoinstance,odtoitallofindwsa
sensoryobjectatacertainposition[60,116,131,227,248].Inthecontextofmaps,wecannot
retrieveanythingfromthefiringprofileFig.1.3(B)butanoisybackgroundwhereasFig.1.3(A)
presentsquitenicelytheprobabilityoffindingacorrespondingsensoryobject.Thatis,thelatter
correlateswithapositionalestimatorforsensoryobjects.Whilesensorymapsencodingspatialor
temporalestimatorsarethemostevidentones,maps,especiallymapsinhigherbrainareas,are
notlimitedtospatialortemporaltopology.Theyarefoundtoencodevariousstimuliunderlying
vkindariousoftopmapisologiestopro(reviewvideabyKnsystematicudsenetal.represen[161]).tationSo,ofgettingcomplextostimthepuli.oint,therealpowerofany
sensoryIntheobconjecttextpresenoftedthisabthoevesis,isthecrucial.likelihoBeyodondinthisterpretationissue,ofmapstheprofiringvideotherpatternevokcomputationaledbya
advandanBowtages.er[208One,]assuggestedwehavethatseendifferentpreviously,categoriesrelates(seetotheFig.1.2)degreesofoftoptopologicologicorderorder.aredueNelsonto
minimizingcomputationaleffortsimilartoparallelcomputers.Thegeneralhypothesisisthatmap
overheadorganizationandanreflectsoptimalmapusageloadandbalancing.inputThestructureformerinorderandtotheachievlattereaareminimalfoundtocommbeunicationrealized
andneuronallyscatteredbythemap;tcf.hreeFig.kinds1.2.ofConmapstinmeuousnmtionedaps,abforove,instance,namely,areconfoundtinuoustobemap,usedpatchmainlyymap,for
localoperationsintheproblemspacesuchasfeatureextractionandspatialfiltering.Wehaveto
bearinmindthatcomputationaltasksareperformedmainlybutnotonlybymeansofsynaptic
connections.Chemicalprocessesarealsousedforinformationprocessing,forinstance,toinhibit
acompleteareaofneuronsrepresentingsimilarfeaturesofthestimulus.Thustheconceptofa
mapdoesnotonlyminimizeneuronalwiringeffortbutenablesreasonablechemicalcontrol.“Load
balancing”isachievedbylocalmapdistortionsthatreflectdifferencesinsensorydensities.For
instance,thereisamuchhigherdensityofsomatosensoryreceptorsinthetonguecomparedtothe
skincorrespinourondingback.sensoryThemapincreasedwheresensorymoreneuronssensibilityencoofdethethetonguesamepishysicalreflectedspacebythandistortionsinourofbacthek.
Thewithinpatcahymoremapsglobalseemcontotext.Butadditionallythisisincludestillnonmatter-loofcalopongoingerationsdebate.toTheanalyzeloscatteredcalsensorymapssucdatah
asidenfoundtifiesinthecomplexolstimfactoryuliandsystemanswreflectersthethequestionnon-topofographicwhatrathernatureofthanthethatinput.ofwhere.Here,thesystem

mapscomputationalofCollection1.2.2Thereisanevenmoreconvincingexplanationfortheomnipresenceofmapsinthebrain.The
realpowerofcomputationalmapscanbeappreciatedonlywhentheinterplayofseveralmaps
isarbitraryconsidered;transfforormationreviewinsee[the161].repreInsenthetationconoftextofinformation.thisthesisTowegiveadefinecfamousomputationalexampleofashoanw
informationisprocessedinasequenceofcomputationalmapsweconsidertheauditorysystem;cf.
ofFig.the1.7.Finferiorrequency-depcolliculusend(ICC).ingtimeInanextdifferencesstep,arethismapcomputediswithintranslatedainmaptoianthespatialcentralmapninucleusthe
theexternalopticnucleustectumof(forthereviewinferiorsee[collicul154,us155,(ICX)161]).whereAlthoughthethevinformationariousisearlyfurthersensoryprocuesjectedinintheto
auditorysystemarehardlyrelatedtospace,intheoptictectum(OT)theauditorymaprepresents
spaceandevenconnectssmoothlytospatialmapsofothermodalities.
beautifulTheOTexampleanditsofwtheell-studiedconnectionbmammalianetweendifferenhomologue,tmaps.thesupAlleriorsensorycolliculussystems(SC)that[261pro],isvidea
minformationultisensoryasinwaellasmap-likepredominanformprotlyjecttomonosensorythelaSC/OTyers[are261];foundcf.andFig.are1.4.shownWithintoproctheessSC/OT,spatial

6

1.ProcessingSensoryInformation

Figure1.4:Thesuperiorcolliculus(SC).Thesuperiorcolliculusanditsnon-mammalianhomologue,
thethatproopticvidetectum(informationOT),areinwaellmap-likstudeiedform[mid-brain261].stWithinructuresthetoSC/OT,whichallspatialsensorysensorysystemsinformationproject
ofinformationdifferentisusedmonosensoryfordelivmapseringisunamcomputedbiguousandcommotorbinedincommands,tomforultisensoryrefiningmaps.sensoryThemocomdalities,bined
andSC/OTforarefurthermutuallyprocessingaligned,inwehighercanspbraineakofareasa[166collection,167,of179maps,216that,260].represenBecausetanallalignedmapsinsensorythe
space.Thefigureisadaptedfrom[179].

information.Inmultisensorymaps,mostneuronsrespondtomorethanonesensorymodality.In
doingso,theneuronalresponsetomorethanonesensorymodalitycanbecomplex.Incaseofspatial
andtemporalcoincidencewecanobservesuper-additivity,cross-modalenhancement[134,224,261],
orinverseeffectiveness[224]whereasincaseofnon-coincidencecross-modalsuppressioncanbe
found[140].Multimodalmapsarewidespreadinthebrain[139].
BoththemultisensorymapsandthemonosensorylayersarefoundtobealignedintheSC/OT
togainaunifiedmultisensoryrepresentationofsensoryspace[147,261].Thecombinedsensory
informationcanthenbeusedtogeneratedirectionalmotorresponses[166,167,179,216,260].This
isonlypossiblesince,withintheSC,therearenotonlysensorybutalsomotormaps,i.e.,motor
neuronsorganizedinamap-likestructure.Inmotormaps,systematicvariationsofamovement
arerepresentedtopographicallyacrosstheneuronalarray.Directevidenceforthishypothesishas
recentlybeenfoundineye-trackingexperiments[104].

Inacollectionofalignedneuronalmaps,moreover,externalobjectscanbeidentifiedbytheir
positionencodedbymeansofthefiringpatternindifferentsensorymaps.Thisdoesnotmeanthat
anobjectisadequatelydescribedifonlythepositionisknown.Itratherimpliesthatpositionserves
asfordefiningappropriateaandsensory–inobconject;trastcf.toFig.1.5.higherWhenccomputationalombininglevelsdifferen[65t,98]sensory–necessarysystems,theinformationspatial
informationisneededtobindinformationassociatedwiththesamepositioninthemonosensory
mapsintoonesinglemultimodalperceptforfurtherprocessing.

7

1.ProcessingSensoryInformation

Figure1.5:Objectformationbyspatialinformationbinding.Withinanatlasthereisasimple
waactivitytoydefinepatterna(redsensoryobcolumn).jectbyUptomeansnoofw,sucgroupinghamecspatialhanismisinformation,speculativviz.,e.Hospatialwever,congrueitcannt
bedynamicrelevanent,forvironment.instance,Theretoareselectlinesamofultimoevidencedalsupptargetorting(e.g.therunningexistenceandofsucbarkinghamecdog)hanism.withinAa
andprominenvisualtmapexample[210].aretheTheseneuronalcircuitsare“AND”suppandosedto“OR”idencircuitstifyexternalwithinobthejectsOTsucthathaslinkpreyinfraredand
enhancetheircontext-dependendsaliency.

Bearinginmindtheadvantagesofneuronalmapsthatareduetotopographywenextfacethe
questionofhowaneuronalmapcandevelop.Laterinthisthesis,weadditionallystudyhowa
collectionofmapscanevolve.

developmentMap1.2.3Onemightarguethatneuronalmapssimplyexistbecausetheirneuronalarchitecturefollowsthe
sensorysurfaceoftheirinputmodality.Fromthispointofview,forinstance,thevisuallayersare
retinotopicallyorganizedbecausetheyreceivetheirinputfromtheretina.Similarly,afrequency
mapjustreflectsthetonotopicorganizationofthecochlea.
Interestingly,thisargumentdoesnotholdforeverysensorymapasitalreadybecomesapparent
intheexistenceofauditorymapsforinterauraltimeandamplitudedifferences.Regardingexamples
inthespatialdomain,letusconsiderthesensoryunitsofthefroglateral-linesystem[295]orthe
snakeinfraredsystem[109].Thesesensoryunitsreceiveacomplexsuperpositionofinputfrom
severaldifferentspatiallocations.Computingamapfromsuchacomplexinput[75,76,95,252]is
certainlynotstraightforward.So,howcanmapsevolve?
Themostobviousideaistoimagineanexplicitgeneticmechanismencodingmaptopography,
viz.,connectivity.Inthiscase,however,weha12vetogenetically15encodealargenumberofpossible
connections.Ourbrainconsistsofabout10neuronsand10synapticconnections.Thatis,
wehavetoencodeanumberofpossibleconnectionscomparabletothenumberofatomsinthe
universe[267].Ofcourse,our2−3∙104humangenescannotachievethisdirectly.Consequently,
intrinsicmechanismsmustdeterminemapdevelopment.
Inthelastyears,neurobiologistsformulatedfourmainhypothesesastohowtopographic-correct
synapticmapconnectionsarise;reviewse.g.[34,181,190,271].First,followingtheaddress
labelhypothesis,synapticstructurespossessaseriesofpositionallabelstellingingrowingaxons
orpre-synapticaxonterminalshowtoarrange.Second,neighboringaxonsconservepre-synaptic
topographyduringthegrowthprocess.Therefore,thepost-synapticstructuresaretopographically
arranged.Third,boththeingrowingaxonsandthetargetstructuresdevelopinatopographically

8

1.ProcessingSensoryInformation

arrangedtimesequence.Thetemporallymatchingentailsaspatialmaptopography.Fourth,pre-
synapticaxonterminalsarrangethemselvestopologicallybymeansofelectricalactivity.Although
evidenceforeachofthesehypothesescanbefound[271],severalmechanismsseemtocooperate
toensureapropermaptopography.Togiveanexample,electricalactivity-basedadaptionis
supposedtofine-tunemapconnectivityaftersomeimpreciseprewiringwhichcanbebasedonother
mechanismssuchaschemicallabeling[13,146].
Precisemaptopographyiscrucialforsensoryperformance.Inthenextsection,wetherefore
shedlightonactivity-based,viz.,spike-dependentmapformationbyconsideringlocalmechanisms
andglobalstructuresresponsiblefortheplasticityofmaptopography.

1.3Learninginsensorysystems
InThattheis,consensorytextofactivitsensoryysys(input)tems,leadslearningtoadifferenbasicallytmapmeansactivitacyhange(output).intheinpuConsequent-outputtly,learningrelation.
tak1949,esplaceHebbbyalreadymeansofctranslatedhangingthisthegeneralconnectioninsightofstrengthsactivitbetwy-basedeen(inputplasticitandyintooutput)neuronalneurons.termsIn
byformulatinghisfamousrule2[112].Inshort,ifapost-synapticneuronexcitesaspikeshortly
wafterords,oneHebb’sofitslearningpre-synapticruleneudescribronseshasthatfired,tempthisoralparticularcorrelationsofconnectionneuronalisstrenactivitygthened.–ifIncausalother–
wareorldlearned.andthTusocutHebb’salongrulestoryimpressivshort,elyneuronalenablesusactivittoyrelatesunderstandtypicallyverytoevgenerallyentsinhothewtoexternallearn
correlationintheexternalworldbymeansofalocalneuronalmechanism.
withIntheHebb’sconruletextofarise.thisFirst,thesiscanw“PredictingeSeunderstandnsoryPaspectserformance”oftsensorywopquestionserformancdirectlyebymeansconnectedof
studyinglocalmechanismsofsynapticplasticity?Second,whatprovokeslearning,i.e,theadaption
ofsensoryabilities?Moreprecise,whichglobalandintrinsicmechanismsassureproperguiding
inputalignedforlocollectioncalofsynapticmapsplasticitfoundyintotheevolvSC?eacomplexbrainstructure,forinstance,suchasthe

mechanismscalLo1.3.1Inthelastdecades,neurobiologicalevidencehasbeenfoundtosupportHebb’srule.Various
mechanismsunderlyinglong-termpotentiation(LTP)andlong-termdepression(LTD)ofsynaptic
transmissionhavebeenstudiedingreatdetailevenifnotyetfullyunderstood;reviewse.g.
185].177,128,34,[13,Inaddition,togainfurtherinsightintothistopic,Hebb’sessence,synapticweightdevelopment
duetocorrelatedactivityofcorrespondingpre-andpost-synapticneurons,hasalsobeenformulated
inmathematicalterms;fordetailedreviewseee.g.[115,145].Thedevelopmentofsynaptic
connectionstrengthJijbetweenapre-synapticneuroniandapost-synapticneuronjcanbe
dividedintothreepartsasillustratedinFig.1.6.First,thearrivalofapre-synapticspikeattime
timinducestheweightJijtochangebyanamountηwpre.Second,iftheconsideredpost-synaptic
neuronjexcitesapost-synapticspikeattnj,allconnectionsarechangedbytheamountηwpost.The
quantityη>0isasmallparameterandtheweightchangeswpreandwpostcaneitherbepositiveor
negative.InFig.1.6weassumedwpre>0andwpost<0.Thethirdpartisduetothespike-timing
ofpreandpostspikes.Giventhetimedifferencestim−tnjbetweenpre-andpost-synapticspikes,
JijischangedbytheamountofηW(tim−tnj),wherethelearningwindowW(tim−tnj)denotesa
2gro“WhenwthanproaxoncessoformcelletaAbisolicnearchangeenoughtaktoesexciteplaceincelloBneandorbrepotheatedlycellssucorhperthatsistenA’stlytakefficiencyes,partasinonefiringoftheit,somecells
firingB,isincreased”;[112]p.62

9

1.ProcessingSensoryInformation

Figure1.6:Thesynapticlearningprocess[115,145,146].Thediagramillustratesthelearning
processduetothethreepartsofSTDP.Inthiscase,everyspikeofthepre-synapticspiketrain
(toprow)strengthensthesynapticweightJijbywpre(green).Allpost-synapticspikes(second
upperrow)induceanegativeweightchangewpost(orange).Thecontributionofthethirdpart
ofSpike-timing-dependentplasticity(STDP),thecorrelationterm,dependsonthespiketiming
andthelearningwindow(middlerow).Intheleftpart,thepairofpre-postspikesistemporally
toofarseparatedascomparedtothedimensionofthelearningwindow.Theremainingtwoother
possibilitiesaredepictedintherightpart.Ifapre-synapticspikeprecedesapost-synapticspike,
thesynapsestrengthincreases(blue).Incaseofoppositetemporalorder,thesynapseisweakened
(purple).

ofrealFig.valued1.6.Finallyfunction.,weWecanhavecombinedepictedtheatthreeypicalpartsformandofaderivelearningaHebb-twindoypweinthelearningthirdruleuppbasederroonw
spike-timingforatimeinterval[t,t+Δt]
ΔJij=ηwpre+wpost+W(tim−tnj).(1.4)
timtnjtim,tnj
npTheabost-synapticoveequationspikecantrainsaltsopre;pbeost(t)form=ulatedδ(tin−intin;j)tegralandform.computeWethereforedefinethepre-and
ΔJij=ηt+δtdtwpretipre(t)+wposttjpost(t)+t+δtdtW(t−t)tipre(t)tjpost(t).
tt(1.5)Both,correlateequatipre-onand(1.4)pandost-synaptic(1.5),represenactivittya.Inmathematicalparticular,theHebb-tlearypeninglearningruleisrulebasedsinceonthetheequationsprecise
timingsequenceofpre-andpost-synapticspikes.Suchlearningschemesarethereforedenotedas
Spike-timing-dependentplasticity(STDP);fordetailedreviewseee.g.[1,15,55,78,115,145].
Kempter,preGerstner,andvanpHemmenostdemonstrated[145]thatlearningschemesbasedon
pre-synapticνiandpost-synapticνimeanfiringratessuchas
ddtJij=a0+a1νipre+a2νjpost+a3νipreνjpost+O[(νipre)2]+O[(νipost)2](1.6)
arenotcapabletocatchtheneuronaldynamicsonaplausibletemporallevel.Togiveanexample,
meanratedescriptionsrequirethatallchangesofneuronalactivityareslowatatimescaleofthe
learningwindowW.Thatisnotnecessarilythecase.TheHebbianlearningwindowincortical
areas,forinstance,isabout100msascomparedtothefrequencyof40Hzcorrespondingtoatime
scaleof25msonwhichneuronalactivityvaries.

10

1.ProcessingSensoryInformation

Figure1.7:Siteforplasticityandoriginofguidinginputformapadaptionintheauditorypathway;
adaptedfrom[78,156].Theauditorysystemprocessestheinitialsensorycues,forinstance,interaural
timedifferences(ITD)togainaspatialrepresentationofsoundsourcelocationsintheoptictectum
(OT)thatisalignedtomultimodalmapsintheOT.Inshort,sensorycues,i.e.,frequency-specific
ITDmaps,arepresentinthecentralnucleusoftheinferiorcolliculus(ICC)andareprojected
totheexternalnucleusoftheinferiorcolliculus(ICX).Fromthere,spatial-auditoryinformation
ismappedtotheoptictectum.Experimentsprovidingabnormalvisualinputdemonstratetwo
importantfindings.First,thereisaspecificsiteforplasticity.Thesynapticconnectionsbetween
ICCandICXchangeinsuchawaythattheICXmapacquiresOTtopography.Second,audition
adaptstoinputfromtheOT.Inresult,visionseemstoguideauditorymapdevelopment[159,160].

ThisshortexampleillustratesnotonlytheadvantagesofSTDPascomparedtorate-based
learningschemesbutalsoreflectsanissuealreadymentionedabove.Thewayofneuronalencoding,
forinstanceratecodingversusspikepatternsthatarephase-lockedtotheinputsignals,mayreduce
andrefinetheimbeddedinformationandthuscorrespondtoaneuronalfilterintheprocessof
neuronalencoding;seeSec.1.1.2.
Inthissectionwehavequantitativelyseenthatpre-synapticinputthatdrivespost-synaptic
activitycaninduceplasticityofcorrespondingconnectionstrengths.Tounderstandandpredict
theconsequencesoflocalsynapticeventstosensoryperformancewehavetoconsidertheglobal
contextwhereinthelocaleventstakeplace.

1.3.2Globalstructures
Inthissection,weintroducesome,butnotmanyimportantthoughtsaboutlearningbeyond
localsynapticevents.Locallearningrulescandescribehowthelocalconnectionstrengthchanges
accordingtothecorrespondingpre-andpost-synapticactivity.However,withoutknowingwhat
“corresponding”meansinthecontextofaspecificneuronalsetupofasensorysystem,locallearning
rulescannottellushowthesensorysystemdevelopsasawhole.Therefore,wehavetoaskatleast
threequestionsthatwillguideusthroughthesubsequentlines.First,whatdrivespost-synaptic
activitythatresultsinsynapticplasticity?Second,whereinthestructureofpre-andpost-synaptic
synapticconnectionsactivitdoyestoresplasticitultyinappreasonableear?Third,globalhowconnecdotiesons?theHereunpre-synapticto,wetakinputeanmanipulateexemplaryploost-ok
attheauditorypathway.
Wehaveseenabovethatpre-sypaticinputdrivingpost-synapticactivitycanleadtochangesof
thedrivescorrespthepondingost-synapticconnectionactivity?strength.Thatis,Thewhicfirsthinpquestionutprobeyjectsondtolothecalpossynaptict-synaptevicentsneuron.is:WhatTo

11

1.ProcessingSensoryInformation

giveaconcreteexample,aswecanseeinFig.1.7theexternalnucleusoftheinferiorcolliculus
(ICX)receivessensoryinputfromthecentralnucleusoftheinferiorcolliculus(ICC)andtheoptic
tectum(OT).Consequently,theactivityofneuronsinthespatialmapinICXisdrivenbytwo
inputorigins.Verycloselyconnectedtothefirstquestionisanotherone:Whatconstitutesthe
inputfromOT?AsthisistopicofChap.6,weleaveitforthemoment.
Topredicthowtheauditorysystembehaves,wehavetoconsiderthenextquestionofwhere
plasticityappears,i.e.,whichconnectionsarechangedbymeansofactivity-basedlearning?We
knowthatmapsintheOTaremutuallyalignedandthatthisisaresultofsensoryexperience
duringgrowth.Theanswertothewherequestioncanhencetellus,forinstance,whichinputguides
alignmentandwhichconnectionsadapt.Inourparticularexample,weseethatonlytheICC–ICX
connectionsshowplasticityandadapttoinputfromOT.Thelatterinputisthereforedenotedas
guiding;cf.Fig.1.7.ForadetaileddescriptionoftheICC-ICXconnections,considerKnudsenet
[156].erpapal.’sHavingclarifiedthewhatandwhere,“only”thehowremains.Incaseofourexample,Friedel
andvanHemmen[78]havemathematicallyanalyzeddifferentkindsofguidinginputandtheir
practicalconsequencestothelearningprocess.Theyderived,forinstance,thatonlyinhibitory
inputfromtheOT,calledselectivedisinhibition,canreorganizemaptopography,viz.,theICC–ICX
mappingcanbereorganizedsoastocompensateevenaspatial180◦shiftbetweenguiding(OT)
andsensory(ICC)input.ThiscorrespondstofamousprismglassexperimentsdonebyKnudsen
andothersinthebarnowl[156,159]–performedsimilarlyinotheranimals,suchashamster
[200],cat[278,280],clawedfrog[44],ferret[148,150],andsnake[96].Incontrast,excitatoryinput
leadstoalessprecise,lessstable,andslowerlearningprocesswhichresultsadditionallyinan
ambiguousmapping[78].However,inotherscenariosexcitatoryguidinginputfaithfullydescribes
174].[146,yrealitneurobiological

Insummary,derivingmathematicaldescriptionsoflocalsynapticevents,thatis,makingthem
quantitativelypredictive,isimportantbutnotenoughtogaininsightintothedevelopmentof
brainstructuresandtheirfunctionasawhole.Wehavetoconsidertheglobalenvironmentin
enwhichvironmenthetissynapticevconstitutedentsbappyearmeanstoofpredictneuronalwhatmapstheythatcause.bInelongthetoconsensorytextofsthisystems.thesis,Wesucprohvidane
anewmodelforintrinsicmapdevelopmentinthelastchapterofthiswork.Inthenextchapter,
wewillfocusourattentiononageneralframeworkofoptimalmapformationtounderstandhow
sensoryinformationcanbeprocessedbymeansofmapsandhowtheneuronalconnectivitypattern
predicted.ebcan

12

2.OptimalMapFormation

Mansollteallessoeinfachwiem¨oglichsehen-
einfacher.nichtaucherabertAlbEinstein

Inthestruggleforsurvivalinacomplexanddynamicenvironment,naturehasdevelopeda
multitudeofsophisticatedsensorysystems.Inordertoexploittheinformationprovidedby
thesesensorysystems,highervertebratesreconstructthespatio-temporalenvironmentfromtheir
sensorymodalities.Asweintroducedinthelastchapter,mostanimalsareabletocompute
aneuronalrepresentationoftheoutsideworldinformofaneuronalmap.Herewepresenta
universalframeworkthatallowstocalculatethespecificlayoutoftheinvolvedneuronalnetwork
bymeansofageneralmathematicalprinciple,viz.,stochasticoptimality.Thatis,givenaknown
physicalsignaltransmissionandrudimentalknowledgeofthedetectionprocess,ourapproach
allowstoestimatethepossibleperformanceandtopredictneuronalpropertiesofbiological
systems.rysenso

ointprtingSta2.1Amousehearsarustlinginthegrass,seessomeleavesmovingand–escapesfromthepredator.
Thus,theperceptionoftheoutsideworldbysensorysystemsandtheconsequenttranslationof
theirresponseintoareliableneuronalrepresentationthatallows,forinstance,directionalmotor
commandsisanessentialconceptforsurviving.
Inthelastchapter,weintroducedtheadvantageousconceptofaneuronalmapforrepresenting
theexternalworld(seeSec.1.2),forinstance,theauditoryspaceintheexamplejustmentioned
intheaboveparagraph.Moreover,wesubdividedthesensoryinformationprocessingintothree
fundamentalsteps.Thesearesignalmapping,sensoryencoding,andneuronaldecoding;seeSec.1.1
andtheexampleinFig.1.1.Inthischapter,wewillnowtranslatetheseconsiderationsintoa
mathematicalframeworkofoptimalmapformation.

delmoMathematical2.2Followingtheideaofmapping,encoding,anddecoding,wewanttounderstandhowtoextract
thestimulusatthebestfromthesensoryresponses.Mathematicallyspeakingwehavetoderive
theinversetransferfunctionl(1.3)thatcanperformanoptimalreconstructionofaparticular
stimulusgiventhesensoryresponsesandcomputationallimitations.Thisinversetransferfunction
canthenbetranslatedintoaneuronalconnectivitypattern(seeSec.1.1).
Thederivationweprovidebelowisbasedonthreereasonablesimplifications.First,weassume
thatallsensorymapsarepurelymonosensory.Althoughithasbeenquestionedwhetherthereare
mapswithoutinfluencefromothersensorysystems[262],ourassumptioncanbejustifiedbythe
findingthatmanyspatialmapsareclearlydominatedbyasinglesensorymodality[280].

13

OptimalMapFormation

receptorSecond,inputwewithsimplifyitstherespinonse.ternalThatprois,cesseswesetwithintheasptransferecificsensoryfunctionkreceptor(1.2)btoybeidenthetifyingidentitthey
connectsfunction,toandthehencecorresphondi(1.1)ngconnectssensorydrespirectlyonsestheonastimulusone-to-onecbasis.haracteristics,Althoughi.e.,thethismasensoryynotinputbe
givenforarealisticsetting,theframeworkofoptimalmapformationiswithoutlossofgenerality
btheecausefirstwetransfercanincorpfunctionorateh.theInreturn,filteringwepropgetaertiessetofoftheequationssecondthattransferaremorefunctionkbunderstandableyredefiningand
comparablewithcommontechniques;seee.g.Chap.5.
respTheonseofthirdtheimportansensorytsystem.assumptionThatisthatmeansofalinethatartherelationdetecbtoretwrespeentheonsesstimchangeulusandproptheortionallyreceptor
tothesignalstrength.Forexample,themultipolesofasubmergedmovingobject,i.e.the
signalstrengths,translatelinearlytothedetectablewatervelocitiesatthelateral-lineorgans
[[169251,,75213].,172,Nonlinear136,53],relationcansinbetwprincipleeenstimbeulustreatedandwithdetectorourrespmodelonses,aswe.g.,ell;aseelogariAppthmicendixrespA.2onsefor
details.

2.2.1Definitionoftheproblem
Aswehaveintroduced,anobjectgeneratesasignalsx(t)varyingintimetandpositionxinthe
externalworld.Thesignalmaybe,forinstance,thetime-dependentsoundpressureataparticular
locationormaydenotethepresenceofedgesormovementataparticularpositionwithinthevisual
field.Thesignalinducesaresponseri(t)inasetofNsensorydetectors.Dependingontheproblem
atorparthandofaasingledetectordetectorarrayisucwithhas0a≤spi≤ecificNincantervbaleaofbestcompletefrequsensoryrnciesinorgan,thecocsuchhlea.asIntheleftprinciple,ear,
thedetectorcombinesinformationfrompastsignalswithinthewholesensoryspace.Theresponse
isthereforedescribedby
tri(t)=dxdτsx(τ)hix(t−τ)(2.1)
spaceall−∞xThewheretransferthetrfansferunctionfcanunctionbehi(different)tincorpforeacorateshthedetectorphi.ysicsofAuditorysignaltransfertransmissionfunctions,andfordetection.example,
differincorpbetorateweentherighptositionandleftofear.soundInsourcegeneral,andweearcanwithsafelyrespeassumecttothatthehixhead(t)=0midlineforandlargevaluesthereforeof
|xstate|andofta.Thissensor.reflectsWewillourinneedtuitionthispropthatertevyentslateroccuon.ringMoreofaravwer,ayorsincelonganyagodetectorwillnotcaninfluenceonlyreactthe
xtofunctiontemp(2.1)oral-causal,withi.e.,adaptedpastinsignalstegrationwesetlimitshi(tas)a=0conforvtolution<0.Wwithercanespthenecttorewritetime,theresponse
ri(t)=dx∞dτsx(τ)hix(t−τ)=:dx(sxhix)(t).(2.2)
−∞theTheabodetectorverespequationonseisdescriblimitedesthebyrespatonseleastofthreeanidealfactors.system.Inbiologicalsystemsthequalityof
system.First,Second,informationnoisemayinfluencesgetlostalldurinstepsginthethetransferdetectionfromandtheoutsidereconstructionobjecttoprothecess[inside71].Finalsensoryly,
limitationsoftheneuronalhardware,forinstance,thelimiteddynamicrangeofreceptors,constrain
possiblesolutions;seeSec.2.4fordetails.
additionalWithinournoiseterms.mathematicalAccordinglymodel,waetermincorpdesoratecribingthesebacthreekgroundrestrictivnoiseeξx(tfactors)mbustybinetroaddedducintog

14

FMapOptimalrmationo

Figure2.1:Physicalmapping:signalsx(t)withbackgroundnoiseξx(t)ismappedontoanoisy
receptorresponseri(t)+χi(t)throughthenoisytransferfunctionhix(t)+ηix(t).Optimalmap
formation:the(possiblynoisy)inversetransferfunctionlix(t)+λix(t)givesanestimatesˆx(t)ofthe
signal.Signalsx(t)+ξx(t)
Transferfunctionhix(t)+ηix(t)
Receptorresponseri(t)+χi(t)
Inversetransferfunctionlix(t)+λix(t)
Estimatedsignalsˆx(t)
Table2.1:Functionsanderrortermsdescribingdetectionandprocessingofsensoryinformation.

theadditionalsignal.Fnoiseteurthermore,rmsηix(wt)eandassumeχi(t),thatresptransferectively.functionConsequeandntly,sensory(2.2)isrespmoonsedifiedaresohampastoeredreadby
ri(t)=dx[(sx+ξx)(hix+ηix)](t)+χi(t).(2.3)
Toreconstructtheestimatedsignalfromthedetectorresponsesri(t),theabovetransformation
musttransferbe“invfunctionsertedl”x(int)bsomeetweenappropridetectorateiwayand.Wtheethemaprefoatrepositcalculateionx.theWhentime-depapplyingendlexn(tt)intoversethe
iireceptorresponsesati,weobtainaccordinglyto(1.3)theestimate
sˆx(t)=[ri(lix+λix)](t)(2.4)
ioftheoriginalsignalsx(t).Herethehatonsˆx(t)denotesareconstructionandthetermλix(t)
noterepresenthattsintheconnoisetrastduetoeltostheewhere[concrete217,229]realizationthepreofsentthemodeltheoreticalisinvnon-iterativersee.transferThiswillfunction.resultWine
apurelyfeedforwardnetworkstructurewhenitcomestoaneuronalrealizationinSec.2.4.
Figure2.1illustratesthewholemathematicalprocedureofsensoryinformationprocessingas
describsection,edweinwillthesteindicatepsofhoFig.wto1.1.Allcalculatetheinvreleverseanttransfertermsaresfunctionsumlixmarized(t)thatinTenableable2.1.optimalInthesignalnext
reconstruction.2.2.2Optimalreconstruction
Wewanttotuneoursensorysystemtooptimallyreconstructnotonlyonespecificsituationbut
thethatwtypicealdenoteenasvironmen“tt.ypical”.InotherwConsequenords,tly,abiologicallyspecificrelevsensoryantsignsignalsalisbaelongconcretetoaclassofrealizationsignalsof
aminimizeclassofttheypical,expectationbiologicallyvalueofrelevtheantsquaredsignals.Thatdifferenceis,bitetiswaeenstocsignalhasticandquanretity.constructionWe.therefore
ofphThisysicalispossiblemappingbandecausetheallneuronalquantitiesprocesandsoffunctionsoptimal(seemapFig.formation2.1)inv(Secolv.ed2.4)inbareoth,self-athevproeragingcess
aswedemonstratein[35].Themathematicaldefinitionofself-averagingallowsforadescriptionin
termsofexpectationvalues.

15

OptimalMapFormation

Toderivetheinversetransferfunctionslix(t)thatenableoptimalsignalreconstructionfora
classoftypicalsignals,wecannextminimizetheexpectationvalueofthesquarederrorbetween
signalrealandestimatedE{lx(t),t}:=dtdx[sx(t)−sˆx(t)]2=dtdx[sx(t)−sˆx(t)]2.(2.5)
tt
t−Tt−T
Herethebrackets.denotetheexpectationvaluewithrespecttothedifferenttypesofnoise,and
Tisatypicalprocessingtime.
Tobemathematicallyprecise,anexpectationvalueisanintegralonaprobabilityspacewith
respecttoaprobabilitymeasurep.Forarbitraryfunctionsfandg,if|f−g|2=0thenf=g
withrespecttopor,physically,lookingattheworldthroughp’sglasses:whatpfindsimportant
popsupclearlywhereaswhatpfinds“irrelevant”hashardlyanyweight.Thelatterneednot
correspondtowhatwe“think”ourselves;seevanderWaerden[276].
Aquadraticformoftheerrortermhasbeenproventobeareasonableandpracticalchoicein
manyphysicaloptimizingproblems;see[193].IncaseofindependentGaussianerrorterms,the
formulationviaaquadraticerrorisundercertainconditionsidenticaltoresultsobtainedbymeans
ofmaximum-likelihoodestimates[135,144];seeSec.2.3.2.
Mathematically,theerror(2.5)isafunctionalassigningtoeverysetofinversetransferfunctions
onespecificvalue.Minimizationoffunctionalsintheaboveintegralformisacentralandwell-
studiedaspectofthecalculusofvariations[45,85,138,32].Forthepresentsituation,thefirst
variationwithrespecttoeveryinversetransferfunctionlj(x,t)istovanish.Thatis,
∂[sx(t)−sˆx(t)]2
∂ljx(t)=0foreveryj.(2.6)
Inordertosolve(2.6),wehavetosubstitute(2.4)fortheestimatesˆx(t)andreplaceri(t)byits
description(2.3).Expandingthesquare,weencounterexpectationvaluesofproductsconsistingof
varyingcombinationsofnoiseandsignalterms.Hereweassumethatallnoisetermsaswellasthe
signalitselfarestochasticallyindependentofeachothersothattheexpectationofaproductof
independenttermfactorizes;forinstance,
sx(t)ηix(t)=sx(t)ηix(t).(2.7)

Foraproductconsistingofthexsamekindoftermweneedtoconsiderthedefinitionofthe
autocorrelationofaquantityf(t)asgivenby
fx(t)fx(t)=δ(x−x)δ(t−t)(µf2+σf2)(2.8)
withµfthemeanandσfthevarianceofthequantityfx(t).Thatis,weassumeinafirststepthat
thevaluesfordifferentspatio-temporalpositionsarecompletelyuncorrelated,akindofworst-case
analysis.Sincethemeansofallnoisetermsµfvanishwegetthefollowingcorrelationterms
ξx(t)ξx(t)=δ(x−x)δ(t−t)σξ2,(2.9a)
χi(t)χj(t)=δijδ(t−t)σχ2,(2.9b)
ηix(t)ηjx(t)=δijδ(x−x)δ(t−t)ση2(2.9c)
with|x|<xmaxand0<t<tmax.(2.9d)

16

rmationoFMapOptimal

Throughthefinalequationwetakeintoaccountthatthenoiseηix(t)vanishesforlargevaluesoft
and|x|,inthesamewayasforthetransferfunctionhix(t).
Theautocorrelation(2.8)ofthesignalsx(t)itselfdependsontheproblemathand.Eitherthe
detectorsofthesensorysystemmeasureabsolutesignalstrengths(µS),e.g.,vision,ormodulations
ofameanvalueofthesignal(deviationσS),e.g.,audition.Inanycase,onehastochoosethe
correspondingbiologicallyrelevanttermandputtheotherequaltozero.Inthefollowing,we
choosetheexpectationvalueµS2ofthesignalastheappropriatequantityandthereforetakeσS2to
zero,ebsx(t)sx(t)=δ(x−x)δ(t−t)µs2.(2.10)
While(2.9)incorporatesreasonableassumptionsforallnoiseterms,thecorrelation(2.10)forthe
signalisastronghypothesis.Signalsarenamelycharacterizedbyspatio-temporalcontinuity;e.g.,
objectsandtheircorrespondingsignalsusuallydonotdisappearfromonepointintimetothenext.
termcorrelationGaussianAsx(t)sx(t)=Aexp−|x−x|2/(2σx2)exp−|t−t|2/(2σt2),(2.11)
forinstance,cantakeintoaccountcorrelationsbetweenneighboringpointsinspaceandtime.Here
σxandσtaretypicalspatialandtemporalcorrelationscales.TheapplicationofsuchaGaussian
correlation,however,doesnotgreatlyalterthefurtherderivationbutonlysmoothensthefinal
estimatedsignal.Forreasonsofclarity,wewillthereforesticktotherelation(2.10);seeAppendix
details.for[35]andA.4Returningto(2.6)wehavetosolveitandinsodoingapplythecorrelations(2.9)and(2.10)so
astoarriveat
maxljx(t)σχ2+(µs2+σξ2)|y|<ymaxdydτση2+(µs2+σξ2)dy(hiylix)◦hjy(−t)=µ2shjx(−t)
i<t<τ0(2.12)fordetailsseeAppendixA.3.Theopencircle◦denotestheautocorrelationintegral
∞(a◦b)(t):=dτa(τ)b(t+τ).(2.13)
−∞Inordertosimplify(2.12),wedefinetwonewnoisemeasures,
2τ2:=σ2ξ(2.14)
µsandσ2:=σχ2+dydτση2(µs2+σξ2).(2.15)
µs20|y<τ|<y<tmaxmaxµs2
Theparameterτrepresentsaninversesignal-to-noiseratio.Therefore,itisoftenreasonableto
assumeasmallvalueofτ.Theparameterσ,ontheotherhand,describestheoverallmeasurement
noisebyrelatingdetectionandtransmissionnoise,σχandση,tothesignalmeanamplitudeµs.A
priori,itsvaluecannotbeassumedtobesmallandhastobeadjustedaccordingtothesituationat
hand.Inordertofurthersimplify(2.12)weswitchtoFourierspace,whereconvolution(2.2)and
correlation(2.13)becomeordinarymultiplicationscombinedwithcomplexconjugations.Using
(2.14)and(2.15)anddenotingFouriertransformsbycapitallettersandthecomplexconjugation
byanoverline,(2.12)simplifiesto
Lixσ2δij+(1+τ2)dyHiyHjy=Hjx.(2.16)
i

17

OptimalMapFormation

Equation(2.16)isthemainresultofourderivation.Inprinciple,itallowsustocalculatethe
x–invseeerse(A.16)transferinthefunctionsAppLendixiffororoptimaldetails–signalthenconfirmsreconstruction.thatAthecalcinvuerselationofthetransformationsecondvweariationhave
foundindeedminimizestheerror.Forconveniencewewillintroduceanalternativenotationinthe
section.nextionnotatMatrix2.2.3Torewrite(2.16)inamorepracticalnotationweintroduce“matrices”HandLbyputting
H[ix]=Hix,L[xi]=Lix.(2.17)
Thenotationsillustratethattransferfunctionsandinversetransferfunctionsarelineartransforma-
tionsfromacontinuousspace(theoutsideworld)intoadiscretespace(theneuronalmap)and
viceversa.Therefore,HandLareonlyformallymatriceswithaspatialcoordinatexvaryingin
R.Thematrixmultiplicationinvolvingthespatialcoordinatemustconsequentlybeunderstood
asanintegration.Adiscretizationofspace,asisusualinnumerics,wouldleadtoatruematrix
ulation.formInaddition,weintroducethecovariancematrixC(R)ofthereceptorresponseRasdescribed,
e.g.,in[135,144].Inourcasewefind
C(R):=(R−R)(R−R)T=µs2σ21+τ2HHT(2.18)
wherethesuperscriptTdenotesthematrixtransposeand1theidentitymatrix.Equation(2.16)
nowsimplifiesto
MLT=HwithM:=µs−2C+HHT.(2.19)
GivenMasaninvertiblematrix,denotedasthe‘modelmatrix’,thesolutionforLturnsouttobe
sL=M−1HT=HTµ−2C+HHT−1.(2.20)
Thisequationgivesauniquesolutionfortheoptimalreconstructionforanygivensetoftransfer
functionsandnoiseconstants(σ,τ).Using(2.4)inmatrixformwefind
Sˆ=LR(2.21)
asestimatedsignalfromthemeasuredresponsevectorR.
2.3Relationtocommonmethods
Thechallengeofsignalreconstructionhasalongtradition,and,accordingly,onemayaskhow
theaboveformalismrelatestomethodsthathavebeenestablishedinthisfield.Inthefollowing,
wewilldiscusstherelationofourmodeltomethodsbasedonthepseudo-inverseandtothe
maximum-likelihoodapproach.
Pseudo-inverse.2.3.1Ifthenoisetermscanbeneglected,theinterpretationof(2.19)isstraightforward.Inthiscasethe
covariancematrixCvanishesandtheresultingequationleadsto
L=HT∙H∙HT−1.(2.22)
Thepseudo-inversereconstruction1ofH[12functions].AtLthathindsighwet,havethisjustmakesfoundsensefulfillsincetheptheropertiespseudo-inofvtheerseMoore-Pgeneratesenrosean
1TheexplicitexpressionforLin(2.22)onlyholdsif(H∙HT)isinvertible.

18

rmationoFMapOptimalinvapproersionximatemayinvnotersebepmatrixossiblethatforaminimizesmatrixHthethatquais,dratice.g.,error;rectangularseeAppinsteadendixofA.5.squareAnorexactof
Butincompleteevenrank.inthemoregeneralsituationofnon-vanishingnoiseterms,wecanobservestrong
therelationscalculationbetweofentheourframewpseudo-inorkverse,andamethodsregularizationbasedonhasthtoebeinpseudo-introvducederse.toThesuppresspointisnoisethat,terms,for
ttermypicallyα1tomakehigh-frequencyitmorevstableariations.[193,The268,269,so-called228].TheTikhonov-Milregularizedlerreequationgularizationthenaddsreadsapositive
α1+H∙HT∙LT=H.(2.23)
2Hence,Comparinginthisthispsecialequationcaseourwithgeneral(2.19)weapproacseehthatisαidencorrespticaltoondsmethoexactlydsusingtoourthetermTikhonoσifτv-Miller=0.
regularization.2.3.2Maximum-likelihoodapproach.
Thedata.Withinmaximum-likelihothemaximodum-lanalysisikeliho[o135d,sc144heme]isaonecommoncomputestooltheinthestiminulusthatterpretationistheofmostlikmeasuremenelyonet
givenasetofdetectorresponsesR.Experimentshaveshownthatoptimalornear-optimalstimulus
Acommethobinationsdofopticanmalindeedstimulusdescribecomseveralbinationsucphenomenahasofthesensorymaximproum-likcessingeliho[o67d,126approac,165h,is2,113,therefore202].
highlyrelevanttoneuronalinformationprocessingandoughttobeincludedintoourmodel;see
1.Chap.Themaximum-likelihoodapproachtriestofindthemostprobableinputsignalSgiventhe
(σdetector=∞).respWenoonseswR,assumeaknoawnlineartransferrelationfunctionH,andnoaprioriknowledgeaboutthesignal
sR=HS+χ(2.24)
withχrepresentingthenoise.WeassumethenoisetofollowaGaussiandistributionwithzero
meanTheandmethothedstandardminimizesthedeviationnoiseσχ.χ.Thatis,basedonthefundamentaldefinitionsofBayesian
statistics,itmaximizestheconditionalprobabilitydensityfunction
p(R|S)∝exp−2σ1χ2(R−HS)T(R−HS)(2.25)
withrespecttothesignalS.Thisleadstoalinearsystemofequations
S=HTH−1HTR.(2.26)
=:LML
Usingtheaboveassumptionsforourmodel,viz.,σs=∞,η=0,andξ=0,(2.19)reducesto
HHTLT=H.(2.27)
Totestwhetherthetwofiltersareequal,weinsertLMLinto(2.27).Applicationofthetransposition
rulesshowsthatwiththeassumptionsweusedL=LML,thetwostrategiesareidentical;for
lodetailscalizationwerefercapabilittoyofelsewherethe[135,lateral-line144,242sys,te236m,].forWewillinstance,useofthisfishminethothednext(2.25)ctohapter.quantifythe
19

OptimalMapFormation

Figureconnects2.2:tosevNeuronaleralmaprealizationneurons.ofTheunimomapdalneuronsmap(encoformation.dingtheEachlocationsensorx)(heremayhairreceivcellse(mlabeledultiple)i)
thiswconnectionsay,thefromeactransformationhsensor.lix(Eact)hcanbeconnectionreliablyhasawrepresenell-definedtedinastrengthneuronalandnetwtemporkoral[76].delayt.In

2.4Neuronalrealizationofthemodel
Inthissectionwerelatethegeneralmathematicalalgorithmofoptimalstimulusreconstructionto
ainSec.concrete2.2.2neuareronalfulfilledconintext.Wneuronalethereforeprochaessing.vetoThatverifyis,wfirsteneedwhethertochecthekwhethassumptionsertheweneuronalmade
quantitiesandfunctionsofoptimalmapformationareself-averaging.Tothisendwenoteonthe
onedescribhandedbthatyastofirincghasofticproneuronscess,isforcorrelatedinstance,withaGaussianneuronalone;inputweandwillthatseeinneuaronalminutenoisewhy.canOurbe
frameworkcancopewithanydistributionofneuronalxnoiseaslongasthemeaniszero.Onthe
theothermapshandassotheciatedoptimalwithinvdifferenersetmotransferdalitiesfunctionsandlihence(t)arereflectlearnepropdertiessynapticoftheconnectionsunderlyingbetlearnweening
protimecess.scalesforEffectivelearninglearningandissloindividualwbecauserealizationsitneedsofanmanyexternalindepsignalendentcanrepbesetitions.eparated.AccordInotheringly,
ofwtheords,prolearningcess;seeis[a145self-a].Asvmeneragingtionedprocessbefore,wherequanonlytitiesaverandagedfuquannctionstitieswithinenterthebyphtheysicalverymappingnature
proexploitcesstheareself-amathematicalveragingasframewwell;orkaspleasederivseeedalinso[Sec.35].2.2Inareconclusion,fulfilled.theconditionsneededto
xInsuchConsequenanarctly,whitecture,ecanthenowactualtranslateprothecessinginveisrseptransfererformedbyfunctionstheli(synaptict)intoconnectionsneuronalbhardwetweenare.
neuronsanddetectors.Spatialprocessingisgovernedbythetopographicstructureofthenetwork;
thatdeterminedis,whicbyhthedetectordistributionisconnectedofdelatysowhicwithinhtheneuron.setofTempconnections.oralproFigurecessing2.2onshothewsotheranhandexampleis
ofsuchaneuronalsetup.
andInthetheabensuingovederivmapationthroughweahavediscretealreadynumtakberenofintoinverseaccounttransferthefdiscreteunctions.charFacurthtereofrmore,detectorsthe
discrete,“spiky”characterofresponseandreconstructionbytheneuronalrealizationisalready
xthetakeninvcareerseofbtransferythefunnoisectionstermslx(tχ).iandThisλi.Thatdiscretizationis,weisarerealizedleftwithbyathesamplingtemporalprodiscrcedureetizationwhereofa
ixbnumeenbershoofwnthatdendritesawithlimitedapnumpropriateberofdelayssynapticischosenconnectionstorepresensufficestthetocompletesamplelithe(t).timeIthascourseindeedof
lix(t)[76].Evenmoreso,themap-neuronresponseisrobustwithrespecttothesamplingmethod
ofthetemporaldelays[178]aswell.

20

FMapOptimalrmationo

asimpleConsequenfeedforwtly,ardasnetwillustratedorkofinFig.excitatory2.2,ourandunifiedinhibitoryframeworkconnectionscanbineorderimplementoformtedbayunimomeansdalof
mappatternfromisarbitraryestablishedininpuat[real76,252,biological178].Itsystem.doesHerenot,thehowevcorrecter,explainsynaptichowconnecsuchtaionshavconnectivitetobye
learned.Ithasbeenshown[74,78]thatateachersuchasthevisualsystemcangeneratecorrect
synapticstrengthssothatamapcanindeeddevelopinothermodalitiesbymeansof(supervised)
STDP;compareforthedetailslearnedseeSec.connectivit1.2.3,ypSec.attern2.5.3,withandtheChap.optimal6.oneThanksastogiventhebypresen(2.16)tandmethod(2.20).wecan
Abiologicalmeansetup,ingfulthough,comparisonmayofnotthebestraighmathematicallytforward.Inoptimalrealnetwbiologiorkcalarcsystems,hitectureerrorwithanreductionactual
asin(2.5)toitsminimum–thatis,realizingtheoptimalconnectivity–maynotbepossible
becauseofneuronallimitations.Thelimitedneuronalaccuracythatresultscanbeincludedinto
ourframeworkbyreducingtheerroronlybelowacertainerrorthreshold,whichmayevenvaryin
space.Forinstance,thesamplingarraysofanimaleyesarenon-uniform,withdifferentpartsof
fothecusonvisualspfieldecificbeingspatio-tempsampledoralwithdomainsdifferentcanspatialmathematicallyandspectralbererealizedsolutionby[in125tro,257ducing,293a].pSucositivhea
wcertaineightingfuthreshold,nctiontheintoareathesinwithintegraltheoffo(2.5).cusoftheAccordinweighglyt,whenfunctionhareducingvetothereachglobalahighererrorbleveloelwofa
optimization,i.e.,ofresolution,thantherest.
AsindicatedinSec.1.2,theconceptofreceptivefieldsisincludedinourformalism.Itmaybe
goodtoremember,though,thattherearetwomappingfunctions(LH)andLprojectingdirectly
ontothemap.Sincetherowsofthesemappingfunctionscontaintheinformationfromwhichareas
aofspreceptivecificemapfieldsneuronwhenwreceiveescomeintoput,theauditoryrowsdessignalcribproethecessingreceptivineChap.fields.3.WTheere,willweusetaktheeaconceptcloser
lookattheneuronalimplementationbymeansofaconcretesettingandexploittheconnection
betweenoptimalmapformationandreceptivefields.
Takentogether,theformalismofoptimalmapformationiscapabletodeliveranoptimal
neuronalconnectivitypattern,justasillustratedinFig.2.2,andherebydirectlygivesaforecastof
howexampletheinthereceptivconetextfieldsofaretheshapvisualed.Insystemthetonextillustratesection,theweprocapabilitiesvideavoferythesimplepresentbutframewconcreteork.

Spatialexample:visualprocessing
Withinthevisualsystemeachsensoryneuronisbasicallytunedtoaparticularspatialposition.In
mathematicalterms,everyretinalneuronireceivesinputfromaspatialpositionxi,itspreferred
position,andneighboringpositionswithinaregiondeterminedbyresolutionρ.Thetransfer
functioncorrespondingtosuchasensorysystemis
2hix(t)=exp−|x−xi|δ(t),(2.28)
2ρ2

anditsFouriertransformreads

(2.28)

2Hix=exp−|x−x2i|.(2.29)
ρ2Withinourexemplarysetupweassumethatthesignalpositionx=(u,v)encodespositions
u,v∈[−1/2,1/2].Asaremind2er,wehaverescaledpositionssoastomakethemdimensionless
andfitinthesquare[−1/2,1/2].Fromtheaboveansatz(2.28)and(2.16)wecalculatethematrix

21

OptimalMapFormation

Figure2.3:Spatialreceptivefield.Connectionstrengthstoamapneuronencodingtheposition
(u,v)=(0.1,−0.2).Thesensoryneuronsaredistributedona40×40gridwithpreferredpositions
u,v∈[−1/2,1/2]andatuningcurvewidthρ=0.9.Wechoseσ=1andτ=0.Aclearcenter-
surroundreceptivefieldemerges.Receptorneuronsthathaveapreferredpositionmatchingthat
ofthemapneuronhaveexcitatoryconnections(whitespot).Receptorneuronshavingaslightly
off-setpositioninhibitthemapneuron(darkcircle).Neuronswithpreferredpositionsfaraway
fromthemapneuronhaveconnectionstrengthzero(gray).

tsonencomp2
Mij=σ2δij+(1+τ2)exp−|xi4−ρ2xj|×erfui+2uρj−1−erfui+2uρj+1
(2.30)vi+vj−1vi+vj+1
×erf2ρ−erf2ρ
whereerf(x):=√2π0xexp−y2dyistheerrorfunction.Tofindtheconnectionstrengthslix,
wenumericallycalculatethemodelmatrixMforadiscretizedspaceandparametersσ=1and
τ=0.transformationWiththewematrixcannMumerwiceallythenobtaindeterminelxfortheeachmapconnectionpositionstrengthxassshoL.wnByinanFig.inv2.3.erseFHereourithere
iconnectionsfromallreceptorstoamapneuroni,i.e.,itsreceptivefield,areplottedforanarbitrary
strongpreferredpropjectionsositiontoxithe=(0.map1,−0.neuron2).(brighClearlyt,sptheotinFig.receptors2.3)encobut,dingintheterestinglypreferred,thepreceptorsositionhathatve
encodeslightlydifferinglocationscontributenegatively(darkcircleinFig.2.3).
Suchacenter-surroundprofileiscalled“Mexicanhat”andis,e.g.,realizedbylateralinhibition,
awell-knownphenomenonfirstdescribedbyMach[184]inthevisualsystemin1866.Uptonow
forthismecinstance,hanism,insescttudiedvisionin[the130],snakmammalianeinfraredvisualvisionsystem[255[,281252,],143],electrichasbeenfielddiscodetectionveredinaswellelectricin,
fish[250],andsurfacewavedetectioninthebackswimmer[203].
surroundIncontrastreceptivtoemanfieldyismoadelsnaturalsuchasconsequencepop-outof[153our]moordelsaliencyandthusdetectionexplains[129,lateral175]ainhibitioncenter-
asandoptimalapplyforourconceptmap-formationofoptimalpurpmaposes.Informationthefollotowingelabcoratelyhapter,wderiveevisanitarcthehetypauditoryemodelsystemfor
signalenhancementandechosuppression.IntheAppendixA.1,weadditionallyprovideaneasy
step-by-step“recipe”soastofindtheoptimalconnectivityinanarbitrarybiologicalsetup.
Next,weleavetheframeworkofoptimalmapformationforthetimebeinguntilweconsider
thethesensoryinfraredpsystemerformanceofsnakofesthe(Chap.auditory5)sandystemrevisit(Chap.general3),asptheectsoflateral-linecross-mapsystem(Chapcomputation.4),thatand
followfromcombiningmono-sensorymaps.

22

MapOptimalrmationoF

interactiondalMultimo2.5Inthelastsection,wegainedadeeperunderstandingofmonosensorymapsandtheircomputational
power.Thatis,wecanrevisitatopicofSec.1.2,Cross-mapcomputation,andtakeacloserlookat
thisimportantissue.Onthelevelofmaps,wegenerallydistinguishtwocategoriesofmultimodal
interaction,viz.,cross-mapcomputation:integrationandpoolingofinformation.

Integration2.5.1Congruentspatialinformationfromdifferentsensorysystemscanbeintegratedintoasinglemerged
and,hence,multimodalmap.Suchanintegratedmap,ascomparedtounimodalinformation
processing,featuresincreasedinformationreliabilityandsaliencyaswellasanimprovedsensitivity
inbothspaceandtime[166,167,104,237].Forexample,ifvisualandauditorysensorysystems
bothregisterasignal,e.g.,“brownahead”and“barkingahead”,itisveryprobablethatthesignal
correspondstoanactualobjectratherthantoasensoryartefact.Atthesametime,theintegrated
signalwillbestrongerandallowsforfasterreactions(e.g.,“escape!”).Insomecasesanintegrated
signalisevenoptimal[102,202].
Moregeneralneuronalmodelsdescribingmultimodalintegrationandbasedonstatistical
methodshavebeenpresentedelsewhere[59,60,61].Concretetheoreticalmodelsofmultimodal
integrationwithintheSChavebeendevelopedaswell[7,220,238,239,186,274].

olingoP2.5.2Notonlycanthemonosensorymapsbemergedintoamorereliablemultisensorymap,butthe
diverseinformation,thus,signalcharacteristicswithinthemonosensorymapscanbeaccessed
simultaneouslyaswell;rememberFig.1.5.Thissimultaneousaccessingisonlypossiblesince
allmonosensorymapsarealignedandconsequentlyspace-timecanservetolinkthedifferent
modalities.Consequently,anobjectatonespecificpositioncanbeidentifiedandcharacterizedin
ordertoselectmotorresponsesinacomplexenvironment.Forexample,arattlesnakemaydetect
spatialcoherentactivityinitsvisualand/orinfraredmap.Onlyiftheencodedobjectisvisible
andwarmitwillbeidentifiedasalivingpreyobject.Ifitisvisibleandnotwarmthesnakewill
discardneuronaltheANDandinformation.ORproExpcessingerimenstepstalforevidencethecomforsucbinationhapofoolingvisualofandinformationinfraredismappro[210vided,211by].
TheseprominentexamplesofpoolingintheSC/OTcouldenabletargetselectionandthusensure
appropriatemotorcommandsinacomplexenvironment.
Despiteincreasedreliabilityofanintegratedmap,itsindividualinputstreamscannotbe
distinguishedanymore.Thatis,theinformationaboutwhichmonosensorymaphasdetermined
thepositionislost.Withintheaboveexamplethemultimodalmapmayindicateamultimodal
eventahead,butthetriggeringmodality,thatis,visual,auditory,oryetanothermodalityremains
ed.unresolvInsummary,integrationofinformationallowsforareliablespatialdeterminationofanobject,
thekeytaskofobjectformation.Ontheotherhand,poolingofinformationassuresanaccesstothe
detailscorrespofondsantoobjaectswitchnecessarybetweenforobjeparallelctandidentificserialation.dataSwitcprohingcessingbettowbeenestinfittegrationdifferentandtasks.pooling
Toenableefficientmultimodalinteractionsuchasintegrationandpooling,alignmentofthe
differentmono-andmultisensorymapsisofcrucialimportance.Onlythencanamultimodal
stimulusataspecificspatiallocationbeidentified.Analignmentofsensorymaps,however,isnot
presentatbirthandmustbelearned[116,262,156],asdiscussedinthenextsection.

23

OptimalMapFormation

Development2.5.3Wenextcomebacktothequestionofhowsensorymapscanbealigned,i.e.,howacommon
multimodalspacecanevolvethatwehavebrieflyintroducedinSec.1.2andSec.1.3.Anobvious
solutiontosuchanalignmentprocesswouldbetheexistenceofonedominantmodalityasreference
forallothermodalities[158,156].Thisreferencemapwouldthenautomaticallyleadtomodifications
maps.otherallofAndindeed,experimentalandphysiologicalstudieshaveshownthat,inmanyanimals,destruc-
tionordisturbanceofthevisualpathwayleadstodisorganizedandabnormalsensorymapsin
non-visualmodalities.Thesefindingshavebeenobtainedinhamster[200],cat[280,278],clawed
frog[44],ferret[150],barnowl[159],andinsnakes[96].Psychophysicalexperimentswithcongeni-
tallyblindandnormallysightedhumanshaveshownthatvisualinputearlyinlifeisnecessaryfor
multimodalinteractiontooccur[122,230,233].Consequently,visionseemstoserveasa“teacher”
dalities.monon-visualforAplausibleargumentsupportingtheideaofvisionasteacherinputistheintrinsictopographic
orderoftheretina.Itisknownthatlayersofneuronscanself-organizeintotopographicmaps,
proreadervidedisthatreferredtoinitially[271a].smThisallsettogetherofcorrecwithtlytheorganizedsubsequentneuronsdevelopmenexists[t284of].layFeorrsainthereviewvisualthe
cortex(formice,see[133])mayallowtheintrinsictopographyoftheretinatostep-by-stepdictate
theorganizationandalignmentofhighervisualand,potentially,alsomultimodalmaps.
Thegeneralmechanismfacilitatingtheprecisionofsuchanalignmentofmapsisspike-timing-
dependentplasticity(STDP)[16,17,55,87,115,145,188,254,294].Anexamplewherethe
alignmenthasbeenstudiedindetail,bothexperimentallyandtheoretically,isaudio-visualintegra-
tionwithintheOTofthebarnowl.Hereexperiments[127,158]haveshownthattheauditorymap
followssystematicchangeswithinthevisualinput.Althoughtheprecisenatureofthisteaching
signalhasnotbeenclarifiedexperimentally,selectiveneuronaldisinhibition,orgating,seemsto
playakeyrole[103,285].Althoughexcitatoryandinhibitoryteachinginputcantheoretically
accountforpropermapalignment[78,56]onlyinhibitoryteachinginputisabletore-alignalready
[78].connectionsmapestablishedInsummary,theabovestudiessupporttheideaofvisionasteachermodalitytoalignother
monosensorymaps,buttherearecontradictingfindingsaswell.Wecansummarizethesefindings
intotwomajorpoints.First,visionisnotneededatallasteacherinputforthelearningprocess
ofsensorymaps.Second,visionshowsplasticityasitisinfluencedbyothermodalitiesandasit
improvesduringdevelopment.Concerningthefirstpoint,ithasbeenshownboththeoreticallyand
experimentallythat,althoughimprecise,amapofazimuthalsoundlocationcanbelearnedwithout
anyvisualinput[146,158]thoughadmittedlyonageneticallydeterminedsubstrate.Inaddition,
non-visualmodalitiescaninfluenceeachotheraswell,e.g.,auditioncaninfluencehaptics[31].
Moreover,somatosensoryreceptivefieldsalreadyshrinkinapostnatalphasewhenonlyauditory,
butnovisualneuronsarepresent[279,277].Forthesecondaspect,behavioralandpsychophysical
studiesshowthatvisualperceptioncanevenbeinfluencedbyothermodalitiessuchashaptics[68]
oraudition[249,77,258].Moreimportantly,visionitselfcanimprove,respectivelysharpen,as
foundinthevisualsystemofyoungcats[279,277].
Altogethertheexperimentalandtheoreticalfindingswehavepresentedabovequestionthe
currentpictureofvision-guidedmapalignment[148].WallaceandStein[279]havepointedout
thatthedevelopmentofdifferentmodalitiesstartsinparallelandintemporalcoincidencewiththe
appearanceofmultimodalintegration.Theyherebysuggestacommonmechanismdrivingboth
mapdevelopmentandmultimodalintegration.InChap.6,wewillsticktothisidea,namelythat
acombinationofallavailablesensoryinputintrinsicallyguidesitsalignmentbydevelopingthe
conceptofintegratedmultimodalteachingthataddressestheissuesmentionedabove.
Inthenextchapters,weturntomoresophisticatedsettingsthaninSec.2.4andgainnew
insightsintothecapabilities,characteristics,andlimitationsofthesessenses.

24

Esh¨ortdochjedernur,waserversteht.
JohannWolfgangvonGoethe

3.Auditoryprocessing:Exemplifyingoptimality

Inthischapterweapplyourmodelofoptimalmapformationtotheauditorysystem.That
is,signalweprodeducecessing,anaviz.,rchetypehandlingmodelechoandes.coBasedrresponondinourgmotempdel,oralwepreceptiveredictthefieldsabilitforyofauditobothry
mathematicalandneuronalimplementationstosuppressechoesandextractsoriginalsignals
foinrm.vaInriousaddition,echo-richwecscenaomparios,reevenstate-of-the-ainthertabsencetechnicalofexactalgoinforithms,rmationbiologicalonthesolutionsspecificandechoour
atherchetcompaypemorisondelhintsderivedtofromlimitationstheofgeneralourgeneralframewoframewrk.orkOriginandatingpromisingfromthissitesparfoticrulafuturerwsetting,ork.

ductionIntro3.1inIntheChap.pres2toentachapterconcretewesettingapplyinthetheframewtempororkalofdomain.optimalInmapordertoformationdoso,wewetakepreviouslytheauderivditoryed
pathwayasareferenceforanarchetypemodelofsignalenhancementandechosuppressionthat
yieldoptimaltemporalreceptivefields.Togetstarted,weinitiallyreviewtheessentialfactsand
specificquestionsofinterestaccordingtothissetting.
Innaturalenvironments,auditorysignalsarefollowedbyechoes,orreflections,thatdegradethe
signal.Inthenon-biologicalfield,thissettingiscalledreverberation.Notsurprisingly,biological
systemsprocessingacousticsignalshavedevelopedneuronalmechanismsofechosuppression
[20after,221the].originalHumans,signal,fordepexample,endingdoonnottheconsciouslysignalcpharactererceive[ec20ho].esWearrivingthereforelessexpthanectabdelaoutys20andms
suppression,inneuronalterms,inhibition,toshapethereceptivefieldsofourmodel.Fora
ofquanthetitativreceptiveundeerfieldstanourding,mohodelwever,predicts?moreHospwdecificoesthisquestionsshapeneedconnecttobetoasked.biologicalWhatisrealitthey?shapCane
differentfunctionsbeattributedtodifferentregionsofthereceptivefield?Canweidentifyuniversal
characteristicssharedbydifferentreceptivefields?
Wehavealsochosenthecontextofauditorysignalprocessingbecausethisisasitewhere
hatecvehnicalalreadysolubeentions,realized.viz.,Thisalgorithmspromisesthataaddressfruitfulsomecomparisonoftheofabothevemoquedelsstionsandtoabiology.certainAlthoughextent
filters,artificialsimplealgorithmsandarerobustquitealgorithmssophisticatedmaywoutperemarkerformthatmore“[...]forsophisticatedacousticecsolutionsho[...]”cancellation[105].
So,thereareartificialalgorithmswithgoodperformancethatprovideenoughsimplicitytobe
settingcomparableandtoconthetrastarcourhetypmoedelmodelandatitshand.collectionIndoingofso,receptivweeapplyfieldsourtowhatpresenwtemofinddelintoanatureneuronaland
technics.Finally,similaritiesanddifferencesofthederivedarchetypemodelmaypointoutinwhich
wayalgorithmscanbeimprovedbymeansofbio-analogouscognitivearchitectures.

25

SignalEnhancementandEchoSuppression

3.2Archetypemodel
Inthissectionwebrieflyrecapitulatetheessentialstepsofourframeworkofoptimalmapformation
intheconcretesettingofauditorysignalprocessing.Thatis,wederiveanarchetypemodelof
signalenhancementandechosuppressionbasedonlyonthemathematicalprincipleofoptimality.

3.2.1Optimality:Anarchitecturalprinciple
Themodelderivationisbasedontheideaofsignalreconstructionbyoptimalreceptivefieldsthat
inversethecorruptionprocessofechoesandnoise.Byderivingthenecessaryneuronalconnection
strengthsweformthebasisforasmoothintegrationofourarchetypemodelintotheexisting
knowledgeonauditoryprocessingsuchasrelatedphysiologicalandpsychophysicalresults.
Forthestartingpoint,themodelminimizestheexpectationvalueofthequadraticerror
E:=(st−sˆt)(st−sˆt)betweentheoriginalauditorysignal,s=(...,sµ,...),anditsoptimal
reconstructionsˆ=(...,sˆµ,...).ThereconstructedsignalsˆminimizesEandisthusoptimalin
theleast-squaresense.Thisisthedefinitionofoptimalitywehavealreadyintroducedpreviously.
Theoptimalreconstructionsˆiscomputedbythetensorl.Itscoefficientssufficethesetof
equations(∂E/∂lµν=0).Theresultingreconstructionalgorithmcomputessˆonlybymeansofthe
sensoryresponser=(...,rµ,...)oftheauditorysystem,sˆt=ltτrτ.Thecoefficientsltτofthe
matrixlaredescribinghow(inneuronalterms:withwhichconnectionstrength)avalueofthe
incomingsignalatthetimet−τismappedontotheoutputsignalattimet.Neuronallythiscan
berealizedthroughadelaylinewithdelaytimeτandasynapseofsuitablestrength.So,lcontains
thesetofoptimaltemporalreceptivefieldsthatcompensateforthecorruptionoftheoriginalsignal
onse.respsensorythewithinThesensoryresponseitselfwillbe,inprinciple,aconvolutionoftheoriginalsignalandthe
transferfunctionhthatinthissettingwedenoteasechofunction,viz.,rt=htτsτ.Inthetechnical
fieldtheechofunctionhismorecommonlyspecifiedas“roomimpulseresponse”(RIR).Itdepends
onthephysicalsurrounding,e.g.,the“room”.Inanalogy,thescenariobiologicallydenotedasecho
suppressionistechnicallyreferredtoasde-reverberation.
Asinthegeneralcase(seeChap.2),anyauditorysensorysystemalsohastocopewith
uncertaintiesnomatterwhethertheycomefrommeasurementerrors,varianceswithintheassumed
physicaltransmissionprocess,orsimplythroughthefactthatspaceandtimearecontinuous
quantitiesthatsensorsandespeciallyneuronscannotcontinuouslyrepresent.Artificialauditory
sensors,forinstance,typicallyaverageoveraspecificamountoftimeandgiverisetomeasurement
errors.Thetemporaldynamicsofneuronsareeveninherentlylimitedbycharacteristicssuchas
refractoriness.Inconsequence,theinput-outputrelationwithinanysystemwillbecorruptedby
errorsornoise.Wethusrewritethesimpleconvolutionrt=htτsτbyaddingthenoisetermς
accountingformeasurementandtransmissionfailurestooursensoryresponse.Inaddition,our
signalofinterestmay,andingeneral,willbedisturbedbycomplementarysignalswepoolandrefer
toasbackgroundnoisewemodelbyaddingarandomvariableαtothesignal.Wehencearriveat
rt=htτ(sτ+ατ)+ςt.(3.1)
Beforecalculatingthereceptivefieldslweneedtodiscussiftheassumptionsregardingthe
expectationvalueswehavemadeinderivingthegeneralframework(2.16)arevalidinthepresent
contextofauditorysignalprocessing.Ofcourse,theinputsignalstisdeterministicandwewould
notexpectanyproblem.However,wedonotknowexantetheexactvaluesofst.Wecanovercome
thisproblembylookingat“biologicallyrelevant”signals.Suchsignalsbelongtoaclassofsignals
thatwedenoteas“typical”.Consequentlyaspecificsensorysignalisaconcreterealizationofa
classoftypicalsignals.Thatis,theinputsignalstisastochasticquantitywithadefinedmean
µSandweuseanexpectationvaluetocomputeEinthesamewayasinthegeneralframework.

26

SignalEnhancementandEchoSuppression

Furthermore,wealsotakeallappearingcross-correlationstobezero,thatisnowthesignalvalue
atonespecificpointintimedoesnottellusanythingaboutthesignalvalueinthenexttimeframe.
Thesameholdstrueforbothtypesofnoise.Wetakethenoisetobeindependentatdifferent
pointsintime,eachwithstandarddeviationσς/α.Pleasenotethattheallocativefunctionforthe
noiseneednottobeGaussian.We,asexpected,canthereforeusethesameassumptionsasincase
of(2.16).Thatis,onlytheautocorrelationsremainandwithδµνasKroneckerdeltaaregivenby
sµsν=µs2δµν,ςµςν=σς2δµν,αµαν=σα2δµν.(3.2)
Wefinallyobtainalinearequationforthecoefficientsofthereceptivefieldsldependingonly
onthetransferfunctionhandthedimensionlessparametersσ:=σς/µSandη:=σα/µS,
lµγhνδhγδ1+η2+σ2δνγ=hνµ(3.3)
wherehνδisthetransposeofthematrixhγδ.Theparametersσandηcorrespondtoinverse
signal-to-noiseratiosofσ–themeansignalstrengthtothevarianceofthedetectormessurement
errors–andofη–themeansignalstrengthtothevarianceoftheacousticnoise.Thereceptive
fieldslnowmatchtheechofunctionhandallowcalculatingthereconstructionsˆoftheoriginal
auditorysignals.Sincewehaveneitherspecifiedtheoriginalsignalnorusedanyinformation
suchasinputcorrelations(3.2),ouralgorithmcanreconstructanyarbitrarysignal.Togetan
optimalreconstructionperformanceinthecaseofnoise,wesimplyneedtoadjustthetwomodel
parametersσandη.Ifwehaveaccesstothenoiselevelsσς/αandthetypicalvalueoftheoriginal
inputstrengthµS,thedefinitionofσandηgivesusagoodestimateofthesevalues.Incaseprior
knowledgeofintrinsiccorrelationsoftheoriginalsignal,wecanincorporatethisintoµSthathence
transformsintoamatrixwithoff-diadonalelementsandleavetheaboveminimalansatz.
Wehavenowderivedanarchetypemodelfromourframeworkofoptimalmapformationina
concretetemporalsetting.Inthenextsubsequentsectionwewillanalyzetheperformanceandthe
robustnessofthemodelbyapplyingittoexemplaryechofunctions.
3.2.2Receptivefields:Acomputationalbuildingblock
Byanalyzingthemathematicalmodelderivedintheprevioussectionwecanobservetheintrinsic
capabilitiesandcharacteristicsofourapproach.Wedealwiththreeexemplarybutexplicittemporal
transferfunctions,namely,theechofunctionshd;r;e;cf.Fig.3.1.Indoingso,wetakeadelta
functioninthetimedomainasinputsignal.Thisfunctioncorrespondstoaclick.Itappears
respslightlyonse.Thesmearedsimplestoutandenvirisfolloonmewntedbyfeaturinganecanhoecashowdefinedouldbbyeathesingleechosolidfunctionwallininathefreesensoryspace.
ecHere,hothefunctionechowewouldassumebeaasingle,simple,wdiscreteeakenedrreflectionepetitionofaofclicktheandoriginlabelalthesignal.echo“d”Henceforforthe“discrete”.first
Thesecondechofunction,“r”for“realistic”,mimicsatypicalroomimpulseresponse(RIR)where
wehavenotonebutmanywallsandwherethesignalisfollowedbyashortsilenceandmany
thesubsequenclick.Ftorthereflectionsthird[ec170ho].Wefunctiontakewetheseassumereflectionstheclosedtohavespaceatomaximbeevumenatmoreabout20restrictedmsafterand
hencedonotsupposeanygapbetweensignalandreflections.Herewetakeanexponentialdecayof
thereflectionsandthereforemarkitwith“e”.So,roughlyfollowingsomeexemplaryRIRs[105],
wecovertherangeofpossiblecomplexechofunctionsbythreesimplifiedscenarios,i.e.,explicitly
assumingthereflectingboundariesatτveryfar(case“d”),normal(case“r”),andclose(case“e”)
distance.Intensornotation(rt=htsτ),theyare
hdµν=exp−(µ−2ν)+exp−(µ−ν+2βd),(3.4)
22
2κ2κ
hrµν=exp−(µ−ν)2/2κ2+A(ν,µ),(3.5)

27

SignalEnhancementandEchoSuppression

Figure3.1:Threevariationsofechofunctionshwiththeoriginalsignalsandtheresultingsensory
respodiscretizednser;echosettingsfuncti“d”,ons“r”,hd,r,eand(bl“e”ack=from0,leftwhiteto=righ1t.)inThematrixuppernotationgraphs(asA−Cdefined)depictby(3.4)the
–(3.6).Thelowergraphs(dD,r,−eF)showtheoriginalsignals(adeltafunction,filledline)and
resultingexemplaryecsensoryhorespfunctions,onsesvriz.,(soliddiscreteline)(left),inrealisticarbitraryunits.(middle),andThroughoutexponenthistialpaper(righwet)useecho,threeas
.2.3.2Sectioninexplained22heµν=expexp[−−((µµ−−νν))/κ/2]κififµµ−−νν<>00(3.6)
2wheredefinesalltheconstancommontsarerealsynaptic(∈Ralph).aThefunction,functionµAand(ν,νµ):=denote(µ−theν+roβrws)expand−(µcolumns−ν+ofβr)h/,2γand
(µdela−yνb)etweindicatesensignaltheandrelativechoefordiscretizedconditiontime“d”anddifference.“r”.TheTheconstanparameterstsκβanddγandβdenoterdenotehowthethe
signalsandechos,respectively,getbroadened,andκisameasureforthedecayoftheexponential
“e”.onconditiintailWenowwanttoarriveatathoroughunderstandingofthereceptivefieldsinvolvedinour
theoptimalthreemoechodel.Tofunctionsthisend,definedwebhayve(3.4)–calculated(3.6);cf.sensoryFig.3.1respandonsesFig.and3.2.Wreceptivehaevefieldsalsoforeveacaluatedhof
theincludingrobustnessnoiseofandtheamo“mismdelbatcyh”comparingconditionwherereconstructedechofuandnctionoriginalandsignreceptivaleforvfieldsariousdonotconditionmatch;s,
cf.Figs.3.2and3.3.Importantly,wehavetestednotonlythecompleteoptimalreceptivefieldsas
derivedintheprevioussectionbutalsoanabridgedversionthatdoesnotrequireanintegration
withwindoandwandwithoutthereforenoise,allowwsorkpropreal-timeerly;profordetailscessing.seeBothFig.3.2.abridgedandcompletereceptivefields,
andl.MoreoInver,doingwesohawveeusetestedthethewrongcollectionreceptivofefieldsreceptivforefieldsinreconstructingcaseoftheasignalmismatchandbetwcompareeenh
Fig.3.3reconstructionandshopwthaterformancethewithreceptivthee“matcfieldshing”(mainlylrcondition.)areAllableninetopcopeossiblewithcasesextremearevdepictedariationsin
–namelythewrongecho.Thesignalcanbereconstructedbymeansofreducingthenoiseand
effectivelysuppressingtheecho;cf.Fig.3.3.Thisflexibilityisaprerequisiteforanybiologicalsystem
sincescenario.suchItastysurnstemoutcannotthataffordincreasingaspσecificmakesneuronalthewirindifferengtforevreceptiverypeossiblefieldseclesshosensible,(suppression-)more
robust,andthusmakesthemperformbetterinthe“mismatch”condition,cf.Fig.3.3.

28

SignalEnhancementandEchoSuppression

toFigureright.3.2:GraphsReceptiv(Ae−C)fieldslrepresenandtthereconstructedcompletesignalsreceptivsˆ;eforfieldscondild,r,etionas“d”,defined“r”,inand(3.3)“e”(grafromy)andleft
theabridgedversions(black).Theversionsd,r,andealwaysappearasleft,middle,andright.The
areshoreconstructedwningraphssignals(Dobtaine−F)d.Thethroughbcompleteothabridgedreceptive(blacfieldsk)landleadtounabridgedhigherp(graeaky)receptivamplitudesefieldsand
lessdepictedartifactsin(Gin−theI)withreconstructed10%noiseassignals.comAparednoisytosensorinputstillstrength.allowsAllforagraphsreliablehavebeenreconstructionexpressedas
inarbitraryunits.OriginalsignalandsensoryresponseareasinFig.3.1.

Toanalyzewhichpartofthereceptivefieldscorrespondstowhicheffectinsignalreconstruction,
wehavesetselectedentriesinthereceptivefieldstozeroandobservetheconsequences.Inresult,
e.g.forthe“realistic”receptivefields,thefirst,fastinhibitoryregionsharpensthecontourofthe
signalpeakandreducestheexponentiallydecayingtailoftheecho.Thesecond,slowinhibitory
regionreducesthesteepnessoftheechoonsetthatisimportantforauditoryobjectformation.

Tosummarizethepresentsection,wehavederivedandanalyzedasimple,yetgenericarchetypeof
modelforauditorystimulusreconstruction.Inthenextsectionwebrieflyreviewthestate-of-the-art
approachesintechnicalauditoryprocessingandrelatethemtothemodelathand.

3.3Technicalprospect
Havingderivedanddiscussedaveryfundamentalmodel,wenowrelateittomorerecentapproaches.
Fromthetechnicalpointofview,whatwehavederivedisasinglechannelde-reverberationalgorithm
thatusesonlyonefrequencychannel.De-reverberationiswellstudiedandanotyetcompletely
solvedproblemintechnicalaudioprocessing.
Ourapproachcoversawiderangeoftypicalde-reverberationalgorithms.Forexample,htτ=δtτ
correspondstoaconventionaldenoisingscenarioasdescribedby[66,290].Foraspecificecho
functionandσ=η=0themodelinverts,or,foraσ=0approximatelyinvertstheequivalentRIR
intheleast-squaresenseasdiscussedbyMiyoshietal.[195].Aspecialcaseoflinearfilteringisthe
parametersetofη=0andσ=1thatcorrespondstoasimpleWienerfilter[135,228,232,236];
forareviewonlinearfilteringsee[141].

29

SignalEnhancementandEchoSuppression

actualFigureec3.3:hoFlexibilitfunctionyhof(gratheymoarea)del.inAllarbitrarygraphsshounits.wtheIneachreconstructedcolumnasignaldifferensˆ(blacteckholine)functionandtheh
isusedtocalculatethesensoryresponsewhereasthereceptivefieldslarevariedineachrow.
Echomismatcfunctionhingtheandactualreceptivechoefieldsfunctionmat(Bc,hC,onD,Fthe,G,Hdiagonal)can(A,leadE,toI).Evresonableenusingresultsifrecepσtivisechosenfields
appropriately.Hereσ=5.Inthecasemismatchincreasedσleadstoresultsmuchbetterthanthe
initialσ=1whichcorrespondstotheWienerfilter[135,228,232,236].

Forabriefsynopsis,wehavetriedtosubdividethevarioustechnicalalgorithmsforde-
reverberationinthreegenerations.Thefirstgenerationconsistsofstraightforwardmethods,
thesecondcombinesseveralmethodsfromthefirstgeneration,andthethirdgenerationtriesto
exploitthesamesignalcharacteristicsthatareexploitedbybiologicalsystems.Singlechannel
(usingonlyonefrequencychannel)de-reverberationofthefirstgenerationcanbedistinguishedinto
threemajorgroupsofalgorithmsbasedonde-convolution,blindde-convolution,andsuppression.
Thefirstgroupassumesknownechofunctions(i.e.RIR)anddealswithapproximateandfast
methodsforinvertingit[105,195,206,215],asrealechofunctionsareusuallynotexactlyinvertible
[206].Thesecondgrouptriestofindafilterwhichaccountsforacertaincriterion,forinstancethe
meanpowerspectraldensityofspeechornoisereduction,andindirectlyreducesthereverberation
[66,73,90,105,273,290,291].Asinthepreviouscasethisfilterconvergestoanapproximation
oftheechofunctioninversion.Thethirdgroupestimatestheamountofechopowerspectral
density[22,105,173]andusessuppressionmethodstoperform,forinstance,minimummeansquare
estimationoftheoriginalsignal.Inthestateofartalgorithmsforechosuppression,theoneswerefer
toassecondgenerationechosuppression,differentmethodsarecombined[79,105,110,288].An
exampleforthethirdgenerationareattemptstorelyechosuppressioninanunknownenvironment
onthepitch[272,273].
Butstillthesealgorithmsdonotmeettherequirementsforartificialcognitiveauditoryprocessing
suchassoundsourcelocalizationorspeechrecognitioninadynamicreverberatingenvironment.In
contrast,oureverydayexperienceprovesneuronalalgorithmsthatallowsophisticatedauditory
processingexist.Therefore,inthenextsection,weshowhowthepresentedarchetypeforecho
suppressioncanbetransferredtoaneuronalsystem.Bycontrastingtheneuronalrealizationwith
actualneurobiologyweidentifysimilaritiesanddifferences.Thisallowstopinpointpromisingsites
ofcognitiveprocessingintheauditorypathway,thatis,thesitesforfutureresearch.

30

SignalEnhancementandEchoSuppression

Figure3.4:Influenceofparameterσonthereceptivefieldslfor“d”,“r”,and“e”formlefttoright.
Thereceptivefieldsld,r,eforσ=1(black)andσ=5(gray)inarbitraryunits.Asσincreases,the
receptivefieldsgetsmearedoutandlosefinestructure.Simultaneouslythereconstructiongains
generality,whichisadvantageousforreconstructingasignaldeformedbyanunknownechofunction,
seeFig.3.3.OriginalsignalandsensoryresponseareasinFig.3.1.

ectrosppBiological3.4Inthecourseofthisthesiswehavestatedthattheframeworkofoptimalmapformationcan
beimplementedtoaneuronalsettinginastraightforwardway.Inthefollowingweprovethis
statement.Ourpresentmodelofauditoryprocessingallowsaneasyone-to-onematchofthe
optimalreceptivefieldswehavederivedabovetotheneuronalconnectionsincludingtheexplicit
valuesofdelayandsynapticstrength.Indoingso,wemapthereceptivefields(seeFig.3.4)toa
feedforwardnetworkasdepictedinFig.3.5.Suchaneuronalnetworkisthencapableofextracting
theoriginalsignalandsuppressingechoes.Theperformanceoftheneuronalrealizationisdepicted
inFig.3.6fordifferentsituations,viz.,receptivefields,andvaryingσ.Justasexpected,thesignal
isextractedreliably.Thatis,weobtainaneuronalmodelthatsuppressestheechoandextractsa
signaljustasinthemathematicalmodel.Weherebydonotstatethatechosuppressionisdone
inbiologybymeansofasinglefeedforwardnetworkbutgiveanexistenceproofofourmodelin
are.hardwneuronalAcomparativelookontheshapeoftheoptimalreceptivefieldsrevealsthattheresulting
synapticstrengthsofthedelaylinesfollowveryplausiblepatterns.Themostobviousfeatureisthe
emergenceoftwodistincttimescalesofsuppression.First,adirect,fastinhibitionwithaclear
maximum(e.g.atabout2mslastingca.5ms;cf.Fig.3.4B).Second,adelayedshallowinhibitory
region(inFig.3.4Bfromabout10-17ms)thatisthereforecalled“slowinhibition”.Thesetwo
findingsfitverywellintotoday’spictureofauditoryprocessing.Gapdetection,auditorycontrast
enhancement,andechosuppressionexploitpropertiesoftemporalreceptivefields,moreprecisely,
delayedinhibition[36,37,100,101,282].Asforechosuppression,itisunderstoodthatitisin
partmonauralandinpartbinauralasvonB´ek´esyandKoenigalreadystatedmorethan50years
[20].agoThemonauralpartofthesuppressioncorrespondstemporallyaswellasfunctionallytothefast
inhibitioninourmodel.Ithasamaximumatsimilartimes(about2ms[108]).Physiologicallyitis
locatedinthecochlearnucleusandhard-wiredneuronallywithoutsignificantvariability[283],just
asinourmodelwherethefastinhibitionisindependentoftheactualechoscenario.Inthemodel
thefastinhibitionsharpensthecontourofthepeakandreducestheexponentiallydecayingtailof
theechobutisnotsufficientforcompleteechosuppression.Similarfunctioninghasbeenfoundin
andtheoreticallyexplainedforthecochlearnucleus[36].
Asforthebinauralpartofechosuppression,thingsaremorecomplicatedbynature.First,in
binauralprocessingmorenucleiareinvolved.Second,theyhavetodealnotonlywiththeinput
fromone,buttwoears.Third,theinvolvednucleireceivemanyefferentfibers,thatis,top-down
controlplaysarelevantrole.Evenifitishardtodisentangletheindividualeffects,theapparent
resultsarethefollowing.First,binauralechosuppressionisslowerthanthemonauralone[283].
Second,itismoreflexible.Moreprecise,ithasbeenshownthatabinauralinhibitorymechanism
suppressesandchangestheconsciousperceptualprocessesofechoesdependingonthesituation

31

SignalEnhancementandEchoSuppression

Figure3.5:Neuronalsetting.AnoisyresponseR(t)incontaininganechoisusedtostimulateaset
ofPoissonneurons[115]A,theso-calleddetectorneurons.Theresultingspikespropagatethrough
delaylineswithdifferentdelaysτ1,..,nandstimulatetheLeaky-Integrate-and-Fire-neuron(LIF)[88]
outputpopulationBthroughthecorrespondingsynapticconnectionstrengthsSc1,...,n.Synaptic
connectionstrengthsanddelaysaregivenbytheoptimalreceptivefieldscontainedinl.Theoutput
Sˆ(t)outoftheneuronsBrepresentstheoriginalsignalwithsuppressedechoandreducednoise.

[221similar].timeComparedcoursetothatthetimestronglycoursedepofendstheonslothewisituation,nhibitioncf.foundFig.in3.4.nature,Foururthermore,modelitshoreduceswsa
thewherethesteepnessflexibilitoftheyecofhotheonset.binauralThisdepsuppressionendencyisupcononthetrolledbysituationthefitstop-dothewneffebiologicalrents.counterpart
theInnaturalsummaryecho,asuppression.neuronalTheimplemenbinauraltationpofarttheofecarchohetypesuppressionmodelwthatorksisveryhighlywell.Itadaptableconnectscouldto
beasiteforfutureresearchtoidentifyandmathematicallydescribecognitiveprocessingsteps.

Discussion3.5Basedonthearchitecturalconceptoferrorminimizationandreceptivefields,wehaveappliedour
frameworkforoptimalmapformationtoaconcretesettingbydeducingaunifyingarchetypemodel
forreal-timeextractionofanacousticsignalfromacorruptedsensoryresponse.Wehavestudied
themodelperformanceinextractingtheoriginalsignalforthreeexemplaryechoscenarios.
Ontheonehand,wehaveshownlinkstocommontechniquesthatsolvetheinverseproblem
[242],applylinearfiltering[141]suchase.g.theWienerfilter[135,228,232,236],pseudo-inverse
techniques,andtheMaximumLikelihoodEstimation[135,228,269].Ontheotherhand,wehave
implementedthearchetypemodelintoaneuronalsettingtocontrasttheoptimalandthebiological
approach.Initially,wehaveposedandansweredthequestionastowhichneuronaldelaysplay
aroleinechosuppression,whataretheirsynapticcouplingstrengths,andwhataretherelevant
timescales(seeFig.3.2and3.4).Wenotethattheresultingsynapticstrengthsofthedelay
linesfollowbiologicalpatternssuchasthetwotemporaldistinctinhibitoryregionsandhintto
cognitiveprocessingsuchastheCliftonEffect[46].Thatis,theauditorysystemadaptsquickly
tocharacteristics(echoesatabout∼10ms)ofaparticularsurrounding.Inmathematicalterms,
thesignalprocessingshowsplasticitybymeansofadaptingquicklytoaspecifictransferfunction
onse).respimpulseom(roMoreover,auditoryobjectformation,bothspeechrecognitionandauditoryspacedevelopment,
areimpactedbythemechanismreferredtoasbinauralechosuppression.Thisisbecausetheslow
inhibitionreducesthesteepnessoftheonsetoftheneuronalresponsetotheechoandauditory
objectformationstronglyreliesontheonsetofacousticsignals[29,292].Theslowinhibition
isthusresponsiblefortheidentificationoftheechoasalocalizableobjectandaconsciousecho
perception.Inthislight,itmakessensethatitisrealizedbinaurallywithstrongtop-downcontrol.

32

SignalEnhancementandEchoSuppression

Figure3.6:Neuronalinputandoutputactivity;forsettings“d”,“r”,and“e”fromlefttoright.
AccordingtoFig.3.5,wefeedthesensoryresponseR(t)inwithanoiselevelof5%(ςt)toa
populationof140detectorneuronsA.Theresultingspiketrains(A−C)propagatethrough
thedelaylineswithdelaysτ1,..,nandstrengthsSc1,...,nandstimulateapopulationof140LIF
outputneuronsB.Thesetofdelaylinesrealizesthederivedoptimalreceptivefieldscontainedin
l.TheupperrowdepictstheactivityofthedetectorneuronsAforthedifferentechofunctions
hd,r,e,thelowerrowthecorrespondingactivitySˆ(t)outoftheoutputneuronsBfortwosetsof
reconstructionparametersσ.Moreprecisely,thelowerrowdepictsthenormalizednumberofspikes
oftheoutputpopulationwithσ=10(D),5(E),and10(F)(graybars).Choosingtherightσ
enhancesthemodelperformanceascomparedtoσ=1(blackbars)thatcorrespondstotheWiener
filter.Obviously,neuronalcharacteristicssuchasthespontaneousrate,hereroughly15Hz,donot
degradetheperformancesincethesignalisextractedreliably.ThethresholdoftheLIFneurons
evenenhancesechosuppression.

Comingbacktothestate-of-the-artalgorithmsfortechnicalde-reverberationandtheirpotential
forimprovement,werealizethatthenaturalapproachismorepowerful.Today’sspeechrecognition
andauditoryobjecttrackingalgorithmscannotcopewithnoisecausedbyreverberationindynamic
scenes.Eventheidentificationofoneacousticsourceinthepresenceofothersisaproblemnot
entirelyovercomebytechnicalsystems.However,arobustauditoryrepresentationaspartofa
multi-modalrepresentationasfoundinanimals[147,261]issupposedtobeanecessaryingredient
forcognitiveapplicationssuchasrobotsandcognitiveworkbenches.Consideringtheeasewith
whichevennotfullydeveloppedbiologicalorganismslike,e.g.,fledglingsmanagetoidentifyand
trackan,untilthenunknown,auditoryobjectsuchastheirmother’svoiceinanewenvironment
itisobviousthatthetwofoldviewonboththebiologicalandthemathematicalsideofthesame
problemdisemboguesintoafruitfulmergeofcultures.

Insummary,ourarchetypemodeluniformlytiesupwiththeknownphysiological,behavioral
mapfindings,formationtechnicalisaaspfeasibleectstoandoltomethoprodsvideofaechocommonconsuppression.vergingThpaterspis,ectivoureonframewsensoryorkofprocesoptimalsing
enablingthemultidisciplinaryapproach–bionics.

33

erceptionPLateral-line4.

Wirk¨onnendieWeltnichterkennen,wiesieist,
sondernnur,wiesief¨urunsist.
erretrandersNoorT

Thelateral-linesystemisauniquemechano-sensoryfacilityofaquaticanimalsthatenables
themnotonlytolocalizeprey,predators,obstacles,andconspecificsbutalsotorecognize
hydrodynamicobjects.Inthischapter,wepresentananalyticapproachthatshowstheeffectof
submergedmovingobject(SMO)shapetolateral-lineperception.Basedontheseresultswe
deriveanexplicitmodelexplaininghowaquaticanimalscandistinguishdifferentlyshapedSMOs.
Ourmodeloriginatesfromthehydrodynamicmultipoleexpansionandusestheunambiguousset
ofmultipolecomponentstoidentifythecorrespondingobject.Furthermore,weshowthatwithin
thenaturalrangeofonefishlengththevelocityfieldcontainsfarmoreinformationthanthat
duetoasimpledipole.Finally,theresultswepresentareeasytointerpretbothneuronallyand
technically,andagreeswellwithavailableneuronal,physiological,andbehavioraldataonthe
system.lateral-line

ductionIntro4.1AllfishandsomeaquaticamphibianssuchastheclawedfrogXenopuspossessauniquesensory
facility,thelateral-linesystem.Thissystemiscomposedofmechanosensoryunitscalledneuromasts
locatedonthetrunkoftheanimal.Theneuromastsconsistofsmallcupulae,gelatinousflags
protrudingintothewater,whicharesensitivetothelocalwatervelocity[21].Theabovementioned
animalsusetheirlateral-linesystemtolocalizepredators,prey,obstacles,orconspecifics.The
laterallineissoeffectivethatitenablesevenblindfishtonavigateefficientlythroughtheir
environmentbymeasuringandanalyzingthepressureandvelocityfieldofthesurroundingwater
[52,111].Moreover,itissupposedthataquaticanimalsnotonlylocalizebutalsoextractand
recognizefeaturesfromthehydrodynamicstructureofthevelocityfieldtodeterminesize,speed,
andpresumablyshapeoftheobjectgeneratingit[39,52].Thiscanbedoneevenindirectly,for
instance,bymeansofextractinginformationfromthevortexeswithintheobject’swake[52].

Thequestionwenowpose,andanswer,iswhetherandhowapassivedetectionsystemsuchas
thelaterallinecanbothlocalizeamovingobjectanddetermineitsshapeinanincompressiblefluid
suchaswater,orairatlowvelocities.Moststudies,bothexperimentalandtheoretical[54,75],
usedavibratingortranslatingsphereasstimulus.Duetothespecialsymmetryofthesphere,the
resultingvelocityfieldisexactlythatofadipole–nomatterwhethervibratingortranslating[171].
Ofcourse,notallobjectsinnaturearespheres;cf.Fig.4.1.Onlyforvibratingbodiescanone
identify[107,142]theinfluenceoftheirshapeontheflowfield,showingareasonabledominanceof
thedipole.Theliteraturelacks,however,ageneralexplanationinamoreglobalcontext.Herewe
showandquantifyhowmuchinformationisavailableinthevelocityfield,howonecanextractit
andtowhatextentindependenceuponthedistance.

35

ineLateral-l4.erceptionP

Figure4.1:Submergedmovingobjects(SMOs)appearinwidelyvaryingshapes.Furthermore,
aquaticstimulimaybutneednotmoveatall;forexample,vortexstructuresthataregeneratedin
thewakeofaswimmingfishorattheendofthefins[52].Thesestimuliimprintinformationonthe
flowlateral-linefieldthatorganscanisbemoreaddifiedoutbdueytothequanlateral-linetitiessuchorgans.astheInSMO’saddition,ptositheion,inputmotion,measuredandatshapthee.
Thebodyofthedetectingfishalsoinfluencestheflowfield,forinstance,bymeansofdamping
(FromNOAA’sHistoricFisheriesCollection;IDs:fish3078andfish3039)

Moreinparticular,wefirstderiveanalyticalequationsshowingthatshapeinformationplays
anessentialroleaffectingthevelocityprofileatthelaterallinewhichisthenaturalstimulusto
inthetermsmecofahanosensorymathematicalsystem.Second,descriptionweinthreedemonstratedimensionshowhisydropossibledynamicbyobjectmeanscofamharacterizationultipole
apreexpansiondator,can(ME).readInoutaddition,thevweloecitshoywfieldhowandanaquaticreconstructanimal,theshapcalledeofasdetectingtimulus,animalviz.,(DaA),subsmuchergedas
movingobject(SMO),suchasprey.Finally,wegivelinesofevidencethattherearebiologically
therelevanrangetuplimitstoofwhichstransmittinghapeandinformationextractingisimportaninformationt.The–forlatterboth,hasthenotbeencapabilityexploredtoloupcalizetonoandw.

4.2Analyticdescriptionofshape
Asexplainedabove,disturbancesofthesurroundingwaterformthenaturalinputtothissensory
system.Theinputmay,thoughneednot,begeneratedbysubmergedmovingobjects(SMOs)and
dependsontheirposition,motion,andshape.Thebodyofthedetectingfishalsoinfluencesthe
flowInfieldthis,cforhapterinstance,wefobycusonmeansandofisolatedampingthewaterinfluenceofdisturbances,theSMOviz.,shapthee.signal;Tocf.outlineFig.the4.1.basic
poideawoferfulthetwpresento-dimensionalsection–anmethodsanalyticalbasedonthedescriptioncomplexoftheforminfluenceulationofofstimpotenulustialshapfloew–devweeloprecalled
forairshipdesign[270].Potentialflowisavalidhydrodynamicdescriptionofthelateral-line
perceptionalthoughitneglectseffectsofviscosity[95,111].Acomplexpotentialflowdescription
ofthree-dimensionalhydrodynamicsettings–includingevenrotational-symmetricbodies–is
ellipticallyanalyticallyshapnotepdossibleprofiles(seewithinApptheendix(x,yB).)Wplane.eInthereforedoingso,reducewethegainobjectsanalytictoalaccesscircularlytoandthe
essentialeffectsofobjectshapeontheflowfield.

36

4.Lateral-linePerception

a=Figure10−43.2:,2.5Flo,3w.5;aroundtheradiusthreer=differen4,andtlyV0shap=ed1.Weprofiles.indicateFromthelefttoprofilerighbyt:grashapy.eOneparametercanseeho(4.2)w
theprofilebecomesellipticandthestreamlinescorrespondingtotheinputtothelaterallineadapt
accordinglytoobjectshape.

rmulationfoMathematical4.2.1ThecomplexvelocitypotentialF(z)candhencethecomplexvelocityw(z)c:=dF(z)c/dzwith
x+iy=:z∈C,aroundacircularprofile[218]isgivenby
F(z)c:=V0z+r2(4.1)
zthewhereJoukV0owskidenotesthetransformationmovingspζ(eedz):=andzr+athe2/zradiuswithofshaptheecircularparameterprofile.a∈InR[the218],nexti.e.,step,wconformaleuse
mapping,toobtaintheflowfieldwe(z)aroundanellipticallyshapedprofile.Herethesuperscripts
ctheandfloewreferfieldwtoeincirculartheζand-planebutelliptical,intherespectivoriginalely.planeInconoftrtheasttosphere,thetheusualzwa-plane.y,weThwillatis,notwestudycan
identifythecoordinatesandthuscomparetheflowofthecircularandellipticalprofile,
edFdzr2a2−1
w(z)=dzdζ=V01−z21−z2.(4.2)
wc(z)ddζz
Inthiswaywehavederivedthecomplexvelocitywe(z)oftheflowaroundanarbitraryelliptic
eandshape.theFuvelocitrthermore,yfieldwveecan=(ve,computeve)byallinmeansterestingof(4.2)quanandtitiesz=xsuc+hiyas.theAllvelothesecityptermsotenaretialΦlisted(x,yin)
Table4.1.Thetransformationxyfromtheflowaroundcircularprofilestotheflowaroundelliptical
onescanbedescribedbyonlyoneparametera∈R.Asa→0,allellipticalquantities(seeTable
4.1)convergetothecorrespondingquantitiesofthecircularflow.
Wesummarizethesethoughtsbystatingthatthecircularflowisonlyaspecialcaseofthe
moregeneralellipticflow(4.2).Asaconsequence,weobviouslycannotexpectaprioristatements
derivedfromcircularscenarios,i.e.,thespecialcase,toholdgenerally–ifatall.

eshapofEffects4.2.2Havingderivedageneralexpression(4.2)fortheflowaroundanellipticallyshapedprofile,wenow
exhibitfundamentalmodificationsofthevelocityprofileasitismeasuredbythelaterallinedue
tostimulusshape.Inthepresentliterature,onedistinguishestwoclassesoftheoreticalmodels
thatexplainthedetectioncapabilityofthelateral-linesystem[54,75,95,251].Modelsofthefirst
thecategoryfish’sboexplaindy.Thethepseerfcondormanceclassofofmothedelslateral-lineexploitschsystemaracbteriysticusingpointhetssucwholehasvelocitextremyaprofileandzerosalong

37

erceptionPineLateral-l4.

Figure4.3:TherelativequadraticerrorΔofthevelocity-profileextremaatthelaterallinexEin
dependenceuponthedistanced/randshapeparametera/rnormalizedbytheradiusr.Thegraph
showsΔ:=[xE(0)−xE(a)]2/xE(0)2ford/r=0.5,1,1.5.Fordistancesdthatarecomparable
totheSMOsize(radiusr),viz.,inmathematicalterms(d≈r⇔d/r≈1),thepositionsofthe
extremaxEclearlychangewiththeshapeparametera.Presentmodelsignoringshapeeffects
calculatingthedistancedfromalinearrelationd∼xEthereforeoverestimatetherealdistance
beyondd≈r.

ofthevelocityprofilealongthelaterallinesincethesepointsaresupposedtobeeasilyaccessible
.neuronallyWefirstanalyzethelattercategory.Knowingtheorientation,i.e,thedirectionofsensitivity
g=realizedg(x,byy)andmeansposofivtiong(rr)==r(dΦx,/ydg).ofIntheotherwlateral-lineords,weorgans,canevcalculateerythelateral-linewatervelogeometrycityincanevberye
directiongandatanypointroftheflowfield.Here,foreaseofcomparisonwithcommonscenarios
inthethedetectingliterature,animal.weInstudythisansetup,SMOcenthevteredelocitaty(0,profile0)ispassingidenticalalongwiththevx-axisandatyadenotesdistanceyexactlyto
xtheWedistancestartbdybresetwcalineengtheourcentersexpressionoftheforvSMOxbyandmeanstheoftrunknormalizinglateralline.allquantitiesbytheSMO
size,i.e.,theradius.Thisresults,forinstanceina→(a/r)r=ar.Thenormalizationmakesv
explicitlyindependentofr.Computingthex-valuesxEoftheextrema(skippingxE=0)bymeansx
ofcomputing∂vx/∂x|x=xE=!0givesxE(a)=±a2+dd±2√a2+d21/2.Obviously,the
extremadependontheshape;inmathematicalterms,ontheshapeparametera∈[0,1].Onlyin
thecaseofacircularprofile(a→0)dotheextremaencodethepuredistancexE(0)∼d.Common
tomothedelszerosexplaind∼the(xloE1−calizationxE2)[p54,75,erformance95].oftheWithoutlateral-linetakingthesystemshapbeyintomeansaccounofat,linearthepresenrelationt
modelsthereforeoverestimatetherealdistance.
detectingAtthesamelateral-linetimesystemtheaseffectwellofasshaptheeSMOalsobdepodyendslengthonthethatdistancedirectlybetrelatesweentotheSMOandparameterthe
rangeradiusrdefinedintheastheequations.distanceAsupwetocantheseeSMOinbFig.ody4.3,lengththeandinfluvenceanishesofforshapelargerdominatesdistancesthednearb1.y
Nextweturntotheotherclassofmodelsthatnotonlyexploitparticularpointsbutthewhole
pvelootencitytialfieldofatellipttheicallateralprofilesline.Fe(Tzo)inanalyzeaTaylortheseriesinfluencearounofdshapa=e,0,weexpandthecomplexvelocity
Fe(z)=V0z+rz+rz3V0−Vz0a2+O[a]4.(4.3)
22
Fc(z)
Throughthesubstitutionc:=r2/a−awecantransform(4.3)intoapowerseriesinz,
Fe(z)=V0z+V0ca+a3+a5+O[a]7.(4.4)
zzz

38

4.Lateral-linePerception

Table4.1:Hydrodynamicetoolbox:Comparison2oftheflowaroundcircular(denotedbyc)and
theellipticalflowaround(denotedanbyellip)ticalprofiles.profilebDefiningymeancs:=ofr/aonly−a,oneweshapcanederiveparametersimplea.equationsdescribing
Fc(z)=V0z+rz
Complexvelocityp2otential
Fe(z)=V0z+2c[ln(z+a)−ln(z−a)]
Hydrodynamicpotential
22Φc(x,y)=V0x+x2r+2xy2
Φe(x,y)=V0x+4cln((xx−+aa))2++yy2
Velocityoftheflowinx-direction
vxc(x,y)=V01+x2r+2y2−(x22r+2yx22)2
vxe(x,y)=V01+2[(xc+(xa)+2a+)y2]−2[(xc−(xa)−2a+)y2]
Velocityoftheflowiny-direction
vyc(x,y)=−V0(x22r+2yy2x)2
22vye(x,y)=−V0a4−2a2(2(x2r−−y2a))+(yxx2+y2)2
Equation(4.4)showsmoreclearlythan(4.3)thattheshapeparameteraisthenaturalexpansion
parameterforthecomplexmultipoleseriescharacterizingshape.Wewillexploitthederivedfinding
thatmultipolescanbeusedtodefineandcharacterizehydrodynamicobjectshapelateron.
Thedifferencebetweenthefloworiginatingfromacircularandanellipticallyshapedprofile,
andthustheimpactofshape,canbeexpressedtofirstorderby
22ΔF(z)=[Fe(z)−Fc(z)]=−arV0+O[ar]3(4.5)
zwherea=a/risagainnormalizedandrangesbetween0and1.Itiseasytoseethatthecomplex
velocitydifferenceΔw(z)=dΔF(z)/dzandthusthemeasuredvelocitydifferencescalesas(r/z)2.
ThebodylengthoftheSMO(≈r)thereforedefinesanaturallengthscaleuptowhichtheeffectof
shapecontributessubstantiallytotheflowfield.Withinthisrangethedipoleapproximationis
onlyavaliddescriptionforcircularshapes.
Insummary,shapeshiftsthepositionofcharacteristicpointsofthevelocityprofileandalso
modifiesitasawhole.BotheffectsdependontherelationbetweendistanceandSMOsize.Since
underpotentialflowconditionwehavep∼vv[95],theSMOshapeaffectspaccordinglyand
thusalsothecanallateral-lineprocessing.Moreoverwehavegivenananalyticallineofevidence
thatthenaturalrangeuptowhichinformationaboutshapeistransmittedistheSMObodylength.
Havingdiscussedreallysimplebutneverthelessnon-trivialshapes,wenowwanttogivean
outlookonpossibleextensionsofourmodel(4.2).Itisstraightfrowardtoadapttheequations
inTable4.1tomorefish-likeshapes.Substitutingz→(z+b)withb∈RintheJoukowski
transformationleadstomorecomplicatedbutricherexpressionswheretheadditionalshape
parameterbinreturnallowsformorerealisticshapes.Wehavedepictedsuchshapesandresulting
velocityprofilesinFig4.4.
Inthenextsection,however,weturntothree-dimensionalshapes,theirinfluenceonlateral-line
perception,and,moreimportantly,howthelaterallinecanrecognizesuchdifferentlyshapedSMOs.

39

erceptionPineLateral-l4.

Figure4.4:Effectofthesecondshapeparameterb.Thegraphsdepicttheflowfieldaroundthree
differentprofileswithvaryingthesecondshapeparameterb=0,−0.1,−0.3(fromlefttoright).
TheradiusoftheSMOisr=1andtheshapeparametera=0.5from(4.2);bothrandaare
constant.Theprofileitselfisindicatedbythegrayarea.Duetovariationofbtheprofilebecomes
moredrop-shapedandthestreamlinesadaptaccordingly.

recognitionobjectdynamicHydro4.3mInultipthisolespart,iswecapablewanttotopicdescribkeupthetheinfluencefindingsooffSMO(4.4),shapnamee,lyandthatderivaeseriesbothofahydromathematicaldynamic
definitionanddescriptionofhydrodynamicobjects.Inaddition,wederiveanoptimalobject
inthreerecognitioncoordinates.algorithmInthatdoingcanso,lowcalizeeareandabletoreconstructpredictptheositionandcapabilitiesshapofeoftheSMOlateral-linesimultaneouslysystem.

4.3.1Definitionofhydrodynamicobjects
Aswediscussedintheprevioussection,avelocityfieldrepresentsanadequateandnaturalstimulus
tothelateral-linesystemandcanbedescribedbyamultipoleexpansion(4.4).Now,weleavethe
two-dimensionalspaceandfocusonthreedimensions.Becausetherelevantfluiddynamicsarewell
realdescribveloedcitbyyptheotentialEulerv(r)=equation−[Φ(95r])w[e171tak]efortheathree-startingdpimeoinnst.ionalUsingvelothecityrealfieldsphericalderivedfromharmonicsthe
YlRmandthesphericalmultipolemomentsq=(...,qlm,...),wecanexpandthevelocitypotentialΦ,
llR∞∞Φ=qlmΦlm=qlm1Ylml(θ+1,ϕ)(4.6)
l=1m=−ll=1m=−l2l+1r
describForedthesakthrougheofcosimplicitordinatesyweς∈fo[0cus,α]onandηrotationally∈[0,2π],symmetricalbodieswhosesurfaceScanbe
−1/3
ςπ√2γsin(α/2)sin(ς)cos(η)
S(ς,η)=Ncosα3×√2γsin(−1/α/32)sin(ς)sin(η)(4.7)
γ2/3cos(ς)
Thethatvsurfaceortexstructuparametersresαcan∈be(0,πdesc/2]ribandedbγy∈(0means,∞)ofadeterminemultipoletheshapexpanesofionthe(ME)surface.aswellWe[171note].
BecauseSMOssuchasfishleavecharacteristicvorticesintheirwake,thefindingswederivein
thisandthefollowingsectionsarealsoapplicableforidentifyingSMOsindirectlybymeansof
characteristicvorticesintheirwake.
BecauseoftheEulerequation,onlytheNeumannboundaryconditionv∙nS=0isrealizable
wherenSdenotesanormalvectortothesurfaceS.Intermsofsphericalcoordinateswecalculate

40

4.Lateral-linePerception

Figure4.5:Differentshapeparametersresultindistinguishablesetsofmultipolecomponents.We
depictanapproximationforonlythreemultipolemomentsl≤k=3in(4.8).TheSMOsurface
parameterαincreasesalongtheverticalaxisandtheSMOsurfaceparameterγalongthehorizontal
one.Wehavecalculatedthesetofmultipolesqˆthrougharasterof30,000randomlydistributed
positionsoneachmovingobject’ssurfaceandusedobjectswithlengthsofabout5cmanda
matchingmovingspeedv∞=0.01m/s.Eachvolumeisnormalizedtothevolumeoftheunitsphere.
Itisthereforefairtosaythatthedipoleq10staysalmostconstantwhilethequadrupoleq20varies
accordingtoαandtheoctupoleq30accordingtoγ.Therelativeerror(k):=|Tqˆ−w|/|w|on
theSMOsurfaceSascalculatedforrotationallyinvariantellipticbodiesgives(k)<0.4fork=3
andγ<2.Itconvergesslowlytoabout0.15fork→∞.Forγ2amultipoleexpansioncannot
describethecorrectvelocityfield.This,however,isnottoorestrictive.Forexample,thefishin
Fig.4.1canbedescribedbyγ≈1(leftfish)andγ≈2(rightfish).

thecomponentsofv(r)=−Φ(r)=(vr,vθ,vϕ)atrthroughcoefficientsarlm,etc.,sothat
∞l∞l∞l
vr=l=1m=−lqlmarlm,vθ=l=1m=−lqlmaθlm,andvϕ=l=1m=−lqlmaϕlm.(4.8)
Thegeometry-depequationsending(4.8)cancobeefficienwrittentsarlinm.Tmatrixoexplformicitlyv=AcalculateqwiththethesinglematrixmAultipconoletainingmomenthetsqlonlym,
weuniformhavesptoeedspvecifyofthethebSMOoundaryintheconditionstationarywithrespframeectoftonSreference.and,forThethebsakoundaryeofsimplicitconditiony,thenthe
∞readsv(rS)∙nS=nS∙v∞=nSTA(rS)qwithrS∈S.(4.9)
theThevelocitinfluenceyfielofdtheonlymupultiptoaolesincertain(4.8)numbdecreaseserkandwithsetqlmincreasing=0lfor;lsee>k(4.6)..WeHencearethwuseexpanddealing
withanapproximatedsetofmultipoles,denotedbyqˆ.Tofindanappropriateapproximation
wassoetakciatedesevsurferalacpeositionsnormals1n≤i,i≤thenmatrixrandomlyAi,anddistributedwriteitonlineanbySMOlineinsurfacetotheandtransfercalculatematrixthe

41

ineLateral-l4.erceptionP

T:=(...,niTAi,...).Theindexiofthevelocitycomponentswi:=v∞∙niofthevectorw
spcorrespeedvondsoftotheniSMOandleadsthereforetolabincreasingelsthevsamealuespofosition.thecorrespHere,weondingnotemthatultipanolemomenincreasingts.moInvingthe
∞nextsection,thiswillbecomeimportantinthecontextofthedetectororganssignal-to-noiseratio.
TocalculatethemultipoleswesimplysolvethelinearequationTqˆ=w.Themultipolemoments
followfromqˆ=(TTT)−1TTw[228].Thesolutionapproximatesthemultipolecoefficientsuptoa
givenkoptimallyinthesenseofminimalquadraticerror.Introducingacharacteristiclengthscale
λ,saythebody-lengthoftheSMO,intoequation(4.6)weget
v(r)=qlm(λ/r)l+2flm(θ,ϕ)(4.10)
,mlwhereflm(θ,ϕ)arefunctionsdependingonlyontheangularcoordinatesθandϕandcanbe
calculatedbymeansof(4.6).Indoingso,weobtaindimensionlessmultipolemoments.Onthe
onehandwecancomparethemindependentlyoftheSMOsize.Ontheotherhandweseefrom
v(r)=l,m(λ/r)l+2qlmflm(θ,ϕ)howtheinfluenceofeachmultipolevariesindependenceupon
thetellsussizeλthatofthetheshapSMOeofandthetheSMOisdistanceonlytoimptheDortanAtlabiftheledebsizeyλtheandradialthecodistanceordinaterarer.ofThistheresultsame
willnotmagnitude.transmitItisane.g.yobshapeviousthatinformationplankton,throughwhicthehisflowtenfieldtoh–andundredinthistimescasesmallershapethantheinformationDA,
isnotneededeither.Thisresultisinperfectcorrespondencywithwhatwehavederivedpreviously
startingfromdifferentequationsinthecomplextwo-dimensionalspace;cf.Fig.4.3.
However,asstatedbefore,inschooling,matefinding,orpredator-preybehaviorobjectinforma-
tioncanmakeadifference,andasshowninFig.4.5,differentlyshapedobjectsarerepresentedby
differentsetsofmultipolemoments.Consequently,multipolesareabletoquantitativelydefineand
describehydrodynamicobjectsaswehavedemonstratedforthefirsttime[251]–asfarasweknow.
4.3.2Objectrecognitionperformance
How,then,canaquaticanimalssuchasfishreconstructtheabovemultipolemomentsfromwater
velocitymeasuredthroughtheirlateral-linesystem?Ofcoursetheanimalhastofulfillthistask
undertheinfluenceofomnipresentnoiseandthelimitationsofitsneuronalsystem.
Ourreconstructionmodelforthree-dimensionalshaperecognitionisbasedonamaximum-
likelihoodestimator[276]andtheframeworkofoptimalmapformationthatwehaveintroduced
inChap.2.Thatis,wearelookingforthemultipolesqwiththehighestprobabilitygiventhe
measuredvelocitieswontheDAbody.Sotospeak,thisestimatormaximizestheconditional
probabilityp(q,r0|w)fortheSMOcenterofmassr0,whichhasalsotobedeterminedbymeansof
theinformationavailablethroughtheDAsuperficialneuromasts,thewatervelocities.Thevelocity
sizewiatorganiisgivenby
wi=T(r0,ri)q+ni.(4.11)
WenotethatthetransfermatrixT(r0,ri)dependsontheSMOpositionr0andtheposition
riofthelaterallineorgani(DA).Asintroducedinthefirstchapter,thetransfermatrixmaps
thesignalontothedetectors.InthiscaseT(r0,ri)projectsthemultipolesqlmontothevelocity
componentswimeasuredatlateral-lineorgani.NoiseismodeledbyaddingindependentGaussian
randomvariablesniwithmean0andstandarddeviationσntothewi(4.11).Allpositionsr0areof
equalprobability.WethereforetakeaGaussianprobabilitydistributionp(q,r0)∼exp(−q2/2σq2).
InsteadofmaximizingthecombinedprobabilitybasedonBayeslaw,wemaximizeitslogarithm
L(q,r0):=ln[p(q,r0|w)].Definingthedimensionlessparameterσ:=σn/σq,wejusthaveto
maximizeL(q,r0)=−[w−T(r0,ri)q]2+σ2q2(4.12)


42

4.Lateral-linePerception

Figure4.6:Neuronalimplementationof(4.14)isstraightforward.Inafirststep(A)→(B)the
vmeloultipcitiesolewicompattheonentsqˆlmneuromasts(r0)((BA)).areThiscalculatedcorrespforondstodifferenanettwporkositionsofr0synapticfromtheconnectionsmeasuredbetwweenater
thelcomputedateralfromlineandtheenthetriescenoftralthenervrighoust-handsystem.sideTheof(4.14)strengthorofcanthebelearnedindividualneuronallyconnections[74].canInbae
secondconnections;step(Bcf.)→(4.12).(C),LThe(q,mrax0)imisumL(calculatedqˆ,rˆ0)fromindicatestheqˆrthe0,whiccorrecthpcanositionalsobofethedoneSMObyandfeedforwselectsard
theoptimalqˆr0fromstep(C)→(B);cf.(4.13).

thewithlikrespelihoectodtoL(theqˆ,rˆm0)ultipattheole-momencorrectlytvectorestimatedqandpr0.ositionTherˆ0isnecessaryconditionforamaximumof
∂L(q,rˆ0)|q=qˆ=TTT+σ2q−TTw|q=qˆ=0,(4.13)
q∂whichleadstoalinearsystemofequationssolvablebymeansofthepseudo-inversetechnique[228],
qˆ=TTT+σ2−1TTw.(4.14)
AWeneuronestimatealtheimplemensetofmtationultipofolesthqˆesegivenonlycalculationstheismeasuredstraighvtforwelocitiesardwandatcanthebedonelateral-lineeasilyorbgans.y
neuronalhardware;seeFig.4.6.Thusthepositionandtheappropriatemultipolemomentscanbe
.neuronallycalculatedTotestwhetheritispossibleforaquaticanimalstodetermineSMOpositionandshapeusing
theabovemethod,wehaveappliednoisetotheinputsignalwi.Wehaveusedσn=10−4m/sas
standarddeviationofthenoise,whichcorrespondstothevelocitythresholdofXenopus’lateral
lineorgan[21].Furthermore,wedefineasignal-to-noiseratio,SNR:=v∞/σn.Plausiblemaxima
ofthemovingspeedofSMOsrangebetweenonebody-lengthpersecondandzero.Tobefair,in
theexamplesinthefigureswehaveusedavelocityaslowasv=1cm/scorrespondingtoonly
1/5to1/10ofthetypicalmaximummovingspeed.Thatistosa∞y,wehavedealtwithaworse-case
scenariosinceincaseofhighermovingspeedv∞thereconstructionperformanceisevenbetter
thanpresented.Anotheraspectregardinglateral-lineperformanceisthenumberofdetectororgans.
Withinabiologicalrangetheinfluenceoftheexactnumberoflateral-lineorgansismarginal.We
butshowstainysFig.almos4.9tthatconstanthetfordiffreconstructionerentnasumwbellersasoftheloneuromasts.calizationOferrorcourse,increasestaking,withfortheinstance,distanceten
timesmoreneuromastscancompensateahigherSNRvalue–butonlywithinmarginalranges.In
summaryAnother,theissueSNRis,thedominatesmoremtheultippoleswerformanceeuseasthemorecomparedptoositionsthearenumofbherighofprobabilitneuromasts.ybecause
onethehasshapemoreestimateparametersimprovtoes.fitWtheecancmeasuredhoosewian’s.Thusappropriatetheσestimatedforapcertainositionktogetscompambiguousensatebutthis

43

ineLateral-l4.erceptionP

Figure4.7:Thelateralline(A,grayline)iscenteredat(0,0).Thedistancebetweentheobject
(SMO,grayarea)centerofmassr0andthelateralline(DA)isjustbelowthelengthofthelateral
line.Usingthedipoleterm(k=1)only,wecaneasilyreconstructthepositionrˆ0(A,cloudof
mblacultipkpoleointsmomenaroundts(rk0=),3a)asresultanefficienconsistenttwiminimalththatmoofdel,Fourranoschmethoetdal.[allo74ws].Ifaweproptakereonlyestimatethreeof
bothposition(A,smallopencircles)andform,cf.Fig.4.8–eventhoughmorepositionsarenow
likelytooccur.Forbothcases,40reconstructionshavebeendepictedwithσn=10−4m/smoderate
SmapNRof:=pv∞osition/σn=100estimation.andIt500showsneuromasts.noamBbiguity;depictscf.logFig.[L4.(qˆ,6C.r0)]correspondingtotheneuronal

effectandenableafaithfullocalization;fordetailsofhowtotuneσconsiderChap.2.InFig.4.7
weshowtheabilityofourmethodtoestimatetheSMOpositionrˆ0.
Followingtheansatzoftheneuronalmodel(Fig.4.6)wehavecalculatedthemultipolemoments
attheestimatedposition,i.e.,notonlyatthecorrectpositionbutalsoatrˆ0+δrwhereδraccounts
forthecontinuityofrealspace,alimitednumberofmapneuronsaswellasaslightlocalization
uncertainty(Fig.4.7).Ourmodelcanreconstructqˆr0evenundernoisyconditionsandatslightly
wrongpositions(Fig.4.8);forexample,thefishbodiesofFig.4.1(seealsoFig.4.5)canbe
recognizedanddistinguished.Thusobjectlocalizationaswellasobjectrecognitionbasedonthe
estimationofmultipolesispossiblesimultaneously.
Inthesubsequentlineswediscussthelimitationsoftheproposedmethodandlikewisethe
theoreticallimitationofaquaticanimals’abilityofobjectrecognition.Tocomparethequalityof
differentmultipoleestimatesqˆatdifferentdistancesd,wedefineδq:=(qˆlm−qˆlm)2/qˆlm
asqualitymeasure.Hereδqlabelsthenormalizedstandarddeviationduetowronglyestimated
positionsrˆ+δrandnoise.Forsmallδqtheestimationerrorinqlmissmalland,inspiteofaslightly
wrongposition,theshapeparameterγcanberecognized.Forvaluesδq>1ashapereconstruction
basedontheestimatedqˆlmisimpossible.
Theresultsofournumericalexperiments,thatis,thelateral-lineperformanceourmodelpredicts,
supportthepreviousanalyticalfindingsasfollows.Moreprecisely,fortheoctupoleq30thecritical
distancewheretheerrorδqstartstogrowextremelyfastistheSMOsizeitself.Forlocalizationthe
criticaldistancecanbetakenasthelengthoftheDAlateral-linesystem,sayafishlength,which
isincaseofapredatorhuntingforpreylargerthanthecriticallengthofshapereconstruction
4.9).(Fig.

44

4.Lateral-linePerception

qˆ30Figure(I−4.8:L)normalizedReconstructedbythedipoledipqˆole10(Astrength−D),q10forquadrupα=oleπ/qˆ220(andEv−Hariable),andshapoesctup1ole≤γ≤strength2at
withdifferentnoiseatdistancesthed.mostThelikelygraypositioncolored(lighregiontgrey)depictsandtheatvslighaluetlyofincorrectlyestimatedmultipestimatedolepstrengthsositions
(darkgrey)asexplainedinthemaintextandcorrespondingtoFig.4.7.Sincewehaveusedan
approximationwithonlythreecomponents,thestrengthsofthemultipolesdifferfromtherealqlm
(blacdistinguishkline).differenRemarktlyablyshap,qed30isobanjects.approWeseeximately(A,B,linearE,F,Ifunction,J)ofthatγforandthsmallusdialmoststancesbsufficesetweento
ME.SMOIfdandDAapproacdifferenhestlySMOshapsizeed(5bocm),diesthecanbereconstructiondistinguished,startsevengettinginspiteblurredofnoise(B,Cand,F,G,Jtruncated,K).
Atlargerdistances(D,H,L)nostrongcorrelationbetweenqˆ10,qˆ20orqˆ30andγcanbefound,and
thusnoshapecanberecovered.NeverthelessthetwofishinFig.4.1with,say,γ=1andγ=2
canbedistinguishedatleastatd=3cm(A,E,andI)andd=4cm(B,F,andJ)bymeans
ofestimatingthewholesetofmultipolemomentsoronlythestrengthofqˆ30.Inbothcasesqˆ30
discriminatesγ=1andγ=2.

Insummary,ourmultipoleexpansion(ME)methodcanquantifystimuluscharacteristicsby
meansofreconstructingthemultipolemomentsqlm.Becausetheflowfieldneednotbegenerated
byasubmergedmovingobjectalone,themethodcanalsobeappliedtocharacterizecomposite
depictedsituationsinsucFig,has4.8schoandolingFig.orv4.9ortexwehavestructuresansweredfoundtheinthecompwakeellingofafish.question,Inpresenwhethertingatheresdetectingults
animal(DA)orartificialsystemcanlocalizeandrecognizeaSMOgivenonlythemeasuredwater
velocitiesattheDAlateral-lineorgansand,andifso,howwell.Thatis,theMEisusefulonlyif
theSMO,ordistanceshorbteetr.weentheOtherwiseobjectthe(SMEMO)reducesandtodetectoritsdip(DoleA)isapprosimplification.ximatelyofthesamesizeasthe

45

erceptionPineLateral-l4.

Figure4.9:Thetwographsdepicttherelativereconstructionerrorδq30(dashedline)oftheoctupole
momentq30asdefinedinthemaintextfordifferentnumbersofneuromasts.Theyshowthat
fordistancesdlargerthanamovingobject’ssize(5cm)multipolereconstructionisnotpossible.
Thesolidlinedepictsthelocalizationerror||rˆ0−r0||,whichstartstogrowfastatadistance
comparabletothelengthofthedetectinganimal(10cm).Thisagreeswellwithexperimental
findingswherelocalizationperformancestartsdecreasingatdistancesbeyondthefishbodysize
[54].Fordiscriminationtasks,fishhavetobeveryclosetotheobjectunderinvestigation[39].The
leftgraphshowstheerrorsemploying250(left)and500(right)neuromasts.Asexpected,inboth
plotserrorsincreasewithdbutthedifferenceduetothenumberofneuromasts(withinbiological
marginal.isrange)

okoutloandDiscussion4.4Inthischapter,wehavefocusedontwomajoraspectsoflateral-lineperformance.First,nomatter
whethertheanimal’scapabilitytolocalizeprey,predators,conspecifics,orobstaclesisbasedonthe
processingofcharacteristicpointsofthevelocityprofileorthewholevelocityfield,theSMOshape
playsanessentialrole[253].Assumingthatpotentialflowisanadequatetoolforapproximating
thecorrespondinghydrodynamics,wehavequantifiedanalyticallytheeffectofshapetogainfurther
insightsintomechanosensoryprocessing.Inadditionandforthefirsttime,wehavepointedoutthat
differentlyshapedobjectsresultindifferentboundaryconditionsforthehydrodynamicflowfield
andthuscausedistinguishablemultipolemomentsqlm.Wehavedevelopedamultipoleexpansion
method(ME)thatpredictsthepossibleobjectrecognitionperformanceofthemechanosensory
sensorysystem–thelateralline.
Second,inthepresentliteraturethereisanaturalcharacteristiclength-scaleuptowhichthe
animalcanlocalizeobjects,viz.,itsbodylength,ormoreprecisely,thelengthofitstrunklateral
line[95,75].Here,wehavegivendifferentlinesofevidencethatthereisasecondlengthscale
withinwhichshapeinformationistransmitted,namely,thebodylengthoftheSMOtobedetected.
IfthedistancebetweentheDAandtheSMOisbelowthatlengthscaleeventhesimplestobjects
transmitmeasurableshapeinformation.Thisinformationcanbeusednotonlytolocalizebuteven
torecognizehydrodynamicobjects[251].

46

4.Lateral-linePerception

Promisingstepsforfurtherenlighteningthisinterestingsensorysystemaremanifold.Inconcreto,
wesuggesttoexperimentallytesttheabilityofthelaterallinetorecognizeshapeindifferent
settingstoevaluatethepredictedrecognitionperformance.Onecouldclarify,forinstance,the
numberofmultipolesthatisnecessarytorecognizeshapeinnaturalconditions.Anotherinteresting
aspectwouldbetoanalyzewhetherthesensorysystemis“blind”whenitcomestodistinguish
SMOswithspecialsymmetries.Moreover,thepossibleexperimentalsetupsmaynotbelimitedto
thelateralline,viz.,hydrodynamics.Air-dependentmechanosensorysystemssuchasthesensory
systemofcricketscanalsobeconsidered.
Onthetheoreticalside,itwouldbeveryinterestingtoextendourmodeltothetimedomain.
Thatis,ontheonehandmodelsthatenhancethepresentMEmodelshoulddealwithatemporal
varyingmultipolepatternq(t)tofaithfullyidentifyhydrodynamicobjects.Namely,themoving
bodyorthevorticesinthewakeofswimmingfishproducespecifichydrodynamicstructuresthat
aretime-dependent,whichistrueforthecorrespondingmultipolesaswell.Presumably,the
time-dependentmultipolesareanimalspecific.Ontheotherhand,itispossibletoextendthe
presentMEmodeltonotonlyreconstructthemostprobableSMOpositionatonespecificpointin
timebuteventoestimatethemostprobableSMOtrajectory.Giventhetrajectoryandassumptions
onthemovingobjectsuchasitsinertmassitiseasytopredictitsfuturepositions.Thedipole
momentincludestheactualmovingdirectionandspeedoftheSMOandwouldthereforealready
with.starttosufficeInsummary,ourobject-recognitionmodelagreeswellwithbiologicalfindingsandprovidesa
theoreticalunderstandingofhydrodynamicobjectperceptionthroughthelateralline.Furthermore,
wenowunderstandthefundamentalrestrictionsforany(sensory)evaluationoflateral-linedata.
Finally,thesefindingscanalsobeappliedtobiomimetics[289];e.g.,toimprovepassivenaval
ms.systevigationna

Figure4.10:Sometimes,itmakesadifferencetoknowwhatiswhere.Frommyposterpresentedat
the2009Sloan-SwartzSummerSchoolatHarvardUniversity.

47

ihrDieeGeWahrheitgnersterbtrienumphiertnuraus.nie,

5.InfraredSystemofSnakes

kPlancMax

Somesnakespossessaninfrareddetectionsystemthatisusedtocreateaheat-imageoftheir
chaenvironment.racteristicsInofthissensorychapter,en-andwedecoexploding,retheandinvolvedaddressthephysicsquesoftionsignalofhowmapping,theinfratheredneuronalsystem
cancreateafunctionalrepresentationofthethermalpropertiesoftheenvironment.

ductionIntro5.1Twogroupsofsnakes,pitvipersandboids,possessasensorysystemtodetectinfrared(IR)
radiation[198]whichallowsthemtoperceiveatwo-dimensionalimageoftheheatdistributionin
theirsurroundings.Thedetectionsystemconsistsofasetofcavitiescalledpitorgans.Inpitvipers,
apitorganislocatedoneachsideofthesnake’sheadneartheeyes;cf.Fig.5.1A.Suspendedin
eachcavityisaheatdetectingmembrane,whichissensitivetomKtemperaturedifferences[33,63].
TheopticalprincipleunderlyingthedetectionofIRradiationisthatofapinholecamerawithlarge
aperture;cf.Fig.5.1B.
Radiationenteringthroughthepitholehitsthemembraneatacertainspotdependingonthe
sourcedirection;seeFig.5.2A.Theresultingtemperaturechangeatthisspotisdetectedbyheat

Figurecourtesy5.1:of(A)GuidoHeadWofaesthoff;pitviperadaptedwithfromnostril,[252].(large)(B)pitApihole,tvipander’seye,leftinfrared-sensitoright.tivepitPhotographorgan
wmemorksbrane,likeasusppinholeendedcamera.freelysoasRadiationtomiennimizeteringheatthroughlosstothetheopeningsurrounding(left)hitsatissue.Toheat-sensitivensuree
alargeenoughenergyinflux,theaperturehastobequitelarge(∼1mm),comparabletothe
distanceaperture-membrane.Theimageofapoint-likesourcethusformsadisc-shapedimageon
brane.memthe

49

InfraredSystemofSnakes

sensitivecells,freenerveendings(FNEs)thatareorganizedwithindensearchitecturescalledthe
terminalnervemasses(TNMs)[6,121].TheTNMsrepresenttheheatsensitivesensoryareasthat
aredistributedthroughoutthemembrane;cf.Fig.5.2B.Thereareabout40×40sensoryareason
themembrane[198],andthefieldofviewisabout100◦wide,whichimplies◦thatinputtotheorgan
couldberepresentedinthebrainwitharesolutionofapproximately2.5.Sincetheradiationflux
enteringtheorganmustbelargeenoughtoquicklydetectmovingprey,theapertureoftheorgan
iswide,approximately1mm,andcomparabletotheorgandepth.Thusincomingradiationfroma
pointsourcedoesnotstrikeapoint-likeregiononthemembrane,asinanidealpinholecamera,
butrathermuchlargerdisc-shapedregion;cf.Fig.5.1.Itisnonethelesspossibletodetermine
thedirectionoftheincomingradiationiftheboundaryofthedisc-shapedregionremainsnarrow
enough.Thisapproachbreaksdownifmultipleornon-pointlikeheatsourcesarepresent.Thenthe
resultingheatdistributiononthemembranewillbeheavilyblurred(seeFig.5.10),precludinga
directevaluationoftheinput.
TheIR-detectionsystemofsnakespresentsaparadox.Theopticsofthepitorganaspinhole
cameraensuresthatimagesonthemembranewillbedistendedandblurry.Yettheinformationfrom
theIRsystem,combinedwithinputfromthe“normal”visualsystem,allowsformationofarather
preciseneuronalmap[116]inthebrain’soptictectum[109,210,211];cf.seeFig.5.2C.Thismap
issharpenoughtoserveasatopologicalrepresentationoftheoutsideworldinwhichneighboring
neuronsrepresentneighboringregionsoftheoutsideworld.Evenamultimodalinteractionbetween
theinfraredmapandthevisualmaphasbeenfound[109,210,211].Experimentalstudieshave
shownthatasnake’sorientationtoapointsourceofheatvariesbutthatanaccuracyof5◦is
neverthelessattainable[211],whichapproximatesourroughestimateabove.Wearethusfaced
withaparadox,inthattheopticalqualityofapitorganislowbuttheneuronalperformanceis
high.Howdoesthathappen?
TheparadoxinvolvestheopticalqualityofIR“vision”.Wethereforeexploretheoptical
qualityofthepitorganandaddressthethermodynamicphenomenainvolved.Thisenablesus
tocalculatetheheatdistributiononthemembraneforagivenheatdistributioninspace.Ina
secondstep,weaddressthequestionofhowthemappedheatdistributionisencodedneuronally.
Basedonthemembraneimageandtheknowledgeofsensoryencoding,wecontinuewiththe
thirdstepofsensoryprocessing,thatistoneuronallydecodetheoriginalspatialheatdistribution
fromthemeasurementsonthemembrane.Wedeveloptworeconstructionmodelswhichareable
toexplainhowtheneuronalsystemcancompensatetheoversizedpinhole.Sinceasnakehas
limitedcomputationalresources(all“calculations”mustberealizableinneuronal“hardware”)any
reconstructionmodelmustbesimple.Accordingly,ourmodelsuseonlyonecomputationalstep
(theyarenon-iterative)toestimatetheinputimageonlyfromthemeasuredresponseonthepit
membrane.BothresembleaWienerfilterandareakintobutdifferentfromsomeofthealgorithms
usedinimagereconstruction[229].Ithastobeconstantlyborninmindthattheopticaldevice
discussedherehasverylimitedimagingcapabilitiesincomparisontohuman-deviseddetection
systems.Nonetheless,aswewillshow,itisstillpossibletoreconstructthespatialheatdistribution
inabiologicallyplausibleway.Inaddition,wequantifytheperformanceofourmodelsandgive
reasonablelimitationsforlocalizingwarmobjectsbymeansofthissensorysystem.Beyondvarious
specificissuesofthepitsthatareconsidered,weextendthecurrentunderstandingofthissenseby
meansofenlighteningthequestionofwhattheappropriatestimulustotheIR-detetctionsystemof
snakesis.Finally,wepresentanoutlookanddiscussstillopenquestionsthatshouldbeanswered.

50

InfraredSystemofSnakes

Figure5.2:Schematicdrawingoftheessentialpartsofthesnake’sIR-detectionsystem;adaptedfrom
NewmanandHartline[211].(A)depictsapitorganthatisilluminatedbytwoinfraredsources.The
illuminatedareas1and2onthepitmembraneheatup.(B)Sensoryresponsesaregeneratedwithin
thefreenerveendingsofthebranchingtrigeminalnerveaxonsthatformtheterminalnervemass
(TNM).Thespikesaretransmittedalongthetrigeminalnervetothenucleusofthelateraldescending
trigeminaltract(LTTD).(C)Withinthebrain,theinfraredinformationis,i.a.,representedinthe
optictectumbymeansofaneuronalmap.(D)Withintheoptictectumseveralcomputationalsteps
betweenthevisualmapandtheinfraredmapcanbefound.Theinformationofthesetwosensory
systemsfusesintoacomprehensiveperceptoftheenvironment.Foradetailedreviewofthesensory
pathwaywerefertoelsewhere[162,163,182,191,197,198,199,209,211,210,246,255,256,266].

5.2Warmingup:Signalmapping
Inthissection,wederiveequationsofhowtomapthestimulus,thespatialheatdistributionofthe
externalInitially,wweorldclarifyontowhicthehthesensorycellsappropriatewithinstimtheuluspitformemthebraneinfraredofthedetectionsnakeassystemdepictedisandinFig.whatits5.2.
characteristicsare.Insodoing,wederiveanappropriatemathematicaldescriptionofthesignal
cess.promapping

vistadynamicthermoA5.2.1Tostartwiththelatterissue,wefollowthewidelyacceptedopinionthatthepitmembranereceives
thermalradiationfromtheexternalworldandhenceheatsuporcoolsdown,viz.,itisawarm
receptororgan[48,49,168].Ontheotherhand,themembranedoesnotrespondtospecificwave
lengthsliketheretina[93,196].Wehencehavetoanalyzethemechanismsofhowthemembrane
heatsupandcoolsdown.Wederiveasimplifiedmodelofthecomplexheatingandwarmingprocess
toestimatetheimportanceofthedifferentmechanismsinvolved.
Attheoutset,weconceptuallydividetheheattransportphenomenaintotwocategories,the
heattransportwithinthepitmembraneandtheheattransferthroughthemembrane-airsurfaces.
Thatis,weconsiderheatconductionthroughtheair,energygainsandlossesthroughradiation,

51

InfraredSystemofSnakes

thebloodtempperfusion,eratureofheatasmallconductionpitmemwithinbranetheslicetissue,arescandhematicconvallyection.depictedAllintheseFig.pro5.3.cessesaffecting

Figure5.3:Sliceofthecenterofthepitmembrane.Wecanseethedifferentheatflowsthat
gomemthroughbrane-airthesurfacesdifferenStma=surfaces,lp2.Thei.e,edgethemelengthmlpofbrane-memthebranequadraticsurfacessliceisS25mm×=10l−p6dmpandandthethe
thicknessofthemembranedpisassumedto15×10−6m.

Thestartingpointofourstudyisthebasicheatequationthatdescribestheheatdevelopment
withinthetissueofthepitmembrane.Itreads
ρpCp∂T(x,t)=kp2T(x,t)+Qi.(5.1)
t∂iInthissection,wedenotequantitiesassociatedwiththepitswithasubscriptp,forinstance,thepit
membranedensityρp(≈1080kgm−3).TheotherquantitiesareCp(≈3850Jkg−1K−1),theheat
capacitanceandkp(≈0.49Wm−1K−1),thethermalconductivity.Bothrefertothecharacteristics
ofthetissue.Asreasonablevaluesfortheparametersofthepitsarenotavailable,wehavetaken,for
thesakeofconsistency,valuesforneuronaltissue,i.e.,thebrain,fromFialaetal[72].Thefirstterm
intheright-handsideofequation(5.1),kp2Tp(x,t)describesheatconductionthroughthetissue.
Thefundamentalsolution,viz.,incaseQi=0isanexponentialdiffusionT(x,t)∼exp[−x2/ct]of
theheatprofilewithc=const..Theeffectofthistermisthatdifferencesintheheatdistribution
getsmearedout[70].Wewillcomebacktothistermlateronand,insodoing,wespecifyand
discussthedifferentsourcesandsinksofenergyQi.
Thesecondtransportphenomenonwithinthetissueistheheattransportthroughblood.Many
experimentalfindings[6,92,121,205]havereportedadensecapillarysystemwithinthepit
membrane.Amemiyaetal.[5]evensuggestedthecapillariesfunctionasa“heatexchanger”;see
Fig.5.7.Pennes[222]firstformulatedandsubsequentinvestigatorsvalidated[50,72,207,286]the
so-calledbio-heatequationdescribingtheheattransportthroughthebloodQbforpartsofthe

52

InfraredSystemofSnakes

bodysuchasanarm,
Qb=ρbVbCb(1−κ)(Ta−T)=ωCb(1−κ)(Ta−T).(5.2)
HereQbistherateofheattransferperunitvolumeofthetissue.Theperfusionrateperunit
volumeofthetissueisdenotedasVb,ρbisthedensityofblood,Cb(≈4000Jkg−1K−1)isthe
specificheatoftheblood,κisafactorthataccountsforincompletethermalequilibriumbetween
bloodandtissue.Typically,itissetκ=0.ThetemperatureofthearterialbloodisdenotedasTa,
andTdescribesthelocaltissuetemperature.Despitethelargenumberofmitochondriaaround
theTNMs,wedonotassumeintheaboveequation(5.2)thatthepitmembraneitselfproduces
essentialthermalenergyaccordingtobiochemicalprocesses.
Next,weturntothephenomenathatappearatthemembrane–airinterfaces;cf.SmainFig.5.3.
Suspendedwithinthecavityofthepitorganthepitmembranedividesthepitorganintoaninner
andanouterchamber[9,198,211];seeFig.5.2.Inotherwords,themembraneissurrounded
byair.Sincewedealwithverysmalltemperaturesandtemperaturedifferences,aircannotbe
treatedasanidealinsulator.Theheattransportthroughtheaircanbedividedintoconvection
andconduction.Thepitmembraneisalmostperfectlyprotectedfromforcedconvectionbecauseof
itslocationwithinacavity.FreeconvectionwithinanenclosureonlyappearsiftheGrashofnumber
Gr:=gβL3(T−T∞)/ν21000[64].Hereg=9.81ms−2,βisthecoefficientofthermalexpansion,
Lisacharacteristiclength,Tdescribesthetemperature,T∞thesteady-statetemperature,andν
denotesthekinematicviscosity.TheGrashofnumberdescribestherelationbetweentemperature
inducedliftingforceandviscosity.BecauseofthesmalldimensionsofthepitorganGr1000we
canneglectfreeconvectionasBakkenandKrochmal[9]havealreadypointedout.Withthehelp
ofFourier’slawweapproximatetheamountofheatenergylostbyconductionthroughairQaby
Qa=kaT−Tp+kaT−Tp=kadi+do(T−Tp).(5.3)
didodido
Heredianddodenotetheeffectivedistancesfromthepitmembranetothewalloftheinner(di)and
outercavity(do),respectively.Thethermalconductivityofairisgivenbyka=0.026Wm−1K−1,
andTpdenotesthetemperatureofthepitcavitywalls.
Theremainingtransportprocess,namelytheradiationprocess,isthemostimportantonesince
itupmapstheallmembrane,informationandofthistheistheexternalfocuswoforldthisontothesection.memThebrane.secondIntheproblemfirst–place,howradiationinformationheats
istransmittedandextractedbythepitorgans–willbediscussedlater.Wedescribetheenergy
transportduetoradiationprocessesQrby
Qr=−2σT4+µpσTe4+µoσTo4+σTp4(5.4)
bothsidesfrontsidebackside
energylossesexternalworldwarmobjectcavitywall
whereTdescribesthelocaltemperatureofthepitmembrane,asalwaysinthissection.The
meantemperatureofthesnake,viz.itscavitywalls,isdescribedbyTp,andTereferstothe
meantemperatureoftheexternalworldprojectedontothefrontsideofthepitmembrane.In
thefollowing,weassumeTe=Tpbecausethesnakeisacold-bloodedanimal.Thedimensionless
theparametersareaswithµpandmeanµotempdenoteeraturetheTpratioandtheunderwarmwhichatargetsmallobjectsurfacewithoftempthepiteraturememTo;braneobviously“sees”
µp+µo=1.Wecalculatedµo:=A/(2πd2)withA,thevisiblesurfaceofthewarmobject,and
thedistancedbetweenmembraneandobject.Wenotethattheinformationoftheexternalworld
iscontainedintheexactvaluesofµpandµoforeachsensoryareaonthemembranebutforthe

53

InfraredSystemofSnakes

heatingprocess;itsufficestoassume,forinstance,µo≈0.025(d=0.5m)or0.1(d=0.25m).
Furthermore,weassumeallsurfacestobeblackbodyradiatorwiththeconstantσdenotingthe
Stefan-Boltzmannfactor(σ=5.67×10−8Wm−2K−4)emittingtheirenergyintoahalfspace.We
cansummarizethetermsandrewrite(5.4),
Qr=−2σ(T4−Tp4)+σµo(To4−Tp4)
Qr≈−8σTp3(T−Tp)+4σµoTp3(To−Tp).(5.5)
Itcanclearlybeseenin(5.5)thattheenergyinputandtheenergyoutputdependonheatcontrasts,
namely,theexternalandtheinternalheatcontrast.
Thepit-heatequationanditssimplification.Wecannowcombinethedifferentsourcesand
sinksofheatenergywehavederivedaboveintoapitheatequationthatreads
T∂ρpCp∂t=kp2T+Qb+Qa+Qr.(5.6)
Givenallparameters,theaboveequationcanbesolvednumerically.Next,however,wefollowan
analyticapproachtoderivethecharacteristicsofanddependenciesupontheheatstimulus.
Aswehaveseenpreviously,alloftheQitermsdependontemperaturedifferencesinvolvingT.
Wecanreasonablysimplifythesituationbyassumingallpartsofthesnakeandthebackground
oftheexternalworld,exceptthepitmembraneandthetargetobject,tohavethesamemean
temperatureTp.Thisreduces(5.6)toanequationwithonlytwovariables,namely,theheat
contrastintheexternalworldξext:=To−Tpandtheinternalheatcontrastξint:=T−Tp
onthepitmembrane.Furthermore,wecanidentify∂ξint/∂t=∂T/∂t.Inaddition,weleave
thedifferentialdescription(5.6)andconsiderasmallpieceofthepitmembranelocatedinthe
centerofthemembranewithasquaredbaseoflp×lpandathicknessdp;cf.Fig.5.3.Wetake
lp=25µmanddp=15µm.Thelengthlpcorrespondsapproximatelytothesizeofasingle
independentreceptorarea(terminalnervemass).Figure5.3depictstheheatflowsthroughdifferent
surfacesofthemembranecuboid.Themembrane–airsurfacesareSma=lp2=625×10−12m2,the
membrane–membranesurfacesareSmm=lpdp=375×10−12m2,andthevolumeofthecuboidis
Vp=lp2dp=0.01×10−12m3.Thesimplifiedpit-heatequation(5.6)foroursmallcuboidnowreads
τp∂∂ξtint=−Qoutξint+Qinξext,(5.7)
whereτp=ρpCpVp(≈40×10−9JK−1).WecanestimatethevaluesofthetermsQoutandQinby
consideringthecorrespondingmembranecuboid;cf.Fig.5.3.
3di+do4k
Qout=Smaσ8Tp+Smakadido+Smmrˆp
≈8×10−9WK−1≈15×10−9WK−10−10×10−6WK−1
+ωCbVp25×10−9WK−1
≈0.4×10−9WK−1
1Qin=Smaroσ4Tp3100Qout(5.8)
≈0.1−0.4×10−9WK−1

54

(5.8)

InfraredSystemofSnakes

Thestaticpit-heatequationand“optical”contrast.Wenowset∂ξint/∂t=0in(5.7)and
computesimilarlyto[9]themaximalvalueoftheinternalheatcontrastξmaxandthesteady-state
relationbetweeninternalandexternalheatcontrast
ξmax=Qinξout1ξextandξmax1.(5.9)
Qout100ξext100
Wecanseeclearlyfrom(5.9)thatthequalityofthe“opticalsystem”pitorganispoor,i.e.,it
representsexternalcontrastsroughly100timeslessstronglythantheyareintheexternalworld.
Inotherwords,todetectobjectswithabout1Ktemperaturecontrast,snakeshavetomeasure
temperaturecontrastonthepitmembraneintheorderof10mK.Thispurelytheoreticallyderived
resultcorrespondsto,andexplains,manyexperimentalandbehavioralfindings[9,33,63,211].

Figure5.4:Thesteady-stateinternalheatcontrastξmax.Abovewehavestudiedthesteady-stateof
theinternalheatcontrastξmaxofasmallsliceinthecenterofthepitmembrane(seeFig.5.3)by
meansofvarying(A)theexternalheatcontrastξext,(B)thedistancebetweenmembraneanda
warmobjectdMO,(C)thebloodperfusionrateofthemembraneωb,and(D)thedistancebetween
theinnercavitywallandthebacksideofthepitmembranedICM.Exceptifstatedotherwise,
parametersareQin=0.4×10−9WK−1,Qout=50×10−9WK−1,andξext=5K.

discussTogettheatermsdeeperin(5.8).understandingInitiallyof,thetheimpheatortanceconductionofthetermdifferen∼tgradeffects,Tiswsetrnextonglyanalyzesimplifiedand
to∼conduction1/rˆp.Ifwithinweusethethistissuewouldsimplificationthenandclearlytakerˆdominateptoballetheotherradiusterms.oftheThispitapproacmemh,branehowevheater,
istoosimple.Foranumericalcalculationrˆpreferstothedistancelpbetweenthecenterofthe
cuboidandtheneighboringmembranecuboids.Inrealsituations,theneighboringcuboidswill
butshowalmaostsimilarzero.tempNeveratuertheless,reasinthetheobservcaseedthereone.isanHenceexternalthetempheateraturesource,thedifferencesradiatedwillheatnotbeenergyξint
willgradienheattsupatthethememedgesofbranethenotilluminatedhomogeneously;memcf.braneFig.area5.10.thatThewillwbearmingerasedproquiccessklygbyeneratestheheatheat
conductionterm.Inabsenceofsteeptemperaturegradientsonthepitmembrane,i.e.,between
tworeceptorareasinthecenteroftheilluminatedmembranearea,wecanneglectthemembrane
term.conductionheatdepTheendsseoncondthemaanatomjorytermofoftheQpitoutinorgan.(5.8)Thedescribgreateresheatthedistanconductioncebetweenthroughthetheinnerair.caThivitsywtermall

55

InfraredSystemofSnakes

andthebacksideofthepitmembranethesmallerthistermandthebettertheheatcontraston
themembrane[9];seeFig.5.4D.
ThenextimportanttermofQoutin(5.8)isduetoradiationlosses.Amemiyaandothers
[4,38],forinstance,havereportedaspecialsurfacestructureofthepits.Itseemstobeplausible
thattransmittanceandreflectioncouldbeminimizedthroughsuchasurfacestructure.However,
conservabsorbance.ationSiofnceenergythey,aredescribassumededbtoybKirecnearlyhhoff1’s[9la]w,wesetequalsboththethermalthermalradiationradiationemittanceemittanceand
andtheabsorbanceto1.Accordingly,thepossiblevariationofthesetwoparametersisverysmall.
Moreover,experimentalquantificationismissing.
brainThetissuelast[72],termthisofQtermoutisdescribquiteessmalltheheatandlossesunimportanthroughtasbloweodhavfloew.IfdepictedwetakineFig.the5.4valuesC.Bufort
asreportedinmanyexperiments,themembraneshowsagreatamountofcapillariesanditisof
verysmallscalecomparedtothetissueusedforderivingthebio-heatequation[222].Moreover
andunfortunatelywithoutgivingnumbersontheinterestingtimescale,Gorisandcolleagues
[92,121,205]havereportedthatthebloodflowwithinthecapillariesfollowsthetemperature
development.Theyalsospeculatedaboutare-coolingmechanismoftheTNMbymeansofthe
bloodflow,whichevenseemstobecontrolledlocallybytheTNM.Accordingly,weeitherhaveto
modifytheequationbymeansofconsideringω=ω(T)(seeFig.5.4C)orwehavetomodelthe
effectofbloodflowotherthanPenne’sbio-heatequationbecausethedimensionsofourproblem
aremoretwo-dimensionalinsteadofthree-dimensional.
Ontheotherside,Qinandthusξmaxisinfluencedbytheexternaltemperaturecontrastξextas
wehavedepictedinFig.5.4A.Thegeometryofthesetting,inmathematicaltermsµo,isinfluenced
bythesizeofthevisiblesurfaceAandthedistancedMObetweenthemembraneandthewarm
objectofinterest.ThedistancedMOlimitsthemaximalheatcontrastξmaxandthusthedetection
rangeaswehaveillustratedinFig.5.4B.
Thetime-developmentofthepit-heatequation.Anotherimportantaspectof(5.7)isthe
timedevelopmentoftheheatcontrastandthusatheoreticalapproachtotheresponsetimeofthe
infraredsystem.Wecanrewrite(5.7)tothedifferentialequation
∂∂ξtint=−τ1ξint+Qˆin,(5.10)
wherewehavesubstitutedτ:=τp/QoutandQˆin:=Qinξext/τp.Thegeneralsolutionoftheabove
equationisgivenbyasuperpositionofthehomogeneoussolution(Qˆin=0)andaparticular
solution.Combined,thesolutionreads
ξint(t)=ξmax1−exp−t,(5.11)
τwithξmax:=τQˆin=(Qin/Qout)ξext–theresultforthesteady-statecondition(5.9).Underthe
assumptionsandsimplificationswehavemadeabove,wereceiveatime-dependendequationthat
allowsustostudythequestionofwhatinfluencesthewarmingprocessofthepitmembrane(5.11).
InFig.5.5wehaveillustratedthedependenciesofthewarmingprocessupontheinfluencing
variables.Asaspin-offwefindphysicalevidenceforexperimentalresults[33]thatsupposea
tempneuronaleraturerespcononsetrastbeloonwthe100msmemusingbrane,mKisnottempconstaneraturetduringresolution.thatpTheeriodsignal,oftimehowbutever,i.e.,increasing.the
Inthissection,wehaveforthefirsttimederivedanequationincludingallaspectsofenergyin-
andoutputofthepitmembranewarmingprocess(5.6).Bysimplifyingthisequationto(5.7)we
havequantifiedboththedependenciesoftheheatcontrast(5.9)andthetime-dependentwarming
process(5.11).Thusweareabletoidentifytheimportanteffectsandshowwhichdataarecurrently
missing.Fromtheexperimentalpointofview,wehavearrivednotonlyatatheoreticalvalidation

56

InfraredSystemofSnakes

Figure5.5:Transientwarmingprocessofthepitmembrane.Abovewehavestudiedthewarming
ofasmallsliceinthecenterofthepitmembrane(seeFig.5.3)bymeansofvarying(A)the
externalheatcontrastξext,(B)theamountofenergylostbythemembraneQout,(C)theamount
ofenergyreceivedbythemembraneQin,and(D)parametersofthepitmembraneτpincludingfor
instancethemembranethickness.Thebasicgraph(red)correspondsinallpanelstotheparameters
Qin=0.4×10−9WK−1,Qout=50×10−9WK−1,τp=40×10−9JK−1,andξext=5K.

and,moreimportantly,ataclarificationofthepossiblesensoryperformancesbuthavealsoprovided
mathematicaltoolsforfurtherinvestigations.
Inthenextsection,weturntotheinformationcontentthatistransmittedtothepitorgan.
Thatis,wewillclarifytheradiationprocessinmoredetailbymeansofdisentanglingµoforevery
sensoryareaonthewholemembrane.

heattheMapping5.2.2Haandvingthattheclarifiedaccuracythatonlyofthetheheatradiationconoftrasttheplaysexternalacritiwcalarmrole,objewctsehacanvetocauseaanalyzereasonablecarefullystimwhiculush
inphysicaltegratedspdescriptionectralfitsradiancebestu(toλ,Tm)ap=πtheI(λ,T)externalderivheatedbyconMaxtrastPlantocktheinthe1901pit[226]membrane.The
u(λ,T)=2hc52πhc1Js−1m−3,(5.12)
λexpλkT−1
describestheamountofenergyablackbodywithtemperatureTemitsintothehalfspaceper
time,perwavelengthλ,andperunitsurface.AswecanseeinFig.5.6,almostall(99,5%)ofthe
tenthradiatedoftheenergytypicalfrompitablacorganskbapodyerturewith[47T,=137303].KWeisecanmittedthereforebelowignore100µmeffectswhichthatisareaboutdueoneto
thethatwais,vethecharactermappingofoflighthetandspatialinsteadheatusedistribgeometricution.Asopticsatoside-remark,describethephatmosphericysicsoftranoursmittanceproblem,
canbeignoredoverdistancesofmeters[8];sowecanneglectthiseffect,too.

Next,wecondensethephysicsofgeometricalopticsintoaformalismthatmapstheemitted
signalsfromtheexternalenvironment,forinstance,apreyontothepitmembraneofasnake.In
theprevioussubsection,wesawin(5.7)thattheexternalheatcontrastξextisanappropriate

57

InfraredSystemofSnakes

Figure5.6:Thespectralradianceu(λ,T)(left)andthenormalizedintegralofu(λ,T)(right)asa
functionofthewavelengthλattemperaturesofT=303K.Weseethatthewavecharacteroflight
canbeignoredandwecanusegeometricoptics.

descriptionforthestimulusbecauseQin–aconstant,mainlygeometricfactor–timesξextisthe
receivedenergyonthemembrane.WenowleavethesimplifiedQin,whichisadescriptionforone
areaofthepitmembranebymeansofµoandµpin(5.5),andderiveadescriptionforallsensory
areasionthepitmembraneaccountingforanarbitraryspatiallydistributedexternalheatcontrast.
WethereforediscretizetheexternalspaceinfrontofthepitintoplainsurfacesjofequalspaceSext
withtheheatcontrastξext;jfromnowondenotedbyξj.Usingthegeneralformalismderivedin
Chap.2,thereceivedenergyriduetoradiationofasensoryareaionthepitmembranenowreads
ri=hijξj.(5.13)
Themappingismathematicallymodeledbymeansofthetransferfunctionhijcomponent-by-
componentwherethecomponentsofthetransferfunctionare
jSmaSextσTp31/d2ijcos(ϕ),ifiisvisiblefromj
hi=0,otherwise.(5.14)
Toaccountfortheconservationofenergy,theemittedenergyscalesas1/d2ijcos(ϕ)wheredij
denotesthedistancebetweenthesurfacesjintheexternalworldandionthemembrane.Werefer
toϕastheanglebetweenthenormalvectorsofthesurfacesiandj,andthereforejthecosineterm
compensatesfordifferentorientationsofthetwosurfacesiandj.Thetensorhiisonlydetermined
bythegeometryofthepitorgansandthemeantemperature.Wecanassumebothtobeconstant.
So,thetensorprojectseveryheatdistributionoftheexternalworldinthesamewayontothepit
brane.memToexplicitlycalculatetheresultinginternalheatcontrastonthemembrane,onehastosolve
(5.7)foreachsensoryareaigivenri,oronecouldlinearize(5.7)andjustmultiplyribythe
appropriatefactorderivedfrom(5.7).Aswewanttostudyinformationcontents,wewillusethe
pureheatintensitydistributionriassensoryinputinthefollowing;cf.Chap.1.
Forrealisticallymodelingthemappingprocessoftheheatintensitydistributionontothesurface
ofthesensorycellsiofthemembrane,wealsohavetoconsideruncertaintiesandnoise.Two
sourcesofnoisecanbeidentified.Noiseofthesignalandmeasurementerrorsofthedetector,
respectively.Wemodelbothbymeansofaddingstochasticvariablesαjandχitotheinputand
thedetectorresponse
ri=hij(ξj+αj)+χi.(5.15)
Thenoiseofthesignalcanbecausedbymeansofdisturbancesofotherobjectssuchasgrassand
fastmovements.Sinceeverydetectionprocessisblurredbynoise,itisreasonabletoassumethe
samehere,too;cf.Chap.2andChap.3.

58

InfraredSystemofSnakes

Insummary,wehaveidentifiedtherelevantphysics,i.e.,geometricaloptics,andderiveda
modelthatdescribestheheatintensitydistributionacrossthepitmembrane.Inthenextsection
weturntothedetectionprocessitself.

Figure5.7:Capillarynetworkandterminalnervemass(TNM)ofthepitmembrane;adapted
from[5,6].Theupperpanels(A−B)depictscanningelectronmicrographsshowingthecapillary
vlosingasculaturetheirofsheaththepit(smallmemarrobrane.w)andThe(Dlo)wtheerfreepanelsnervilleustrateendings(C)(nf)theofmayeterminallinatednervnerveefibmassersarra(mfy).
Acollectionoffreenerveendingsformsareceptorunitandrespondstotemperaturestimulias
modeledinSec.5.3.Experimentaldataonthenervous-controlledcapillarynetworkanditseffect
onthewarmingprocessaremissing.

dingencorySenso5.3Inthissectionwewanttoslightlytouchuponthequestionastohowthestimulusisencoded
neuronallytoextractimportantideasfortheoverallfunctioningandperformanceoftheinfrared
es.snakofsystemdetectingFreenerveendings–amulti-purpusedetector.Weknowthatthelocationwherethe
stimwithinulustheisfreedetectednerveistheendingspit(FNE)memofbrane.theMoreterminalinnervparticularemass,the(TNM)sensoryofpitrespmemonsesbarerane.Ingeneratedother
words,thefreenerveendingsencodethetemperatureoftheirsurrounding(staticresponse)and
theyfoundinrespondothertoanimalstempersucaturehascmice,hangesvampire(dynamicbats,resporonscatse)[[3324,,117118].,F243ree,244nerv].eInendingsthesearesettings,also
vFNEsalidit[y.41]First,areknothewnstaticasresp(mammalian)onsecurvcoldesorw(neuronalarmreceptors.activityvs.Twotempfindingserature;seemcf.tobeinlaofyunivFig.ersal5.8)
ofandthe[24,235thermo].Second,receptorsinttohedifferenanimalstlevaboelsveof–constanalthoughtnottempaseraturepronouncedareassimilar;inthecomparesnake’s[117pit]

59

InfraredSystemofSnakes

membrane–thereisalsoadynamicresponsetotime-dependentthermalstimuli;cf.Fig.5.8and
e.g.[243,244].InpassingwenotethatFNEsseemtobeusedalsoinspecificwatersnakesfor
detectingwatermovements[43].

Figure5.8:Averageimpulsefrequencyofapopulation(N=9)ofwarmfibersfromthetrigeminal
branchwhenapplyingthermalstimulitotheskinofaBoaConstrictor;adaptedfrom[117]and
[25].Duringthisexperiment,Henseletal.haveincreasedthetemperature(redcoloredareaand
righty-axis)step-wise(ΔT=5◦C),startingatT=20◦CandendingatT=40◦C.Aswecan
see,themaximaofthedynamicresponseaswellasthesteady-statevaluesofthestaticresponse
(seeinlaygraphC)followabell-shapedcurve.Theinlaygraphshowsthesteady-stateresponse
curvestostatictemperaturestimulusof(A)acat(N.lingualis,bursting),(B)adogfish,(C)aBoa
Constrictor,and(D)acat(N.infraorbitalis,non-bursting).Notonlytheresultingdetectoractivity
isqualitativelycomparablealsotheunderlyingdetectionprincipleseemstobethesame:freenerve
endings.

Amodelforthermoreception.Thestaticresponsecurvesofmammalianthermoreceptors,the
so-called“coldfibers”,andtheunderlyingbiologicaldetectionmechanism–freenerveendings–seem
tobethesameoratleastcomparabletowhatwefindinthepitmembrane;cf.Fig.5.8.Wehave
thereforeadaptedacommonionicmodelformammaliancoldfibers,theHuber-Braunmodel[26,27,
28,124],tostudyFNEresponses.Notsurprisingly,wecanreproducetheexperimentallyobserved
staticresponses,abell-shapedfrequencyvs.temperaturecurvature.Evenmoreinterestinglyand
forthefirsttimewehavebeenabletoreproducethedynamicresponses;seeFig.5.9.Toour
knowledge,nootherthermo-receptormodelhasbeenabletodoso.
TheHuber-BraunmodelforthermoreceptorsisbasedonasimplifiedversionoftheHodgkin-
Huxleymodel.Themodelconsistsoftwosetsofionicconductancesthatoperateattwodifferent
voltagelevelsandtimescales.Thetemperaturedependenceismodeledbytwoscalingfactorsfor
therateconstantsρ(T)andthemaximumoftheconductancesΦ(T).Themainobservableisthe
membranepotentialVthatiscontrolledbyfivecurrentsaccordingtothemembraneequation
VdCMdt=−INa−IK−Isd−Isr−Il,(5.16)
whereCMisthemembranecapacitance.Theioniccurrentsoftherighthandsidecanbegrouped
intothreecategories.ThefirsttwofastcurrentsINa/Kgeneratetheactionpotentials.Thenext

60

InfraredSystemofSnakes

twocurrentsIsd/srareslow,subthresholdde-orrepolarizing,andtheirinterplaycausesmembrane
potentialoscillations.Finally,thelastcurrentIlisthepassiveleakcurrent.Allcurrentsare
modeledinthefollowingway
Ii=ρgiai(V−Vi),(5.17)
withρatemperaturescalingfactor,githemaximumconductance,aitheactivationvariable,and
Vithereversalpotential.Theactivationvariableaiitselfcanbedescribedbyadifferentialequation.
ForadetaileddescriptionoftheHuber-Braunmodelanditsnumericalparameterswereferto
elsewhere[26,27,28,124].Inthelightofourfocus,theHuber-Braunmodelcanreproducethe
neuronalresponsestostatictemperaturelevels,e.g.,thebell-shapedactivityvs.temperature
characteristic;seeFig.5.8.Yetthemodelcannotreproducetheobservedneuronalresponsesto
eratures.ptemhangingcTheHuber-Braunmodelanditsextension.Wemodifytherepolarizingsubthresholdcurrent
Isr=ρgsrasrf(V,T)in(5.17)(correspondstoequation(5)in[27])byintroducingatemperature-
gradient-dependentactivationvariableasrthrough
asr(t>t0)=asr(t)−ΔasrifgradT|t=t0=0.(5.18)
Thismeansthatifatemperaturestepoccurs,theactivationvariableasrisreducedbyΔasr=0.5.
Itismathematicallyunimportantwhetherwemodifyasrortheconductancevariablegsr.The
resultisthatIsrisreducedshortlyafterachangeintemperature,andtheneuronfiresaburst
ofspikes.Thismodificationcorrespondsinprincipletoatransientionicgatingmechanismdue
to,e.g.,thermo-sensitiveTRP(transientresponseprotein)channels[23,176,189,212,275]which
resultsinanadditionalcurrentoccurringattemperaturechanges.Bothatemperature-depending
ionicgatingmechanismbymeansofTRPchannels[97]andanadditionaltransientcurrentsensible
fortemperaturechanges[219]havebeenfoundinthesnakerecently.
TheresultofournumericalexperimentisdepictedinFig.5.9andagreeswellwithbiological
findingsofthesnake’sneuronalresponses;cf.Fig.5.8.Thatis,theextendedHuber-Braunmodel
cannowreproducetheneuronalresponsestotemperaturechanges.Moreprecisely,ourfindings
hinttowardsthreeimportantissues.
•First,evenifthemechanismofthedynamicanswerismuchmorecomplex–aswebelieve–
itcanbeunderstoodatfirstsightbymeansofionicmodelsincludinganadditionalthermo-
gradient-sensitivecurrentorchannel.
•Second,wewanttohighlightthatatemperaturechangemodeledinthewaydescribedabove
synchronizesthedifferentFNEs.WepredictthatsynchronizingeffectsofFNEresponses
willenhancereliability,justasinmanyotherneuronalsensorysystems[80,81].Aprecise
quantification,however,willneedamoredetailedanalysis.
•Third,wecanseeaprominentdifferenceinthequalityofthestaticandthedynamicanswer.
Thetemporalprecisionandthesuddenchangeinfrequency(≈80Hz)atatemperaturestep,i.e.,
thedynamicanswer,ascomparedtotheincrease(≈5HzforΔT=5◦C)inthecorresponding
staticresponseisveryhigh.Inotherwords,wecandefinetherelativequalityΩofhowprecisely
theFNEssystemcanencodeatemperaturechangebymeansofachangeofthecorresponding
neuronalactivityfromATatTtoAT+ΔTatT+ΔT,

Ω=AT+ΔT−AT.
AT

(5.19)

61

InfraredSystemofSnakes

Figure5.9:SimulationofFNEresponses,thesensoryencodingstep.Theneuronalactivityof
uncoupledneurons(N=26),theaveraged(timeintervalsare200ms)firingrate,isplottedon
the(left)y-axis.TheneuronsaremodeledaccordingtothemodifiedHuber–Braunmodelof
thermo-sensitivecoldreceptors[27,28,124]asdescribedinthemaintext.Theneuronssequentially
start50msaftereachother.Duringournumericalexperiment,wehaveincreasedthetemperature
(redcoloredareaandrighty-axis)step-wise(ΔT=5◦C),startingatT=5◦Candendingat
T=30◦C.Aswecansee,themaximaofthedynamicresponseaswellasthesteady-statevaluesof
thestaticresponsefollowabell-shapedcurvecomparabletothebiologicaldata;cf.Fig.5.8.The
x-axisispresentedpiece-wise(e.g.fromt=t1tillt=t1+5s)aroundthetemperaturestepsbecause
thetemperaturestepssynchronizetheoutputoftheFNEsandpreventareasonableaveraging.We
concludethatthemodified(seemaintext)Huber–Braunmodelcanreproducethesensoryencoding
(FNEsresponses)oftemperaturedynamics.ThatisTRPchannelsresultinginasub-threshold
currentsthataresensitivetothermalgradientscancausethedynamicresponse.

Wecannowseethatthedynamicanswerencodestemperaturechanges:temporallypreciselyandvery
reliably;Ωdyn≈80Hz/20Hz=4.Incontrast,thestaticresponsetoabsolutetemperatureislesspre-
cise,relativelyweak,andevenambiguousontheleveloffrequency;Ωsta≈5Hz/20Hz=1/4=Ωdyn/16.
Thisbecomesevenmoreevidentifweadditionallyconsidertheamountoftimeinwhichthechange
ofactivityΔAoccurs.

encodedbAccordinglyythe,wstaticecanrespsponseeculatebutthatinsteadtheusessnakedochangesesnotofusethethetempabsoluteeratureteconmptrasteratureencocondedtrastby
thedynamicneuronalresponse.Thepresentparagraphshaveshownthatthedynamicanswerand
yettempboralelievced.hangesNext,ofwethewillheatexploitcontrasttheonresultstheofmemthebranepresenpttossiblywoplasections.yamoreThatimpis,weortancomtrolbineethanour
knowledgeonthephysicaltransmissionprocessandthefindingsofthesensorydetectionprocessto
deriveaplausiblemodelfortheneuronalreconstructionmechanism.

62

InfraredSystemofSnakes

dingdecoNeuronal5.4Inthissection,westartwiththesimplestmodelwecanimaginethatisabletoreconstructthe
heatcontrastdistributionξintheexternalworld.Thisminimalmodelaccountsforthelimitation
ofaneuronalsystembymeansofrestrictedcomputationalpower.Weextendtheminimalmodel
bymeansofincorporatingpropertiesofthesensoryencodingprocesssoastoarriveataplausible
andmeaningfuldescriptionoftheinfrared-visionsystemofsnakes.Inaddition,wetestourmodels
invariousconditionswithregardtoexternalheatcontrastandomnipresentnoise.Finally,we
showthatandpredicthowfeatureextractionispossibleusingboththestatic(minimalmodel)and
dynamic(extendedmodel)responsesoftheinfrareddetectorsofthepitmembrane.
delmoMinimal5.4.1Wehavetothinkofsimplemodelsbecausethesnakehaslimitedcomputationalresourcesinview
ofproptheosefactamothatdelallthatiscalculationsnon-iterativmustebandeusesrealizableonlyinoneneuronalcomputationalhardwarestepandthatfast.canbWeethdoneerebfyorea
feedforwardnetwork.Thereconstructedexternalheatcontrastdistributionξˆjatlocationjcan
thereforebewrittenasalinearcombinationofthemeasuredintensitiesri,
ξˆj=ljiri.(5.20)
Theoptimalreconstruction.Tofindtheoptimalcoefficientsofthereconstructiontensorlji
wefollowtheframeworkofoptimalmapformationpresentedinChap.2anddefinetheexpectation
valueEofthequadraticerrorofourestimateξˆias
E=(ξˆi−ξi)(ξˆi−ξi).(5.21)
Intheaboveequationweinserttheexpressionξˆi=lijrj=lijhjk(ξk+αk)+χjfrom(5.15)and
(5.20).Furthermore,wecomputetheexpectationvalueoverallpossibleinputαianddetector
χinoises.Thenoise-freepartoftheinputξiisdeterministic.However,wewanttooptimally
reconstructnotonlyonespecificsituationbutthetypicalIRenvironment.Inotherwords,
biologicallyrelevantsignalsbelongtoaclassofsignalsthatwedenoteby“typical”.Consequently
aspecificsensorysignalisaconcreterealizationofatypicalbiologicallyrelevantsignal;henceit
isastochasticquantity.Wethereforehavetodealwiththeexpectationvalueofthesignal;fora
detaileddiscussionofwhywehavetotaketheexpectationvaluewerefertoSec.2.2.2.
ibeInzerothbeecausefollowingthenoisecalculationstermsofareδEindep/δhjenden=0,tofweeachassumeotherallandappofearingtheinputcrossstrengths.correlationsTheto
correlationtermscanbeapproximatedtofirstorder
χiχj=σχ2δij,αiαj=σα2δij,andξiξj=σξ2δij(5.22)
whereδijdenotestheKroneckerdelta.Allthreeequalitiesdefineautocorrelations.
Thefirsttwotermsdenotethenoisetermsontheinputandthereceptormanifold.Herewerefer
tonoiseasaspatiallyindependentadditiverandomvariablewithzeromeanandstandarddeviation
σχ/αthatmodelsthefollowingpoints.Evenifthedetectionmechanismonthepitmembraneis
thefast,asignalsignalwillinbetheavexterneragedaltowtheorldrelevcanbanetfaster.signalSothatwecancanbedefineanreconstructed.integrationThislowindow-passwwithinfilter
effectisinherenttoanysensorysystem,nomatteriftechnicalorbiological.Theinputnoiseαi
accountsforthiseffectandthereforeblursthetruesignalξi.Strictlyspeaking,thenoisewould
thenbespatiallycorrelatedduetoacommonmovementoftherealIRobject.However,wewill

63

InfraredSystemofSnakes

Theignoresecondthisknonoisewledgetermanddescribthusesdealtheinthedetectorfollonoisewingonwiththeawpitmemorst-casebranescenariothatofmayknowingoriginatenothing.from
physiologicalprocessesintheheat-sensitivecells.
Thethirdtermin(5.22)describestheautocorrelationoftheinputthatwehavemotivated
abovdeviatione.Inσofgeneral,theitinputcanξb=e(.w.r.,ξitten,..as.);ξiseeξj=Chap.(µξ22.+Wσeξ2)δwillijonwithlymuseeanoneµξtermandconthetainingstandardthe
iξtnowypicaldependsinformationontheabpoutersptheectivIReofsurrounding,whethertheviz.,relevanteitherthestimulusmeanortheinformationstandardisdefineddeviationas.theIt
te.g.,ypicalpreyh.eatThecontrastformerorcaseastherefersvtoariationtheofmeanatµξypicalandheattheconlattertrastcasetorelatingthetostandanobardjectofdeviationinterest,σξ.
Forwithoutthesaklosseofofuniformitgeneralityybinyσourξandskipmathematicalforthetimedescription,beingwecathenabodenotevethequestionimpofortantwhatquanistitthey
relevantstimulustothepitorgan.Indoingso,weassumethatthecorrelationbetweeninputsat
differentlocationinspacei=jvanishes.Thatis,aworst-caseansatzagainbecauseknowingthe
valueoftheinputatoneparticularpointnowdoesnottellussomethingaboutitsvalueatother
pfromositions.thesameThisobjectassumptionortheofsamecoursewillsurrounding.notholdHeatinrealittranspybortecausephenomenaneighboringpeliminateointstesteepndthoebate
howgradienever,ts,lessandthantheptempossibleeraturemakleveselofneighreconstructingboringthepoininputtswillonlymorethereforebedifficult“similar”.andsufficesAssuming,fora
h.approacminimalHavingclarifiedthemappingoftheinputontothepitmembrane(5.15)andhowwecandescribe
ljtheiasnoisedefinedandininput(5.20).cThatharacteristicsis,weha(5.22),vetowenextminimizeprotheceederrorfinding(5.21)thewithoptimrespalecttoreconstructionthecoefficientensorts
ofthereconstructiontensor,i.e.,δE/δhji=0.Definingthedimensionlessparameterσ:=σχ/σξ
andτ:=σα/σξwecancomputethecomponentsofthereconstructiontensorljisoastofind
likhjvhkv(1+τ2)+σ2δjk=hji.(5.23)
toNextrealwequantakeatities.briefThelookatparameterthetwτomodenotesdeltheparametersinverseτandσ,signal-to-noisetheirmeaning,ratioofandthetheirinputwhicrelationh
ingeneralwillbesmall.Thesecondparameterσrelatesnoiseonthepitmembranetothemean
vki.e.,inputitintenincreasessityinitstheeigenvexternalalues,wtheorld.valueSinceofσσdirectlyinfluencesmothedifiesthereconstructiondiagonaloftensorthematrixsignifican(hjtlyh.v),

64

InfraredSystemofSnakes

Figure5.10:Minimal-modelreconstructionperformance;figuresareadaptedfrom[252].We
discretizeAlbrechtD¨urer’s1502paintingofahare(left)intoapixelizedversionwithan8-bitgray
scaleintendedtorepresentthehare’ssurfacetemperature.Inacomputermodel,thisIR-source
imagewaspresentedtoanidealizedsnakepitorgan.Thedashedcircleindicatestheorgansfieldof
view(100◦).MappingtheIRsourcebymeansofthetransferfunctionhgivenby(5.14)ontothe
pitmembraneresultsinaheavilyblurredmembraneheatimage(middle).Inthenextstep,the
modelreconstructstheIRinput,i.e.,theharefromtheIRimageonthemembraneofthemodelpit
organ.Therightpanelsshowthereconstructedimagebymeansofthereconstructionmatrixlof
(5.23)calculatedfordifferentnoiselevelsofthemembranesensorcells(0.25%,1%,2%,and5%of
themeanvalueofthemembraneintensities).Forfurthermodelparameterssee[252].Despitethe
pooropticalqualityofthepitorgan(centerimage),theoriginalinformationcanstillbeobtained
neuronallyfromthemembraneimage.

Thereconstructionperformanceoftheminimalmodel.Toillustratetheabilitiesofour
model,wepresentanexplicitexampleofhowpreciselythesnakecanreconstructitsIRsurroundings.
Weusethefamous1502drawingofAlbrechtD¨urerofahareconvertedinto8-bitgraylevelsata
resolutionof32×32;cf.leftpartofFig.5.10.Inthenextstep,wecalculatetheheatintensity
distributionrionthemembraneusingthetransferfunctionhkvgivenby(5.14).Wehavechosen
conservativebutrealisticvaluesthatcombinearelativelywideorganaperture(radius0.4mm)and
amoderatenumberofheatreceptors[252].Thecalculatedmembraneheatdistributionisshownat
thecenterofFig.5.10.
Theresultsofapplyingthereconstructionalgorithmtothemembraneimageareshownin
Fig.5.10.Thequalityimprovementisspectacular,providedthereisnottoomuchdetectornoise.
Themostimportantresultisthattheinformationneededtoreconstructtheheatpanoramaisstill
presentalthoughinvisibletooureyeinthemembraneimage.Thepresentmodelhasafairlyhigh
inputnoisetolerance.Forinputnoiselevelsupto50%,thehareisrecognizable.Sensitivityto
measurementerrorsislarger.Inourcalculations,onepixelofthereconstructedimagecorresponds
toabout3◦.Fordetectornoiselevelsuptoabout1%ofthemembraneheatintensity,agood
reconstructionispossible,meaningthattheedgeoftheharemaybedeterminedwithaboutone
pixelaccuracy.Atdetectornoiselevelsbeyondabout1%,theimageisnotsoeasilyrecognizable.
Choosingahigherσcancompensatethemembranenoiseuntilacertainnoise-levelrevealingat
leastthepresenceofanobject;cf.Fig.5.10.Thedrawbackofchoosinghighσvaluesisthatdetails,
viz.,variationsoftheheatimageare“treated”asnoiseandwillbesuppressed.Inaddition,in
thecaseofasimpleandclearinput–say,asmallwarmanimalinfrontofacoldbackground–
reconstructionmaystillbepossibleathigherdetectornoiselevelsaswewillseelater.

Predictedneuronalcharacteristics.Ourmodelisnotonlyabletopredictthesensoryperfor-
manceofthesnake’sIRsystembutcanalsoshedlightontotheneuronaldecodingmechanismused
tosharpenthemembraneimage.Theneuronalimplementationofthemodelisfairlystraightfor-

65

InfraredSystemofSnakes

Figure5.11:Predictedreceptivefields.Asexplainedinthemaintext,thestrengthoftheindividual
connectionscorrespondstotheentrylji.Theneuronsinthetopologicalmapreceiveexcitatory
(positive,red)aswellasinhibitory(negative,blue)inputfromthemembranereceptors.Thespatial
distributionoftheseexcitatoryandinhibitoryregionsonthemembraneiscalledthereceptivefield
ofacorrespondingmapneuron.Aswecanseeinpanel(A−E)thereceptivefieldforasingle
mapneuronconsistsofaexcitatoryringlocatedattheedgesoftheheatprofilesurroundedbyan
inhibitoryring.Thisconstructionconstitutesanedgedetector.Wecanshrinktheradiusrofthe
organaperture(B:0.4mm,C:0.2mm,D:0.1mm,andE:0.05mm)untilwereceiveapin-hole-like
geometrythatiscomparabletothesetupinopticalsystemsemployingalens.Inbothsensory
systems,theIRsenseandthevisualsystem,wefindthesamemechanismsthatfollowthesame
mathematicalprinciplesofoptimality.Reconstructionparameterσ=3,distancebetweenaperture
andobjectis1m,anddistancebetweenapertureandmembraneis0.8mm.

ward.Thereconstructiontensorljicorrespondstoanetworkofsynapticconnectionsbetweenthe
membraneIRdetectorsandneuronsbuildingamapin,forinstance,theoptictectum[109,210,211].
Thestrengthoftheindividualconnectionswouldjustbethevalueofthecorrespondingentrylji,
basedonaratecoding.Theneuronsinthetopologicalmapreceiveexcitatory(positive)aswell
asinhibitory(negative)inputfromthemembranereceptors.Thespatialdistributionofthese
excitatoryandinhibitoryregionsonthemembraneiscalledthereceptivefieldofacorresponding
mapneuron;cf.Chap.1.ItcanbecalculatedbyourmodelasillustratedinFig.5.11.Ring-like
structuresarisethat“detect”theedgesofprojectedimagescorrespondingtothepositionthatthe
mapneuronencodes.Asimilarsystem(calledlateralinhibition)isfoundinmammalianneurons
receivinginputfromtheretina;cf.Chap.2.Althoughtheexcitation-inhibitionpatternthere
isdisc-shaped(“small”aperture)ratherthanring-like(largeaperture)theexcitation-inhibition
patternnaturallyfollowsthesameprincipleofmathematicaloptimalityasthereceptivefieldwe
havederivedhere.

66

InfraredSystemofSnakes

Insummary,thesnakedoesneedaccuratedetectorstogetareasonablereconstructionquality.
Indeed,theexperimentallydeterminedmembraneprecisionishigh.Thepitmembraneresponds
totemperaturechangesdowntoabout1mK[33,63].Aswediscussedinpart5.3,theresponse
ofthedetectors(FNEs)totemperaturechangesismuchmoreprominentandthusreliablethan
tostatictemperaturevalues.Wenowincludethisfindingandadapttheminimalmodelderived
aboveaccordinglybymeansofincorporatingthedynamicdetectorresponses.

delmoextendedThe5.4.2HereweextendtheminimalmodelwithrespecttothemajorfindingofSec.5.3concerningthe
neuronalcharacteristicsofsensoryencoding.Themajorresultisthattemperaturechangescould
beencodedneuronallymorereliably,i.e.,withamuchhighertemporalprecision,andmoreclearly
thanthestaticmembranetemperature(5.19).Wenextdiscussthisfindinginthelightofneuronal
decodingandadaptourminimalmodelderivedaboveaccordingly.

Detectorcharacteristicsinthelightofneuronaldecoding.Theminimalmodelclearly
informationdemonstratescanthatbethedecodedinformationneuronally.transmittedThetominimalthemopitdel,memhowbraneever,isfacesstillatherebigdraandwbacthatk.theIt
needshighlyaccuratesteady-statetemperaturemeasurementsonthemembraneandtheliterature
wlaceksreferanetoxpthelanationfindingsofofwhereSec.the5.2.1necessarywhereweprecisioncouldseecouldthatoriginatethetempfrom.eratureTogivconesometrastnisuminbtheers,
rangeofmKwithinthefirst100msofastimulusonset.Inaddition,themeasurementerrorsneed
is,tobtheelessFNEsthanhav10%etotodetectreconstructabsoluteaheattemppanoramaeraturecasonwetrastshaveofseabenout1previously;mKupcf.toF20ig.mK5.10.withThatat
leastsampling0.1inmKtervalsaccuracyto.encoGivdeenthearelevnoiselessanttempdetectioneratureprovcess,alues.theLookingdetectorsatthewouldneuronalthenrespneedonses200
(seefoundFig.inthe5.8)toeascendingncodenerthevesranofgetheofpitΔξme=m20braneK;ev(Fig.enif5.8)weweconsiderfindathefrequencytemporalrangeofpatternsayof20Hzthe
thecorrespsensoryondingsystemspikeandtrainsalmaosthighunbelievaccuracyableinincaserepresenofatingnoisythestimdetectionulusproseemscess.veryWecthereforehallengingnoforw
focusontheotherneuronalresponse,thedynamicansweroftheTNMstotemperaturechanges.

Theextensionoftheminimalmodel.Wecanthinkoftwodifferentoriginsoftemperature
changesofthemembrane.Thefirstpossibilityisthesudden(dis)appearanceofaheat-emitting
toobjecmotvemencausingts.Wthearmriseoborjectsfallinofthethememexternalbraneworldtemptranslatingerature.Therelativsecondelytoandthepimosttmemnaturalbraneisleaddue
toobjectemptmovorallyes,ctemphangingeraturetempchangeseraturesappateartheatthesensoryedgescofellstheontheresultingpitmemtempbrane.eratureIfpsucrofilehaonwarmthe
membrane.Inencodingtemperaturechanges,theFNEsdonothingbuteffectivelyfilteringthe
edgesWeofcanthepitcalculatememanbraneinedge-filteredtensityversiondistributionfi(seeri.Fig.5.12)ofthemembraneintensitydistribution
rjbymeansoflinearfilteralgorithmssuchastheLaplacefiltermatrixtij[91],
fi=tijrj=tij(hjvξv)=hˆivξv.(5.24)
vˆTheheatabconovetrastξvequationontoandefinesedge-filteredsimilarlytointensit(5.14)yaprofiletransferfionfuncthetionmemhibrane.thatnoSincewmapsthetheexedge-filteredternal
imageapproximatesthetemporalderivativeoftheintensityprofileonthemembrane,wehave
essenextendedtialpropourmoertiesdelofofthethesnakdetectionemecinfraredhanism.senseinUsingathesimplesamebutmethoreasonabledologywaasythatdescribnoedwiinncludes(5.20)

67

InfraredSystemofSnakes

–(5.23),wecancalculatecorrespondinglyto(5.23)anoptimalreconstructiontensorlˆijusingthe
newtransferfunctionhˆivof(5.24)
lˆikhˆjvhˆkv(1+τ2)+σ2δjk=hˆji.(5.25)
Arrivingattheaboveequationwehaveimplementedaveryplausiblesecondstepofsensory
processing(seeChap.1),i.e.,wehaveincludedthepitmembranedetectorcharacteristics.Even
thisseemstobeasmallstep–aswehaveleftoutthederivationstepsanalogueto(5.20)–(5.23)
fortheminimalmodel–itresultsinabigadvantageaswewillseeinaminute.

Figure5.12:TheLaplacefilter.Asexplainedinthemaintext,weusetheLaplacianfilterthat
can“detect”theedgesofthemembraneintensitydistributiontoextendtheminimalmodel.The
Laplacianfiltermechanismworksinthefollowingway:(B)Thefilterkernelsequentiallytakesevery
pDepoinetofndingtheontheoriginalLaplaceimage,filterscaleskernelit,atandhand,subtractsitcantheaffectweighthetedareavofalue3×of3,itsasinneightheboringfigurep5oin×5ts.,
ormorepixelscenteredaroundtheevaluatedpoint.(C)Tousethismethodinthecontextofthe
minimalmodelwehavetoiwritethefilterkernelaccordinglytotheindexingioftheoriginalimage
inimagetothe(A)linesontoofatheLapmatrixtlacianjforfilteredeachimageimage(pBoin)bt.yThemeansofLaplaceonematrixmatrixtmjnowultiplication.mapstheoriginal

advAdvanantagestagesthatofthearisefromextendedusingmothedel.dynamicTheinsteadextendedofmothedelstaticfeaturesresponse.thefolloFirst,wingthechangeconceptualin
neuronalactivityΔA≈80Hzofthedynamicresponsecausedbyanincreasingtemperature
contrastξisabout16dyntimeslargerthanΔAsta≈5Hzofthestaticresponsecausedbyanincreasing
tempemployerature16timescontrasmoret;cf.Secsampling.5.3.poinDuetstotothisdecofact,detheusingsignal.theThatdynamicmeans,responsetheallodetewsctiontheacsnakcureacyto
es.vimproΩ,theSecond,relativnoteconlyhangetheofabsolutefrequencychangecomparedoftoneuronaltheactivitactualyΔAfrequencyoftheoffreethenervedetector,endingsisbutincreasedalso
(5.19).Third,Thatacis,hangingthequalitIRyenandvironmenreliabilitt–ymoofvingtheobneurjectsonal–rdoespesonsenothevenindermorebutthanactuallyimproenableves.the
detectionprocess.IncaseofusingtheabsolutetemperaturecontrastξoftheIRenvironment
(minimalmodel),ξhastobestatic.Thatis,incaseoftheminimalmodel,movementsaretreated
asnoise.Incontrast,theextendedmodelisbasedonthedynamicresponse,i.e.,detectorresponses
tothechangessignaltoofthethemoIRendel.Thisvironment,decreasesdξ/dt.thTheus,noisemovlevelemenintsthearenotsignalrelatedmappingtoprnoiseocessbut.Ofconstitutecourse,
thereisalsoatemporal“integrationwindow”duringwhichasignalhastobeconstant.Sincethe

68

InfraredSystemofSnakes

dynamicresponse,i.e.,thesudden(about100ms)increaseinfrequencyoftheneuronal-detector
activity,istemporallyquitepreciseandfast,theintegrationwindowisnarrow,andthusthe
precise.aredetectorsFourth,weremembertwofindings.First,inSec.5.2.1wehavederivedthatthetemperature
contrastξonthemembranehasnotreacheditssteady-statevaluewithintherelevantreaction
timeofthesnake.Thesignalξcanthusneverbeconstantandself-averaging,respectively.Second,
thereconstructionmechanismneedsveryaccuratesensorydata.Asignalthatisconstantduringa
certainperiodoftimecanbedeterminedmorepreciselybymeansofmakingmoremeasurements
andaveragingtheresults.Inthecaseoftheminimalmodel,ξisnotconstantandthuscannotbe
improvedbystatisticalmethods.Onthecontrary,thetimederivativeofthetemperaturecontrast
dξ/dtisalmoststaticwithintherelevanttimeframe.Astaticphysicalsignalprovidesaveraging
and√thusreducingadditivedetectornoise.Wecanatleastimprovethemeasurementbyafactor
∼twithmoremeasurements.Thatis,theaccuracyofhowwecandetectandencodethesignal
ishigherincaseoftheextendedmodel.
Insummary,wehavederivedanextensionoftheminimalmodelthatconnectssmoothlytoa
plausibleunderstandingofthedynamicresponsepatternfoundintheFNEs;seeFig.5.8.Toclarify
theoverallsensoryperformanceoftheextendedmodel,wenexthavetoquantifyitsdependency
uponnoise.SincetheunderlyingmathematicsoftheextendedisakintoaLaplacian-filteredversion
ofthemembraneimageonecannotexpectagoodnoiserobustnessbeforehand.Thisfactcould
cancelouttheaboveadvantages.Inthefollowingsection,wethereforecomparetheabilityofthe
twomodelsathandtoreconstructtheIRenvironmentandtoextractrelevantfeaturessuchasthe
objectpositioninvarioussettings.

extractioneatureF5.4.3AfterhavingseenabovethattheIR-sensorysystemofsnakescanreconstructtheheatenvironment,
wobejecnotswandaddresswhichthequanquestiontitiesofinfluencehowthepreciselysensorysnakpes,thaterformance.isourtwomodels,canlocalizewarm

Thenumerical“experiment”setup.Inthefollowingnumericalexperimentswepresenta
warmobjectwithinanon-uniformandnon-zerotemperaturebackgroundtoanartificialpitorgan.
WeassumeanIRenvironmentwith25◦Cmeantemperature.AswederivedinSec.5.2.1and5.3,
theexternalheatcontrastξextandthecorrespondingtimederivativedξext/dt,respectively,are
theappropriatestimuliforthepitorgan.WeaddaGaussian-distributedtemperaturebackground
contrastwithΓ◦CmeanandΓ◦Cstandarddeviationtostudytheeffectsofexternalnoiseand
signalcontrast.ThewarmobjectshowsaGaussian-shapedtemperatureprofilewithmaximalheat
contrastξmax(inrespectto25◦C)andstandarddeviation6.25cmthatislocatedatx=(x,y)
withx,y∈[−1m,1m],1minfrontofthepitorgan;cf.Fig.5.13.Foreveryrunwerandomly
positiontheobjectcenteratx,addanoisybackground,mapthewholeinfraredsceneontothepit
membraneandreconstructtheenvironment.Weusesimpleshapesbecausewenowcanextractthe
estimatedpositionxˆ=(ˆx,yˆ)easilybymeansoftakingtheaverageofthe50%warmestlocations
ofthereconstructedenvironmentandcomparetheestimatedpositionxˆtotherealpositionx;
cf.Fig.5.14.Duringthenumericalexperimentswemonitorthedependenceofthelocalization
performanceuponthebackgroundcharacteristics,namely,Γandthedetectornoiseχ;seeFig.5.15.
WehavetonotethatthefocusofthisworkisontheIRsensorysystemandnotonthemost
cognitivealgorithmforobjectrecognitiongivensensorydata,i.e.,apost-processingstep.Wehave
alreadyshown[251]howtoestimateboth,positionandshapeofobjects,givensensorydataofa
naturalsensorysystem.So,ourfindingspresentabottom-lineperformanceofthesnake’ssensory
abilitiesthatcanbebetterthanderivedhere–intheaspectofobjectrecognition.

69

InfraredSystemofSnakes

Figure5.13:Setupforlocalizationperformanceexperiments.(A)Wesimplifythewarmobjecttobe
atwo-dimensionalGaussian-shapedprofilewithstandarddeviation6.25cm,maximalamplitude
ofξmax=5◦C,andpositionx,andaddaGaussianrandombackgroundtemperaturecontrast
distributionwithmeanandstandarddeviationofΓ◦C.(B)ThisparameterizedIRenvironment
ismapped(minimalmodel:hikorextendedmodel:hˆik)ontothepitmembrane,anddetector
noiseχiisadded;cf.(5.15).Nowitdependsonwhichmoidelweconsider.Eithertheintensity
distributionriandthecorrespondingreconstructiontensorlk(minimalmodel)ortheedgesofthe
intensitydistributionfiandlˆik(extendedmodel)areusedtoreconstructtheIRenvironment.In
thelattercasetheedge-filteredmembraneimagefiapproximatesthetime-derivativefi≈dri/dt
duetoobjectmovements;seeSec.5.4.2.(C)Inthefinalstep,weextracttheestimatedpositionxˆ
ofthewarmobjectfromthereconstructedIRimageandcompareittotherealpositionxduringn
independentruns:seeFig.5.14.Toquantifyhowtheparametersaffectthesensoryperformance,
wepresenttheroot-mean-squareerrorbetweenx=(x,y)andxˆ=(ˆx,yˆ)inFig.5.15.

Sensoryperformance“experiments”.Withinthesetupoutlinedinthelastparagraph,we
varythesignal-to-nosie(SNR)ratiooftheexternalheatcontrastξbymeansofincreasingΓgiven
aconstantξmax=5K.Inaddition,westudyhowthedetectornoiseχionthepitmembraneas
givenby(5.15)affectssensoryperformance.Theresults(Fig.5.14)illustratehowpreciselythe
positioncanbeestimatedfromtheIRenvironment.Incaseofmoderatenoiseboththeminimal
model(leftpanelsinFig.5.14)andtheextendedmodel(rightpanelsinFig.5.14)canreconstruct
thepositionofthewarmobjectquiteprecisely.
Inthissection,thestandarddeviationofthemembranenoiseσχisrelatedtothemaximal
valueofeithertheintensitydistribution(minimalmodel:σχ∼rmax)ortheedgesoftheintensity
distribution(extendedmodel:σχ∼fmax)onthemembrane.Thepreviousdefinitionofmembrane
ofthemeanvalueofthemembraneintensities:σχ∼1/niri;seeFig.5.10.Sinceahigher
noisepresentsthestandarddeviationσχoftheadditiveGaussiannnoisevariableχiinpercentages
temperaturebackground–viz.,increasingΓ–alsoincreasesthemeanvalueofintensitiesrion
membrane,thetwodifferentkindsofnoisewouldnotbeindependentincaseoftheprevious
definition.Soweredefinethenoiseleveltobenowrelatedtothemaximalvalueofthemembrane
intensityriortheedgesofthemembraneintensityfi.Thisre-definitionalmostcompletelydecouples
.ΓandχFurthermore,weseeinFig.5.14thatinbothmodelsthecapabilitytolocalizeawarmobjectis
stillavailablebeyondthemembranenoiselevelof1%butdecreasesfast.Independentlyofeach
otherboththeexternaltemperaturebackgroundΓandthemembranedetectornoiselevelsχ,play
animportantroleinpredictingthesensoryperformanceandinevaluatingexperimentalresults.
Surprisingly,theestimatedpositionsxˆdonotonlybecomemorevaguebutalsoclusteredaround
themiddleofthepitapertureduetoincreasingnoiselevels.Thelattereffectismoreprominentin

70

InfraredSystemofSnakes

respcaseofonsesthetomitempnimaleraturemodel;ccf.hangesleftinpanelsprincipleinFig.access5.14.sensoryThedataextendedwithmohigherdelemploaccuracyyingasthecompareddynamic
totheminimalmodel.Onthecontrary,sinceLaplacianfilterstypicallyenhanceimagenoise[91]
wasecouldcomparedexpetoctthethatmintheimalextendedmodel.moIfdelthisthatisisthebasedcase,onthealessLaplaciannoisefiltertoleranistlessdecodingtoleranttoalgorithmnoise
wouldcancelouttheadvantagesgainedthroughamoreaccuratesensoryencodingmechanism.At
firstglance,theextendedmodel,however,isasrobusttonoiseastheminimalmodel;seeFig.5.14.

Figure5.14:Localizationperformanceexperiments.Thetwocollectionsofninepanelsillustratethe
effectsofthetwoparametersχi,themembranenoise(x-axis)andΓ,parameterizingtheexternal
IRenvironment(y-axis)forboththeminimalmodel(left)andtheextendedmodel(right).Ineach
ofthe18panelswehavedepictedrealpositionsx(panelx-axis)versusestimatedpositionsxˆ(panel
y-axis)of2500trailscorrespondingtothesettingdescribedinFig.5.13.Weseeclearlythatboth
modelscanreconstructthepositionoftheinfraredobjectverypreciselyincaseofmoderatenoise
levels.Inbothmodels,localizingawarmobjectispossiblebeyondthemembranenoiselevelχ
of1%butdecreasesfast.Furthermore,thelevelofcontrastΓwithintheexternaltemperature
backgroundplaysanimportantroleinpredictingthesensoryperformanceasthelocalizationgets
worsewithincreasingΓ(seey-axis).Interestingly,notonlydoestheestimationaccuracydecrease
butforhighΓtheestimatedpositionstendtobemoreclusteredaroundthecenteraswell.Above
all,theextendedmodelrobustnesstonoiseseemsatleasttobecomparablewiththeminimalmodel
–itisevenbetterintheparameterrangedepictedabove;butseealsoFig.5.15.

Tomathematicallyquantifytherobustnesswithrespecttobothtemperaturebackgroundand
detectornoise–theperformanceofthetwomodels–wedefinetheroot-mean-squareerrorErms
betweenrealxandestimatedcoordinatexˆ
Erms=(x−xˆ)2/n(5.26)
nthroughrobustnessnasindepcomparedendenttoruns.theInextendedFig.5.15model.weseeThethatnoisethetoleranceminimalofthmoedelextendedshowsmoadel,higherhowevnoiseer,
isgoodenoughtoallow,incombinationwiththeadvantagesofthesensoryencodingstep,i.e.,the
higherqualityoftheneuronalresponse(5.19),theconclusionthattheextendedmodelisthemodel
e.oichcofConsequently,weagainposethequestionofwhatisthestimulustotheinfraredsystemsincewe
havederivedthesuperiorityoftheextendedmodelemployingthedynamicresponse.InSec.5.2.1

71

InfraredSystemofSnakes

wehavederivedthatthestimulushastobetheexternalheatcontrastξextaccordingtothe
thermodynamicsofthesysteminsteadoftheexternalabsolutetemperature.Thefindingsof
sensoryencodingonthelevelneuronalcompartmentsinSec.5.3havethenhintedtowardsthetime
derivativeoftheexternalheatcontrastdξext/dttobetheadvantageousandappropriateinputfor
thepitorganinsteadofξextortheabsolutetemperature.Thenumericalperformanceexperiments
havenowunderlinedthisassumption.Althoughintheliteraturetheabsoluteexternaltemperature
isreferredtobeingthestimulustothesensorysystemwecanalsofindexperimentalstatements
thatsnakesrespondbesttomovinginsteadofstationarywarmobjects[94].Aswehavediscussed
above,thelatterstatementscorrespondtodξext/dtandhenceprovideanexperimentalevidence
forourtheoreticallyderivedfinding.

Figure5.15:Quantificationofthemodels’localizationperformance.TheplotsshowErmsforthe
andminimalχ(y(left)-axis)andusingtheanextendedoptimalmoσ.delFor(righeacht)insetdofepnoiseendenceupparametersontheΓnoiseandχ,wparametersehaveΓv(xaried-axis)σ
The(2500minimalrunsformoeacdehlcomemploysbination)alargertofindparametertheoptimalspaceinσthatwhichlominimizescalizationtheofloawarmcalizationobjecterrorisErquitems.
wgooelldpenoughossible,toviz.,notincancelwhichoutErmstheisadvsmall.antagesTheofnoiseusingthetolerancedynamicoftherespextendedonsesucmohasdelistheathigherleast
qualityofencodingatemperaturestimulus.

Insummary,wehaveshownmathematicallythattheIRsignalsoriginatingfromanexternalheat
distributionandgettingprojectedontothepitmembranestillsufficetoreconstructboth,animage
oftheexternalheatdistributionandthepositionofwarmobjects.Furthermore,wehaveclearly
demonstratedthatincludingcharacteristicsofthedetectionprocessintothereconstructionmodel,
viz.,focussingonthedynamicresponse,enablesareasonableunderstandingoftheremarkable
performanceoftheinfraredsystemofsnakes.

72

InfraredSystemofSnakes

5.5DiscussionandOutlook
Inthischapterwehaveanalyzedtheinfraredsystemofsnakesbystudyingthethreestepsof
sensoryprocessingthatwehadintroducedinSec.1.1.First,usingtheeyeglassesofphysicswe
havelookedatthemembraneheatingprocessandcorrespondingexperimentalfindingsofthepit
organssoastoidentifytherelevantequationsdescribingthefirststepofsensoryprocessing,viz.,
thesignalmapping.Indoingso,wehavequantifiedhowtheheatingprocessdependsonexternal
quantities–suchasdistanceortemperaturecontrastofawarmobject,anatomicaldata,and
time.Second,wehavepresentedanionicmodelthatcombinesrecentanatomicalfindingsand
presentmodelsofheatreceptors.Thatis,bytakingtheeffectsoftransientresponseprotein(TRP)
channelsandtime-dependentsub-thresholdcurrentsintoaccountwewereabletoreproducethe
neuronalresponsesofthefreenerveendings(FNEs)notonlytostatictemperaturesbutalsoto
temperaturechanges.Ourfindingsofsensoryencoding,furthermorehinttowardssynchronization
effectsoftheFNEsthatcouldberesponsiblefortheoutstandingtemperaturesensitivityofthis
sensoryarchitecture.Third,forthelaststepofsensoryprocessing,i.e.,neuronaldecoding,wehave
providedbothaminimalandanextendedmodelthatareabletoreconstructtheexternalheat
distribution,i.e.,theinfraredimageoftheenvironment,givenonlytheheavilyblurredmembrane
image.Acomparisonofbothmodelswiththeirresponsetodetectorandexternalnoisehasrevealed
theabilitiesandlimitationsofthisIRsense.Ourfindingsbothfitwellintotoday’spictureand
extendtheunderstandingoftheinvolvedprocesses.
Moreover,wehaveposedandansweredthequestionofwhatisthetruestimulustothis
sensorysystem.Inthefirstsectionofthischapterwephysicallyderivedthatitisnottheabsolute
temptemperatureerature.butIntheaddition,temptheeraturecequationsontrasthavofetheshownexternalthatwtheorldtempthateratureeffectivconelytrastaffectsonthethememmembranebrane
doesnotreachitssteady-statevalueduringtherelevantresponsetime.Sincethetimederivativeof
thetemperaturecontrastonthemembranestaysalmostconstant,wehavestartedtoconsiderthe
wetimehavederivfounativdeofmoretheevidenceexternaltoheatsupportconthistrasttothesis.betheThestimsupuluseriortoqualittheyIRofthesystem.sensoryInthisrespconsehapterto
temperaturechangesascomparedtostatictemperaturesisoneofthem.Totesttheassertion,we
haveextendedtheminimalansatzmodelingthereconstructionperformance.Insummary,wehave
thenotbtempeenoralabletoderivfindativeorofderivtheeexternalreasonableheatconargumentrasttsistheagainstbutappropriateonlysintimfavulusorforforthethesnakclaimes’thatIR
system.Withinthiswork,notonlyarenewinsightspresentedbutalsonewquestionsforfurther
researchhavearisen.Inthecontextofsignalmapping,experimentallygainedvaluesoftheinvolved
quantitiessuchasthebloodflowkinematicsortheheatconductanceofthemembranetissue
areneeded.Systematicandquantitativemeasurementsofthewarmingprocessdynamicswould
allowforunderstandingthefunctionsoftheindividualphenomenasuchasbloodflowandsurface
structureone-by-oneandininterplaywithoneanother.
dynamicsOnthearelevelneededoftosensoryfurtherencoenlighdingtenmothedelsquestionincorpoforatingwhatexpmakeserimenthetalpitmemdataofbraneTRPsocsensitivhannele.
ApromisingissuecouldbesynchronizationeffectsoftheFNEs.
Inthecontextofreconstructionmodelsweseetwofutureapproaches.Ontheonehand,
theoreticalmodelsshouldbecomemorebiological.Thatis,theyshouldbebasedonneuronaldata
–ifpresent–todescribesensoryperformanceonthelevelofspikingneurons.Theextendedmodel
basedoncharacteristicsofthesensoryencodingprocess,i.e.,thedynamicresponse,providesagood
startingpoint.Ontheotherhand,nobodyhaseveraddressedthequestionofhowtheaccuracyof
theIRsenseandthusthetopographyofthecorrespondingneuronalsynapticconnections,arises.
Thereisaclearlackofexperimentaldataonthisissue.Fromthetheoreticalpointofview,onecan
imagineinstructivelearningasfoundintheauditorypathway,orsomekindofauto-focusalgorithm

73

InfraredSystemofSnakes

thatusesimagestochasticstoimproveitself,tocausethedevelopmentofthissensorysystem.In
thenextchapterwewillstudyamodel,calledintegratedMultimodalTeaching,thatcouldbea
reasonableexplanationofhowtheinfraredmapandthevisualmapintheoptictectumgetaligned.
Thefindingsderivedinthischapterarenotpurelyofbiologicalinterest.Asourmathematical
modelsshowagoodperformancewithoutemployingcomplexcalculations,wehavealreadystarted
tosetupaprojecttoimplementourfindingsintohardware;cf.Fig.5.16.Theoverallgoals
aretoevaluateourmodelsinrealsettings,toisolateindividualaspectsofthemodelsandthe
involvedphysicsthatarenotpossibletostudyinabiologicalcontext,andtodevelopandtest
learningalgorithms.Onthehardwaresidethegoalsaretoclarifythepossibilityofimplementing
bio-inspiredcognitivecircuitsinsilicontoimproveopticalIRinstruments.Firststepsarepromising,
andwearelookingforwardtothisinterdisciplinary,bionicapproach.

Figure5.16:Bionicexperimentsetup.Asdescribedinthemaintext,wearegoingtosetupan
artificialpitorganexperimentincooperationwithErikJungandAlexanderHilgarthfromthe
FraunhoferIZM,Berlin.Theinputprojectormapsthesignalontothedetectorsoftheartificialpit
organ.Werealizeafullyflexiblemotor-controlledgeometry(flexiblepositionofthepitorganwith
respecttotheinput,variableorganaperture,adjustabledistancebetweenapertureanddetector)
toevaluatethepredictionswehavederivedinthiswork.Moreover,wewilldevelopandimplement
“reality-feedbacked”learningalgorithmsandminiaturizethissetupintheprogressoftheproject.

Afinalremarkregardingthesensoryperformanceoftheinfraredsystemofsnakes:Itseemsthat
notonebigdevelopmentbutratherthecomplexinterplayofseveralspecificstepssuchasavery
thinandisolatedmembrane,adensecapillarynetwork,FNEsemployingtemperaturesensibleionic
channelsthatareorganizedtoacompactnetworkcalledterminalnervemass,neuronaldecoding
mechanisms,etc.haveledtooneofthemostspecializedandremarkablesensorysystemsfoundin
nature.Wewereabletoanswersomebutnotallofthequestionsconnectedwiththesesteps.
Inthenextchapter,wefocusonthegeneralquestionofhowaprecisealignmentofdifferent
sensorymapscandevelopintrinsically.Thealignmentofdifferentmapsisfundamentalfor
multimodalintegrationthatisalsofoundinthesnake’soptictectum.There,theinfraredmapand
thevisualmapinteractfaithfully,forinstance,bymeansofperforming“AND”(e.g.visibleand
warm)and“OR”(e.g.visiblebutnotwarm)operations[109,210,211].

74

Millionssawtheapplefall,butNewtonwasthe
onewhoaskedwhy.
hBarucesMannBernard

6.IntegratedMultimodalTeaching

Inthischapterwecomebacktothetopicoflearningandaddressglobalstructuresunderlying
theevolvementofsensorysystemsandtheirpreciseinteraction.Weintroducetheconcept
ofintegratedMultimodalTeaching(iMT)thatisbasedonsupervisedspike-timing-dependent
plasticity(sSTDP)andmultimodalintegration.Weshowthatamultimodalteachercanensure
properintrinsicmapformationandalignment.Thedominanceofvisioninguidingmapalignment
isdemonstratedtobeanaturalconsequenceoftheexceptionalprecisionofthevisualsystem.

ductionIntro6.1Ourabilitytoactinadynamicandcomplexenvironmentreliesfirstandforemostontheprecise
interactionofthedifferentsensorymodalitiesthatformacongruentperceptionoftheexternal
world.Thispreciseinteractionofthedifferentmodalitiesisnotpresentatbirthbuthastobe
learnedduringthefirstperiodofsensoryexperienceinlife[271,280].

Figure6.1:Thebasicideaofintegratedmultimodalteaching(iMT).Allmodalitiesareintegrated
aintoteacanhinginsignaltegratedm(blackultimoarrodalws)mapthat(graleadsyarrotoanws).Theadaptationactivitofyofthetheunimomultimodaldalmaps.mapWethenexplaininducesthe
detailedmodelsetupemployedinthepresentchapterinSec.6.2.2.

Theprocessofsensorycalibrationiscommonlyseenasvision-guided;cf.King’sreview[148].
Severalremarkableexperimentsshowthatdestructionordisturbanceofthevisualpathwayleads
todisorganizedandabnormalsensorymapsinnon-visualmodalities.Thesefindings,obtained
insupporthamsterthe[200idea],ofcat[vision278,as280teac],clahingwedmofrogdalit[y.44],Howferretever,[150],vision-guibarndeowld[map159],andcalibrationinsnakisesnot[96as],
evidentasitseemsatfirstglance.First,sensorymaps,althoughimprecise,canevolvewithoutany
visualdespiteainput[missing146,158visual,271].system,Exptheerimentsauditoryonandvisuallydeprivsomatosensoryedanimalssystemsandcanhumansdevelophaveshonormally;wnthatsee
[152,231,234]fordetails.Second,behavioral[30,68,77,249,258]andphysiological[202]studies

75

IntegratedMultimodalTeaching

haWveallaceshownandthatStein[vision277,can279]halsoavebepoininflutedencoutedbythatotherreceptivmoedalities.fieldsinThird,thesupanderiormostimpcolliculusortan(SC)tly,
thatalreadyallmoshrinkdalitiesatshotimeswawheretempornoallyvisualcoincidingneuronsdevareelopmenpresentt.inFwhichurthermore,evenvisiontheyhavimproeves.observButed
ifhowvisioncanitshouldimprovacteasitselfteac?Thingogether,modalitthey,abhoowvecanparitadoguidexicalcaexplibrationerimentalwithoutandbeingtheoreticalpresenfintdingsand
questionthecurrentpictureofvisionasthepredominantguidingmodality.Thatis,whatisa
coherentalternativetovisionastheteachingmodality?
moIndalitiesthiscbeginshapter,atwtheetaktimeetofwoappfindingsearanceasofmstartinultimogpdaloinints.tegrationFirst,[the279].formationSecond,ofthedifferenmotor,t
hasi.e.,bameenultimosuggesteddalmaptoinprothevideSCaasteacwhellingasinsignalitsfornon-mammalianauditoryspacehomolog,alignmenthet[optic180];cf.tectumFig.(OT),1.4
forSCconnectivity.Therefore,wesuggestanintegratedMultimodalTeachingconcept(iMT);
seefindingsFig.as6.1.toWhoewexplaindifferentthemwoell-prodalitiesvencanbedominanalignedceofwiththerespvisualecttosystemeachandother.unifyIncondoingtradiso,ctorwey
inontroaduceconceptualaninlevtegratedel.Tmoultimothisdaend,lteacweherfocusandondiscussmapitsinformationtrinsiccwithinharacteristicstheSC/OTandbyimplicationsmeansof
long-termpotentiation(LTP)andlong-termdepression(LTD)ofsynapsesthatcanbedescribedby
movsSTDPe.theirWeleasensoryveoutorgansthe(eyquestiones,ears,ofwhiskdynamicers,etc.)adaptationindepasendenapptlyeariofngtheirinbmanodyy.Theanimalsmecthathanismcan
underlyingthisadaptationisnotknown,butretinotopiccoordinatesystemsseemtoplayakey
horolewev[83er,].mAttemptsultisensorytoandmodelthispredominanissuetlycanmonbeosreviewensoryedlayiners[82are,99m,240utually,241].alignedWithintogaintheaSC/OT,unified
multisensoryrepresentationofsensoryspace[147,261].Thecombinedsensoryinformationcan
Inthenotherbewusedords,toatgeneratethisstagedirectionalallsensoryresponsesmapsinsharetheaSC/OTcommonmotorrepresenmap[tation166,167system.,179,Sim216il,ar260to].
othercalibrationtheoreticalappearingmodelsin[the78,con287],texttheofthesuggesteverydiMTcommonconceptsensoryconcenspace.tratesInonadditionthetoquestionderivingofmapthe
generalconceptforiMT,weapplyourmodeltoconcreteexamples.Usingthismodel,wereanalyze
ofinWdetailallacetheandconcreteStein[277exp,er279ime]nintsoftheKnlightudsenofouretal.iMT[159]concept.andcompareFinally,wthemewithconcludethebyfindingsgiving
experimentallytestablepredictionsofourapproach.

6.2IntegratedMultimodalTeaching(iMT)
Ourconceptofintrinsicsensorymapcalibrationstructurallyconsistsofastackofunimodal
maps,eachconnectedwiththemultimodalmap(cf.Fig.6.1).InouriMTconcept,different
unimounisensorydalmapsmocdalitiesancurrenarefirsttlybemergedfoundintowithinthemtheultimoSC/OT,dalmap.whereSucthishamsituationultimoisdalexpinerimentegrationtallyof
wellunimodaldescribmaps.ed[261Th,us,279].theInmtheultimonextdalstep,maptheinducesinategratedteacmhingultimosignaldalthatmapproguidesjectsthebackcalibrationtothe
does(formationnotoriginatandealignmenfromat)singleprocessmoofdalitythe,fordifferenexample,tunimovisiondal,butmaps.fromtheTherefore,integratedtheteacmultiherminodalput
map.InForgeneral,adetailedmapcalibrationunderstandingcanofbeourdividedmodelintodesign,twopleasedifferentseeSec.classes.6.2.2.Thefirstdescribesthe
situationinwhichsensorymapshavetobelearnedfromscratch.Connectionsbetweensensory
systems(inputmaps)andrepresentingmapscontainnoinformationandarerandomlydistributed.
Thesynapses;inconnectionshamathematicalvetoevtermsolveduringdescribedthebySTDPcalibration.WeprodenotecessbythismecaseansasofLTPformationandofLTDtheofmathpse.
Thesecondclassrequiresacollectionofalreadypre-wiredmaps[271]andcorrespondsto

76

eachingTdalMultimoIntegrated

anstimulusalignmentpositionofismapheres,dforisplacedinstance,induringcomparisongrotowththeorpinositionshiftingoftheexpteacerimenhertsinput.[158,By159].meansTheof
activity-basedsynapticplasticity,thesynapticpatternofthemapismodifiedsoastocompensate
forthemisalignmentbetweenthemapandteacherinput.
TocarefullyanalyzetheconsequencesofapplyinganiMTsignalwehavetounderstandmap
proformationcess(asandshowninalignmenFig.tf6.2)romaproconstitutesceduralapclosedointoffeedbacview.kloToopthisthatend,consistsweofnotethreethatreptheeatingiMT
ansteps.inFirst,tegratedmduringultimomultimodaldalmap.integrSecond,ation,duringdifferentmultimosensorydaltemoachingdalities,theareinoptimallytegratedmultimergedmoindalto
mapalignmeninducest,ofanthediffereninhibitorytteacsensoryhingmaps.signalThird,thatguidesduringtheunimodalcalibrationmapproadaptioncess,,i.e.theformationadaptingandand
adaptedunimodalmapsagainmodifytheintegratedmultimodalteacher.
Thethreephaseswehavedescribedaboveareinherentlyconnectedbymeansofthreemajor
questions:1)HowdounisensorymapsdetermineiMT?2)HowdothecharacteristicsofiMT
together,influencethesemapthreeadaptation?questionsandpro3)videHoawdostructureesiMTforourcalibrateiMTdiffeconceptrentandunimwilloadaldmaps?ditionallyTakguideen
usthroughtheanalysisoftheiMTprocessinthefollowing.

Figure6.2:MainproceduralaspectsofiMT.Combiningdifferentunimodalmapsintoamultimodal
mapleadstocertaincharacteristics.Whatarethesecharacteristicsandhowdotheyforman
instructiveteachingsignalforthemapcalibrationprocessthatcanbedescribedbymeansof
supervisedspiketiming-dependentplasticity(sSTDP)?Toclosethefeedbackloop,wediscusshow
differentunimodalmapsadapttoacommonteachingsignal.

6.2.1HowdounisensorymapsdetermineiMT?
aTosensoryanswermothedalitfirstymasaitsjorneurquestion,onalwerepreseninitiallytationhavonetoaclarifyspatialthemap;essenseetialSec.terms.1.2.WOnerefersuchtoa
maponecaninterpretthefiringrateasthelikelihoodtofindastimulusatthepositionthatthe
neuronsencode.Inthismanner,sensorymapsrepresentpositionestimatorsorpopulationcodes
[60,116,131,227].Dependingonthestatisticsofthefiringprofile,wecanattributeameanµ
correspondingtotheestimatedpositionandastandarddeviationσdescribingtheestimator’s
accuracytoeachsensoryestimator.Theintegrationofdifferentsensorymapscanthereforebeseen
tobeequivalenttoacombinationofpositionestimators[2,67,102,113,126,165,202].
Althoughtheexactneuronalmechanismastohowdifferentmodalitiesareintegratedintoa
multimodalmap[60,61]isstilllargelyterraincognito,thefamilyofBayesiancombinationschemes
mathematicallydescribestheresultofoptimalmultimodalintegration.Forthesakeofsimplicity,
wewillfocusonthemaximumlikelihoodcombinationofonlytwomodalitiesinthefollowingtext,

77

IntegratedMultimodalTeaching

Thenamelyvisual,visionand(V)auditorandymapsauditionare(cA).Theharacterizedcasebofythreetheirmomeans,i.e.dalitiesisthepdescribositionedinestimates,AppendixµVC.2.for
visionandµAaudition.Furthermore,theaccuracyofthepositionestimatesisgivenbyσV(vision)
andσA(audition),andillustratedbyFig.6.3.

forFigureaudition,6.3:Multimovision,dalandinthemtegrationultimooftdalwomapunisensoryassumedasmaps.TheGaussianfiringfunctionsprofileswithofmeanneuronalµA/Vmaps/M
inandtegratedstandardmapisdeviationassumedσtoA/Vb/eMincantegratedbeinbymterpretedeansofasapmositionaximumlikestimatorselihoodofthemethodstimurepresenlus.Theted
byand(6.1)isthandusthe(6.2).most(A)Theprecisemofultimoalldalmaps.map(Bsho)IfwsthethesmalleunisensoryststandardmapsaredeviationshiftedσµMV<=σVµA,<σtheA
multimodalmapµM(redprofile)islocatedmorecloselytothemoreprecisemodality(blueprofile)
.(6.3)(see

ThemultimodalpositionestimateµMasamaximumlikelihoodcombinationfromcoinciding
visualandauditoryinputsisgivenby[135,144]
22σσµM=σA2+VσV2µA+σA2+AσV2µV(6.1)
inwithtegration,thestandardthemultimodeviatidalonσMestimatorofthemfeaturesultimothedalpsmallestositionpesossibletimate.Asstandardaconsequencedeviationofσgivoptimalen
MybσσVAσM=σA2+σV2.(6.2)
Foradetailedderivationofthetwoaboveequations,pleaseseeAppendixC.1.
pWositionsecanµalsobycomputemeansofthe(6.1)distancetobetweenthemultimodalpositionµMandtheunimodal
A/V|µM−µA/V|=σ2A/V|µA−µV|/σA2+σV2.(6.3)

78

dalMultimoIntegratedeachingT

Thatis,themultimodalandthereforetheteachingsignalislocatedmorecloselytothepositionof
themoreprecisemodality,e.g.,visionincomparisontoaudition,asshownhere.
Theaboveequations(6.1)to(6.3)illustratehowwecandescribeanintegratedmultimodal
mapataspecificpositioninspaceandtime.Nextwehavetotakeintoaccountthattheintegrated
multimodalmapwillprovidetheguidingsignalforunimodalmapformationandalignment.The
equationsofoptimalcombination,however,donotdescribehowunimodalmapschangeduringthe
learningprocess.Mapformationandalignmentareaneffectofsynapticplasticity(mathematically
describedbySTDP),aswewilldiscussinSec.6.2.2.Inthissectionwefocusonfundamental
characteristicsofthemultimodalmapthatfollowdirectlyfromoptimalmultimodalintegration.
Fornow,wepresumethattheunimodalmapsthatformthemultimodalmaparevariableintime.
Moreprecisely,ontheonehand,thepositionoftheunimodalmapsµA/V(t)canandwillvary
duringthecalibrationprocess.Ontheotherhand,themapscanimproveinprecisionsuchthat
σA/V(t)decreasesaswell.ThesetwoeffectsleadtoavariableiMTwithtime-dependantµM(t)
andσM(t)duringthecalibrationprocessoftheunimodalmaps.
Inshort,themultimodalmapispositionallyflexibleoverthecourseofthealignmentprocess
becauseitisthecombinationoftwoormoresensorymaps(6.1)to(6.3)thatthemselveschange
duringthelearningprocess.Wewillnowdiscussingreaterdetailwhatpositionalflexibilitymeans.
Wi.e.,ehathevecmeanµharacterizedofitsfiringeachprofile.mapinThthisus,wpapeerfirstbycitshangestandσard(Fig.6.4deviationA)andσandseethatestimaatorsensoryposition,map
Vthatbecomesmoreprecise(blueprofile)“attracts”thepositionofthemultimodalmap,i.e.,the
learningsignal.SensorymapsthatincreaseinaccuracygetmorestablepositionallyduringsSTDP
learningprocesses.Second,ashiftingsensorymap,forinstance,vision(blueprofile)inFig.6.4B,
inducesashiftofthemultisensorymaptowardsthestationarysensorymap(yellowprofile).The
upshotofiMTevenwithoutincludingeffectsofsSTDPinthecontextofmapalignmentisalready
thatamorestaticsensorymapattractstheteachingsignalandwillthereforehardlyadaptatall.
Insummary,wehaveanalyzedcharacteristicsoftheoptimallyintegratedmultimodalmapand
havetherebyshownthreeessentialaspects.First,themultimodalmapisalwaysmoreprecisethan
anyunimodalmap;cf.(6.2).Second,theoptimalmultisensoryinstructivesignalislocatedmore
closelytothepositionofthemoreprecisemodality;see(6.3).Third,onthebasisofanadaption
ofthemonosensorymapsthroughSTDPlearningrules,(6.1)and(6.2)describehowposition
andprecisionofthemultimodalteachervariesforgivenµA/V(t)andσA/V(t)oftheinputmaps.
Equippedwiththesecharacteristicsoftheteachersignal,wenowanalyzehowthecharacteristicsof
iMTinfluenceunimodalmapadaptationonthelevelofsynapticplasticity,thatis,inphysiological
andmathematicaltermsforsSTDPmapalignment.
6.2.2HowdoiMTcharacteristicsinfluencemapadaptation?
Intheprevioussectionwecouldseethatanintegratedmultimodalteacherhasaspecificvariance
andpositionthatbothdependonthecharacteristicsoftheinputmaps;cf.(6.1)and(6.2).In
addition,thesecharacteristicswillvaryduringmapcalibration.Inthissection,wethereforediscuss
howteacherswithdifferentaccuraciesandteachersthatareshiftingduringthelearningphase
t.alignmenmapaffectTheresultsaspresentedinthefollowingarebasedonnumericalsimulationsofaninhibition-
mediatedSTDPmapalignmentprocess.Thebasicstructureofourmodelconsistsofthethree
moteacherdalitiesmoadalituyditionTviaA,STDPsomatosensorylearningSrules;,andseevisionFig.V6.5.thatTheadaptfundamentothetalmprinciplesultimodalofinhibition-inhibitory
mediatedSTDPmapalignmenthavebeendescribedelaboratelybyFriedelandvanHemmen[78].
Allsensorymapsareone-dimensionalmapsofNPoissonneurons[145].Eachmodalityconsistsof
aninputmapwithaGaussianfiringprofile
fiA/S/V(y):=SA/S/Vexp−(y−xi)2/2σ2A/S/V(6.4)


79

IntegratedMultimodalTeaching

Figure6.4:Characteristicsofanintegratedmultimodalmap.DuringtheiMTlearningprocessthe
differentmodalities(µA/Vchanges)alignwiththeinstructivesignalprovidedbythemultimodal
map,andtheirreceptivefieldsbecomemoreprecise(σA/Vdecreases).Giventhatthemultimodal
mapisacombinationoftheunimodalsensorymaps,itsprecisionandpositionaredeterminedby
thesensorymapsandconsequentlychangeaswell.Therefore,wevaryonemodality(blueprofile)
inprecision(A)andposition(B)andleavetheothermodality(yellowprofile)unchanged.The
resultingdevelopmentofthemultimodalteacher(redprofile)isdepictedintheuppergraphs.(A)
Themodality(blueprofile)thatgetsmorepreciseattracts(6.1)andsharpens(6.2)themultimodal
teacher(redprofile).(B)Inaddition,themodality(blueprofile)thatalignswiththeteacher(red
profile),inducesthemultimodalmaptoshiftaswell.Thedistancebetweenthemultimodalteacher
andthestationarymap(yellowprofile)thereforechangesaswell(cf.(6.3)).

for0≤i≤N,withstimuluspositiony,thei-thmapneuron’spreferredpositionxi,standard
deviationσA/S/V,andSA/S/Vbeingthemaximalamplitudeoftheactivity.Fortwodifferentmaps,
e.g.audition(A)andvision(V),theaccuraciesdifferandσA=σV.Theinputlayerprojectsvia
all-to-allsynapsesJjA/S/V(t)ontoneuronjofthemaplayer.Inaddition,eachneuronofthemap
layerreceivesinhibitoryinputfromonecorrespondingneuronoftheteachermodalityT.The
synapsesconnectingteacherandmaparemodeledasstaticone-to-onesynapticconnections.
artificialWeinteacterruptherthatthemisnotultimodalgeneratedfeedbacfromklotheopmultimocompareddaltorepresenFig.6.1tationandbut6.2shobywssimtheessulatingentialan
characteristicsthatwederivedinSec.6.2.1.Aneuronpoftheteachermapthereforerespondstoa
stimulusatpositionyinformofan“inverted”Gaussianfunction
fpT(y):=ST1−exp−(y−xpT)2/2σT2(6.5)
withxpbeingthepreferredpositionofneuronp,STbeingthemaximalfiringactivityoftheteacher

80

eachingTdalMultimoIntegrated

Figure6.5:Modelarchitecture.Amultimodalobjectsuchasawaterdrophittingtheskinproduces
sensoryinputforaudition,somatosensory,andvision.WemodeltheinputbymeansofaGaussian
fiA/S/V(y)soastostimulatejneuronioftheinputlayer.Neuronsoftheinputlayerprojectthrough
all-to-allsynapticconnectionsJ(t)toneuronsjofthemaplayer.Thesesynapticconnections
willadapttoaligntheoutputofA/Sthe/VmaplayertotheinhibitoryteacherinputbymeansofsSTDP.
HereweusethismodelsetuptostudytheconsequencesofaniMT,whichisthemultimodal
representationofallsensorymodalities.

neurons,andσTbeingthewidthoftheteacherinfluence.Theprocesswehavejustdescribed
mathematically,whereateachersuppressestheactivityinthemaplayersatlocationsdifferent
fromtherealobject’spositiony,iscalledselectivedisinhibition[78,127].
Toanalyzethealignmentprocessweperformnumericalexperiments.Weintroduceanartificial
shiftΔinthesensorypositionofthemodality,e.g.,realizedthroughprismsintheKnudsen
[158,159]experimentonthebarnowl.Inmathematicalterms,wereplaceybyy+Δin(6.5).
Thesensorymodalities,say,auditionA,somatosensoryS,andvisionV,subsequentlychangetheir
synapticconnectionsbetweentheinputandthemaplayertocompensate(follow)theshift;see
6.5.Fig.Thesituationinournumericalexperimentscorrespondstothesituationofshiftingvision
bylenses[158,159].AsexplainedinSec.6.3.1,ashiftofvisioninducesadisplacementofthe
multimodalteacher.Consequently,ashiftofallsensorymapswithrespecttothepositionof
thecombinedmultimodalteacheroccurs,thatis,againstintuition,lensesshiftingvisionhardly
displacevisionwithrespecttotheteacherbutdisplaceallothersensorymapswithrespecttoour
multimodalteacher.Therefore,thebiologicalexperimentsandournumericalexperimentsagree.
Thechangeswithinthesynapticconnectivitypatternduringthealignmentprocesscanbe
describedandexplainedbySTDP[78,145].Toquantifychangeswithinthesynapticpatternwe
introducethemapestimationerrorEasaleast-squareerror.Itmeasureshowwellamapcan
reproducepositionalinformationofmsystematicallyvariedincomingsignals,

m
E=xi−xiT
=0i

2/m.(6.6)81

IntegratedMultimodalTeaching

Here,xidenotesthemeanpositionofthefiringprofileofthemap,andxiTrepresentsthetheoretical
position.ThesmallerthevaluefortheerrorE,thebetterthemapreproducesthereal(theoretical)
positionofanobject.TheparametersusedinournumericalexperimentsaregivenintheTableC.1.
Multimodalteacherwithvaryingvariance
Oneoftheessentialcharacteristicsofthemultimodalteacherisitsabilitytoimproveinprecision
duringthealignmentprocessasaconsequenceofdynamicunimodalmaps;see(6.2).Afterbirth,
mostanimalsfirsthavetolearnhowtointerpretthesensoryinformationtheyreceive.Allavailable
mapsasgivenbygeneticsareveryinaccurate[181,271].Furthermore,WallaceandStein[279]
observedthatmultisensoryneuronsincatsyoungerthan28daysreliablyrespondtoinputsfrom
morethanasinglemodalitybutdonotshowmultimodalenhancementordepression.These
observations,however,aremeasurementsonthelevelofsinglecells.Subsequentconclusionsinthe
contextofneuronalmapsandestimatorsareelusive.Therefore,itisfundamentallynecessaryto
showthatlearningstillfunctionsoratleaststartswithaninaccurateteacher.Beforewestudy
theeffectofadynamicteacheronsSTDPmapalignmentwewillclarifyhowteacheraccuracy
isdefined.Wenotethatthelessaccuratetheunimodalmaps,thelargerthevarianceofthe
multimodalteacher(6.2),thatis,theestimatedpositionandthustheteachingsignalisspatially
lessaccurate.Consequently,wemodeltheteacheraccuracybyaddingaGaussiandistributed
randomshiftΔTtothecorrectteacherposition.Asaresultofthenumericalexperimentswitha
varyingteachershiftΔT,weseethattheunimodalmapsadaptmoreslowlyandwithalargermap
estimationerrortoalessaccurateteachersignal;cf.Fig.6.6.Withinagiventimewindow,an
inaccurateteacherthereforeslowsdownbutdoesnotpreventthemapalignmentprocess.Wecan
thereforeconcludethattheiMTconceptdoesnotdependcruciallyonaconcreterealizationofthe
tegration.indalultimom

Figure6.6:InfluenceofteacherprecisionΔTonmapalignment.Wedepictthemapestimationerror
E(6.6)formapalignmentwithamultimodalteachervaryinginprecision.Themapestimation
errorbetweenteacherpositionandrepresentedpositiononthemapdropsfasterandtoalower
levelwithamoreprecise(lowerΔT)teacher.Theprecisionoftheteacherthereforeinfluencesthe
temporaldevelopmentofmapalignment.

Multimodalteacherwithdynamicallychangingposition
AsillustratedinFig.6.4,theintegratedmultimodalteacherpositionshiftsduringthealignment
process.Therefore,wemodelteacheractivitiesthatcontinuouslyshiftinspaceandanalyzehow
theirshiftingvelocitiesinfluencemapalignment.Forthetimestept=2000s,Fig.6.7showsthe
temporaldevelopmentofsensorymaps,i.e.theirsynapticpattern,modifiedbymeansofteaching

82

IntegratedeachingTdalMultimo

signalsshiftingwithdifferentvelocities.Wenotethatthedynamicsoftheteacheressentially
determinestowhichdegreeamapcanadapttotheteacher.Theslowertheteacherpositionshifts,
thebetteramapcanadapttotheteacher.Moreover,thecriticalvelocityuptowhichmapadaption
worksproperlydependsontheprecisionofthesensorymodalitythatisreflectedintheprecisionof
theinputmap.Alessprecisemodalitycanstilladapttoateacherwithagivenvelocity,whereasa
precisemapcannotfollowanymore;cf.Fig.6.7,changefromtoptobottom.

perfFigureorm6.7:edwithInfluenceinputofdynstandardamicallycdeviationhangingσ=teac0.03herp(topositionsline),σon=map0.05(balignmenottom)t.Simandteaculationsher
standarddeviationσT=0.02.Fordetailsregardingtheparameters,seeAppendixC.3.Weshift
theteachersignal,thatis,theinverseGaussianprofile,inspace.Theredlinesindicatethemean
positionoftheteacher.Inournumericalexperimentstheteacherpositionstartsshiftingfromthe
simdashedulatedredtimeline(ofΔ=20000.1s.).TheThedistancestraightbredetwlineeentheindicatestworedhowlinesfarthedepteacendsheronhastheshifteddifferentwithinshiftingthe
vtheelocitdiagoyofnal.theTheteacboherxatsignal.thetopTheleftshosynapticwsthepatternsynapticofthepatternsensorycausedmapsbyareasloinitiallywlyloshiftingcatedteacaroundher.
toWiththearighcertaint,thedelateacy,herthevelosynapticcitypatterndoubles.ofAstheamapconsequehasnce,adaptedthetomaptheadaptsteachermucmap.hlessByunsteptilpingthe
velocityreachesacriticallevel(toprightbox)wherethemapdoesnotadaptatall.Thevelocity
oftheteacherthereforeessentiallydeterminestowhichdegreeamapadaptstotheteacher.This
effectcanbecompensatedbydecreasingmapprecisionasrealizedfromthetoptothebottomline.
Wmap.ethenTheseecriticalthatvtheelocitysynapticofpatternnon-adaptationofalessisthereforeaccuratemaphigherstillforaloadaptsw-pvreerycisionwellmaptothethanteacforhera
map.high-precision

meansWeofconcludesSTDP.thatHowevdifferener,ttheteacteacherheraccuraciesaccuracywillnotinfluencespreventhettempalignmenoraltdevofdifferenelopmenttmapsand,btoy
somedegree,theaccuracyofthefinalstate.Wehaveseenthattheshiftingvelocityoftheteacher
pflexibleositionthandeterminesanaccuratetowhicmhapbdegreeecauseaitmapcancanadaptadapttoatothfasteretedyacher.namicAallesslycaccuratehangingteacmapher.ismoreThe
precisionofthedifferentcontributingunimodalmapsthereforeseemstodeterminehowtheyadapt
totheteachersignal.Inthenextsectionwewillstudytheseeffectsindetail.

83

IntegratedMultimodalTeaching

6.2.3HowdoesiMTcalibratedifferentunimodalmaps?
Havinganalyzedfirstthecombinationofanintegratedmultimodalmapasitarisesfromunimodal
andmapslastandstepthe(seeeffectFig.ofa6.2)howdynamicallythecessentialhangingcteacharacteristicsheronofmapthealignmendifferentt,wmoesimdalities,ulateinintheparticular,third
thedifferentaccuraciesofthesensorysystems,influencemapformationandalignment.
ResultsofoursimulationsofmapformationandalignmentareshowninFig.6.8.Basedon
thesenumericalexperimentsweobservethatmapformationandalignmentbymeansoftheiMTis,
inofmapgeneral,pformationossibleandforinputsalignmenoftdepdifferenendtontheaccuracies.accuracyHowevofer,thethesensoryqualityandsystem.temInporalpassing,progresswe
notethatwithinthealignmentprocess,ahigh-precisionmapreachesamodifiedsynapticpattern
thatisagainmoreprecisethanthatofalow-precisionmap.Moreimportantly,ahigh-precision
mapadaptsmoreslowlytotheteacherinputthanamapwithlow-precision.Inconclusion,the
highertheprecisionofasensorymap,themorestaticthemap.Wewillseeinthenextsectionhow
thesefindingscanexplainexperimentsofsensorymapformationandadaptation.

plotsFiguresho6.8:wtheInfluencemapofestimationvaryingerrorinputEasmapgivenprecisbyion(6.6)(σ)fromonmaunimopdalformationmapsinandcomparisonalignment.toThethe
teacher.Themapsvaryinprecision(inputstandarddeviationσ)asindicated.Thegraphsshow
isthemapenlargedtoestimationclearlyerrorshowEforwhichmapmapformationadapts(A)faster.andforBothmapplotsshoalignmenwtthat(B).mapsThebwithoxedahigherregion
precisionadaptmoreslowlytotheteacherinputthanlower-precisionmaps,butregainhigher
precisionafterthemapcalibration.

6.3Modelresultsandexperimentalfindings
AfterdetailedanalysisandbeingequippedwithadeeperunderstandingoftheiMTconcept,we
returntotheexperimentspresentedintheintroductionandreviewtheminthelightofouriMT
concept.Indoingso,wedistinguishbetweenthepreviouslymentionedexperimentsthatcanbe
explainedbyavision-guidedconceptofmapcalibrationandthosethatcannot.TheiMTconcept
wepresenthereoffersexplanationsforboth.
Inthefollowingwewillanalyzeagroupofexperimentsthatmotivatedtheideaofvisionas
thedominantguidingsensorymodality:theprismglassexperimentsofKnudsenandcoworkers
[127,158].Wereviewtheremainingexperimentswhiledenyingavision-guidedmapalignment.We
thenillustrateusingexperimentalevidenceasexamplesfromSteinetal.[277,279]howaniMT
conceptcouldwork.Finally,wemakesuggestionsandtestablepredictionsfornewexperiments
concept.iMTtheonbased

84

eachingTdalMultimoIntegrated

AnimalσAσVµµVA−−µµMAµµVV−−µµAM
Human1◦0.02◦99.96%0.04%
Barnowl2◦0.3◦97.8%2.2%
Cat8◦0.2◦99.93%0.06%
Table6.1:Visualandauditorylocalizationcapabilitiesofhuman[57,194],barnowl[106,157],and
cancat[b19e,42].distinguishedThevisualinoneresolutionsdegreeofaretheoriginallyvisualfield.givenWeinhavcyclesepassumederdegree,thei.e.,resolutionshowmantobyelinesthe
inverseofthesevalues.Onthebasisofthesevarianceswecombinethetwounimodalmapstoa
multimodalteacherandcalculatethedistanceofeachmaptothemultimodalmapdueto(6.3)in
comparisontothedistancebetweenthetwomodalities.

6.3.1Experimentsprovision-guidedmapformation
Themostprominentexperimentssupportingvision-guidedmapformationaretheshiftingexper-
imentsbyKnudsenetal.[127,158].Here,prismsshiftthevisualsystemofowlsbyacertain
angle.Afteralearningperiodtheneuronalprojectionsfromtheauditorymaparerearrangedto
compensateforthemisalignmentbetweenvisualandauditorymaps;seeSec.1.3intheintroduction.
Thevisualmap,incontrast,remainsconstantduringthewholealignmentprocess.
CanouriMTconceptreproducethedominanceofvisionwithinthelearningprocess?Toanswer
thisteacherquestionmapaswegivhaenvebysim(6.3)ulated.Wenotealignmenthattofthemapsvisualwithsystemdifferentusuallyvarianceshasaanmducdhistancehigherstospatialthe
resolution(aboutafewarcminutes)thanallothersensorysystems,e.g.theauditorysystem
(aboutafewdegrees);seeTable6.1fordetails.Wenowrevisitsynapticplasticityduringaprism
shiftexperimentfromthepointofviewofiMT.

•Initialstate(Fig.6.9,top)
Auditoryandvisualmapsareshiftedwithrespecttoeachotherbecauseofanartificialshift
ofthevisualsystemthatisintroducedbyprisms.Giventhatthemultimodalteachermap
isacombinationoftheunimodalmaps,itsactivitymeanhasshiftedaswell.Anadaptation
processofallunimodalmapsisinduced.Againstnormalintuition,shiftingvisiondoesnot
onlyresultinashiftbetweenvisionandthemultimodalteachingsignal;itinsteadresultsina
shiftofallsensorymapscomparedtothemultimodalteachermap.Asaconsequenceofthe
greaterprecisionofthevisualsystem,themultimodalteachermapismainlydeterminedby
thecharacteristicsofthevisualmap,thatisσML≈σVandµML≈µV;cf.(6.1)and(6.2)).
Equation(6.3)neatlyillustratesthattheshiftbetweenvisionasthemostprecisesensorysystem
andthusthemultimodalteachermapissmallerthanforanyothersystem.
•Intermediatestate(Fig.6.9,middle)
Synapticconnectionstotheauditorymapstartchangingtocompensatefortheshiftbetween
auditionandthemultimodalteacher.BasedonourpreviousstudiesaspresentedinSec.6.2.2
and6.2.3,weknowthatbecauseofitshigherprecision,thevisualmapadaptsmuchmoreslowly
thantheauditorymap.Theauditorymapthereforestartsadaptingtothemultimodalteacher.
Theteacherasacombinationofthetwomodalitiesthusshiftstowardthevisualmap.The
auditorymapadaptsagaintotheteacherinputandhencecontinuestoshifttowardthevisual
map,whichstaysalmoststatic;seeFig.6.4,B.
•Finalstate(Fig.6.9,bottom)
Thetwomapshavebeenrealigned.Thepositionofthevisualmaphashardlychanged,whereas

85

86

IntegratedMultimodalTeaching

Figure6.9:Numericalrealignmentexperiments.Weillustratetheshiftingprocessfortheauditory
map(A)withvarianceσA=0.07andthevisualmap(V)withσV=0.03.Thedistancesbetween
teacTheherarrayandplotsunimoshodalwthemappsynapticositionsareconnectivitchosenypatternaccordinglyfortovision(6.3)(leftforthecolumn)correspandondingauditionvariances.(right
mapcolumn).A,theThevisualsynaptmapicV,strengthsandthehereteacbyherranmgeapbetareweensho0wnandin1.theThemiddleactivitycolumn.profilesTheofthecolorcoauditorydeis
specifiedinthebottomrightcorner.Theplotillustrateshowsynapticplasticitydependsonthe
faraccuracylessthanofathemolessdality.Inaccurateconcreteauditoryterms,maptheA.moreprecisevisualmapVshiftsmoreslowlyand

eachingTdalMultimoIntegrated

theauditorymaphasshiftedalmostthewholedistanceinducedbytheprismsatthebeginning
t.erimenexptheofKnInudsensumandmary,othersour[iMT127,158concept].Hocanwever,reproducesimilarthenumericaldifferentstepsresultsofcanthebeshiftingobtainedexpforerimenanytstwofo
systemarbitraryonlysensoryreflectsmapsthewithmoregeneraldifferingvconceptariancesof.aTherefore,dominancetheofacobsecurrvacyedfollodominancewingofnaturallythefvisualrom
concept.iMTthe

6.3.2Experimentscontravision-guidedmapformation
Intheintroductionofthischapterwereviewedsomeexperimentsthatquestionavision-guided
ofitslearninghighprocess.precisionThebutanalysisthatofvisionourisnotconceptstatic.hasshoItwncanbthatemovisiondifiedplabysyaadyndominanamticallyrolecbhangingecause
multimodalteachermapandthereforebyanyunimodalmap[68,77,202,249,258].
Furthermore,theiMTconceptshouldbeabletoreproduceformationofunimodalmapsin
lovisuallycalizesoudeprivndasedpreciselyanimals.asStudiesindividuals[152,231with,234]normalshowvision.thatblindMeasuremenhumans,tsofcats,WallaceandandferretsSteincan
[277,279]supportthecriticalviewofvision-dominatedlearningbyshowingthatreceptivefields
butalreadyisshrinkdominatedatbytimesvisionwheninnocasesvisualinwhichneuronsitisaarevailabpresenle.t.Learningispossiblewithoutvision
butInitthehascaonmuctexthoflowoureriMTprecision.conceptFaormationmultimoofdaltheteacotherhermapsmapiscanthereforeexiststillwithoutpaossiblevisualwithinmap,
certainlimits;cf.Fig.6.6and[146,158,271].
animal,Inthetherebfolloywingstudyingtext,wewhetherwillsomreproduceatosensoryformationandofauditoryunimomdalapsmapscandevinealopvisuallywithoutdeprivvisioned,
i.e.,withaveryimpreciseteacher.
•InitialSomatosensorystate(Fig.and6.10,auditorytop)mapsareonlycoarselyprewired,andthevisualmapdoesnot
existimpreciseatall.aswTheell;mseeultimoSec.dal6.2.2mapforasadetails.combinationofsomatosensoryandauditorymapsisvery
•Intermediatestate(Fig.6.10,middle)
Evwhileentheirwithvaveryariancesdecimpreciserease.teacThehermapearlierthesimauditoryulationsandillustratethatsomatosensorythelearningmapsstartprotcessowithimprovane
impreciseteacherstartsveryslowly,correspondingtothetopmostcurveinFig.6.6.Nevertheless,
twtheounimosensorydalmapsmapsasaimproveconsequence.slightly,asWithdoesthethemimproultimovingdalteacmapherthat(seeisFig.acom6.6),thebinationoflearningthe
accelerates.cesspro•Finalstate(Fig.6.10,bottom)
Thetwomapsarealignedandhavereachedtheirbest-possibleresolutions.
Together,multimodalteachingillustratesthattheiMTconceptisasuccessfulconceptfor
incalibratingparallel,diasffreperentortedunimoindalexpmaps.erimentsInsucourhinastrinsic[277,279].learningprocessallmapsdevelopandimprove

6.4DiscussionandOutlook
IntheapplicationpresenoftciMThaptertowmapefhaveormationintroandducedthealignmencontceptleadsoftointhreetegratedfundamenMultimotallydalnewTeacideas.hing.First,The

87

88

IntegratedMultimodalTeaching

Figure6.10:Numericalmapformationexperimentswithanimprovingteacher.Activityprofilesof
theauditorymapA(σA=0.03),thesomatosensorymapS(σS=0.045),andtheteachermapare
showninthemiddlecolumn.Theblurredteacherprojectsontothemapswithaprecisionthatstarts
atΔT=0.1andthenreduceslinearlyintimetoΔT=0.02fort=7500s.Theblackbarsbelow
thefiring-activityplots(middlecolumn)denotetheregionswithintheteacherisprojectingforthe
correspondingtimestep.ThearrayplotsshowthesynapticconnectionpatternJAforaudition
(leftcolumn)andJSforthesomatosensorymodality(rightcolumn).Thesynapticstrengthsrange
between0and1.Theinitialsynapticpatternisonlycoarselyprewiredanditisadditionallyblurred
bymeansofrandomlydistributedsynapsestrengths.Thecolorcodeisspecifiedinthebottomright
corner.Despiteanimpreciseteacher,bothmapscandevelopproperly.

eachingTdalMultimoIntegrated

theintegratedteachercontainsallavailableinformationofthemapsinvolved.Second,theteacher
isintrinsictothesystembecauseitisgeneratedonlyfrominformationavailablewithinthesystem
ofmaps.Third,asaconsequence,theteacheritselfisdynamicwiththepotencytoshiftinposition
andtoimproveinprecisionduringthelearningprocess.
Moreover,thepresentedresultsprovidethebasicinsightthatthemorepreciseasensorysystem
is,themoreslowlyitadaptstoinstructivesignals.Thisnaturalconsequence,whichwedenoteas
dominanceofaccuracy,explainsobservationswhereadominanceofvisionwithinthealignment
processmaybewronglyinterpretedaspurelyvisuallydriven.
state-of-the-artthetoarisonCompHere,wehavesimulatedmultimodalteachercharacteristicsformapcalibrationonthelevelof
spikingneuronsandsynapticplasticity,namely,sSTDP.Theteachersignalinfluencestheunimodal
mapsbyselectivedisinhibitionasproposedbyanincreasingnumberofexperimentalfindings,
forinstance,intheICC-ICX-OTpathway[14];seealsoSec.1.3oftheintroduction.Taken
together,ouriMTconceptthereforeconnectstwofundamentalneurophysiologicalphenomena:
inhibition-mediatedmapcalibrationandmultimodalintegration.
Previoustheoreticalstudieshavefocusedononlypartsoftheseaspects.GelfandandMysore
[86,204]proposedmodelsforHebbianmapcalibrationbasedontheassumptionthatvisionserves
asteachersignaltomodify,forinstance,synapsesoftheauditoryprojectionpathway.Asalready
pointedoutintheintroductionofthischapter,visionasteachersignalneverthelesshastobe
questioned.SteinandWallace[277,279]thereforesuggestedaconnectionbetweenmultimodalmap
andmapformationincats.ThelatestexperimentalstudiesonbarnowlsfromBerganandKnudsen
[14]alsoshowthatcross-modaleffectsexistintheICXandmayinfluenceexperience-dependent
calibrationofconvergingrepresentations.
Includingtheseexperimentalfindings,Wittenetal.[287]developedatheoreticalmodelfor
bimodalmapalignment.Intheirmodel,thesynapsesofboththeauditoryandthevisualmap
areadaptedduetoHebbianlearning.Bythesemeans,themodelcanreproducethedominanceof
visionsuchthatthechannelwiththeweakerorbroaderreceptivefieldsalwaysexhibitsmostorall
oftheplasticity.However,thetheoreticalmodelofWittenetal.isexclusivelybasedonexcitatory
inputsandratecodes.Thelatterassumptionsarebothquiteimplausibleinthecontextofmap
alignmentinICXandOT.Concretely,experimentaldatafromBerganandKnudsen[14]suggest
thatinstructivesignalstotheICXaregatedbyinhibition.Inaddition,Kempteretal.[145]
demonstratedthattheassumptionsunderlyingrate-basedHebbianlearningarenotnecessarily
valid,especiallyintheauditorysystemofthebarnowl.
modelofIncludingmapthealignmenlastmentbytionedmeansexpoferimenselectivtaleresults,disinhibitionFriedelonettheal.lev[78el]ofdevspikingelopedaneurons.theoreticalThe
modelshowsthataninhibitoryteacherisessentialtorealignalreadycalibratedunimodalmaps.
However,themodelonlycontainsplasticitywithintheauditorysystem,thatis,thevisualsystem
isstaticandprovidestheguidingsignalforauditoryspacealignment.
ThepresentiMTmodelcombinesthefundamentalaspectsofthemodelsfromWittenetal.
[287]andFriedeletal.[78]:spike-timing-dependentlearningbasedonbothselectivedisinhibition
andfullplasticitywithinallcontributingmodalities,thatis,allavailableinformationisintegrated
intoamultimodalmapasafirststep.Inasecondstep,themultimodalmapprovidesaninstructive
signalformapcalibrationtoallmodalities.Therefore,theiMTmodelcanexplainthedominant
roleofvisioninmapalignment,plasticityinmodalitieswithoutvision,plasticitywithinthe
visualmodalityguidedbyothermodalities,plasticityofnon-visualmodalitiesguidedbyanother
non-visualmodality,andanycombinationofthesesettings.
TestablepredictionsoftheiMTconcept
GiventhatwehaveshownthatouriMTconceptcanreproducebothexperimentssupportingand
questioningvision-dominatedlearning,itisnowtimetooffersomesuggestionsregardinghowthe

89

IntegratedMultimodalTeaching

iMTconceptcanbeexperimentallyverified.Todistinguishvision-dominatedteachingfromiMT
wehavetobypassthedominanceofvision.Wecaneitherstudyartificialsituationsinwhichthe
precisionofsensorymapsisequal,investigateanimalswithapoorlydevelopedvisualsystem,or
observemapformationwhilemultimodalmapsareexperimentallydeactivated.
Tofirstequalizetheprecisionofsensorymaps,onecould,forinstance,performshiftingexperi-
mentswithprismsthatdisplaceandblurthevisualinputatthesametime.Second,studyingmap
formationandalignmentforanimalswithamoreprecisesensorysystemthanvisioncouldalso
identifyifthereisageneralprinciplesuchasdominanceofaccuracy.Athirdpossibilitywouldbe
todeactivatethemultimodalintegrationduringthelearningprocess.GiveniMT,deactivationof
theandmalignmenultimotdalshouldmap(s)failincorrespthisond(s)case.toInacondeactivtrast,ationforaoftheteacvision-dominatedhingmodalitteachy.ingMapscenario,formatione.g.
vision-guidedmapalignment,nonegativeeffectwouldbeobservedonthelearningprocess.
okOutlo

iMTFiguresetup6.11:shoOutlownoink:Fig.iMT6.5netbwyorkmeanssetupofwithincludingfeedbacktheconmultimonections.dalinOnecantegrationextendprocesstheoncurrenthet
levelpreviouslyofspikinginSec.6.2neurons.andThatdepictedis,weinFig.automatically6.2.First,incorptheporatetheost-synapticfeedbackcurrenmectshαanismgenerateddiscussedby
itheincomingspikesPfromtheneuronsiofthedifferentsensorymapsA/S/Varelogarithmically
summedup;Ij=ilogαi.Second,theprobabilitytoevokeaspikeinneuronjofthemultimodal
jmapofthecanmbeultimomodeleddaltomap.relateThird,expweonenusetiallyantointheterneuroncompletlayeerinputto“invcurrenert”tIthejinjectedexcitatoryintoactivitneurony
profiletoaninhibitorydisinhibitionsignalthat,fourth,guidesmapcalibrationofthesensorymaps;
sensorypathwayandinputisfadedascomparedtoFig.6.5.Inotherwords,withthenetwork
setupdescribedabove,wecanneuronallycalculateanoptimalmultimodalmapemployingallthe
characteristicswehaveusedfortheiMTinthischapterbymultiplyingtheactivityprofilesof
thesensorymapsonthelevelofspikingneurons;fordetailsseeAppendixC.Wehavealready
inimplemenprincipletedpandossibletestedbutthisthesetupfeedbacandkmecthehanresultsismloneedsokfurtherpromising.Mapclarification.formationandalignmentis

Insummary,wehaveintroducedthenewideaofanintrinsicmultimodallearningprocessand
insodoinghaverevealedinterestingnewaspectsofmultimodalmapformation.Thequestion
of(multimodal)mapformation,however,isnotyetfullyunderstood.Onthetheoreticalside,
futuremodelsshouldincludeatwo-wayfeedbackbetweenunimodalmapsandthemultimodal
teacher.Wehavealreadyimplementedandstartedtotestamoresophisticatedmodelonthelevel

90

eachingTdalMultimoIntegrated

ofspiking-neurons;seenetworksetupinFig.6.11.Inshort,spikesfromthesensorymapsA/Vare
summedlogarithmicallyprovidingtheinputcurrenttotheneuronsofthemultimodalmapMthat
aremodeledtorespondexponentiallytotheinputcurrent.Thiscorrespondstoamultiplication
ofspikesandmodelsoptimalmultimodalintegrationonalevelofspikingneurons;see(C.6)in
AppendixC.Theactivityofthemultimodalteacheristhen“inverted”viainter-neuronsand
projectedbacktothemapneuronsofthesensorysystemsA/Vtoprovideaninhibitoryteaching
signal.WearelookingforwardtoderiveanexplicitmodelofiMTincludingfullneuronalfeedback
dynamicswithoutartificialnormalizationterms.
Ontheexperimentalside,experimentsmaywelldeliverconvincingevidencefororagainst
aconnectionbetweenmultimodalprocessingandmapcalibration.Itwillbeinterestingtosee
whetherasystemofmapssuchaswhatisfoundintheSC/OT(seeFig.1.4)isabletocalibrate
intrinsically,andwhethermultimodalmapsplayakeyroleincalibration.

InpresentingtheconceptofiMT,wehavenotonlyaddressedthequestionofhowglobal
structurescanenablelocalmechanismssuchassynapticplasticitytoformcomplexbrainstructures,
e.g.,theSC/OT,wehavealsogivenanotherlineofevidencethatsensoryexperienceisthedriving
forceforbraindevelopment.Togiveafinalstatementbeyondthischapter,thisisreallyremarkable
sincethebrainexiststoprocessthequantitywhichdeterminesitsfunctioning,viz.,sensory
experience.Thatiswhywebelieveitissoimportanttounderstandsensoryprocessing.

91

App

endices

A.Appendix:OptimalMapformation

A.1Arecipeformakingmaps
TobringtolifethemathematicalframeworkofSec.2.2,wepresentaneasystep-by-step“recipe”
tofindtheoptimalconnectivityinarealisticbiologicalsetup:
•First,wederivethetransferfunctionhix(t)thatdeterminestheresponseofthedetectoritoa
stimuluspulsethatoccurredttimeunitsagoatpositionx.
•Next,wecalculatetheFouriertransformHixofthetransferfunctionhix(t).
•Wechoosesuitablevaluesofτandσ.Ingeneralthenoise-to-signalratioτcanbeassumed
tobemuchsmallerthan1foranymeasurablesignal.Incontrast,σneedstobeestimatedin
dependenceuponthesituationathand[76,75,252].
•WethencalculatethematrixentriesMijasgivenbyEq.(2.19)andinvertthemodelmatrix
.M•WemultiplytheinvertedmatrixM−1bythevectorHixsoastofindtheinputconnection
x.Lstrengthsi•Finally,wecalculatetheinverseFouriertransformofLixsoastofindtheconnectionstrengths
lix(t).

A.2Nonlinearitiesininformationprocessing
Thepresentedmodelassumesalinearrelationbetweenstimulusanddetectorresponse.Fora
numberofsensorysystems,however,wefindnon-linearitiesinthemappingprocess.First,the
transferfunctionhcanbeanon-linearfunctionh˜.Second,theneuronaldetectorresponsecanbe
non-linear,typicallylogarithmic[169,213,172,136,53].Incaseofalogarithmicresponseanda
non-lineartransferfunctionthebiologicaldetectorresponser˜hastoberewrittenfrom(2.1)as
tr˜i(t)=logdxdτsx(τ)h˜ix(t−τ).(A.1)
spaceall−∞Toapplyourmodelwefirsthavetoincorporateanadditionalcomputationalstepcancelingthe
logarithm.Inabiologicalsystemthiscanberealized,e.g.,byneuronswithexponentialfiring
behavior.Assumingsuchaneuronalstepr˜i(t)asin(A.1)reducesto(2.1)withanon-linearh˜.
Wecanlinearizeanon-lineartransferfunctionbyaredefinitionofthesignals→s˜.Thatis,we
identifyappropriatecharacteristicsofthestimulusthatarelinearlyrelatedtor.Forexample,
insteadoflookingattheheatdistributionT(x,t)wecanconsidertheintensitydistributionofthe
correspondingradiation∼T4(x,t)duetotheStefan-BoltzmannLaw.Inthisway,areasonable
redefinitionofdetectorresponseandsignalcanallowforanoptimallinearstimulusreconstruction.

95

A.Appendix:OptimalMapformation

A.3Remainingderivationstepsleadingto(2.16)
Inthefollowingweelaboratesomestepsskippedinthederivationof(2.16)inthemaintext.In
doingsowetakeadvantageofideasduetothecalclulusofvariations[138].Wethereforestartfrom
(2.6)asaconditiontominimizetheexpectationvalueofthequadraticerrorwithrespecttothe
optimalreversetransferfunctions,theconnectionstrengthsljx(t).Thisleadsto
∂[sx(t)−sˆx(t)]2
∂ljx(t)=0foreveryj.
⇔[sx(t)−sˆx(t)]∂sˆxx(t)=0.(A.2)
∂lj(t)
Inordertosolve(A.2),weexpandtheestimatesˆx(t)usingEqs.(2.3)and(2.4)fromthemaintext
givingsˆx=χilix+χiλix+dysyhiylix+syhiyλix+syηiylix
i+syηiyλix+ξyhiylix+ξyhiyλix+ξyηiylix+ξyηiyλix.(A.3)
Thevariationofsˆin(A.3)leadsto
∂∂lsˆx((tt))=χj+dysyhjy+syηjy+ξyhjy+ξyηjy(0).(A.4)
x
jAsbefore,weassumeontheonehandthatallnoisetermsaswellastheexpectationoftheinput
arestochasticallyindependentofeachother.Ontheotherhandweusethefactthatallnoise
termshavezeromeanandthesignalisself-averaging.Withthesetwoassumptions,theexpectation
valuesspartialˆs/∂landsˆ∂ˆs/∂lfrom(A.2)canbewritten
sx(t)∂sˆxx(t)=dysx(t)(syhjy)(0)(A.5)
∂lj(t)
andsˆx(t)∂∂lsˆx((tt))=(χilix)(t)χj(0)+dydy(syhiylix)(t)(syhjy)(0)
x
ji+(syηiylix)(t)(syηjy)(0)+(ξyhiylix)(t)(ξyhjy)(0)

+(ξyηiylix)(t)(ξyηjy)(0).(A.6)
Inthenextstepwesubstitutethecorrelationtermsasgivenin(2.10)ofthemaintext.Toillustrate
thecalculations,whichsimplify(A.5)and(A.6),weanalyzetwoisolatedtermsfrom(A.6)asan
example.Theothertermsaretreatedinasimilarway.Wefirstsimplify
dydy(syhiylix)(t)(syhjy)(0)
i=dydydτdτdτsy(t−τ−τ)hiy(τ)lix(τ)sy(−τ)hjy(τ).(A.7)
i

96

A.Appendix:OptimalMapformation
Exploitingthecorrelationassumptionsthisexpressionbecomes
µs2dydydτdτdτδ(y−y)δ(t−τ−τ+τ)hiy(τ)lix(τ)hjy(τ)
i=µs2dydτdτhiy(t−τ+τ)lix(τ)hjy(τ)
iyy=µs2dy((hilix)◦hj)(t)(A.8)
iwiththeopencircle◦denotingtheautocorrelationintegraldefinedin(2.13).Wefocusonthe
secondtermintheright-handsideof(A.6).Thatis,
dydy(syηiylix)(t)(syηjy)(0)

i=dydydτdτdτsy(t−τ−τ)ηiy(τ)lix(τ)sy(−τ)ηjy(τ),(A.9)

iwhichsimplifiesto
µs2ση2maxdydydτdτdτδ(y−y)δ(t−τ−τ+τ)δijδ(y−y)δ(τ−τ)lix(τ)
<yy||i0<τ<tmax
=µs2ση2maxdydτdτδ(−τ+t)ljx(τ)
|y|<ymax
<t<τ0=µs2ση2maxdydτljx(t).(A.10)
0|<τy|<y<tmax
Altogetherthefinalexpressionsfortheexpectationvaluesbecome
∂sˆx(t)
sx(t)x=µs2hjx(−t)(A.11)
∂lj(t)
andsˆx(t)x=σχ2ljx(t)+ση2(µs2+σξ2)maxdydτljx(t)+(µs2+σξ2)dy(hiylix)◦hjy(t).
∂sˆx(t)
∂lj(t)0|y<τ|<y<tmaxi
(A.12)Equation(A.2)thereforetransformsinto
maxljx(t)σχ2+(µs2+σξ2)|y|<ymaxdydτση2(A.13)
<t<τ0+(µs2+σξ2)dy(hilix)◦hj(−t)(A.14)
yy
i=µs2hjx(−t).(A.15)
InsertingtheparameterσandτandapplyingaFouriertransformationfinallyleadsto(2.16).
97

A.Appendix:OptimalMapformation

Inordertotestwhethertheextremumisindeedaminimum,wehavetocalculatethesecond
readshwhicariation,v∂2[sx(t)−sˆx(t)]2
∂ljx(t)2
∂x∂sˆx(t)x∂sˆx(t)
=2∂ljx(t)s(t)∂ljx(t)−sˆ(t)∂ljx(t)
=0+2σχ2+ση2(µs2+σξ2)|y|<ymaxdydτ+
0<τ<tmax
(µs2+σξ2)dydτhjy(τ)2.(A.16)
Sincethesquaresarepositive,soisthesecondderivative,andthustheextremumisaminimum.

signalblurredGaussianA.4Inthissubsectionwepresentanequationequivalentto(2.16)ofthemaintextsoastoderivean
expressionforaGaussianblurredsignal.Asin(A.17),arealisticsignalwouldfulfillsomekindof
Gaussianrelationfortheexpectationvalue
sx(t)sx(t)
22
=Aexp−|x−x|exp−|t−t|.(A.17)
2σx22σt2
Forthiscasewecan,analogouslytoappendixA.3,deriveanequationlike(2.16).Sinceforthe
signaltheGaussiancorrelations,however,replacethedeltafunctions,e.g.,in(A.10),integralsover
spaceandtimecannotbeevaluateddirectly.Insteadtheycanonlyberestrictedtotheregion
wheretheGaussianisnon-negligible.Denotingthesetemporalandspatiallimitsbytandwe
canderivetheanalogueto(2.16),viz.,
22
ddtAexp−||exp−|t|2hjx+(t+t)
2σx22σt
=σχ2ljx(t)+ση2σξ2maxdydτljx(t)+σξ2dy(hiylix)◦hjy(t)
0|y<τ|<y<tmaxi
t2
+ση2|y|<ymaxdyddτAexp−2σ2ljx(t+t)
tmax<t<τ0+dyddtAexp−||2exp−t2((hiylix)◦hjy+)(t+t).(A.18)
22
i2σx2σt
Theeffectoftheadditionalremainingspatio-temporalintegralsascomparedto(2.16)isasmoothen-
ingofthefinalreconstruction.Notonlyisthevalueataspecificpointinspaceandtime(y,t)
takenintoaccountbutneighboringpointsinanearlyareasurroundingitareincludedaswell.

98

A.Appendix:OptimalMapformation

PseudoinverseA.5ApseudoinversematrixBofanarbitrarym×nmatrixAwithitselements∈R,Cisageneralization
oftheinversematrix.FollowingMooreandPenrose[201,223]thepseudoinverseBisdefinedby
equations,four

ABA=A,(A.19a)
BAB=B,(A.19b)
(AB)T=AB,(A.19c)
(BA)T=BA,(A.19d)
wheretheoverbardenotesthecomplexconjugation.ThepseudoinverseisusedforamatrixA
thatisofincompleterangandthereforecannotbeinverteddirectlybutonlyduetoauxiliary
constructions(A.19a,b).Thelasttwoequations(A.19c,d)tellusthattheproductofmatrixA
withitspseudoinverseBisHermitian.
Oneofthemostfamousapplicationsofthepseudoinverseistocalculatetheleast-squaresolution
ofasystemoflinearequations.

Ax=b→xˆ=Bb(A.20)
GivenamatrixAandavectorb,theabovesolutionxˆ=BbminimizestheEuclideannorm
||Axˆ−b||.Iftheinversematrixexists,thepseudoinversereducestothenormalinversematrix.
Forageneralreviewandapplicationsthereaderisreferredto,e.g.,[3].

99

B.Appendix:Lateral-linePerception

Incalculatethefollothewing,twweo-dimensionalbrieflyenlighflowteninathe(pretendedx,y)-planeparadox:whereasWhywecanwcannoteusecalculatecomplexanalysisanalyticallyto
withonlyrotational-symmetrictwocoordinatessettings,involve.g.ed.flowaroundbodies,eventhoughthiscorrespondsalsotoasetup

B.1Planepotentialflow
Wedefinethepotentialflowasasettingwheretheflowfieldvcanbecalculatedbymeansofthe
velocitypotentialΦ:Rn→Rthroughv=Φ.Inthesesettingsrotv=0;theflowfieldisvortex
free.Aspecialcaseofpotentialflowistheplanepotentialflow.Here,oneoftheflowfieldcom-
ponentsv=(vx,vy,vz)iszero,e.g.vz=0.Inaddition,theremainingtwocomponentsare
independentofthecoordinatewhichcorrespondstotheflowfieldcomponentthatiszero,e.g.,
v=(vx(x,y,t),vy(x,y,t),0).Incaseofplanepotentialflow,thefollowingstatementsholdtrue
forthepotentialΦandthestreamfunctionΨ;thelatterisdefinedbymeansofvx=∂Ψ/∂yand
vy=−∂Ψ/∂x,
∂2Φ∂2Φ∂2Ψ∂2Ψ
∂x2+∂y2=0=∂x2+∂y2.(B.1)
Thatis,boththepotentialΦandthestreamfunctionΨsolvetheplaneLaplacianequationand
bothareconnectedthroughtheCauchy-Riemanndifferentialequations,
∂Φ∂Ψ∂Φ∂Ψ
∂x=∂yand∂y=−∂x.(B.2)
Forthatreason,wecandefineacomplexpotentialF(z):=Φ(x,y)+iΨ(x,y)withx+iy=z∈C
wherewecanderivethecomplexvelocityw(z),
dzdF=w(z)=vx(x,y)−ivy(x,y).(B.3)
Aswecanseeabove,itisalsocomposedofrealvaluedfunctions,namely,vxandvy.Indoingso,
wecananalyticallysolvemanyhydrodynamicproblemssuchastheflowaroundprofiles.

B.2Rotational-symmetricpotentialflow
toThevariouscomplexhydrodescriptiondynamicofhproblems.ydroItdynamicisobsettinviousgsisthatquitonlyeptowwerfulaso-dimensionalitprovidesissuesancanbanalyticaleaddressedaccess
withcompaosedoftwone-dimensionalorealcocomplexordinatesfux,ynctionorr,φ.sinceInthetuitivelycomplex,onecocanordinatethinkofz=xextending+iy=rtheexp(iφcomplex)is
formalismtothree-dimensionsincaseofsymmetriesthateffectivelyreducethethree-dimensional
problemdemonstrateatbhandelowtoaexplicitlytwforo-dimensionalrotational-symmetricsetup.Howevfloer,w.itisnotpossibletodosoaswewill

101

B.Appendix:Lateral-linePerception

Thevelocityfieldforrotational-symmetricflowisgivenbyv=(vr(r,z,t),vz(r,z,t),0).The
continuityequationdivv=0nowreads
1∂(rvr)+∂cz=0.(B.4)
z∂r∂rThecontinuityequationholdstrueforastreamfunctiondefinedbyrvr=−∂Ψ/∂zandrvz=
.r/∂Ψ∂Sincewearestudyingpotentialflowtheflowfieldhastobevortexfree,viz.,rotv=0.Thetwo
components(rotv)and(rotv)ofthecorrespondingvorticityrotvareobviouslyzero.According
tosymmetryreasons,zhowever,rthethirdcomponent(rotv)φisnotobviouslyzero.Thatis,wehave
toclaim2
(rotv)φ=∂∂vzr−∂∂vrz=−r1∂∂zΨ2+∂∂rr1∂∂rΨ=0.(B.5)
tIfwowecomequationsbineforthebconothtintheuitvyelocityequationpoten(B.4)tialΦandandrotvthe=str0eam(B.5)functionsimilarlyΨ,to(B.1)wecanderive
2222∂∂r2Φ+r1∂∂rΦ+∂∂z2Φ=0=∂∂r2Ψ−r1∂∂rΨ+∂∂z2Ψ.(B.6)
Aswecanseethelefthandsideandtherighthandsideof(B.6)arenotofthesametype.The
streamfunctiondoesnotfulfilltherotational-symmetricLaplacianequation,i.e,ΔΨ=0.
Inaddition,thestreamfunctionΨandthevelocitypotentialΦarenotconnectedthroughthe
Cauchy-Riemanndifferentialequations(B.2)but
∂Φ1∂Ψ∂Φ1∂Ψ
∂r=−r∂zand∂z=r∂r.(B.7)
ThusimaginarywepartcannotofinterpretanalyticalthefunctionstreamF(z)function=Φ(Ψx,y)and+itheΨ(x,vyelo),citinypconcreto,otentialofΦtheasthecomplexrealvandelocitthye
potentialF(z).

Insummary,thestreamfunctionΨandthevelocitypotentialΦdonotsolvethesamedifferential
equation(B.6)–theLaplacianequation–and,inaddition,theyarenotconnectedbytheCauchy-
Riemanndifferentialequations.Againstintuition,wethereforecannotusecomplexanalysisto
describerotational-symmetricflowandthusthree-dimensionalsetupsevenincaseofthisspecial
.symmetry

102

iMTendix:AppC.

C.1Optimalcombinationoftwomodalities
Thefiringprofileofaneuronalmapcanbeinterpretedasthelikelihoodoffindinganobjectata
specificposition;seeSec.1.2intheintroduction.Inotherwords,neuronalmapsencodeposition
estimatorsbymeansoftheirfiringprofiles.Amodality,forinstance,auditionAorvisionV,can
berepresentedthroughGaussianfiringprofiles
fA/V(x):=SA/Vexp−(x−µA/V)2/(2σ2A/V)(C.1)
withmappositionxandestimatedposition,i.e.,meanµA/V,standarddeviationσA/V,and
maximalamplitudeSA/Voftheactivityprofiles.Tocombinetwosuchestimatorstoamultimodal
estimateµMwehavetocalculateaweightedsumofthemeanvaluesµVandµAleadingto
µM=gVµV+gAµA(C.2)
withthenon-negativeweightsgVandgAfulfillinggV+gA=1.Thevarianceσ2Mofthecombined
readsestimatorσ2M=gV2σV2+(1−gV)2σA2.(C.3)
Next,wehavetochoosetheweightgVsoastominimizethevarianceσ2MwithrespecttogV;that
is,∂σM/∂gV=0.SolvingthelatterequationleadstogV=1−gA=σA2/(σA2+σV2)andfinallyto
σ2M=[σA2/(σA2+σV2)]2σV2+([σV2/(σA2+σV2)]2σA2=σV2σA2/(σA2+σV2).(C.4)
Wecaninserttheaboveresultinto(C.2)andcalculatetheoptimalcombinationscheme
µM=σA2/(σA2+σV2)µV+σV2/(σA2+σV2)µA.(C.5)
Formathematicaldetailsregardingthederivationseeelsewhere[276].
Theaboveformulaisabletodescriberesultsofneuronalprocessesbutwithoutexplaininghow
theyareachieved.Inalmostallcasesitispracticallyunknownhowtheneuronalsystemcomputes
anoptimalestimate.Inthecontextofneuronalmaps,however,asimplemultiplicationofthe
activityprofilesoftheinvolvedmapscanneuronallyrealizetheaboveoptimalcombinationscheme.
Themultimodalactivityprofileisthengivenby
fM(x)=SAexp−(x−µA)2/(2σA2)SVexp−(x−µV)2/(2σV2)

=SASVexp−1/2(µA−µV)2/σV2+σA2(C.6)
factorscaling2
exp−1/2σV2+σA2/σA2σV2x−σV2µA+σA2µV/σV2+σA2.(C.7)
1/σ2MµM

StandarddeviationσMandmeanµMofthemultimodalprofileareidenticaltotheoptimalvalues
wehavederivedabove;cf.(C.4)and(C.5).

103

iMTendix:AppC.

C.2Optimalcombinationofthreemodalities

WeextendtheoptimalcombinationschemefortwomodalitiesasderivedinSec.C.1tothree
modalities.ThemultimodalestimateµMofthreemodalities,say,auditionA,visionV,and
somatosensorySrepresentedbytheirmeanvaluesµA,µV,andµSwiththeirstandarddeviations
σA,σV,andσSisgivenby
σ2σ2σ2σ2σ2σ2
µM=VSµA+ASµV+AVµS,
σA2σV2+σA2σS2+σV2σS2σA2σV2+σA2σS2+σV2σS2σA2σV2+σA2σS2+σV2σS2
(C.8)

aconvexcombinationofµA,µV,andµS.ThecombinedmaximumlikelihoodestimateµMfeatures
thesmallestpossiblestandarddeviationσMgivenby
MσA2σV2+σA2σS2+σV2σS2
σ=σAσVσS.(C.9)
Thedistancebetweenthemultimodalestimateandaunimodalone,forinstanceaudition,isgiven
yb

22σA2σV2+σA2σS2+σV2σS2
t|µM−µA|=σA2σS|µA−µV|+σV|µA−µS|.(C.10)
FordistancesbetweenthemultimodalestimateandµVorµSonecaninterchangeAwithVorS.

104

AppC.iMTendix:

C.3ModelParameters
InTableC.1wesummarizethemodelparametersasusedinournumericalexperiments(seemain
text).

aluevparameternumberofmapneuronsN=100
synapseslearningminimalstrengthJmin=0.0
maximalstrengthJmax=0.25
maximalinitialstrength(J0)max=0.25
strengthofteachersynapsesJT=−1
timeonserespostsynapticptoinputspikeτTI=10ms
toteacherspikeτ=25ms
inputamplitudeS=50s−1
teacheramplitudeATA/V=/S100s−1
teacherwidthσT=0.025
shiftmaps–teacherΔ=0.2
learningtriallengthT=0.5s
simlearningulationtimparameterestepηΔt==3∙0.105−ms6
weightchangein
upupononinputoutputspikspikeewwout==1.−510.0
TableparametersC.1:ofMothedellearningparameterswindoaswusearedintakourennfromumerical[78]withexperimendifferingtsor(seemainadditionaltext).parametersShapeandas
e.voabindicated

105

Bibliography

Bibliography[1]Abbott,L.F.&Nelson,S.B.(2000)Synapticplasticity:tamingthebeast,NatNeurosci3
1178–1183.[2]Alais,D.&Burr,D.(2004)TheVentriloquisteffectresultsfromnear-optimalbimodal
integration,CurrBiol14257–262.
[3]Albert,A.(1972)RegressionandtheMoore-Penrosepseudoinverse(Academic,NewYork).
[4]Amemiya,F.,Goris,R.C.,Masuda,Y.,Kishida,R.,etal.(1995)Thesurfacearchitectureof
snakeinfraredreceptororgans,BiomedRes-Tokyo16411–421.
[5]Amemiya,F.,Nakano,M.,Goris,R.C.,Kadota,T.,etal.(1999)Ultrastructureofthe
crotalinesnakeinfraredpitreceptors:Semconfirmationoftemfindings,AnatRec254
107–115.[6]Amemiya,F.,Tatsuo,U.,Goris,R.C.,&Kusunoki,Y.T.(1996)Ultrastructureofthecrotaline
snakeinfraredpitreceptors:Semconfirmationoftemfindings,AnatRec246135–146.
[7]Anastasio,T.J.,Patton,P.E.,&Belkacem-Boussaid,K.(2000)Usingbayesruletomodel
multisensoryenhancementinthesuperiorcolliculus,NeuralComp121165–1187.
[8]Bakken,G.S.(2007)AcritiqueofthermalmodelingofsnakeinfraredreceptionbyJones,
LynnandStone(2001),JTheorBiol244167–168.
[9]Bakken,G.S.&Krochmal,A.R.(2007)Theimagingpropertiesandsensitivityofthefacialpits
ofpitvipersasdeterminedbyopticalandheat-transferanalysis,JExpBiol2102801–2810.
[10]Bartels,M.,M¨unz,H.,&Claas,B.(1990)Representationofthelaterallineandelectrosensory
systemsinthemidbrainoftheaxolotl,Ambystomamexicanum,JCompPhysiolA167
347–356.[11]Beauchamp,M.S.,Argall,B.D.,Bodurka,J.,Duyn,J.H.,etal.(2004)Unravelingmultisensory
integration:patchyorganizationwithinhumanSTSmultisensorycortex,NatNeurosci7
1190–1192.[12]Ben-Israel,A.&Greville,T.N.E.(2003)GeneralizedInverses:TheoryandApplications
(Springer,NewYork,NY),2ndedition.
[13]Benson,D.L.,Colman,D.R.,&Huntley,G.W.(2001)Molecules,mapsandsynapsespecificity,
899–909.2NeurosciRevNat[14]Bergan,J.F.&Knudsen,E.I.(2009)Visualmodulationofauditoryresponsesintheowl
inferiorcolliculus,JNeurophysiol1012924–2933.
[15]Bi,G.Q.(2002)Spatiotemporalspecicityofsynapticplasticity:cellularrulesandmechanisms,
319–332.87ernCybBiol[16]Bi,DepG.Q.endence&Ponoo,spikeM.M.timing,(1998)synapticSynapticstrength,modificationsandpinostsynapticculturedcellhipptypoe,JcampalNeuroscineurons:18
472.464–1010[17]Bi,G.Q.&Poo,M.M.(2001)Synapticmodificationbycorrelatedactivity:Hebb’spostulate
revisited,AnnuRevNeurosci24139–166.
[18]Bialek,W.,Rieke,F.,deRuytervanSteveninck,R.R.,&Warland,D.(1991)Readinga
neuralcode,Science2521854–1857.
[19]Bisti,S.&Maffei,L.(1974)Behaviouralcontrastsensitivityofthecatinvariousvisual
meridians,JPhysiol241201–210.
[20]Blauert,J.(1999)SpatialHearing(MITPress,Cambridge,MA).
[21]Bleckmann,H.(1994)Receptionofhydrodynamicstimuliinaquaticandsemiaquaticanimals
her).(Fisc[22]Boll,S.F.(1979)Suppressionofacousticnoiseinspeechusingspectralsubtraction,IEEET
AcoustSpeech27113–20.
[23]Braucdynamicshi,S.,andOrio,Pelectroph.,&Latorre,ysiologicalR.analysis(2004)ofCluescoldtoreceptorunderstandingtrpm8,ProcoldcNatlsensation:AcadSciThermo-USA
499.494–1515101

107

Bibliography

[24]Braun,H.A.&Hensel,H.(1980)Staticanddynamicdischargepatternsofburstingcold
[25]fibBraun,ersrelatedH.A.,Sctoh¨hafer,ypK.,othetical&Wissing,receptorH.mec(1984)hanisms,TheorienPflugundArchMoEurdelleJPhzumy¨ub3861–9.ertragungsver-
haltenthermosensitiverRezeptoren,FunktBiolMed326–36.
[26]Braun,H.B.,Dewald,M.,Sch¨afer,K.,Voigt,K.,etal.(1999)Low-dimensionaldynamicsin
sensorybiology2:Facialcoldreceptorsoftherat,JComputNeurosci717–32.
[27]Braun,H.B.,Huber,M.T.,Dewald,M.,Sch¨afer,K.,etal.(1998)Computersimulationsof
neuronalsignaltransduction:Theroleofnonlineardynamicsandnoise,IntJBifurcatChaos
881–889.8[28]Braun,W.,Eckhardt,B.,Braun,H.A.,&Huber,M.(2000)Phase-spacestructureofa
thermoreceptor,PhysRevE626352–6361.
[29]Bregman,A.S.(1990)AuditorySceneAnalysis(MITPress,Cambridge,MA).
[30]Bresciani,J.-P.,Dammeier,F.,&Ernst,M.O.(2008)Tri-modalintegrationofvisual,tactile
andauditorysignalsfortheperceptionofsequencesofevents,BrainResBul75(6)753–
760.[31]Bresciani,J.-P.,Ernst,M.O.,Drewing,K.,Bouyer,G.,etal.(2005)Feelingwhatyouhear:
auditorysignalscanmodulatetactiletapsperception,ExpBrainRes162172–180.
[32]vanBrunt,B.(2000)Thecalculusofvariations(Springer,Heidelberg).
[33]Bullock,T.H.&Diecke,F.P.J.(1956)Propertiesofaninfra-redreceptor,JPhysiol134
47–87.[34]Buonomano,D.V.&Merzenich,M.M.(2004)Corticalplasticity:Fromsynapsestomaps,
AnnuRevNeurosci21149–186.
[35]B¨urck,M.,Friedel,P.,Sichert,A.B.,Vossen,C.,etal.(2010)Optimalityinmono-and
multisensorymapformation,BiolCybern1031–20.
[36]B¨urck,M.&vanHemmen,J.L.(2007)Modelingthecochlearnucleus:Asiteformonaural
echosuppression?,JAcoustSocAm122(4)2226–2235.
[37]B¨urck,M.&vanHemmen,J.L.(2009)Neuronalidentificationofsignalperiodicitybybalanced
inhibition,BiolCybern100261–270.
[38]Campbell,A.L.,deCockBunning,T.,Stone,M.O.,Church,D.,etal.(1999)Surface
ultrastructureofpitorgan,spectacle,andnonpitorganepidermisofinfraredimagingboid
snakes:Ascanningprobeandscanningelectronmicroscopystudy,JStructBiol126105–120.
[39]vonCampenhausen,C.,Riess,I.,&Weissert,R.(1981)Detectionofstationaryobjectsby
theblindcavefishAnoptichtysjordani,JCompPhysiolA143369–374.
[40]Carr,C.E.,Maler,L.,&Sas,E.(1982)Peripheralorganizationandcetralprojectionsofthe
electrosensorynervesingymnotiformfish,JCompNeurol211139–153.
[41]Carr,R.W.,Pianova,S.,McKemy,D.D.,&Brock,J.A.(2009)Actionpotentialinitiation
intheperipheralterminalsofcold-sensitiveneuronsinnervatingtheguinea-pigcornea,J
1249–1264.587ysiolPh[42]Casseday,J.H.&Neff,W.D.(1973)Psychophysicalstudiesofmonkeyvision:III.Spatial
luminancecontrastsensitivitytestsofmacaqueandhumanobservers,JAcoustSocAm
365–372.54(2)[43]Catania,K.C.,Leitch,D.B.,&Gauthier,D.(2009)Functionoftheappendagesintentacled
snakes(Erpetontentaculatus),JExpBiol213359–367.
[44]Claas,B.(1994)Removalofeyesinearlylarvalstagesalterstheresponseoftheclawedtoad,
[45]Clegg,XenopusJ.C.laevis,(1968)tosurfaceCalculuswaofves,PhvariationsysiolBeh(Oliv56erand423–428.Boyd,Edinburgh).
[46]Clifton,R.K.(1987)Breakdownofechosuppressionintheprecedenceeffect,JAcoustSoc
1834–1835.82(5)Am[47]deCockBuning,T.(1984)Atheoreticalapproachtotheheatsensitivepitorgansofsnakes,
509–529.111BiolTheorJ[48]deCockBuning,T.,Terashima,S.I.,&Goris,R.C.(1981)Crotalinepitorgansanalyzedas

108

Bibliography

warmreceptors,CellMolNeurobiol169–85.
[49]deCockBuning,T.,Terashima,S.I.,&Goris,R.C.(1981)Pythonpitorgansanalyzedas
warmreceptors,CellMolNeurobiol1271–278.
[50]Cohen,M.L.(1977)Measurementofthethermalpropertiesofhumanskin.Areview,JInvest
333–338.69Dermatol[51]Cohen,Y.E.&Knudsen,E.I.(1999)Mapsversusclusters:differentrepresentationsofauditory
spaceinthemidbrainandforebrain,TrendsNeurosci22128–135.
[52]Coombs,S.,Anderson,R.,Braun,C.B.,&Grosenbaugh,M.(2007)Thehydrodynamic
footprintofbenthic,sedentaryfishinundirectionalflow,JAcoustSocAm1221227–1237.
[53]StevCopenselli,Mand.,RoWebque,er-FecA.C.,hnerOlivlawseira,areR.F.,transfer&Kinoucfunctionshi,O.of(2002)excitablePhysicsmedia,ofPhpsycysRhophevEysics:65
901.60[54]´Cur˘ci´c-Blake,B.&vanNetten,S.M.(2006)Sourcelocationencodinginthefishlateralline
canal,JExpBiol2091548–1559.
[55]Dan,Y.&Poo,M.M.(2004)Spiketiming-dependentplasticityofneuralcircuits,Neuron44
23–30.[56]Davison,A.P.&Fr´egnac,Y.(2006)Learningcross-modalspatialtransformationsthrough
spiketiming-dependentplasticity,JNeurosci265604–5615.
[57]DeValois,R.,Morgan,H.,&Snodderly,D.(1974)Psychophysicalstudiesofmonkeyvision:
Iii.spatialluminancecontrastsensitivitytestsofmacaqueandhumanobservers,VisionRes
75–81.14[58]Dehaene,S.&Cohen,L.(2007)Culturalrecyclingofcorticalmaps,Neuron56384–398.
[59]Deneve,S.,Latham,P.E.,&Pouget,A.(1999)Readingpopulationcodes:aneuralimple-
mentationofidealobservers,NatNeurosci2(8)740–745.
[60]Deneve,S.,Latham,P.E.,&Pouget,A.(2001)Efficientcomputationandcueintegration
withnoisypopulationcodes,NatNeurosci4(8)826–831.
[61]Deneve,S.&Pouget,A.(2004)Bayesianmultisensoryintegrationandcross-modalspatial
links,JPhysiology-Paris98249–258.
[62]Dr¨ager,U.C.&Hubel,D.H.(1975)Responsestovisualstimulationandrelationshipbetween
visual,auditory,andsomatosensoryinputsinmousesuperiorcolliculus,JNeurphysiol38
690–713.[63]Ebert,J.&Westhoff,G.(2006)Behaviouralexaminationoftheinfraredsensitivityof
rattlesnakes(Crotalusatrox),JCompPhysiolA192941–947.
[64]Eckertf,E.R.G.&Carlson,WalterO.(1961)Naturalconvectioninanairlayerenclosed
betweentwoverticalplateswithdifferenttemperatures,IntJHeatMassTransf2106–120.
[65]Eckhorn,R.,Bauer,R.,Jordan,W.,Brosch,M.,etal.(1988)Coherentoscillations:A
mechanismoffeaturelinkinginthevisualcortex?,BiolCybern60121–130.
[66]Ephraim,Y.&Malah,D.(2004)SpeechEnhancementUsingaMinimumMean-SquareError
Short-TimeSpectralAmplitudeEstimator,IEEETAcoustSpeech321109–21.
[67]Ernst,M.O.&Banks,M.S.(2002)Humansintegratevisualandhapticinformationina
statisticallyoptimalfashion,Nature415429–433.
[68]Ernst,M.O.,Banks,M.S.,&B¨ulthoff,H.(2000)Touchcanchangevisualslantperception,
69–73.3NeurosciNat[69]Ernst,M.O.&B¨ulthoff,H.(2004)Mergingthesensesintoarobustpercept,TrendsCognSci
162–169.8[70]Evans,L.C.(1998)PartialDifferentialEquations,volume19ofGraduateSeriesinMathe-
matics(AmericanMathematicalSociety,Providence,RI).
[71]Faisal,A.A.,Selen,L.P.J.,&Wolpert,D.M.(2008)Noiseinthenervoussystem,NatRev
292–303.9Neurosci[72]Fiala,D.,Lomas,K.J.,&Stohrer,M.(1999)Acomputermodelofhumanthermoregulationfor
awiderangeofenvironmentalconditions:thepassivesystem,JApplPhysiol871957–1972.

109

Bibliography

[73]Flanagan,J.L.&Lummis,R.C.(1970)Signalprocessingtoreducemultipathdistortionin
smallrooms,JAcoustSocAm47(6)1475–81.
[74]Franosch,J.-M.P.,Lingenheil,M.,&vanHemmen,J.L.(2005)Howafrogcanlearnwhatis
whereinthedark,PhysRevLett95078106.
[75]Franosch,J.-M.P.,Sichert,A.B.,Suttner,M.D.,&vanHemmen,J.L.(2005)Estimating
positionandvelocityofasubmergedmovingobjectbytheclawedfrogXenopusandby
fish—Acyberneticapproach,BiolCybern93231–238.
[76]Franosch,J.-M.P.,Sobotka,M.C.,Elepfandt,A.,&vanHemmen,J.L.(2003)Minimalmodel
ofpreylocalizationthroughthelateral-linesystem,PhysRevLett91158101.
[77]Frens,M.A.&vanOpstal,A.J.(1998)Visual-auditoryinteractionsmodulatesaccade-related
activityinmonkeysuperiorcolliculus,BrainRes46211–224.
[78]Friedel,P.&vanHemmen,J.L.(2008)Inhibition,notexcitation,isthekeytomultimodal
sensoryintegration,BiolCybern98597–618.
[79]Furuya,K.&Kataoka,A.(2007)Robustspeechdereverberationusingmultichannelblind
deconvolutionwithspectralsubtraction,IEEETAudioSpeech151579–91.
[80]Galan,R.F.,Ermentrout,G.B.,&Urban,N.N.(2007)Reliabilityandstochasticsynchroniza-
tionintypeivs.typeiineuraloscillators,Neurocomputing702102–2106.
[81]Galan,R.F.,Fourcaud-Trocme,N.,Ermentrout,G.B.,&Urban,N.N.(2006)Correlation-
inducedsynchronizationofoscillationsinolfactorybulbneurons,JNeurosci263646–3655.
[82]Gancarz,G.&Grossberg,S.(1999)Aneuralmodelofsaccadiceyemovementcontrolexplains
task-specificadaptation,VisionRes393123–3143.
[83]Gardner,J.L.,Merriam,E.P.,Movshon,J.A.,&Heeger,D.J.(2008)Mapsofvisualspacein
humanoccipitalcortexareretinotopic,notspatiotopic,JNeurosci283988–3999.
[84]Geisler,C.D.(1990)FromSoundtoSynapse:PhysiologyoftheMammalianEar(Oxford
Oxford).Press,yersitUniv[85]Gelfand,I.M.&Fomin,S.V.(1963)Calculusofvariations(Prentice-Hall,EnglewoodCliffs,
NY).[86]Gelfand,J.J.,Pearson,J.C.,Spence,C.D.,&Sullivan,W.E.(1988)Multisensorintegration
inbiologicalsystems(IEEEPress,NewYork).
[87]Gerstner,W.,Kempter,R.,vanHemmen,J.L.,&Wagner,H.(1996)Aneuronallearning
ruleforsub-millisecondtemporalcoding,Nature38376–78.
[88]Gerstner,W.&Kistler,W.(2002)SpikingNeuronModels(CambridgeUniversityPress,
bridge).Cam[89]Gerstner,W.,Kreiter,A.K.,Markram,H.,&Herz,A.V.M.(1997)Neuralcodes:Firingrates
[90]andGillespie,beyond,B.W.,ProcMalvNatlar,AcadH.S.,Sci&USA94Florencio,12740–12D.A.F.741.(2001)Speechdereverberationvia
maximum-kurtosissubbandadaptivefilltering,ProcIEEEIntConfAcoustSpeechSig-
3701–4.6cesspronal[91]Gonzalez,R.C.&Woods,R.E.(2002)DigitalImageProcessing(PrenticeHall,UpperSaddle
edition.2ndNJ),er,Riv[92]Goris,R.C.,Nakano,M.,Atobe,Y.,Kadota,T.,etal.(2000)Nervouscontrolofbloodflow
microkineticsintheinfraredorgansofpitvipers,AutonNeurosci-Basic8498–106.
[93]Goris,R.C.&Nomoto,M.(1967)InfraredReceptioninOrientalCrotalineSnake,Comp
BiochemPhysiol23879–892.
[94]Goris,ReceptorsR.C.in&aTCrotalineerashima,SnakS.I.e,T(1973)rimeresurusCentralFlaRespvovionseridis,toJExpInfra-redBiolStim58ulation59–76.ofthePit
[95]Goulet,J.,Engelmann,J.,Chagnaud,B.P.,Franosch,J.-M.P.,etal.(2008)Objectlocalization
throughthelaterallinesystemoffish:Theoryandexperiment,JCompPhysiolA1941–17.
[96]Grace,M.S.,Woodward,O.M.,Church,D.R.,&Calisch,G.(2001)Preytargetingbythe
infrare-imagingsnakePython:effectsofexperimentalandcongenitalvisualdeprivation,
BehavBrainRes11923–31.

110

Bibliography

[97]Gracheva,E.O.,Ingolia,N.T.,Kelly,Y.M.,Cordero-Morales,J.F.,etal.(2010)Molecular
basisofinfrareddetectionbysnakes,Nature4641006–1012.
[98]Gray,C.M.,K¨onig,P.,Engel,A.K.,&Singer,W.(1989)Oscillatoryresponsesincatvisual
cortexexhibitinter-columnarsynchronizationwhichreflectsglobalstimulusproperties,Nature
334–337.338[99]Grossbadaptiveerg,sS.,accadicRobeyerts,emoK.,vemenAguilar,tconM.,trol&byBullosupceriork,D.colliculus,(1997)AJneuralNeuroscimodel24ofm9706–9725.ultimodal
[100]Grothe,B.,Covey,E.,&Casseday,J.H.(2001)Medialsuperioroliveofthebigbrownbat:
Neuronalresponsestopuretones,amplitudemodulations,andpulsetrains,JNeurohysiol86
2219–2230.[101]Grothe,B.&Neuweiler,G.(2000)Thefunctionofthemedialsuperioroliveinsmallmammals:
[102]Gu,tempY.,oralAngelaki,receptiveD.E.,fieldsin&auditoryDeAngelis,analysis,G.C.J(2008)CompNeuralPhysiolAcorrelates186of413–423.multisensorycue
integrationinmacaqueMSTd,NatNeurosci111201–1210.
[103]Gutfreund,Y.,Zheng,W.,&Knudsen,E.I.(2002)Gatedvisualinputtothecentralauditory
1556–1559.297Sciencesystem,[104]Hafed,Z.M.,Goffart,L.,&Krauzlis,R.J.(2008)Superiorcolliculusinactivationcausesstable
offsetsineyepositionduringtracking,JNeurosci288124–8137.
[105]H¨ansel,E.&Schmidt,G.(editors)(2004)AcousticEchoandNoiseControl–APractical
Approach(Wiley,Hoboken).
[106]Harmening,W.M.,Nikolay,P.,Orlowski,J.,&Wagner,H.(2009)Spatialcontrastsensitivity
andgratingacuityofbarnowls,JVision9(7)1–12.
[107]Harris,G.G.(1964)Considerationsonthephysicsofsoundproductionbyfishes,inW.N.
Tavolga(editor),MarineBio-Acoustics,pp.233–247(PergamonPress,Oxford).
[108]Harris,G.G.,Flanagan,J.L.,&Watson,B.J.(1963)Binauralinteractionofaclickwitha
clickpair,JAcoustSocAm35672–678.
[109]Hartline,P.H.,Kass,L.,&Loop,M.S.(1978)MergingofmodalitiesintheopticTectum:
Infraredandvisualintegrationinrattlesnakes,Science1991225–1229.
[110]Hasan,K.M.,Salahuddin,S.,&Khan,R.M.(2004)Reducingsignal-biasfromMADestimated
noiselevelforDCTspeechenhancement,SignalProcess84151–62.
[111]Hassan,E.l.-S.(1985)Mathematicalanalysisofthestimulusforthelaterallineorgan,Biol
23–36.52ernCyb[112]Hebb,D.O.(1949)Theorganizationofbehavior:aneuropsychologicaltheory(Wiley,New
ork).Y[113]Helbig,H.B.&Ernst,M.O.(2007)Optimalintegrationofshapeinformationfromvisionand
touch,ExpBrainRes179595–606.
[114]Helmholtz,H.(1879)DieThatsacheninderWahrnehmung(AugustHirschwald,Berlin).
[115]vanHemmen,J.L.(2001)Theoryofsynapticplasticity,inFMoss&SGielen(editors),
HandbookofBiophysics,pp.771–823(Elsevier,Amsterdam),4thedition.
[116]vanHemmen,J.L.(2002)Themapinyourhead:Howdoesthebrainrepresenttheoutside
world?,ChemPhysChem3291–298.
[117]Hensel,H.(1975)StaticanddynamicactivityofwarmreceptorsinBoaconstrictor,P߬ugers
191–199.353hivArc[118]Hensel,H.&Wurster,R.D.(1970)Staticpropertiesofcoldreceptorsinnasalareaofcats,J
271–275.33ysiolNeuroph[119]Herz,A.V.M.,Gollisch,T.,Machens,C.K.,&Jaeger,D.(2006)ModelingSingle-Neuron
dynamicsandComputations:ABalanceofDetailandAbstraction,Science31480–85.
[120]Hilis,J.M.,Ernst,M.O.,Banks,M.S.,&Landy,M.S.(2002)Combiningsensoryinformation:
Mandatoryfusionwithin,butnotbetween,senses,Science2981627–1630.
[121]Hisajima,T.,Kishida,R.,Atobe,Y.,Nakano,M.,etal.(2002)Distributionofmyelinated
andunmyelinatednervefibersandtheirpossibleroleinbloodflowcontrolincrotalinesnake

111

Bibliography

infraredreceptororgans,JCompNeurol449319–329.
[122]H¨blindotting,hK.,umans:R¨Anosler,evF.,en&R¨t-relatedoder,pB.oten(2004)tialstudyAltered,ExpBrainauditory-tactileRes159interactions370–381.incongenitally
[123]Hubel,D.H.&Wiesel,T.N.(1977)Functionalarchitectureofmacaquemonkeyvisualcortex,
ProcRoySocLondB1981–59.
[124]Huber,M.T.&Braun,H.A.(2006)Stimulus-responsecurvesofaneuronalmodelfornoisy
subthresholdoscillationsandrelatedspikegeneration,PhysRevE730419291–04192910.
[125]opticsHughes,andA.retinal(1977)Thetoporganization.,ographyinofF.visionCrescitelliinmammals(editor),ofconHandbtrastingookoflifestsensoryyle:physiComparativologye,
volumeVII/5,chapter8,pp.613–756(Springer,BerlinHeidelbergNewYork).
[126]H¨urlimann,F.,Kiper,D.C.,&Carandini,M.(2002)Testingthebayesianmodelofperceived
speed,VisionRes422253–2257.
[127]Hyde,P.S.&Knudsen,E.I.(2001)Atopographicinstructivesignalguidestheadjustmentof
theauditoryspacemapintheoptictectum,JNeurosci218586–8593.
[128]Ito,M.(1989)Long-termdepression,AnnuRevNeurosci1285–102.
[129]Itti,L.&Koch,C.(2001)Computationalmodelingofvisualattention,NatRevNeurosci
194–203.2(3)[130]J¨arvilehto,M.(1985)Theeye:Visionandperception,inG.A.Kerkut&L.I.Gilbert(editors),
NervousSystem:Sensory,volume6ofComprehensiveInsectPhysiology,Biochemistry,and
Pharmacology,chapterFINDCHAPTER,pp.355–429(PergamonPress,Oxford).
[131]Jazayeri,M.&Movshon,J.A.(2006)Optimalrepresentationofsensoryinformationbyneural
populations,NatNeurosci9690–696.
[132]Jeffress,L.A.(1948)Aplacetheoryofsoundlocalization,JCompPhysiolPsychol4135–39.
[133]Jiang,B.,Trevi˜no,M.,&Kirkwood,A.(2007)Sequentialdevelopmentoflong-termpotentia-
tionanddepressionindifferentlayersofthemousevisualcortex,JNeurosci279648–9652.
[134]Jiang,W.,Wallace,M.T.,Jiang,H.,Vaughan,J.W.,etal.(2001)TwoCorticalAreasMediate
MultisensoryIntegrationinSuperiorColliculusNeurons,JNeurophysiol85506–522.
[135]Johnson,D.H.&Dudgeon,D.E.(1993)ArraySignalProcessing:ConceptsandTechniques
(Prentice-Hall,UpperSaddleRiver,NJ).
[136]Johnson,K.O.,Hsiao,S.S.,&Yoshioka,T.(2002)Neuralcodingandthebasiclawof
psychophysics,Neuroscientist8111–121.
[137]Jones,B.S.,Lynn,W.F.,&Stone,M.O.(2001)Thermalmodelingofsnakeinfraredreception:
Evidenceforlimiteddetectionrange,JTheorBiol209201–211.
[138]Jost,J.&Li-Jost,X.(1998)Calculusofvariations(CambridgeUniversity,Cambridge).
[139]Kaas,J.H.&Collins,C.E.(2004)Theresurresctionofmultisensorycortexinprimates:
Connectionpatternsthatintegratemodalities,inG.Calvert,C.Spence,&B.E.Stein
(editors),TheHandbookofMultisensoryProcesses,chapter17,pp.285–293(MITPress,
MA).bridge,Cam[140]ofKadunce,Within-andD.C.,VCross-Moaughan,dalitJ.W.,yWSuppreallace,ssioninM.T.,theSupBenedek,eriorG.,etColliculus,al.J(1997)NeurophMecysiolhanisms78
2834–2847.[141]Kailath,T.(1974)Aviewofthreedecadesoflinearfilteringtheory,IEEETInformTheory
146–181.20[142]Kalmijn,A.J.(1988)Hydrodynamicandacousticfielddetection,inJ.Atema,R.R.Fay,A.N.
Popper,&W.N.Tavolga(editors),SensoryBiologyofAquaticAnimals,chapter4,pp.83–130
NY).ork,YNew(Springer,[143]Kandel,E.R.,Schwartz,J.H.,&Jessell,T.M.(2000)Principlesofneuralscience(McGraw-
Hill,NewYork),4th(international)edition.
[144]Kay,S.M.(1993)FundamentalsofStatisticalSignalProcessing(PrenticeHall,UpperSaddle
NJ).er,Riv[145]Kempter,R.,Gerstner,W.,&vanHemmen,J.L.(1999)Hebbianlearningandspikingneurons,

112

Bibliography

PhysRevE594498–4514.
[146]Kempter,R.,Leibold,C.,Wagner,H.,&vanHemmen,J.L.(2001)Formationoftemporal-
featuremapsbyaxonalpropagationofsynapticlearning,ProcNatlAcadSciUSA98
4166–4171.[147]King,A.J.(1999)Sensoryexperienceandtheformationofacomputationalmapofauditory
spaceinthebrain,Bioessays21900–911.
[148]King,A.J.(2009)Visualinfluencesonauditoryspatiallearning,PhilTransRSocB364
331–339.[149]King,A.J.&Hutchings,M.E.(1987)Spatialresponsepropertiesofacousticallyresponsive
neuronsinthesuperiorcolliculusoftheferret:Amapofauditoryspace,JNeurophysiol57
596–624.[150]inKing,theA.J.visual,Hutcandhings,auditoryM.E.,repreMosenore,tationsD.R.,in&theBlakemore,mammalianC.sup(1988)eriorDevcolliculus,elopmentalNatureplasticit332y
73–76.[151]King,A.J.&Palmer,A.R.(1983)Cellsresponsivetofree-fieldauditorystimuliinguinea-pig
superiorcolliculus:Distributionandresponseproperties,JPhysiol342361–381.
[152]King,A.J.&Parsons,C.H.(1999)Improvedauditoryspatialacuityinvisuallydeprived
ferrets,EurJNeurosci11(11)3873–3883.
[153]Knierim,J.J.&vanEssen,D.C.(1992)Neuronalresponsestostatictexturepatternsinarea
V1ofthealertmacaquemonkey,JNeurophysiol67961–980.
[154]Knudsen,E.I.(1982)Auditoryandvisualmapsofspaceintheoptictectumoftheowl,J
1177–1194.2Neurosci[155]Knudsen,E.I.(1999)Mechanismsofexperience-dependentplasticityintheauditorylocaliza-
tionpathwayofthebarnowl,JCompPhysiolA185305–321.
[156]Knudsen,E.I.(2002)Instructedlearningintheauditorylocalizationpathwayofthebarn
owl,Nature417322–328.
[157]Knudsen,E.I.,Blasdel,G.G.,&Konishi,M.(1979)Soundlocalizationbythebarnowl(Tyto
alba)measuredwiththesearchcoiltechnique,JCompPhysiolA1331–11.
[158]Knudsen,E.I.&Brainard,M.S.(1991)Visualinstructionoftheneuralmapofauditoryspace
inthedevelopingoptictectum,Science25385–87.
[159]Knudsen,E.I.&Knudsen,P.F.(1985)Visionguidestheadjustmentofauditorylocalization
inyoungbarnowls,Science230545–548.
[160]Knudsen,E.I.&Knudsen,P.F.(1989)Visioncalibratessoundlocalizationindevelopingbarn
owls,JNeurosci93306–3313.
[161]Knudsen,E.I.,duLac,S.,&Esterly,S.D.(1987)Computationalmapsinthebrain,Annu
41–65.10NeurosciRev[162]Kobayashi,S.,Amemiya,F.,Kishida,R.,Goris,R.C.,etal.(1995)Somatosensoryandvisual
correlationintheoptictectumofapython,Pythonregius:ahorseradishperoxidaseand
golgistudy,NeurosciRes22315–323.
[163]Kobayashi,S.,Kishida,R.,Goris,R.C.,Yoshimoto,M.,etal.(1992)Visualandinfraredinput
tothesamedendriteinthetectumopticumofthepython,Pythonregius:electron-microscopic
350–352.597ResBrainevidence,[164]Koch,C.(1999)BiophysicsofComputation(OxfordUniversityPress,Oxford).
[165]K¨ording,K.&Wolpert,D.M.(2004)Bayesianintegrationinsensorimotorlearning,Nature
244–247.427[166]Krauzlis,R.J.,Basso,M.A.,&Wurtz,R.H.(1997)Sharedmotorerrorformultipleeye
movements,Science2761693–1695.
[167]Krauzlis,R.J.,Liston,D.,&Carello,C.D.(2004)Targetselectionandthesuperiorcolliculus:
Goals,choicesandhypotheses,VisionRes441445–1451.
[168]Krochmal,A.R.,Bakken,G.S.,&LaDuc,T.(2004)Heatinevolutionskitchen:evolutionary
perspectivesonthefunctionsandoriginofthefacialpitofpitvipers(viperidae:Crotalinae),

113

Bibliography

4231–4238.207BiolExpJ[169]Krueger,L.E.(1989)ReconcilingFechnerandStevens:Towardaunifiedpsychophysicallaw,
BehavBrainSci12251–320.
[170]Kuttruff,H.(1991)RoomAcoustics(ElsevierAppliedScience,LondonandNewYork),3rd
edition.[172][171]LamLaming,b,H.D.(1932)(1997)HydrTheoMedynamicsasurement(CamofbridgeSensationUniversit(OxfordyPress),Univ6thersityed.,Oxfedition.ord).
[173]Lebart,K.,Boucher,J.M.,&Denbigh,P.N.(2001)Anewmethodbasedonspectral
subtractionforspeechdereverberation,ActaAcoustica87(3)359–366.
[174]Leibold,C.,Kempter,R.,&vanHemmen,J.L.(2001)Temporalmapformationinthebarn
owlsbrain,PhysRevLett87248101.
[175]Li,Z.(2002)Asaliencymapinprimaryvisualcortex,TrendsCognSci6(1)9–16.
[176]Liman,E.R.(2006)ThermalgatingofTRPionchannels:Foodforthought?,SciStke326.
[177]Linden,D.J.&Connor,J.A.(1995)Long-termsynapticdepression,AnnuRevNeurosci18
319–357.[178]Lingenheil,M.(2004)TheoriederBeuteortungbeimKrallenfrosch,Master’sthesis,Technische
Universit¨atMu¨nchen.
[179]U.Luksch,Windhorst,H.(2008)&M.C.SensorimotorHirschin(editors),tegration:EncyclopOpticediaoftectum,NeurinM.osciencD.eBinder,(Springer,N.NewHirokYawork,a,
NY).[180]Luksch,H.,Gauger,B.,&Wagner,H.(2000)Acandidatepathwayforavisualinstructional
signaltothebarnowl’sauditorysystem,JNeurosci20RC70.
[181]Luo,L.&Flanagan,J.G.(2007)Developmentofcontinuousanddiscreteneuralmaps,Neuron
284–300.56[182]Lynn,G.W.(1931)Thestructureandfunctionofthefacialpitofthepitvipers,AmJAnat
97–133.49[183]Ma,W.J.,Beck,J.M.,Latham,P.E.,&Pouget,A.(2006)Bayesianinferencewithprobabilistic
populationcodes,NatNeurosci91432–1438.
[184]Mach,E.(1866)¨UberdiephysiologischeWirkungr¨aumlichvertheilterLichtreize,Sitzungsber
AkadWissWienII54393–408.
[185]Madison,D.V.,Malenka,R.C.,&Nicoll,R.A.(1991)Mechanismsunderlyinglong-term
potentiationofsynaptictransmission,AnnuRevNeurosci14379–397.
[186]Magosso,E.,Cuppinia,C.,Serino,A.,DiPellegrino,G.,etal.(2008)Atheoreticalstudy
ofmultisensoryintegrationinthesuperiorcolliculusbyaneuralnetworkmodel,Neural
Networks21817–829.
[187]Manley,G.A.,K¨oppl,C.,&Konishi,M.(1988)Aneuralmapofinterauralintensitydifferences
inthebrainstemofthebarnowl,JNeurosci82665–2676.
[188]Markram,H.,L¨ubke,J.,Frotscher,M.,&Sakmann,B.(1997)Regulationofsynapticefficacy
bycoincidenceofpostsynapticAPsandEPSPs,Science275213–215.
[189]McKemy,D.D.,Neuhausser,W.M.,&Julius,D.(2002)Identificationofacoldreceptor
revealsageneralroleforTRPchannelsinthermosensation,Nature41652–58.
[190]McLaughlin,T.&O’Leary,D.D.M(2005)Moleculargradientsanddevelopmentofretinotopic
maps,AnnuRevNeurosci28327–355.
[191]Meszler,R.M.,Auker,C.R.,&Carpenter,D.O(1981)Finestructureandorganizationofthe
infraredreceptorrelay,thelateraldescendingnucleusofthetrigeminalnerveinpitvipers,J
571–584.196NeurolComp[192]Middlebrooks,J.C.&Knudsen,E.I.(1984)Aneuralcodeforauditoryspaceinthecat’s
superiorcolliculus,JNeurosci42621–2634.
[193]Miller,K.(1970)Leastsquaresmethodsforill-posedproblemswithaprescribedbound,
SIAMJMathAnal152–74.
[194]Mills,A.W.(1958)Ontheminimumaudibleangle,JAcoustSocAm30237.

114

Bibliography

[195]Miyoshi,M.&Kaneda,Y.(1988)InverseFilteringofRoomAcoustics,IEEETAcoust
145–152.36(2)heecSp[196]Moiseenkova,V.,Bell,B.,Motamedi,M.,Wozniak,E.,etal.(2003)Wide-bandspectral
tuningofheatreceptorsinthepitorganofthecopperheadsnake(Crotalinae),AmJPhysiol
598–606.284[197]Molenaar,G.J.(1978)Thesensorytrigeminalsystemofasnakeinthepossessionofinfrared
receptors:II.Thecentralprojectionsofthetrigeminalnerve,JCompNeurol179137–152.
[198]&PMolenaar,.S.UlinskiG.J.(1992)(editors),AnatomySensorimotorandphInteysiologygrationof,infvraredolume17sensitivitofyBioloofgysnakofes,theinRC.eptiliaGans,
chapter5,pp.367–453(UniversityofChicagoPress,Chicago,IL).
[199]Molenaar,G.J.&Fizaan-Oostveen,J.L.F.P.(1980)Ascendingprojectionsfromthelateral
descendingandcommonsensorytrigeminalnucleiinpython,JCompNeurol189555–572.
[200]Mooney,R.D.,Klein,B.G.,&Rhoades,R.W.(1987)Effectsofalteredvisualinputupon
thedevelopmentofthevisualsystemandsomatosensoryrepresentationsinthehamster’s
superiorcolliculus,Neurosci20537–555.
[201]Moore,E.H.(1920)Onthereciprocalofthegeneralalgebraicmatrix,BAmMathSoc26
394–395.[202]Morgan,M.L.,DeAngelis,G.C.,&Angelaki,D.E.(2008)Multisensoryintegrationinmacaque
visualcortexdependsoncuereliability,Neuron59662–673.
[203]Murphey,R.K.(1973)Mutualinhibitionandtheorganizationofanon-visualorientationin
Notonecta,JCompPhysiolA8431–40.
[204]Mysore,S.P.&Quartz,S.R.(2005)Modelingstructuralplasticityinthebarnowlauditory
localizationsystemwithaspike-timedependenthebbianlearningrule,ProcIEEEIntJoint
ConfNeuralNetworks52766–2771.
[205]Nakano,M.,Atobe,Y.,Goris,R.C.,Yazama,F.,etal.(2000)Ultrastructureofthecapillary
pericytesandtheexpressionofsmoothmuscleα-actinanddesmininthesnakeinfrared
sensoryorgans,AnatRec260299–307.
[206]Neely,StephenT.&Allen,JontB.(1979)Invertibilityofaroomimpulseresponse,JAcoust
165–9.66(1)AmcSo[207]Nelson,D.A.(1998)Invitededitorialon”penns1948paperrevisited,JApplPhysiol852–3.
[208]Nelson,M.E.&Bower,J.M.(1990)Brainmapsandparallelcomputers,TrendsNeurosci13
408.–403[209]Newman,E.A.,Gruberg,E.R.,&Hartline,P.H.(1980)Theinfraredtrigemino-tectalpathway
intherattlesnakeandinthepython,JCompNeurol191465–477.
[210]Newman,E.A.&Hartline,P.H.(1981)Integrationofvisualandinfraredinformationin
bimodalneuronsoftherattlesnakeoptictectum,Science213789–791.
[211]Newman,E.A.&Hartline,P.H.(1982)Theinfrared“vision”ofsnakes,SciAm246(3)
98–107.[212]Nilius,B.,Talavera,K.,Owsianik,G.,Prenen,J.,etal.(2005)GatingofTRPchannels:a
voltageconnection?,JPhysiol1235–44.
[213]Norwich,K.H.&Wong,W.(1997)Unificationofpsychologicalphenomena:Thecomplete
formofFechner’slaw,PercPsychophys59929–940.
[214]Olsen,J.F.,Knudsen,E.I.,&Esterly,S.D.(1989)Neuralmapsofinterauraltimeandintensity
differencesintheoptictectumofthebarnowl,JNeurosci92591–2605.
[215]Oppenheim,A.V.,Schafer,R.W.,&Stockham,T.G.(1968)Nonlinearfilteringofmultiplied
andconvolvedsignals,ProcIEEE56(8)1264–91.
[216]vanOpstal,A.J.&Munoz,D.P.(2004)Auditory-visualinteractionssubservingprimategaze
orienting,inG.Calvert,C.Spence,&B.E.Stein(editors),TheHandbookofMultisensory
Processes,chapter23,pp.373–393(MIT,Cambridge,MA).
[217]Oˇguzt¨oreli,M.N.&Caelli,T.M.(1985)Aninverseprobleminneuralprocessing,BiolCybern
239–245.53

115

Bibliography

[218]Panton,R.L.(2005)Incompressibleflow(Wiley),3rdedition.
[219]Pneuronsappas,fromT.C.,pitvipMotamedi,erM.,thermosensors,&Christensen,AmJPhB.N.ysiolCell(2004)PhysiolUnique287temp1219–1228.erature-activated
[220]Patton,P.E.,Belkacem-Boussaid,K.,&Anastasio,T.J.(2002)Multimodalityinthesuperior
colliculus:aninformationtheoreticanalysis,CognitiveBrainRes1410–19.
[221]aPecka,neuronalM.,netZahn,workT.Pfor.,soundSaunier-Reblocalizationori,B.,inSivreveke,erbI.,eranettal.environ(2007)ments.,InhibitingJtheNeurosci.inhibition:27(7)
1782–1790.[222]Pennes,H.H.(1948)Analysisoftissueandarterialbloodtemperaturesintherestinghuman
forearm,JApplPhysiol193–122.
[223]Penrose,R.(1955)Ageneralizedinverseformatrices,CambridgePhilosSoc51406–413.
[224]PNeuronserrault,UseT.J.,VDistinctaughan,OpJ.W.,erationalStein,ModesB.E.,in&theWInallace,tegrationM.T.of(2005)MultisensorySuperiorStimColliculusuli,J
2575–2586.93ysiolNeuroph[225]Pickles,J.O.(1988)AnIntroductiontothePhysiologyofhearing(AcademicPress,London),
edition.2nd[226]Planck,M.(1901)Ueberdasgesetzderenergieverteilungimnormalspectrum,AnnPhys309
553–563.[227]Pouget,A.,Dayan,P.,&Zemel,R.S.(2003)Inferenceandcomputationwithpopulation
codes,AnnuRevNeurosci26381–410.
[228]Press,W.H.,Teukolsky,S.A.,Vetterling,W.T.,&Flannery,B.P.(2007)NumericalRecipes:
TheArtofScientificComputing,3rded.(CambridgeUniversity,Cambridge).
[229]Puetter,R.C.,Gosnell,T.R..,&AmosYahil,A.(2005)Digitalimagereconstruction:Deblur-
ringanddenoising,AnnuRevAstronAstrophys43139–194.
[230]Putzar,L.,Goerendt,I.,Lange,K.,R¨osler,F.,etal.(2007)Earlyvisualdeprivationimpairs
multisensoryinteractionsinhumans,NatNeurosci101243–1245.
[231]Rauschecker,J.P.&Kniepert,U.(1994)Auditorylocalizationbehaviourinvisuallydeprived
cats,EurJNeurosci6(1)149–160.
[232]deRivaz,Peter&Kingsbury,Nick(2001)Bayesianimagedeconvolutionanddenoisingusing
complexwavelets,IEEEImageProc2273–276.
[233]R¨oder,B.,R¨osler,F.,&Spence,C.(2004)Earlyvisionimpairstactileperceptionintheblind,
121–124.14BiolCurr[234]Roder,B.,Teder-Salejarvi,W.,Sterr,A.,Rosler,F.,etal.(1999)Improvedauditoryspatial
tuninginblindhumans,Nature400162–166.
[235]Roper,P.,Bressloff,P.C.,&Longtin,A.(2000)Aphasemodeloftemperature-dependent
mammaliancoldreceptor,NeuralComput121067–1093.
[236]Rosenfeld,D.(2002)NewApproachtoGriddingUsingRegularization,MagnResonMed48
193–202.[237]Rowland,B.A.,Quessy,S.,Stanford,T.R.,&Stein,B.E.(2007)Multisensoryintegration
shortensphysiologicalresponselatencies,JNeurosci275879–5884.
[238]Rowland,B.A.,Stanford,T.R.,&Stein,B.E.(2007)Abayesianmodelunifiesmultisensory
spatiallocalizationwiththephysiologicalpropertiesofthesuperiorcolliculus,ExpBrainRes
153–161.180[239]Rowland,B.A.,Stanford,T.R.,&Stein,B.E.(2007)Amodeloftheneuralmechanisms
underlyingmultisensoryintegrationinthesuperiorcolliculus,Perception36(10)1431–1443.
[240]Rucci,M.,Edelman,G.M.,&Wray,J.(1999)Adaptionoforientingbehavior:Fromthebarn
owltoaroboticsystem,IEEETRobot1596–110.
[241]Rucci,M.,Tononi,G.,&Edelman,G.M.(1997)Registrationofneuralmapsthroughvalue-
dependentlearning:Modelingthealignmentofauditoryandvisualmapsinthebarnowl‘s
optictectum,JNeurosci17334–352.
[242]Sarkar,T.K.,Weiner,D.D.,&Jain,V.K.(1981)SomemathematicalConsiderationsin

116

Bibliography

DealingwiththeInverseProblem,IEEETAntennPropag29373–379.
[243]Sch¨afer,K.,Braun,H.A.,&Hensel,H.(1982)Staticanddynamicactivityofcoldreceptors
atvariouscalciumlevels,JNeurophysiol471017–1028.
[244]Sch¨afer,K.,Braun,H.A.,&K¨urten,L.(1988)Analysisofcoldandwarmreceptoractivityin
vampirebatsandmice,PflugArchEurJPhy412188–194.
[245]Schreiner,C.E.&Langner,G.(1988)Periodicitycodingintheinferiorcolliculusofthecat.
II.Topograhicalorganization,JNeurophysiol601823–1840.
[246]Schroeder,D.M.&Loop,M.S.(1976)Trigeminalprojectionsinsnakespossessinginfrared
sensitivity,JCompNeurol1691–14.
[247]Sejnowski,T.J.(1995)Timeforanewneuralcode?,Nature37621–22.
[248]Seung,H.S.&Sompolinsky,H.(1993)Simplemodelsforreadingneuronalpopulationcodes,
ProcNatlAcadSciUSA9010749–10753.
[249]Shams,L.,Kamitani,Y.,&Shimojo,S.(2000)Whatyouseeiswhatyouhear,Nature408
788.[250]Shumway,C.A.(1989)Multipleelectrosensorymapsinthemedullaofweaklyelectricgymno-
tiformfish.I.Physiologicaldifferences,JNeurosci94388–4399.
[251]Sichert,A.B.,Bamler,R.,&vanHemmen,J.L.(2009)Hydrodynamicobjectrecognition:
Whenmultipolescount,PhysRevLett102058104.
[252]Sichert,A.B.,Friedel,P.,&vanHemmen,J.L.(2006)Snake’sperspectiveonheat:Recon-
structionofinputusinganimperfectdetectionsystem,PhysRevLett9768105.
[253]Sichert,A.B.&vanHemmen,J.L.(2010)Howstimulusshapeaffectslateral-lineperception:
analyticalapproachtoanalyzenaturalstimulicharacteristics,BiolCybern102177–180.
[254]Song,S.,Miller,K.D.,&Abbott,L.F.(2000)Competitivehebbianlearningthroughspike-
timing-dependentplasticity,NatNeurosci3919–926.
[255]Stanford,L.R.&Hartline,P.H.(1980)Spatialsharpeningbysecond-ordertrigeminalneurons
incrotalineinfraredsystem,BrainRes185115–123.
[256]Stanford,L.R.,Schroeder,D.M.,&Hartline,P.H.(1981)Theascendingprojectionofthe
nucleusofthelateraldescendingtrigeminaltract:Anucleusintheinfraredsystemofthe
rattlesnake,Crotalusviridis,JCompNeurol201161–173.
[257]Stavenga,D.G.(2002)Reflectionsoncolourfulommatidiaofbutterflyeyes,JExpBiol205
1077–1085.[258]Steenken,R.,Colonius,H.,Diederich,A.,&Rach,S.(2008)Visualandauditoryinteraction
insaccadicreactiontime:Effectsofauditorymaskerlevel,BrainRes1220150–156.
[259]Stein,B.E.&Gaither,N.S.(1981)Sensoryrepresentationsinreptilianoptictectum:Some
comparisonswithmammals,JCompNeurol20269–87.
[260]Stein,B.E.,Jiang,W.,&Stanford,T.R.(2004)Multisensoryintegrationinsingleneuronsof
themidbrain,inG.Calvert,C.Spence,&B.E.Stein(editors),TheHandbookofMultisensory
Processes,chapter15,pp.243–264(MIT,Cambridge,MA).
[261]Stein,B.E.&Meredith,M.A.(1993)TheMergingoftheSenses(MITPress,Cambridge,
MA).[262]Stein,B.E.&Stanford,T.R.(2008)Multisensoryintegration:Currentissuesfromthe
perspectiveofthesingleneuron,NatRevNeurosci9255–266.
[263]Stryer,L.(1986)Cyclicgmpcascadeofvision,AnnualReviewofNeuroscience987–119.
[264]Sullivan,W.E.&Konishi,M.(1986)Neuralmapofinterauralphasedifferenceintheowl’s
brainstem,ProcNatlAcadSciUSA838400–8404.
[265]Takahashi,T.&Konishi,M.(1986)Selectivityforinterauraltimedifferenceintheowl’s
midbrain,JNeurosci63413–3422.
[266]Terashima,S.I.,Goris,R.C.,&Katsuki,Y.(1970)Structureofwarmfiberterminalsinthe
pitmembraneofvipers,JUltraMolStructR31494–506.
[267]Thomson,R.F.(2001)DasGehirn(Spektrum,Akad.Verl.,Heidelberg,Berlin),3rdedition.
[268]Tikhonov,A.N.,Arsenin,V.Y.,&John,F.(1977)SolutionofIll-PosedProblems(V.H.

117

Bibliography

Winston&Sons,Washington,DC),englishtranslationoftheoriginalRussiantext.
[269]forTikhonotheSv,olA.N.,utionofGoncIll-Poseharskyd,PrA.V.,oblemsStepano(Kluv,werV.V.,&AcademicYagola,A.G.Publishers,(1995)DordrecNumerichalt),Methoenglishds
translationoftheoriginalRussiantext.
[270]Truckenbrodt,E.(1999)Fluidmechanik2(Springer),4thedition.
[271]Udin,S.&Fawcett,J.W.(1988)Formationoftopographicmaps,AnnuRevNeurosci11
289–327.[272]Unoki,M.,Sakata,K.,Furukawa,M.,&Akagi,M.(2004)Aspeechdereverberationmethod
basedontheMTFconceptinpowerenveloperestoration,JAcousticalScienceandTechnology
243–254.25(4)[273]Unoki,Masashi,Furukawa,Masakazu,&Sakata,KeigoanfAkagi,Masato(2004)Animproved
methodbasedontheMTFconceptforrestoringthepowerenvelopefromareverberantsignal,
JAcousticalScienceandTechnology25(4)232–42.
[274]theUrsino,supM.,eriorCuppinia,colliculus:C.,aneuralMagosso,netwE.,orkmoSerino,del,JA.,etComputal.(2009)NeurosciMultisensory26(1)55–73.integrationin
[275]Voets,T.,Droogmas,G.,Wissenbach,U.,Janssens,A.,etal.(2004)Theprincipleof
temperature-dependentgatingincold-andheat-sensitiveTRPchannels,Nature430748–
754.[276]vanderWaerden,B.L.(1957)MathematischeStatistik(Springer,Berlin).
[277]Wallace,M.T.,Carriere,B.N.,PerraultJr,T.J.,Vaughan,J.W.,etal.(2006)Thedevelopment
ofcorticalmultisensoryintegration,JNeurosci26(46)11844–11849.
[278]Wallace,M.T.,PerraultJr,T.J.,Hairston,W.D.,&Stein,B.E.(2004)Visualexperienceis
necessaryforthedevelopmentofmultisensoryintegration,JNeurosci249580–9584.
[279]Wallace,M.T.&Stein,B.E.(1997)Developmentofmultisensoryneuronsandmultisensory
integrationincatsuperiorcolliculus,JNeurosci172429–2444.
[280]Wallace,M.T.&Stein,B.E.(2007)Earlyexperiencedetermineshowthesenseswillinteract,
921–926.97ysiolNeurophJ[281]Wandell,B.A.(1995)FoundationsofVision(SinauerAssociates,Sunderland,MA).
[282]Wehr,Michael&Zador,AnthonyM.(2003)Balancedinhibitionunderliestuningandsharpens
spiketiminginauditorycortex,Nature426442–446.
[283]Wickesberg,R.E.&Oertel,D.(1990)Delayed,frequency-specificinhibitioninthecochlear
nucleiofmice:Amechanismformonauralechosuppression,JNeurosci101762–1768.
[284]Willshaw,D.J.&Malsburg,C.(1976)Howpatternedneuralconnectionscanbesetupby
self-organization,ProcRoySocLondB194431–445.
[285]Winkowski,D.E.&Knudsen,E.I.(2006)Top-downgaincontroloftheauditoryspacemap
bygazecontrolcircuitryinthebarnowl,Nature439336–339.
[286]Wissler,E.H.(1998)’penns1948paperrevisited,JApplPhysiol8535–41.
[287]Witten,I.B.,Knudsen,E.I.,&Sompolinsky,H.(2008)AHebbianlearningrulemediates
asymmetricplasticityinaligningsensoryrepresentations,JNeurophysiol1001067–1079.
[288]Wu,Mingyang&Wang,DeLiang(2006)Atwo-stagealgorithmforone-microphonereverberant
speechenhancement,IEEETAudioSpeech14774–84.
[289]Yang,Y.,Chen,J.,Engel,J.,Pandya,S.,etal.(2006)Distanttouchhydrodynamicimaging
withanartificiallateralline,ProcNatlAcadSciUSA10318891–18895.
[290]Yegnanarayana,B.,Avendano,C.,Hermansky,H.,&SatyanarayanaMurthy,P.(1999)
Speechenhancementusinglinearpredictionresidual,SpeechCommun2825–42.
[291]Yegnanarayana,B.&SatyanarayanaMurthy,P.(2000)EnhancementofReverberantSpeech
UsingLPResidual,IEEETSpeechAudioP8267–81.
[292]Yost,W.A.(1991)Auditoryimageperceptionandanalysis:Thebasisforhearing,HearRes
8–18.56(7)[293]Zeil,J.&Hemmi,J.M.(2006)Thevisualecologyoffiddlercrabs,JCompPhysiolA192
1–25.

118

[294]

[295]

Zhang,

L.L.,

Huizong,

W.T.,

Holt,

Bibliography

W.A.,,sHarriC.E.,

et

al.

(1998)

A

critical

wwindo

for

cooperationandcompetitionamongdevelopingretinotectalsynapses,Nature39537–44.

Zittlau,K.E.,Claas,B.,&M¨unz,H.(1988)Horseradishperoxidasestudyoftectalafferents

withlaevisXenopusin

Beh

olEv

32

208–219.

ecialsp

emphasis

on

their

relationship

to

the

lateral-line

system,

Brain

119

en
Report a problem
playstore
appstore
facebook twitter instagram youtube

YouScribe

The catalogue

Service

Conditions

© 2010-2020 YouScribe
Internship/Work Experience Reports Prose Poetry Online Books The catalogue The whole community