One learns by experience [Elektronische Ressource] : a life-span view on learning and error processing / vorgelegt von Benjamin Eppinger
200 Pages
English

One learns by experience [Elektronische Ressource] : a life-span view on learning and error processing / vorgelegt von Benjamin Eppinger

-

Downloading requires you to have access to the YouScribe library
Learn all about the services we offer

Informations

Published by
Published 01 January 2008
Reads 12
Language English
Document size 3 MB





One learns by experience
A Life-span View on Learning and Error Processing



Dissertation
zur Erlangung des Grades eines
Doktors der Philosophie
der Philosophischen Fakultäten
der Universität des Saarlandes

vorgelegt von

Benjamin Eppinger
aus Sindelfingen



Saarbrücken, 2008






Dekan:
Prof. Dr. Rainer Krause, Universität des Saarlandes
Berichterstatter:
Prof. Dr. J. Kray, Universität des Saarlandes
Prof. Dr. G. Kerkhoff, Universität des Saarlandes

Tag der Disputation: 20. 02. 2008



















II

To Emilia and Caramel, my beloved monsters in heaven. I miss you.
























III
Overview
I Theoretical Part........................................................................................................... 9
1 Introduction ................................................................................................................... 9
2. Literature Review .......................................................................................................... 11
2.1 Overview................................................................................................................. 11
2.2 The Neurophysiological Basis of Reinforcement Learning....................................... 11
2.2.1 A Historical View on and a Definition of Reinforcement Learning................. 11
2.2.2 The Role of Dopamine for Reinforcement Learning ..................................... 13
2.2.3 Dopaminergic Projections to the Basal Ganglia ........................................... 16
2.2.4 Dopaminergic Projections to the Prefrontal Cortex (PFC) ............................ 18
2.2.5 Age-related Changes in the Dopamine System across the Lifespan............ 20
2.2.6 Summary ..................................................................................................... 25
2.3 Neuropsychological Theories of Error Processing ................................................... 27
2.3.1 Theoretical Accounts to the Error-Related Negativity (ERN) ........................ 27
2.3.2 Theoretical Accounts to the Feedback-Related Negativity (FRN)................. 30
2.3.3 Age-related Changes in Error Processing across the Lifespan .................... 31
2.3.4 Summary ..................................................................................................... 35
2.4 The Reinforcement Learning (R-L) Theory of the ERN............................................ 36
2.5 A Computational Account to Altered Error Processing in Older Age........................ 38
2.6 Synopsis ................................................................................................................. 39
II Empirical part .............................................................................................................. 42
3 Experiment 1.................................................................................................................. 42
3.1 Statements of Problem............................................................................................ 42
3.2 Design..................................................................................................................... 42
3.3 Hypotheses ............................................................................................................. 43
3.4 Method.................................................................................................................... 44
IV
3.5 Results ................................................................................................................... 50
3.5.1 Accuracy Data............................................................................................ 50
3.5.2 ERP Data ................................................................................................... 52
3.6 Summary ................................................................................................................ 63
4 Experiment 2 ................................................................................................................. 65
4.1 Statements of Problem ........................................................................................... 65
4.2 Design .................................................................................................................... 66
4.3 Hypotheses ............................................................................................................ 66
4.4 Method ................................................................................................................... 67
4.5 Results ................................................................................................................... 72
4.5.1 Accuracy Data............................................................................................ 72
4.5.2 ERP Data ................................................................................................... 74
4.6 Summary ................................................................................................................ 83
5 Interim Discussion........................................................................................................ 85
5.1 Accuracy Data ........................................................................................................ 86
5.2 Response-locked ERPs.......................................................................................... 89
5.3 Feedback-locked ERPs .......................................................................................... 92
6 Experiment 3 ................................................................................................................. 96
6.1 Statements of Problem ........................................................................................... 96
6.2 An Alternative Account to Reinforcement Learning................................................. 97
6.3 Hypotheses ............................................................................................................ 100
6.4 Methods.................................................................................................................. 101
6.5 Results ................................................................................................................... 108
6.5.1 Accuracy Data............................................................................................ 108
6.5.2 ERP Data ................................................................................................... 112
6.6 Summary and Discussion ....................................................................................... 123


V
III General Discussion .....................................................................................................135
7 Discussion .....................................................................................................................135
7.1 Age differences in Reinforcement Learning across the Lifespan .............................136
7.2 Processing of Internal Error Information ..................................................................141
7.3 Processing of External Error Information.................................................................151
8 Conclusion.....................................................................................................................160
9 Appendix........................................................................................................................163
10 Zusammenfassung........................................................................................................168
10.1 Einleitung ................................................................................................................168
10.2 Experimente............................................................................................................171
10.2.1 Experiment 1 ..............................................................................................171
10.2.2 Experiment 2 ..............................................................................................173
10.2.3 Experiment 3 ..............................................................................................174
10.3 Ergebnisse und Diskussion .....................................................................................177
11 References.....................................................................................................................183
Abbreviations......................................................................................................................198
Annotation...........................................................................................................................199
Danksagung ........................................................................................................................200







VI
Figure Index
Figure 1 Schematic Figure of the mesencephalic dopamine system (MDS) ..................... 14
Figure 2 Results of the study by Schultz et al. (1997)....................................................... 15
Figure 3 Anatomical map of the medial frontal cortex....................................................... 19
Figure 4 The ERN (Gehring et al., 1993).......................................................................... 27
Figure 5 Developmental differences in the ERN (Segalowitz et al., 2005)........................ 32
Figure 6 Age differences in the ERN (Nieuwenhuis et al., 2002) ...................................... 34
Figure 7 The reinforcement learning theory of the ERN (Holroyd & Coles, 2002)............. 37
Figure 8 Experimental conditions in Experiment 1............................................................ 46
Figure 9 Trial procedure in Experiment 1 ......................................................................... 47
Figure 10 Accuracy learning curves in Experiment 1.......................................................... 51
Figure 11 Response-locked ERPs in Experiment 1............................................................ 54
Figure 12 Response-locked ERPs over the course of learning (Exp. 1) ............................. 56
Figure 13 Learning curves for the response-locked positivity and the ERN (Exp. 1)........... 57
Figure 14 Feedback-locked ERPs in Experiment 1 ............................................................ 60
Figure 15 Feedback-locked ERPs over the course of learning (Exp. 1).............................. 61
Figure 16 Learning curves for the feedback-locked positivity (Exp. 1) ................................ 62
Figure 17 Accuracy learning curves in Experiment 2.......................................................... 72
Figure 18 Response-locked ERPs in Experiment 2............................................................ 75
Figure 19 Response-locked ERPs over the course of learning (Exp. 2) ............................. 77
Figure 20 Learning curves for the response-locked positivity (Exp. 2)................................ 78
Figure 21 Correlations between the response-locked positivity and accuracy (Exp. 2)....... 79
Figure 22 Feedback-locked ERPs in Experiment 2 ............................................................ 81
Figure 23 Feedback-locked ERPs over the course of learning (Exp. 2).............................. 82
Figure 24 Learning curves for the feedback-locked positivity (Exp. 2) ................................ 83
Figure 25 The Go-NoGo model (Frank, 2004).................................................................... 98
Figure 26 Learning conditions in Experiment 3 .................................................................. 104
Figure 27 Schematic Figure of a learning block in Experiment 3 ........................................ 105
VII
Figure 28 Mean accuracy in the learning and reversal phase .............................................108
Figure 29 Mean accuracy for positive and negative learners ..............................................110
Figure 30 Mean accuracy learning effects for positive and negative learners .....................111
Figure 31 Response-locked ERPs in Experiment 3 ............................................................113
Figure 32 Learning-related effects in the response-locked positivity (Exp. 3)......................114
Figure 33 Response-outcome relations in the response-locked ERPs................................115
Figure 34 Learning-related effects in the ERN (Exp. 3).......................................................117
Figure 35 The ERN as a function of the learning biases .....................................................118
Figure 36 Feedback-locked ERPs in Experiment 3.............................................................120
Figure 37 Learning-related effects in the feedback-locked positivity (Exp. 3) ......................121


Table Index
Table 1 Sample characteristics Experiment 1.................................................................. 45
Table 2 Mean accuracy in Experiment 1..........................................................................163
Table 3 Mean fit parameters (R!) and mean slope parameters (") in Experiment 1 .........164
Table 4 Sample characteristics Experiment 2.................................................................. 68
Table 5 Mean accuracy Experiment 2 .............................................................................165
Table 6 Mean fit parameters (R!) and mean slope parameters (") in Experiment 2 .........166
Table 7 Sample characteristics Experiment 3..................................................................102
Table 8 Mean accuracy Experiment 3 .............................................................................167





VIII
I Theoretical Part
1 Introduction
“One learns by experience.” What this proverb implies is that we learn from positive,
as well as negative experiences, from our mistakes or errors, as well as from our correct
choices. Whether, under certain conditions, we might learn more from the negative or the
positive outcomes of our decisions (or vice versa) is a matter of current debate (Frank,
Seeberger, & O’Reilly, 2004; Holroyd & Coles, 2002). Recent findings point to the view
that there are considerable individual differences in the way participants learn from their
errors and correct choices, and that these learning biases seem to be associated with
alterations in dopamine levels (Frank, Woroch, and Curran, 2005). Moreover, there are
several findings that indicate that the role of errors for learning changes as a function of
lifespan development. On the one hand, there is data that point to the view that children
are more sensitive to errors and error feedback during learning than younger adults, (see
Crone, Jennings, & van der Molen, 2004; Crone, Somsen, Zanolie, & van der Molen,
2006; Crone & van der Molen, 2007; van Meel, Oosterlaan, Heslenfeld, & Sergeant,
2005). On the other hand, there is considerable evidence for the view that older adults are
impaired in error processing and in the processing of error feedback during learning (Band
& Kok, 2000; Falkenstein Hoormann, & Hohnsbein, 2001; Mathewson Dywan, Segalowitz,
2005, Nieuwenhuis et al., 2002; Themanson Hillman, & Curtin, 2006).
The basic theoretical idea of the present work is to combine neurophysiological
models of reward and reinforcement learning (for reviews see Schultz, 2000; 2002; 2006;
2007) with theoretical accounts and empirical findings on error processing (Holroyd &
Coles, 2002; Nieuwenhuis et al., 2002). In order to investigate the role of error processing
for learning the present work will adopt an event-related potential (ERP) approach, which
allows to investigate brain activity during error commission and the processing of external
error feedback during learning with a high temporal resolution.
The aim of this work is to investigate how children and older adults differ from
younger adults in the way they process errors and negative feedback during learning. The
9
empirical part of this work consists of three ERP experiments. In the first experiment, a
probabilistic learning task has been applied to study the effects of aging on learning and
error processing. In the second experiment, this learning task was adapted, so that it
could be used to study learning and error processing in children. The third experiment was
based on the findings of the first experiment, but takes a slightly different approach. For
this experiment, a learning task was developed that allowed to investigate individual
differences in the way younger and older adults learn from their errors and correct
choices.
The theoretical background of the present research could be broadly separated into
two modules. The first module consists of neurophysiological models of reinforcement
learning, which have been developed based on electrophysiological findings in primates
and functional imaging findings in humans (for reviews see Montague, Hyman, & Cohen,
2004, Schultz, 2007, Wise, 2004). These models assume that reinforcement learning
depends on learning signals from the dopamine system, which are projected to subcortical
and particularly to prefrontal target areas. Moreover, there are several theoretical
accounts and empirical findings in this field that show that the dopamine system and
especially its projections to the prefrontal cortex are subject to pronounced changes
during childhood development, as well as aging (Braver & Barch, 2002; Diamond, 1996;
Diamond, Briand, Fossella, & Gehlbach, 2004; Goldman-Rakic & Brown, 1982; Bäckman,
Nyberg, Lindenberger, Li, & Farde, 2006).
The second module consists of electrophysiological findings and neuro-
computational models on error processing in humans (Botvinick, Braver, Barch, Carter, &
Cohen, 2001; Coles, Scheffers, & Holroyd, 1998; Holroyd & Coles, 2002; Yeung,
Botvinick, & Cohen, 2004). This field of research is mainly based on the observation of the
error-related negativity (ERN), a component of the event-related potential (ERP) that can
be observed when participants commit errors on reaction time tasks (Falkenstein
Hohnsbein, & Hoormann, 1990; Gehring, Goss, Coles, Meyer, & Donchin, 1993). The
amplitude of the ERN has been suggested to increase with learning (Holroyd & Coles,
10