Discourse cohesion in text and tutorial dialogue
16 Pages
English
Downloading requires you to have access to the YouScribe library
Learn all about the services we offer

Discourse cohesion in text and tutorial dialogue

-

Downloading requires you to have access to the YouScribe library
Learn all about the services we offer
16 Pages
English

Description

InformationDesig nJournal15(3), 199–213 ©2007JohnnjaminsBe PublishingCompanyArthurC.Graesser,MoongeeJeon,YanYan&ZhiqiangCaiDiscourse cohesion in text and tutorial dialogueKeywords: cohesion, discourse types, readability, software tools, automatic analysis There has been a dramatic increase in computer analyses of large text corpora during the last decade. This can Discourse cohesion is presumably an important facilitator partly be explained by revolutionary advances in compu-of comprehension when individuals read texts and hold tational linguistics (Jurafsky & Martin, 2000; Walker conversations. This study investigated components of et al., 2003), discourse processes (Pickering & Garrod, cohesion and language in different types of discourse 2004; Graesser et al., 2003), the representation of world about Newtonian physics: A textbook, textoids written by knowledge (Lenat, 1995; Landauer et al., 2007), and experimental psychologists, naturalistic tutorial dialogue corpus analyses (Biber et al., 1998). Because thousands of between expert human tutors and college students, and texts can be quickly accessed and analyzed on thousands AutoTutor tutorial dialogue between a computer tutor of measures in a short amount of time, data mining is and students (AutoTutor is an animated pedagogical emerging as a standard methodology in a broad spec-agent that helps students learn about physics by holding trum of fields.conversations in natural ...

Subjects

Informations

Published by
Reads 34
Language English

Exrait

Arthur C. Graesser, Moongee Jeon, Yan Yan & Zhiqiang Cai Discourse cohesion in text and tutorial dialogue
Keywords: cohesion, discourse types, readability, software tools, automatic analysis Discourse cohesion is presumably an important facilitator of comprehension when individuals read texts and hold conversations. This study investigated components of cohesion and language in different types of discourse about Newtonian physics: A  textbook, textoids  written by experimental psychologists,  naturalistic tutorial dialogue between expert human tutors and college students, and  AutoTutor tutorial dialogue  between a computer tutor and students (AutoTutor is an animated pedagogical agent that helps students learn about physics by holding conversations in natural language). We analyzed the four types of discourse with Coh-Metrix, a software tool that measures discourse on different components of cohesion, language, and readability. The cohesion indices included co-reference, syntactic and semantic similarity, causal cohesion, incidence of cohesion signals (e.g., connectives, logical operators), and many other measures. Cohesion data were quite similar for the two forms of discourse in expository monologue (textbooks and textoids) and for the two types of tutorial dialogue (i.e., students interacting with human tutors and AutoTutor), but very different between the discourse of expository monologue and tutorial dialogue. Coh-Metrix was also able to detect subtle differences in the language and discourse of AutoTutor versus human tutoring.
Information Design Journal () , 99–23 ©  John Bnejamins Publishing Company
ere ha een a dramati inreae in omputer analye of large text orpora during the lat deade. i an partly e explained y reolutionary adane in ompu -tational linguiti (Jurafky & Martin, 2000; Walker et al., 2003), dioure proee (Pikering & Garrod, 2004; Graeer et al., 2003), the repreentation of world knowledge (Lenat, 1995; Landauer et al., 2007), and orpu analye (Bier et al., 1998). Beaue thouand of text an e quikly aeed and analyzed on thouand of meaure in a hort amount of time, data mining i emerging a a tandard methodology in a road pe -trum of field.  Reearher at the Unierity of Memphi hae reently deeloped a ytem alled Coh-Metrix ( http:// ohmetrix.memphi.edu, Graeer et al., 2004), a ompu -tational tool that produe meaure of the linguiti and dioure harateriti of text (oth printed text and tranript of oral dioure). e alue on the Coh-Metrix meaure an e ued to inetigate the oheion of the expliit text and the oherene of the mental repreentation of the text. Our definition of cohe-sion onit of linguiti harateriti of the expliit text that play ome role in onneting idea in the text. Coherence inlude harateriti of the text (i.e., apet of oheion) that are likely to ontriute to the oherene of mental repreentation.

Arthur C. Graesser et a l. Discourse cohesion in text and tutorial dialogue
 Reearher at the Unierity of Memphi hae alo deeloped an intelligent tutoring ytem alled Auto -Tutor (Graeer et al., 2005; Graeer, Lu et al., 2004). AutoTutor i a learning enironment that tutor tudent y holding a oneration in natural language AutoTu -. tor tutor tudent in Newtonian qualitatie phyi, omputer literay, ritial thinking, and other topi that exhiit explanation and eral reaoning. e dialogue of AutoTutor i uffiiently tale that it hold it own in onering with tudent for hour. It i alo deigned to mimi the dioure pattern of human tutor (Graeer, Peron, & Magliano, 1995).  e purpoe of the preent tudy wa to ue the Coh-Metrix tool to analyze the omponent of ohe -ion and language in different type of dioure aout Newtonian phyi. We analyzed a ample of hapter in a textbook , textoids written y experi-mental pyhologit, naturalistic tutorial dialogues  etween expert human tutor and ollege tudent, and AutoTutor tutorial dialogues etween a omputer tutor and tudent. One trong irtue of thi tudy wa our attempt to ahiee information equialene with repet to the ontent oered in the four orpora. We did thi y filtering ontent that oered the ame et of ore ontrut in phyi, namely Newtonian law of fore and motion. Gien thi ontrol oer informa -tion equialene, we inetigated how the four type of dioure differ with repet to language and oheion. We might expet the two type of expoitory mono -logue (textook and textoid) to e different from the two type of interatie dialogue (human tutor and AutoTutor). We might alo expet differene etween the two monologue type or etween the two dialogue typer. For example, textook and textoid preumaly differ from eah other eaue textook are written y profeional writer, wherea textoid are generated y experimental pyhologit to atify methodologi -al ontraint. What i le lear i the nature of thee

idj (), , -
differene. If the Coh-Metrix tool i alid and ueful, it hould detet utle and explainale differene in the four type of dioure. e preent tudy inetigated whether thi i indeed the ae. Coh-Metrix ere are approximately 60 indie in the Coh-Metrix erion (. 2.0) that i aailale to the puli. Aſter the uer of Coh-Metrix enter a text into the We ite, it print out meaure of the text on indie that pan different leel of dioure and language. Coh-Metrix wa deigned to moe eyond tandard readaility formula, uh a Fleh-Kinaid Grade Leel (Klare, 1974–1975). Suh formula rely exluiely on word length and entene length. For example, in the Fleh-Kinaid Grade Leel index (Figure 1) words refer to mean numer of word per entene and syllables refer to mean numer of yllale per word.  (1) Grade Leel = .39 * Word + 11.8 * Syllale - 15.59 Sentene length and word length do in fat routly predit reading time (Haerlandt & Graeer, 1985), ut ertainly there i more to reading diffiulty than word and entene length. ere mut alo e important deeper meaure of language and oheion. Coh-Metrix aim to proide thee deeper meaure.  e Coh-Metrix indices (i.e., meaure, metri) oer multiple leel of language and dioure. Some indie refer to harateriti of indiidual word, a ha een ahieed in many other omputer failitie, uh a WordNet (Fellaum, 1998) and Linguiti Inquiry Word Count (Penneaker & Frani, 1999). Howeer, the majority of the Coh-Metrix indie inlude deeper or more proeing-intenie algorithm that analyze yntax, referential oheion, emanti oheion, and dimenion of the ituation model. Coh-Metrix i the only omputer faility aailale to the puli for free
Arthur C. Graesser et a l. Discourse cohesion in text and tutorial dialogue
that analyze language and dioure on a road et of omponent at multiple leel.  A naphot of the landape of indie i proided in thi etion. Reearher at the Unierity of Memphi hae oer 600 indie in their internal omputer ytem, of whih 60 are aailale on the puli We ite ( http:// ohmetrix.memphi.edu /). i artile foue exlu -iely on the et of pulily aailale meaure. Reearh -er at the Unierity of Memphi hae alo ealuated the auray of the Coh-Metrix indie in oer 60 pulihed tudie, whih an e aeed at the puli We ite. Howeer, it i eyond the ope of thi tudy to reiew the reearh in thee aement.  Word measures . Coh-Metrix meaure word on a large numer of harateriti, mot of whih will not e defined in thi artile (ee the help ytem on the We ite http://ohmetrix.memphi.ed u). ere are meaure of word frequeny in the Englih language, whih i aed on the CELEX lexion (Baayen et al., 1993) and other imilar lexion. Coh-Metrix alo ditinguihe etween ontent word (e.g., noun, main er, adjetie) and funtion word (e.g., prepoition, artile), aed on tandard part-of-peeh ategorie that are aepted in the omputational linguiti ommunity.  Seeral word indie are diretly releant to oheion, oherene, and omprehenion diffiulty. In partiular, there are word lae that hae the peial funtion of onneting laue and other ontituent in the text (Halliday & Haan, 1976; Louwere, 2002; Sander & Noordman, 2000). e ategorie of onnetie in Coh-Metrix inlude additie ( also, moreover ), temporal ( and then, aſter, during ), aual (because, so ), and logial operator ( therefore, if, and, or ). e additie, temporal, and aual onnetie are udiided into thoe that are poitie ( also, because ) or negatie ( but, however ). e word indie inlude negation ( not, n’t ) that pan different leel of ontituent truture and ariou onditional expreion ( if, given ). Negation, onditional
idj (), , -
expreion, and negatie onnetie are predited to e affiliated with omplex oneptualization and rhetorial truture, uh a ounterfatual, hypothetial world, multiple perpetie, qualifiation, hedge, and argu -mentation. A higher inidene of thee word hould therefore predit text diffiulty.  e incidence of eah word la i omputed a the numer of ourrene per 1000 word. An inidene ore i neeary for omparing text of different ize. A text with higher oheion would hae a higher ini -dene of word lae that onnet ontituent.   Syntax . Coh-Metrix analyze entene yntax with the aitane of a yntati parer deeloped y Char -niak (2000). e parer aign part-of-peeh ategorie to word and yntati tree truture to entene. Our ealuation of eeral parer howed etter perfor -mane of Charniak’ parer than other major parer when omparing the aigned truture to judgment of human expert (Hemphill et al., 2006). Coh-Metrix ha eeral indie of yntati omplexity, two of whih (the mean numer of modifier per noun-phrae, and the numer of word efore the main er of the main laue) are reported in thi artile. e mean numer of modifier per noun-phrae i an index of the omplexity of referening expreion. For example, very large accel-erating objects i a omplex noun-phrae with 3 modifier of the head noun objects . e numer of word efore the main er of the main laue i an index of yntati omplexity eaue it plae a urden on the working memory of the omprehender (Graeer, Cai, Louwere, & Daniel, 2006).   Referential and semantic cohesion . Referential ohe-ion our when a noun, pronoun, or noun phrae refer to another ontituent in the text. For example, in the entene As the earth orbits the sun, it exerts a force, the word it refer to the word earth y irtue of a ynta-ti rule of pronoun aignment. A referring expreion (E) i the noun, pronoun, or noun-phrae that refer to

Arthur C. Graesser et a l. Discourse cohesion in text and tutorial dialogue
another ontituent (C). C i deignated a the referent of E. In the example entene, the word it i the referring expreion E, wherea the referent C i the word earth . One form of o-referene that ha een exteniely tud -ied i argument oerlap (Kinth & an Dijk, 1978). i our when a noun, pronoun, or noun-phrae in one entene i a o-referent of a noun, pronoun, or noun-phrae in another entene. e word “argument” i ued in a peial ene in thi ontext, namely it i a ontrat with prediate in propoitional repreentation. e argument oerlap index of Coh-Metrix urrently onid -er exat mathe of argument etween two entene. e alue of thi metri, whih arie from 0 to 1, i the proportion of adjaent entene pair that hare a ommon argument in the form of an exat math. Another form of o-referene i tem oerlap, where  a noun in one entene ha a imilar morphologial root (i.e., lemma) a a ontent word in another entene. For example, onider the two entene As the earth orbits the sun, it exerts a force. e orbit is not perfectly round. Orbits and orbit hae ommon tem, o there i tem oerlap, een though one i a main er and the other a noun. e alue of thi metri i the proportion of adja -ent entene pair that hae a tem oerlap.  Yet another form of o-referene i anaphori pronominal o-referene. A pronoun ( he, hers, it ) in one entene refer to a referent in another entene. A pronoun an preent a oherene prolem when the omprehender doe not know the referent of the pronoun. Pronoun oſten require a onerational or oial ontext to reole their referent, a oppoed to their referring to other text ontituent. Coh-Metrix ompute the referent of pronoun on the ai of yntati rule, emanti fit, and dioure pragmati y ome exiting algorithm in omputational linguiti (ee Jurafky & Martin, 2000; Lappin & Lea, 1994). e alue of thi metri i the proportion of adjaent entene pair in whih the eond entene ha a

idj (), , -
pronoun that an e uefully linked to a ontitu -ent in the preiou entene y exeuting the pronoun aignment mehanim.  In addition to referential oheion indie, Coh-Metrix ha indie that ae the extent to whih the ontent of entene, turn, or paragraph i imilar emantially or oneptually. Coheion and oherene are predited to inreae a a funtion of imilarity. Latent Semanti Analyi (LSA) i the primary method of omputing imilarity eaue it onider impliit knowledge. LSA i a mathematial, tatitial tehnique for repreenting world knowledge, aed on a large orpu of text. e entral intuition i that the mean -ing of a word i aptured y the ompany of other word that urround it in naturaliti doument; two word hae imilar meaning to the extent that they hare imilar urrounding word. LSA ue a tatitial teh -nique alled ingular alue deompoition to ondene a ery large orpu of text to 100-500 tatitial dimen -ion (Landauer et al., 2007). e oneptual imilar -ity etween any two text exerpt (e.g., word, laue, entene, text) i omputed a the geometri oine etween the alue and weighted dimenion of the two text exerpt. e alue of the oine arie from 0 to 1. LSA-aed oheion wa meaured in two way releant to the preent tudy: (1) LSA imilarity etween adjaent entene and (2) LSA imilarity etween adjaent para -graph.  Lexial dierity proide a imple, ut le ompu -tationally expenie, approah to omputing emanti oheion of a text. e lexial dierity metri in Coh-Metrix i the type-token ratio ore. i i the numer of unique word in a text (i.e., type) diided y the oerall numer of word (i.e., token) in the text. A low alue mean there i a large amount of redundany in the ontent word of a text. Coheion and oherene hould inreae inerely with type-token ratio.  Situation model dimensions . M any apet of a text
Arthur C. Graesser et a l. Discourse cohesion in text and tutorial dialogue
an ontriute to the  situation model  (or mental model), whih i the referential ontent or miroworld of what a text i aout (Graeer et al., 1994; Kinth, 1998). Text omprehenion reearher hae inetigated at leat fie dimenion of the ituational model (Zwaan & Radanky, 1998): auation, intentionality, time, pae, and protagonit. A reak in oheion or oherene our when there i a diontinuity on one or more of thee ituation model dimenion. Wheneer uh diontinuitie our, it i important to hae onnetie, tranitional phrae, ader, or other ignaling deie that oney to the reader that there i a diontinuity; we refer to thee different form of ignaling a particles . Coheion i failitated y partile that larify and tith together the ation, goal, eent, and tate oneyed in the text.  Coh-Metrix 2.0 analyze the ituation model dimen -ion on auation, intentionality, pae, and time, ut not protagonit. ere are many meaure of the ituation model, far too many to addre in thi artile. e preent tudy onentrated on three indie that meaure oheion on the dimenion of auality, intentionality, and tempo -rality.  For aual and intentional oheion, Coh-Metrix ompute the ratio of oheion partile to the inidene of releant referential ontent (i.e., main er that ignal tate hange, eent, ation, and proee, a oppoed to tate). e ratio metri i eentially a onditionalized inidene of oheion partile: Gien the ourrene of releant ontent (uh a laue with eent or ation, ut not tate), what i the denity of partile that tith together the laue? For example, the referential ontent for intentional information inlude intentional ation performed y agent (a in torie, ript, and ommon proedure); in ontrat, the intentional oheion partile would inlude infinitie and intentional onnetie ( in order to, so that, by means of ). Similarly, the referential ontent for auation information inlude ariou lae of eent that are identified y hange-of-tate er and
idj (), , -
other releant lae of er in WordNet (Fellaum, 1998); the aual partile are the aual onnetie and other word lae that denote aual onnetion etween ontituent. In the ae of temporal oheion, Coh-Metrix ompute the uniformity of the equene of main er with repet to tene and apet. e Coh-Metrix help faility i aailale at the We ite for more detail.
AutoTutor Student oneration with AutoTutor were one of the four type of dioure analyzed y Coh-Metrix. Auto -Tutor i a pedagogial agent that help tudent learn y holding a oneration in natural language (Graeer et al., 2005; Graeer, Lu et al., 2004). e learning gain of AutoTutor hae een aeed in the area of omputer literay (Graeer, Lu et al., 2004) and Newtonian phyi (VanLehn et al., 2007). AutoTutor inreae learning y approximately one letter grade when ompared to read -ing textook for an equialent amount of time.  AutoTutor’ dialogue are organized around diffi -ult quetion and prolem that require reaoning and explanation in the anwer. e example elow i one of the hallenging quetion on Newtonian phyi. Phyi quetion: If a lightweight ar and a maie truk hae a head-on olliion, upon whih ehile i the impat fore greater? Whih ehile undergoe the greater hange in it motion, and why? Suh quetion require the learner to ontrut approxi -mately 3–7 entene in an ideal anwer and to exhiit reaoning in natural language. e dialogue for one hallenging quetion typially require 50-200 oner -ational turn etween AutoTutor and the tudent. e deep learning expeted to our during thi proe i ditriuted oer many turn.  e truture of the dialogue in oth AutoTutor and human tutoring (Chi et al., 2001, Graeer et al., 1995;

Arthur C. Graesser et a l. Discourse cohesion in text and tutorial dialogue
Van Lehn et al., 2007) an e egregated into three leel or apet: (1) expetation and mioneption-tailored dialogue, (2) a fie-tep dialogue frame, and (3) ompoi -tion of a onerational turn. ee three leel an e automated and produe repetale tutorial dialogue.  Expectation and misconception tailored dialogue. i i the primary pedagogial method of affolding good tudent anwer. Both AutoTutor and human tutor typially hae a lit of expetation (antiipated good anwer) and a lit of antiipated misconceptions aoiat-ed with eah main quetion. For example, expetation E and mioneption M are releant to the example phyi prolem.  E: e magnitude of the fore exerted y A and B on eah other are equal.  M: A lighter or maller ojet exert no fore on a heaier or larger ojet. AutoTutor guide the tudent in artiulating the expe -tation through a numer of dialogue moe: generi pumps (what ele?) to get the tudent to do the talk -ing, hints , and prompts for the tudent to fill in mi -ing word. Hint and prompt are arefully eleted y AutoTutor to produe ontent in the anwer that fill in miing ontent word, phrae, and propoition. For example, a hint to get the tudent to artiulate expeta -tion E might e “What aout the fore exerted y the ehile on eah other?”; thi hint would ideally eliit the anwer “e magnitude of the fore are equal.” A prompt to get the tudent to ay “equal” would e “What are the magnitude of the fore of the two ehile on eah other?” AutoTutor adaptiely elet thoe hint and prompt that fill miing ontituent and therey ahiee pattern ompletion. For thoe tudent who annot fill in the ontent of an expetation aſter multiple onerational turn, AutoTutor tep in a a lat reort and imply expree the expetation a an assertion. AutoTutor end up generating a high proportion of

idj (), , -
pump and hint for artiulate tudent with high knowl -edge, ut a high proportion of prompt and aertion for low eral, low knowledge tudent. e lit of expeta -tion i eentually oered aſter the multi-turn dialogue and the main quetion i ored a anwered.  AutoTutor adapt to the learner in other way than affolding them to artiulate expetation. AutoTutor orret mioneption that periodially arie in the tudent’ talk. When the tudent artiulate a mionep -tion, AutoTutor aknowledge the error and orret it. AutoTutor gie feedak to the tudent on their ontri -ution in mot onerational turn. AutoTutor gie hort feedak  on the quality of tudent ontriution: poitie (ery good, rao), negatie (not quite, almot), or neutral (uh huh, okay). AutoTutor attempt to anwer the tudent’ quetion when they are aked. e anwer to the quetion are retrieed from gloarie or from paragraph in textook ia intelligent information retrieal.  Five-step dialogue frame . i dialogue frame i pre -alent in human tutoring (Graeer et al., 1995; VanLehn et al., 2007) and i alo implemented in AutoTutor. e fie tep of the dialogue frame are: (1) Tutor ak main quetion, (2) tudent gie initial anwer, (3) tutor gie hort feedak on the quality of the tudent’ anwer in #2, (4) tutor and tudent ollaoratiely interat ia expetation and mioneption tailored dialogue, and (5) tutor erifie that the tudent undertand (e.g., Do you understand? )  Managing one conversational turn . Eah turn of Auto-Tutor in the onerational dialogue ha three information lot (ontituent). e firt lot of mot turn i hort feedak on the quality of the tudent’ lat turn (i.e., poitie, negatie, or neutral). e eond lot adane the oerage of the ideal anwer with either pump, hint, prompt for peifi word, aertion, orretion of mioneption, or anwer to tudent quetion. e third lot i a ue to the tudent for the floor to hiſt from
Arthur C. Graesser et a l. Discourse cohesion in text and tutorial dialogue
AutoTutor a the peaker to the tudent. For example, AutoTutor end eah turn with a quetion or a geture to ue the learner to do the talking. Dioure marker (e.g., and also, okay, well ) onnet the utterane of thee three lot of information within a turn.  e three leel of AutoTutor go a long way in imulating a human tutor. AutoTutor an keep the dialogue on trak eaue it i alway omparing what the tudent ay to antiipated input (i.e., the expeta -tion and mioneption in the urriulum ript). Pattern mathing operation and pattern ompletion mehanim drie the omparion. ee mathing and ompletion operation are aed on latent emanti analyi (Landauer et al., 2007) and ymoli inter -pretation algorithm (Ru et al., 2006) that are eyond the ope of thi artile to addre. AutoTutor annot interpret tudent ontriution that hae no mathe to ontent in the urriulum ript. For example, AutoTutor annot explore topi hange and tangent a tudent introdue them. Howeer, aailale tudie of naturali -ti tutoring (Chi et al., 2001; Graeer et al., 1995) reeal that (a) human tutor rarely tolerate true mixed-initia -tie dialogue with tudent topi-hange that teer the oneration off oure and () mot tudent rarely hange topi, ak quetion, and pontaneouly gra the onerational floor. Intead, it i the tutor that drie the dialogue and lead the dane. Using Coh-Metrix to analyze four types of text on Newtonian physics Coh-Metrix wa ued to analyze the language and dioure of four type of dioure on Newtonian phyi: Textbook chapters , textoids , AutoTutor tutorial dialogue, and naturalistic tutorial dialogue . e text-ook orpu wa the firt 8 hapter from Hewitt’ 1998 textook on Conceptual Physics . e textoid orpu wa 12 phyi paage prepared y an den Broek and hi
idj (), , -
olleague for reearh on the ognitie proee that our during iene omprehenion (Kendeou & an den Broek, in pre). e AutoTutor orpu wa dialogue tranript from a pulihed experiment onduted on 10 phyi prolem with 22 tudent (Experiment 1 of VanLehn et al., 2007); there were 213 oneration total eaue een of them were inomplete. e human tutoring orpu inluded dialogue tranript on the ame 10 phyi prolem ut with a different group of 16 tudent, alo in Experiment 1 of VanLehn et al. (2007). e human tutor held oneration with the tudent through omputer mediated oneration. at i, the tudent and tutor were in different room and interated on omputer. e fie human tutor had Ph.D. in phy -i and were highly trained in pedagogy. e tudent in oth tutoring orpora were ollege tudent enrolled in a phyi oure.  e ontent of the four orpora were ery imilar in the ene that they oered Newtonian law of fore and motion. e goal wa to ahiee information equialene in domain knowledge among the four type of dioure o that differene in language and dioure ould e attriuted to the type of text (i.e., genre or regiter).  One way of iewing our eletion of the four text type i to ro two dimenion. One dimenion ditinguihe expoitory monologues that are deigned to e read (textook and textoid) from onerational dialogues (tutoring with human and AutoTutor). e language of the former i expeted to e more ompat, literate, and truturally dene than the language of dialogue that ha an affinity to the oral tradition (Bier, 1988; Tannen, 1982). Orthogonal to thi monologue-dialogue dimenion i a eond dimenion that ontrat natural and artifiial dioure. Textook and human tutoring are natural, eologially alid dioure ample, wherea textoid and AutoTutor interation hae ome modium of artifiiality. e reearher attempted to make the textoid and AutoTutor interation well-

Arthur C. Graesser et a l. Discourse cohesion in text and tutorial dialogue
trutured and oherent, of oure, ut in truth they are ontrained y a reearh agenda or omputational algo -rithm. It i an empirial quetion how loe the artifiial dioure ample are to the naturaliti ample.  We ued Coh-Metrix 2.0 to analyze the four orpora. Eah onerational turn wa treated a a paragraph in the analye of the two tutoring orpora. erefore, a turn in a dialogue i analogou to a paragraph in an expoitory monologue. Means and standard deviations of the Coh-Metrix indices Tale 1 preent the mean and tandard deiation of the Coh-Metrix indie, egregated y the four orpora. Tale 2 preent a follow-up analyi that egregate tutor turn from tudent turn within the two tutoring orpo -ra. In order to ae whether the mean ignifiantly differ from eah other, one an ompute 95% onfidene interal around eah mean. e general formula i Mean ± [1.96 * SD / SQRT(N)]. For example, the mean numer of negation per 1000 word in AutoTutor dialogue i 11.5 and the tandard deiation (SD) i 6.2. e 95% onfidene interal would e 11.5 ± [1.96 * 6.2 / SQRT(213)]. at i, ore etween 10.6 and 12.4 are not ignifiantly different from the mean of 11.5. It follow that the mean ore for negation in textook (7.9) and textoid (6.8) are learly outide of the range for AutoTu -tor, wherea the mean ore for human tutoring (10.6) i within the range for AutoTutor. e 95% onfidene interal for the textook i 7.9 ± 1.2, or 6.7 to 9.1; the negation in the textoid are within thi range, ut not the AutoTutor dialogue. e 95% onfidene interal for textoid i 2.3 to 11.3, o the textook are within thi range, wherea the AutoTutor dialogue are outide of the range.  T-tet an alo e mathematially deried from the mean and tandard deiation in thee tale. e et

idj (), , -
of t-tet would upport the following omparion on the inidene of negation: AutoTutor = human tutoring > textook = textoid. Our diuion of the data elow do not expliitly report inferential tatiti eaue the large numer of tatitial tet would e umerome. Howeer, our laim are upported y tatitial tet at alpha = .05 without adjutment for alpha inflation from multiple tet.
Simple measures of texts e top luter of indie in Tale 1 and 2 preent imple meaure of text. e expoitory monologue (textook and textoid) had a higher Fleh-Kinaid grade leel than the tutoring dialogue (AutoTutor and human tutoring). e higher grade leel an e attriuted to longer entene in the monologue eaue there were mall differene in yllale per word (reall that grade leel i aed on entene length and word length). e monologue alo had more entene in the paragraph than the tutoring eion had entene in the onera -tional turn. e flow of information in tutoring learly ha maller pakage of information (entene, para -graph) ditriuted oer more turn (i.e., paragraph) ompared with the expoitory monologue that are deigned to e read. Stated differently, tutoring i more fragmented and distributed than the dioure deigned for print.  A more detailed analyi of the tutoring an e deried from the data in Tale 2. e AutoTutor dialogue wa more fragmented and ditriuted than the human tutoring. AutoTutor had omparatiely more turn, fewer entene per turn, and fewer word per entene. For oth type of tutoring, the ontriution of the tudent were muh horter than the tutor. Mot of the tudent turn were only one entene with 8 or 9 word. It wa the tutor who did mot of the talking in oth AutoTutor and human tutoring. Eduational reearh -
Arthur C. Graesser et a l. Discourse cohesion in text and tutorial dialogue idj (), , - Table 1.  Means and standard deviations for the measures of Coh-Metrix by physics corpora Textbook Textoids AutoTutor Human  tutoring tutoring SIMPLE MEASURES OFTEXTS Number of texts 8 12 213 160 Total number of words in the t ext 5967 (2333) 177 (15) 913 (369) 406 (215) Total number of sentences in the t ext 329 (136) 14.4 (2.5) 104.6 (49.7) 36.9 (21.2) Total number of paragraphs/turns in the t ext 76.6 (27.7) 3.9 (1.08) 46.5 (22) 15.5 (11.8) Average words per sentenc e 18.2 (.86) 12.5 (1.97) 9.3 (1.73) 11.5 (2.83) Average sentences per paragraph/tur n 4.3 (.58) 4 (1.54) 2.3 (.17) 2.7 (.84) Average syllables per wor d 1.51 (.03) 1.45 (.13) 1.45 (.06) 1.43 (.08) Flesch-Kincaid Grade level (0-12 ) 9.3 (.46) 6.4 (1.55) 5.2 (.8) 5.8 (1.53) WORD LEVEL Logarithm of frequency of content words 2.15 (.05) 2.3 (.14) 2.27 (.11) 2.31 (.16) Incidence score of all connectiv es 69.3 (5.9) 71.1 (20.4) 59.9 (13.4) 71.5 (52.8) Incidence of positive causal connectiv es 9.4 (2.2) 12.4 (11.8) 11.9 (5.5) 18.4 (16.7) Incidence of negative causal connectiv es 1.35 (.52) 2.72 (4.35) .53 (.8) .36 (1.04) Incidence of positive additive connectiv es 22 (2.8) 29.5 (11.1) 22.5 (8.7) 22.7 (21.2) Incidence of negative additive connectiv es 10.7 (1.8) 6 (5.4) 3.9 (2.6) 6.1 (5.6) Incidence of positive temporal connectiv es 10.4 (2.9) 10.6 (8.9) 14.6 (6) 12.3 (11.8) Incidence of negative temporal connective s .29 (.29) .48 (1.66) .1 (.38) .56 (1.77) Incidence of all logical operators (and +if+or+cond+n eg) 38 (2.7) 32.6 (17.6) 34.6 (9.9) 36.9 (27.6) Incidence of conditionals in the te xt 5.19 (.83) 1.33 (2.41) 4.82 (3.03) 5.38 (4.26) Incidence of negations in the t ext 7.9 (1.7) 6.8 (7.9) 11.5 (6.2) 10.6 (8) SYNTAX Words before main verb of main clause in sentences 5.21 (.35) 3.56 (1.11) 2.52 (.64) 2.76 (1.18) Average number of modifiers per noun phr ase .93 (.07) .75 (.17) .87 (.07) .86 (.13) REFERENTIAL AND SEMANTIC COHESIO N Argument overlap of adjacent Sentences .66 (.03) .53 (.26) .24 (.07) .35 (.12) Stem overlap of adjacent sentenc es 64 (.03) .49 (.25) .23 (.07) .3 (.11) LSA cosine of adjacent sentence to sente nce .36 (.02) .28 (.12) .19 (.08) .21 (.11) LSA cosine of paragraph/turn to paragraph/tu rn .48 (.07) .45 (.18) .32 (.09) .26 (.16) Anaphor pronominal coreference of adjacent sentenc es .26 (.07) .29 (.21) .09 (.04) .21 (.1) Type-token ratio of all content wo rds .36 (.03) .73 (.09) .37 (.09) .57 (.1) SITUATION MODEL DIMENSION S Causal cohesion: Causal particles divided by causal verbs .33 (.08) .26 (.24) .26 (.12) .4 (.22) Intentional cohesion: Intentional particles / intentional actions 1.49 (.2) 1.04 (1.62) .58 (.29) .94 (.8) Temporal cohesion:Tense and aspect repetition scores . (.) . (.) . (.) . (.) 
Arthur C. Graesser et a l. Discourse cohesion in text and tutorial dialogue
Table 2.  Means and standard deviations for students and tutor turns  AutoTutor  (Tutor  Turns)   SIMPLE MEASURES OFTEXTS Number of texts 213 Total number of words in the t ext 725 (329) Total number of sentences in the t ext 77.7 (38.8) Total number of paragraphs/turns in the t ext 23.4 (11) Average words per sentenc e 9.8 (1.77) Average sentences per paragraph/tur n 3.3 (.31) Average syllables per wor d 1.44 (.06) Flesch-Kincaid Grade level (0-1 2) 5.2 (.84) WORD LEVEL Logarithm of frequency of content words 2.29 (.11) Incidence score of all connectiv es 59.4 (14.2) Incidence of positive causal connectiv es 10.1 (5.6) Incidence of negative causal connectiv es .61 (.99) Incidence of positive additive connectiv es 22.9 (9.4) Incidence of negative additive connectiv es 3.9 (2.4) Incidence of positive temporal connectiv es 15.6 (6.7) Incidence of negative temporal connective s .0 (.0) Incidence of all logical operators (and +if+or+cond+n eg) 32.8 (11.2) ncidence of conditionals in the te xt 5.16 (3.01) Incidence of negations in the te xt 9.8 (6.9) SYNTAX Words before main verb of main clause in sentences 2.72 (.72) Average number of modifiers per noun phr ase .87 (.07) REFERENTIAL AND SEMANTIC COHESIO N Argument overlap of adjacent Sentences .24 (.07) Stem overlap of adjacent sentenc es .21 (.06) LSA cosine of adjacent sentence to sente nce .15 (.08) LSA cosine of paragraph/turn to paragraph/tu rn .44 (.11) Anaphor pronominal co-reference of adjacent sentenc es .09 (.05) Type-token ratio of all content wo rds .42 (.12) SITUATION MODEL DIMENSIONS Causal cohesion: Causal particles divided by causal verbs .21 (.11) Intentional cohesion: Intentional particles / intentional actions .53 (.27) Temporal cohesion:Tense and aspect repetition score s . (.)

idj (), , -
AutoTutor Human Human (Student Tutoring Tutoring Turns) (Tutor (Student  Turns) Turns)  213 160 160 190 (91) 309 (153) 102 (92) 26.9 (11.7) 26.7 (14.1) 10.2 (7.9) 23.1 (11) 7.9 (5.9) 7.5 (5.9) 7.6 (3.04) 11.9 (3.04) 9.4 (5.13) 1.2 (.2) 3.9 (1.38) 1.4 (.56) 1.51 (.11) 1.44 (.07) 1.41 (.17) 5.2 (1.48) 6.0 (1.51) 4.9 (2.84) 2.2 (.17) 2.3 (.13) 2.36 (.3) 59 (21.9) 64.1 (19.7) 64.3 (57.2) 17.6 (11.4) 15.4 (8) 19.3 (25.3) .19 (.95) .29 (1.03) .45 (1.86) 19.8 (14.4) 20.5 (10.6) 18.1 (21.7) 4 (6.5) 5.4 (5.1) 7.2 (10.9) 10.6 (10) 11.2 (6.8) 11 (18) .43 (1.87) .29 (1) 1.05 (4.75) 40.4 (16.6) 31.1 (12.2) 44.6 (39.8) 3.27 (5.91) 5.36 (4.55) 3.76 (7.28) 18.4 (12.5) 7.5 (5.8) 24.5 (35.7) 1.92 (1.23) 2.91 (1.43) 2.06 (1.91) .85 (.18) .86 (.14) .68 (.32) .24 (.15) .37 (.14) .28 (.25) .25 (.16) .31 (.14) .23 (.25) .23 (.09) .19 (.09) .23 (.15) .20 (.08) .34 (.16) .25 (.17) .10 (.08) .23 (.12) .17 (.19) .49 (.12) .62 (.1) .76 (.16) .47 (.7) .36 (.23) .39 (.41) .84 (1.25) .82 (.74) .78 (1.12) . (.) . (.) . (.)
Arthur C. Graesser et a l. Discourse cohesion in text and tutorial dialogue
er enourage atie learning on the part of the tudent, with attempt to get the tudent to do the talk and ation. Howeer, thi i a hallenge een in one-on-one tutoring. Word-level indices e word in the four type of dioure did not appre -ialy ary in word frequeny ut differene did emerge in onnetie, onditional, and negation. e oerall inidene of onnetie wa lower in AutoTutor than the other three genre, whih were approximately the ame. e ditriution of uategorie of onnetie differed among the dioure type. Howeer, it i diffi -ult to diern any imple piture from the data.  Differene that emerged etween AutoTutor and human tutoring appear in Tale 2. e tudent learning from AutoTutor had fewer negatie onnetie, nega -tion, logial operator, and onditional expreion, ut approximately the ame numer of poitie aual, addi -tie, and temporal onnetie. Although thi ugget that the human tutor extrated more omplex analytial ontent from the tudent than did AutoTutor, the ditri -ution of word ategorie wa ery imilar for the tutor turn in AutoTutor and the human tutor. i upport the laim that the automated tutor did a reaonale jo imulating the human tutor, at leat from the perpe -tie of the ditriution of word ategorie. Syntax e yntati ompoition of entene ytematially differed among the four type of dioure. e expoi -tory monologue had more omplex yntax than the tutoring dialogue when we examined the mean numer of word efore the main er of the main laue, whih reflet a greater load on working memory. e textook learly had the highet ore on thi yntati index. Within the tutoring dioure, the tudent ontriution
idj (), , -
were not different on thi index for AutoTutor eru human tutor; the tutor ontriution were alo not ignifiantly different for AutoTutor eru human tutor. e other meaure of yntati omplexity, namely the mean numer of modifier per noun-phrae, wa not remarkaly different among the dioure type. Referential and semantic cohesion Referential and emanti oheion wa onitently higher for the expoitory monologue than the tutor -ing dialogue when we examined argument oerlap, tem oerlap, and LSA ore. erefore, in addition to tutoring eing more fragmented and ditriuted, the dioure alo ha lower oheion on thee referential and emanti indie. When the two type of expoi -tory monologue were ompared, oheion wa higher for the textook than the textoid. Unfortunately, it i diffiult to interpret the anaphor pronominal referene index eaue the meaure onfound the inidene of pronoun and the likelihood that the pronoun refer -ent an e reoled; additional analye will need to e onduted to differentiate thee two omponent. Neerthele, the ore were higher for expoitory monologue than the tutoring dialogue.  e type-token index of lexial dierity howed extremely high ore for the textoid, followed y human tutoring, and the lowet for the textook and AutoTutor. erefore, there i more redundany in the ontent word of the textook and AutoTutor. e textoid are ery paked with information. Apparently experimental pyhologit fill their timulu text with a large amount of new information, muh more than the profeional writer of textook.  In-depth analye of the tutoring eion unoered a few informatie reult. e tudent ontriution in AutoTutor had lower argument oerlap, LSA-turn imilarity, anaphor pronominal referene, and lexi -
