Effective-Learning Assignment Paper

Available online at www.sciencedirect.com

www.elsevier.com/locate/actpsy

Acta Psychologica 128 (2008) 110–118

Why do delayed summaries improve metacomprehension accuracy?

Mary C.M. Anderson a, Keith W. Thiede b,*

a College of DuPage, 425 Fawell Boulevard, Glen Ellyn, Illinois 60137, United States

b Boise State University, 1910 University Drive, Boise, Idaho 83725-1745, United States

Received 23 March 2007; received in revised form 30 October 2007; accepted 31 October 2007 Available online 19 December 2007

Abstract

We showed that metacomprehension accuracy improved when participants (N = 87 college students) wrote summaries of texts prior to judging their comprehension; however, accuracy only improved when summaries were written after a delay, not when written imme- diately after reading. We evaluated two hypotheses proposed to account for this delayed-summarization effect (the accessibility hypoth- esis and the situation model hypothesis). The data suggest that participants based metacomprehension judgments more on the gist of texts when they generated summaries after a delay; whereas, they based judgments more on details when they generated summaries immediately after reading. Focusing on information relevant to the situation model of a text (the gist of a text) produced higher levels of metacomprehension accuracy, which is consistent with situation model hypothesis. Published by Elsevier B.V. Effective-Learning Assignment Paper

PsycINFO classification: 2343

Keywords: Metacomprehension; Metacognition; Self-regulated learning

1. Introduction

Models of self-regulated learning (e.g. Dunlosky & Thi- ede, 1998; Metcalfe, 2002; Nelson & Narens, 1990; Thiede & Dunlosky, 1999; Winne & Hadwin, 1998) describe learn- ing as a dynamic process that involves setting a goal for learning, monitoring whether that goal has been met, and regulating study in order to achieve that goal. Although there may be debate about what constitutes optimal regu- lation of study (for a description of regulation according to a discrepancy-reduction perspective, see Dunlosky & Thiede, 2004; and for a region of proximal learning per- spective, see Metcalfe & Kornell, 2003), it is generally accepted that accurate monitoring of learning is essential for effective regulation.

Thiede, Anderson, and Therriault (2003) showed the important role that accurate monitoring plays in learning by manipulating monitoring accuracy and examining the

0001-6918/$ – see front matter Published by Elsevier B.V.

doi:10.1016/j.actpsy.2007.10.006

* Corresponding author. Tel.: +1 208 426 1278; fax: +1 208 426 4006. E-mail address: KeithThiede@boisestate.edu (K.W. Thiede).

effect this played on regulation of study, and learning. Participants who more accurately monitored their com- prehension made better decisions about which texts to reread than did participants who less accurately moni- tored their comprehension. That is, for the group with higher accuracy, the mean proportion correct on tests of texts selected for rereading versus those not selected for rereading was .27 versus .78, respectively – indicating that participants compensated for poorer comprehension with additional restudy of texts. By contrast, for groups with lower accuracy, the mean proportion correct on tests of texts selected for rereading versus those not selected for rereading was .43 versus .53, respectively – indicating that participants compensated for poorer comprehension with additional restudy of texts but to a lesser degree than did the group with higher accuracy. More important, more effective regulation of study (selection of texts for rereading) led to higher overall reading comprehension. Given that improving monitoring accuracy can improve learning, it is important to find ways to improve monitor- ing accuracy. Effective-Learning Assignment Paper

 

 

M.C.M. Anderson, K.W. Thiede / Acta Psychologica 128 (2008) 110–118 111

Although research has revealed ways of improving the accuracy of metacognitive monitoring, most of this research has involved monitoring of learning in associative learning tasks (e.g. learning paired associates). For instance, Nelson and Dunlosky (1991) found that delaying judgments of learning produced high levels of monitoring accuracy. Shaughnessy and Zechmeister (1992) showed that practice tests produced a substantial improvement in monitoring accuracy in associative learning tasks. However, many of the factors that have improved monitoring accuracy in asso- ciative learning tasks have not produced the same effects on accuracy of comprehension monitoring (called metacom- prehension accuracy). For instance, Maki (1998a) failed to replicate the delayed judgment of learning effect for a reading task. Furthermore, Maki and Serra (1992) showed that practice tests had only a modest effect on metacompre- hension accuracy. Thus, more research is needed to identify techniques to improve metacomprehension accuracy.

One method used to measure metacomprehension accu- racy is to have participants read several texts and then judge how well they will perform on a test over the content of each one. After all texts have been read and judged, par- ticipants take a test of comprehension for each text. Meta- comprehension accuracy is then estimated by correlating each participant’s judgments with his or her own test per- formance. Higher correlations indicate that the person can more accurately discriminate between texts that are more versus less understood. Effective-Learning Assignment Paper

Although metacomprehension accuracy is often quite poor (for a review, see Maki, 1998b), several techniques have recently been discovered that can substantially improve accuracy. Namely, accuracy is better (a) after rereading than after a single reading (e.g. see Dunlosky & Rawson, 2005, for caveats and review), (b) when keywords are generated for each text, but only if generation is delayed after reading (Thiede, Dunlosky, Griffin, & Wiley, 2005), and likewise, (c) when summaries are written, but only after a delay (Thiede & Anderson, 2003). In the latter case of summarization, which is the focus of the current research, metacomprehension accuracy reached a level of +.57 when summaries were written after all texts had been read (delayed-summarization) but was only +.26 when summaries were written immediately after each text had been read (immediate summarization).

Thiede and Anderson (2003) established summarization as a technique for improving metacomprehension accu- racy. Moreover, they speculated as to the possible cause of the delayed-summarization effect, using construction- integration model of text comprehension (Kintsch, 1988). According to this model, while reading, readers construct several levels of representation for a text: a lexical or sur- face level, a textbase level, and a situation model level. The lexical level contains the surface features of the text. The textbase level is constructed as segments of the surface text are parsed into propositions, and as links between text propositions are formed based on argument overlap and other text-explicit factors. Deeper understanding of the text

is constructed at the level of the situation model, which involves connecting text information with the reader’s prior knowledge. Thiede and Anderson (2003) suggested that writing summaries after a delay focused readers on their situation model, and this produced the boost in accu- racy (see also Rawson, Dunlosky, & Thiede, 2000 for a similar explanation of why rereading improved metacom- prehension accuracy). However, it is important to note that Thiede and Anderson (2003) did not evaluate possible hypotheses proposed to explain the delayed-summarization effect – in large part because their between-subjects design was insensitive to difference in summary length and quality. A primary purpose of the present investigation was to show the delayed-summarization effect using a within-subjects design and, more important theoretically, then conduct content analyses of summaries to evaluate two hypotheses proposed to explain the effect. Effective-Learning Assignment Paper

One hypothesis proposed to account for this delayed-sum- marization effect is the accessibility hypothesis, which states that metacognitive judgments are based on the amount of information accessed from memory (Baker & Dunlosky, 2006; Benjamin, Bjork, & Schwartz, 1998; Koriat, 1993; Morris, 1990; Serra & Dunlosky, 2005). According to this hypothesis, accessibility of information at the time of sum- marization provides cues to be used to judge subsequent test performance. For instance, if a person can retrieve a lot about a text, he or she will judge the text as better under- stood; whereas, if the person can retrieve very little about a text, he or she will judge the text as less understood. It is important to note that according to this hypothesis, judg- ments are based on the amount retrieved from memory and not the content of that which is retrieved (i.e. details or gist).

When summaries are written immediately after reading, information about a text has not had a chance to fade from working memory; therefore, a person has access to infor- mation for writing a summary even for texts that were not well understood. As a result, summaries of less under- stood and more understood texts are more similar when written immediately after reading than after a delay. There- fore, immediate summarization may produce a more homogeneous set of cues, which makes it more difficult to discriminate between less understood and more under- stood texts, thereby reducing metacomprehension accu- racy. By contrast, when writing summaries after a delay, information has had a chance to fade from memory; there- fore, the person has access to relatively little for texts that were not understood and more for texts that were under- stood. As a result, delayed-summarization may produce a less homogeneous set of cues, which makes it easier to dis- criminate between less understood and more understood texts, thereby improving metacomprehension accuracy. Effective-Learning Assignment Paper

A key prediction derived from the accessibility hypothe- sis is that metacomprehension judgments will correlate with the total number of idea units contained in summaries. As the amount produced is more homogeneous in immediate summaries than delayed summaries, it follows that the cor- relation between metacomprehension judgments and the

 

 

112 M.C.M. Anderson, K.W. Thiede / Acta Psychologica 128 (2008) 110–118

total number of idea units contained in summaries will be weaker for summaries written immediately after reading than after a delay – due to attenuation of variance in amount produced by an individual. Given that the total number of ideas is correlated with test performance, this would account for the difference in metacomprehension accuracy for immediate and delayed-summarization conditions.

An alternative to the accessibility hypothesis is the situ- ation model hypothesis, which states that metacomprehen- sion accuracy will increase as the judgment of comprehension for a text is based on information relevant to one’s situation model for the text. According to the sit- uation model hypothesis, because an appropriate situation model is important for test performance (McNamara, Kin- tsch, Songer, & Kintsch, 1996), metacomprehension judg- ments based on information relevant to the situation model for a text will support higher levels of metacompre- hension accuracy. Effective-Learning Assignment Paper

The situation model hypothesis has been proposed to account for the findings of a number of studies. In particu- lar, Rawson et al. (2000) suggested that metacomprehension accuracy was greater for participants who read texts twice (versus once) because those participants who were given an opportunity to reread could allocate resources not only to developing a better situation model but also to monitor- ing their situation model. This produced the boost in accu- racy. The situation model hypothesis has also been proposed to account for boost in accuracy associated with generating summaries (Thiede & Anderson, 2003) and key- words (Thiede et al., 2005) prior to judging comprehension.

In contrast to the accessibility hypothesis, this hypothe- sis suggests that accuracy is driven less by the total number of ideas in the summaries and more by the number of ideas produced that are relevant to the situation model. A pre- diction derived from this hypothesis is that the correlation between metacomprehension judgments and the number of gist idea units will be weaker for summaries written imme- diately after reading than after a delay. Given that the number of gist ideas produced is correlated with test performance, this would account for the difference in metacomprehension accuracy for immediate and delayed- summarization conditions.

It is important to note that the situation model hypoth- esis makes predictions specifically about metacomprehen- sion accuracy (versus metamemory, which addresses metacognitions related to memory of the details contained in texts, see Wiley, Griffin, & Thiede, 2005, for a discussion of the difference between metacomprehension and metame- mory for text learning). Thus, in the present experiment, we assessed comprehension in a way consistent with compre- hension research (e.g. Graesser & Bertus, 1998; Royer, Carlo, Dufresne, & Mestre, 1996), namely, using inference questions. This practice is consistent with the early research in metacomprehension – Glenberg and Epstein (1985) and Weaver (1990) used only inference tests in order to tap comprehension as defined by comprehension theory. Effective-Learning Assignment Paper

2. Method

2.1. Participants

Ninety-six undergraduate students enrolled in an intro- ductory psychology course began the study; nine (9.4%) failed to complete all three sessions and were dropped from the study. Of the 87 participants with complete data sets, 48 were female and 39 were male; 63 (72%) were Caucasian and 24 (28%) were minority. ACT scores (the ACT is a standardized college entrance examination, American Col- lege Testing Program, 1986) ranged from 19 to 36, with a mean of 22.7 (SEM = .42).

2.2. Materials

The texts were adapted from ACT test preparation materials. They ranged in length from approximately 600 to 800 words, and had an average Flesch–Kincaid Grade Level of 11.4. The Flesch–Kincaid provides an index of reading ease (Flesch, 1948; for a discussion of this and other readability measures, see Klare, 1974–1975). We con- structed three sets of texts with a balance of topics from three general categories: natural science, social science, and humanities. Tests were developed for each text, which contained 10 multiple-choice items designed to assess com- prehension (inference making ability or application), rather than memory of details contained in the text. A sample text and test item can be found in the Appendix.

2.3. Design

In this study, summarization condition was a within- subjects variable. That is, each participant completed each condition of the comprehension monitoring tasks: no-sum- mary, immediate-summary, and delayed-summary. Exper- imental sessions were separated on average by five days. A Latin-square design was used to counterbalance the order of conditions. A set of preliminary analyses revealed there were no significant order effects or interactions with outcome variables, all Fs < 1.1, p > .10. Therefore, we col- lapsed across order for all subsequent analyses. Computers controlled text presentation and data collection, including summaries and responses to interview questions. Effective-Learning Assignment Paper

2.4. Procedure

All participants were instructed that they would read texts on a computer screen, rate their comprehension for each text, and then answer test questions for each text. They were also instructed that they might be asked to write a summary for some of the texts. Finally, they were instructed that they would respond to some open–ended questions regarding the tasks in the experiment, which were designed to ascertain the cues used by participants to judge their comprehension of texts.

Following instructions for the first session, the partici- pants read a sample text and rated their comprehension

 

 

Table 1

M.C.M. Anderson, K.W. Thiede / Acta Psychologica 128 (2008) 110–118 113

of the text. The metacomprehension judgment was prompted (as in Glenberg & Epstein, 1985) with the title of the text at the top of the computer screen and the ques- tion, How well do you think you understood the passage whose title is listed above? 1 (very poorly) to 7 (very well).

After judging their comprehension, participants were asked to describe what they used to judge their comprehension of the practice text. After typing their response, they answered sample test questions.

For the critical trials, the order of text presentation within the condition was randomized for each participant. In the no-summary condition, participants read five texts. After reading, they judged their comprehension for each of those texts. After judging their comprehension of the last text, par- ticipants responded to two open–ended questions regarding the cues used to judge comprehension of the set of texts. The participants then answered the 10 multiple-choice questions for each text. The texts were judged for comprehension and tested in the same order as they were presented for reading. Effective-Learning Assignment Paper

In the immediate-summary condition, participants read the first text displayed on the screen. They were then shown the title of the text and instructed to write a summary of that text. Once they finished writing the summary, they were presented the next text. They read and immediately wrote a summary of each of the five texts. After writing the summary of the last text, participants judged their com- prehension of each text, they then responded to the two open–ended questions regarding the cues used to judge comprehension of the set of texts. Finally, they answered the 10 multiple-choice test questions for each text.

In the delayed-summary condition, participants read all five texts. They were then shown the title of the first text they had read and were instructed to write a summary of that text. When they finished writing this summary, they were presented the next title and summarization instruc- tions. After writing a summary of the last text, participants judged their comprehension of each text and responded to the open–ended questions regarding the cues used to judge comprehension of the set of texts. Finally, participants answered the 10 multiple-choice test questions for each text. Effective-Learning Assignment Paper

For each condition, after answering the last multiple- choice test question, participants were presented with the number of questions they had correctly answered over all five tests. That is, they received feedback regarding overall performance; they did not receive feedback regarding per- formance on a test for a particular text. For the immedi- ate-summary and delayed-summary conditions, the participants were then asked whether they had used sum- maries to judge their comprehension of the texts.1

1 A major goal of the present investigation was to evaluate the accessibility hypothesis and the situation-model hypothesis – two hypoth- eses proposed to explain the effect of delayed-summarization on meta- comprehension accuracy. The interview questions were not used to evaluate these hypotheses, rather these data were used to examine the relationship between metacomprehension accuracy and self-reported cue use; therefore, these data will not be discussed in this paper.

3. Results and discussion

3.1. Metacomprehension accuracy

3.1.1. Test performance and comprehension ratings

Metacomprehension accuracy describes the relations between metacomprehension judgments and performance on a test of reading comprehension; therefore, descriptive statistics of these variables are reported first. The median proportion of correct test response and comprehension rat- ing across the five texts was computed for each participant. The median is the recommended measure of central ten- dency for small sets of scores where extreme scores could affect the mean (Gravetter & Wallnau, 1999). The mean of the medians was then computed across participants within each summarization condition. A one-way ANOVA revealed that test performance differed across conditions, F(2, 148) = 3.9, MSE = .01, p = .02, g2 = .05. Follow-up analyses showed that test performance was greater in the immediate-summary condition than in the delayed-sum- mary condition, t(74) = 2.8, p < .01; no other differences were significant. Metacomprehension judgments differed across conditions, F(2, 148) = 2.9, MSE = .75, p < .05, g2 = .04. The rightmost column of Table 1 shows that par- ticipants were more confident about their comprehension in the immediate-summary condition than in the no-sum- mary condition, t(74) = 2.3, p = .03; no other differences were significant. Effective-Learning Assignment Paper

3.1.2. Metacomprehension accuracy

Metacomprehension accuracy was operationalized as the Goodman–Kruskal gamma correlation between com- prehension judgments and performance on a test of reading comprehension computed across texts (for a rationale for using gamma see Nelson, 1984). Three gamma correlations were computed for each participant, one for each summa- rization condition. The mean intra-individual correlation was then computed across participants for each condition. Twelve participants had indeterminate correlations due to invariance in comprehension judgments. A one-way ANOVA revealed that metacomprehension accuracy dif- fered across conditions, F(2, 148) = 19.6, MSE = .20, p < .001, g2 = .21. Follow-up tests showed that metacom prehension accuracy was significantly higher for the delayed-summary condition than for the immediate-sum- mary condition, t(74) = 5.0, p < .001, or the no-summary

Test performance and metacomprehension judgments

Conditions Test performance Metacomprehension rating

No summary .62 (.02) 4.39 (.15) Immediate-summary .65 (.02) 4.72 (.13) Delayed-summary .60 (.02) 4.61 (.16)

Note: Entries are mean across individual’s median test performance and median metacomprehension ratings. Values in parentheses are standard errors of the means.

 

 

0.0

0.2

0.4

0.6

0.8

No Summary

Immediate Summary

Delayed Summary

M et

ac om

pr eh

en si

on A

cc ur

ac y

Fig. 1. Mean metacomprehension accuracy by condition. The error bars represent the standard error of the mean.

Table 2 Content of summaries by condition

Gist Detail All ideas

Immediate summaries 1.1 (.07) 1.8 (.12) 2.8 (.17) Delayed summaries 0.7 (.05) 1.2 (.13) 2.0 (.16)

Note: Entries are mean across individual’s median proportion of main ideas, supporting ideas, and all ideas. Values in parentheses are standard errors of the means.

114 M.C.M. Anderson, K.W. Thiede / Acta Psychologica 128 (2008) 110–118

condition, t(74) = 6.3, p < .001 (see Fig. 1)2. Thus, we rep- licated the delayed-summarization effect using a repeated- measures design. The advantage to this design is that it allows us to evaluate the accessibility hypothesis versus the situation model hypothesis – two hypotheses proposed to explain the delayed-summarization effect. To evaluate these hypotheses, we conducted a content analysis of the summaries produced in the immediate- and delayed-sum- mary conditions.

3.1.3. Content analysis

For each text, a research assistant, who was blind to the condition, parsed summaries into idea units and then coded each idea unit as a gist or detail unit. Summaries received three scores: (a) the number of main ideas written, (b) the number of details written, and (c) the total number of ideas written. We randomly selected 26 participants (approximately 30% of cases) and had a second research assistant coded their summaries. The raters agreed on cod- ing 98% of the time. For each participant, the median of each score was then computed across the five summaries

2 We computed the correlation between ACT scores and metacompre- hension accuracy for each condition. The correlation was not significantly different from zero in the delayed-summary condition (r = �.04, p > .10), the immediate-summary condition (r = �.12, p > .10), or the no-summary condition (r = .07, p > .10). These findings are consistent with Maki, Jonas, and Kallod (1994), and Maki, Shields, Wheeler, and Zacchilli (2005) who also failed to show a significant relation between standardized reading scores and metacomprehension accuracy. Given the weak relation between ACT scores and metacomprehension accuracy, it was not included as a covariate in subsequent analyses.

written in the immediate-summary condition as well as the delayed-summary condition. The mean of the median was computed across participants for the summarization conditions (see Table 2). Participants produced more idea units when summaries were written immediately after read- ing than after a delay; this held for gist, t(74) = 6.2, p < .001, and details, t(74) = 5.1, p < .001. Effective-Learning Assignment Paper

3.1.4. The accessibility hypothesis versus the situation model

hypothesis

To evaluate the degree to which summaries may have served as cues for metacomprehension judgments, for each participant, we computed three gamma correlations across the five texts in each summarization condition. That is, for both immediate summaries and for delayed summaries, we computed an intra-individual correlation between meta- comprehension judgments and the number of gist idea units, the number of details, and the total number of idea units.

As noted above, the accessibility hypothesis suggests that metacomprehension judgments are based on the amount retrieved from memory and not the content of that which is retrieved. The key to evaluating this hypothesis is the comparison of the correlation between metacompre- hension judgments and the total number of idea units for the immediate-summary and the delayed-summary condi- tions. This hypothesis predicts that the correlation will be weaker for summaries written immediately after reading than after a delay. As seen in Table 3, the magnitude of the correlation between metacomprehension judgments and total number of idea units did not differ substantially across conditions, t(74) = 0.4, p > .10. This finding is not consistent with the accessibility hypothesis. Effective-Learning Assignment Paper

An alternative to the accessibility hypothesis is the situ- ation model hypothesis, which suggests that accuracy is dri- ven less by the total number of ideas in the summaries and more by the number of ideas produced that are relevant to the situation model. The key to evaluating the situation

Table 3 Mean intra-individual correlation between metacomprehension judgments and the number of gist, supporting, and total idea units

Gist Details All ideas

Immediate summaries .11 (.08) .57 (.05) .49 (.05) Delayed summaries .63 (.06) .22 (.08) .46 (.07)

Note: Entries are mean across individual’s gamma correlations. Values in parentheses are standard errors of the means.

 

 

Table 4 Mean intra-individual correlation between test performance and the number of gist, supporting, and total idea units

Gist Details All ideas

Immediate summaries .51 (.06) .20 (.07) .41 (.05) Delayed summaries .61 (.05) .14 (.07) .33 (.06)

Note: Entries are mean across individual’s gamma correlations. Values in parentheses are standard errors of the means.

M.C.M. Anderson, K.W. Thiede / Acta Psychologica 128 (2008) 110–118 115

model hypothesis is the comparison of the correlation between metacomprehension judgments and the number of gist idea units. This hypothesis predicts the correlation will be weaker for summaries written immediately after reading than after a delay. A 2 (timing of summariza- tion) · 2 (kind of idea: gist versus detail) ANOVA revealed that neither main effect was significant, both F(1, 69) < 1.6, MSE = .32, p > .10. However, the interaction was signifi- cant, F(1, 69) = 41.3, p < .001, g2 = .37.

The interaction suggests that different aspects of sum- maries are related to metacomprehension judgments in the immediate-summary condition than in the delayed- summary condition. As seen in the first and second columns of Table 3, for delayed summaries, metacompre- hension judgments were related to a greater degree with gist idea units than with details, t(70) = 5.1, p < .001. By contrast, for immediate summaries, metacomprehension judgments were related to a greater degree with details than with gist idea units, t(73) = 4.7, p < .001. Thus, metacom- prehension judgments were related to different aspects of the summaries depending on whether the summaries were written immediately after reading or after a delay. Effective-Learning Assignment Paper

Consistent with the situation model hypothesis, in the delayed-summary condition, in which metacomprehension accuracy was superior, participants appeared to have based metacomprehension judgments on how well they could produce the gist of a text, which presumably are related to one’s situation model. By contrast, in the immediate- summary condition, in which metacomprehension accu- racy was inferior, participants appeared to have based metacomprehension judgments on details of a text, which are not related to one’s situation model.

These findings might suggest that writing immediate summaries would support higher levels of monitoring accu- racy had the tests consisted of items that tapped memory of details (i.e. transfer appropriate monitoring, Dunlosky & Nelson, 1992). As noted above, we did not include detail questions on our tests (but rather included only inference questions, which are critical to evaluating the situation model hypothesis); however, Thiede and Anderson (2003) did and they found that monitoring accuracy was not influ- enced by the kind of test. In particular, when tests were inference questions, accuracy was greater for the delayed- summary group (M = .55) than for the immediate-sum- mary group (M = .28). Likewise, when tests were detail questions, accuracy was greater for the delayed-summary group (M = .57) than for the immediate-summary group (M = .29). Thus, the kind of test did not affect the delayed-summary effect.

As Thiede and Anderson did not evaluate hypotheses proposed to explain the delayed-summary effect, they did not conduct the content analyses conducted here. How- ever, we went back to that data set and conducted these analyses. In the immediate-summary condition, we found a similar pattern of results. That is, metacomprehension judgments were related to a marginally greater degree with details (mean c = .34, SEM = .08) than with gist idea units

(mean c = .10, SEM = .09), t(28) = 2.0, p = .06. Thus, even when metacomprehension judgments are related to the number of details generated in summaries, this does not promote highly accurate predictions of how well one will recall details on a subsequent test of details.

Although it may seem surprising that writing immedi- ate summaries did not improve monitoring of details, Dunlosky and Lipko (in press) argued that a global judgment of comprehension will not accurately reflect memory of specific details because of a mismatch in grain size between the judgment and content being mon- itored. Even with the mismatch reduced, Dunlosky, Raw- son, and Middleton (2005) showed that students are not adept at monitoring memory of specific details contained in texts, but with very specific feedback and directed self- monitoring, students can become quite accurate (Rawson & Dunlosky, 2007). However, it may be that left to their own devices, people are simply not adept at using a sin- gle global judgment of comprehension to reflect memory of details contained in texts. Effective-Learning Assignment Paper

The finding that writing delayed summaries improved the accuracy of monitoring even when tests were tapping details (in Thiede & Anderson, 2003) may also seem sur- prising. It may be that having a well-formed situation model for a text may provide a structure for better remem- bering details contained in a text (Bransford & Johnson, 1972). Therefore, monitoring the quality of one’s situation model may also provide an assessment of one’s knowledge of details. Thus, monitoring the situation model will improve the accuracy of monitoring regardless of the kind of test (as shown by Thiede & Anderson, 2003).

Although the analyses of monitoring memory of details is interesting, it is important to note that the situation model hypothesis makes no prediction regarding the accu- racy of this kind of monitoring. Instead, it predicts greater metacomprehension accuracy for conditions that focus readers on their situation model during monitoring – given that tests will tap into the reader’s situation model. Effective-Learning Assignment Paper

To understand how the above results might affect meta- comprehension accuracy, we have to consider the validity of the summaries as cues for judgments (i.e. examine the relation between summaries and test performance). We computed three correlations across the five texts in each summarization condition. That is, for both immediate sum- maries and for delayed summaries, we computed an intra- individual correlation between test performance and the number of gist idea units, the number of details, and the total number of idea units (see Table 4).

 

 

116 M.C.M. Anderson, K.W. Thiede / Acta Psychologica 128 (2008) 110–118

To further evaluate which aspects of summaries are more predictive of test performance, we conducted 2 (tim- ing of summarization) · 2 (kind of idea) ANOVA. As seen in Table 4, gist idea units were more predictive of test per- formance than were details, F(1, 69) = 41.1, MSE = .24, p < .001, g2 = .37. This is a critical finding because it indi- cates that gist idea units would serve as a more valid cue for metacomprehension judgments than would details. The main effect for timing of summarization was not signifi- cant, F(1, 69) < 1, nor was the interaction, F(1, 69) = 1.9, p > .10. Thus, as writing delayed summaries focuses read- ers on cues that are more predictive of test performance, it follows that metacomprehension accuracy will increase (Koriat, 1997).

4. General discussion

The present study extended the findings of (Thiede & Anderson, 2003) and showed that generating summaries after a delay (versus immediately after reading) improved metacomprehension accuracy – even in a within-subjects design. This is important because it suggests that people can change their monitoring behavior from one context to another (e.g. from an immediate-summary condition to a delayed-summary condition). Unfortunately, the boost in accuracy appears to be context bound. That is, even when participants more accurately monitored their comprehen- sion after writing summaries after a delay, the lack of order effect suggests that they did not apply these skills to other contexts. However, if participants received additional train- ing and evidence of the positive effect of delayed-summariza- tion on monitoring accuracy and learning, they may apply these monitoring strategies to other situations. Effective-Learning Assignment Paper

Many of the recent studies showing improved metacom- prehension accuracy have attributed the improvement in accuracy to getting readers focused on their situation model while judging comprehension (Rawson et al., 2000; Thiede & Anderson, 2003; Thiede et al., 2003; Thiede et al., 2005); however, prior to the present study, this situ- ation model hypothesis had never been empirically evalu- ated. Findings from this study are consistent with this hypothesis. Perhaps more important, the findings from this study disconfirm a rival hypothesis – the accessibility hypothesis (Benjamin et al., 1998; Koriat, 1993; Morris, 1990, see also the modified feedback hypothesis described by Glenberg, Sanocki, Epstein, & Morris, 1987). In partic- ular, given that the correlation between metacomprehen- sion judgments and the total number of idea units in a summary did not differ across conditions, one must look beyond simply accessibility to explain the differences in metacomprehension accuracy of the immediate-summary and delayed-summary conditions.

Understanding that metacomprehension accuracy increases when readers focus on their situation model dur- ing comprehension monitoring suggests new interventions to improve metacomprehension accuracy. That is, instruc- tions that help participants construct a situation model

during reading may also allow readers to monitor the qual- ity of their situation model and improve metacomprehen- sion accuracy. For instance, Chi, DeLeeuw, Chiu, and Lavancher (1994) showed that generating a self-explana- tion while reading helps readers form a more coherent sit- uation model of a text (see also Wiley, 2001; Wiley & Voss, 1999). Recently, Griffin, Wiley, and Thiede (in press) showed that instructing participants to self-explain prior to judging comprehension also improved metacomprehen- sion accuracy. Effective-Learning Assignment Paper

More research is needed to identify additional ways of improving metacomprehension accuracy. Starting with ways that focus readers on their situation model should provide additional tests of the situation model hypothesis and help generate new hypotheses to account for changes in metacomprehension accuracy.

Acknowledgement

The research reported here was supported by the Insti- tute of Education Sciences, US Department of Education, through Grant R305B07460 to Keith Thiede and Jennifer Wiley. The opinions expressed are those of the authors and do not represent views of the Institute or the US Department of Education. We wish to thank John Dunlo- sky, Jennifer Wiley, and Thomas Griffin for their com- ments on drafts of this paper.

Appendix A

A.1. Sample text

One of the major processes that takes place in schools, of course, is that students learn. When they graduate from high school, many can use a computer, write essays with three-part theses, and differentiate equations. In addition to learning specific skills, they also undergo a process of cognitive development wherein their mental skills grow and expand. They learn to think critically, to weigh evi- dence, to develop independent judgment. The extent to which this development takes place is related to both school and home environments.

An impressive set of studies demonstrates that cognitive development during the school years is enhanced by com- plex and demanding work without close supervision and by high teacher expectations. Teachers and curricula that furnish this setting produce students who have greater intellectual flexibility and higher achievement test scores. They are also more open to new ideas, less authoritarian, and less prone to blind conformity. Effective-Learning Assignment Paper

Unfortunately, the availability of these ideal learning conditions varies by students’ social class. Studies show that teachers are most demanding when they are of the same social class as their students. The greater the differ- ence between their own social class and that of their pupils, the more rigidly they structure their classrooms and the fewer demands they place on their students. Students learn

 

 

M.C.M. Anderson, K.W. Thiede / Acta Psychologica 128 (2008) 110–118 117

less when they come from a social class lower than that of their teacher. The social class gap tends to be largest when youngsters are the most disadvantaged, and this process helps to keep them disadvantaged. Effective-Learning Assignment Paper

A.2. Sample test items

The author probably believes that

A. teachers often come from a lower social class than their students;

B. teachers of the disadvantaged should be familiar with the social class of their students;

C. the social class of teachers and students is of little importance and

D. teachers should be hired who are from a higher social class than their students.

The author seems biased in favor of

A. teachers who are less demanding in working with students;

B. discouraging intellectual flexibility in schools; C. encouraging students to think critically and D. giving students less homework.

References

American College Testing Program. (1986). The ACT Assessment Program. Iowa City, IA: Author.

Baker, J. M. C., & Dunlosky, J. (2006). Does momentary accessibility influence metacomprehension judgments? The influence of study- judgment lags on accessibility effects. Psychonomic Bulletin and Review, 13, 60–65.

Benjamin, A. S., Bjork, R. A., & Schwartz, B. L. (1998). The mismeasure of memory: When retrieval fluency is misleading as a metamnemonic index. Journal of Experimental Psychology: General, 127, 55–68.

Bransford, J., & Johnson, M. (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of Verbal Learning and Verbal Behavior, 11, 717–726.

Chi, M. T. H., DeLeeuw, N., Chiu, M., & Lavancher, C. (1994). Elicit self- explanation improves understanding. Cognitive Science, 18, 439–477.

Dunlosky, J., & Lipko, A. R. (in press). Metacomprehension: A brief history and how to improve its accuracy. Current Directions in Psychological Science, 16, 228–232.

Dunlosky, J., & Nelson, T. O. (1992). Importance of the kind of cue for judgments of learning (JOL) and the delayed-JOL effect. Memory and Cognition, 20, 374–380.

Dunlosky, J., & Rawson, K. A. (2005). Why does rereading improve metacomprehension accuracy? Evaluating the levels-of- disruption hypothesis for the rereading effect. Discourse Pro- cesses, 40, 37–55.

Dunlosky, J., Rawson, K. A., & Middleton, E. L. (2005). What constrains the accuracy of metacomprehension judgments? Testing the transfer- appropriate-monitoring and accessibility hypotheses. Journal of Mem- ory and Language, 52, 551–565.

Dunlosky, J., & Thiede, K. W. (1998). What makes people study more? An evaluation of four factors that affect people’s self-paced study. Acta Psychologica, 98, 37–56.

Dunlosky, J., & Thiede, K. W. (2004). Causes and constraints of the shift- to-easier-materials effect in the control of study. Memory and Cogni- tion, 32, 779–788.

Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32, 221–233.

Glenberg, A. M., & Epstein, W. (1985). Calibration of comprehension. Journal of Experimental Psychology: Learning, Memory, and Cogni-

tion, 11, 702–718. Glenberg, A. M., Sanocki, T., Epstein, W., & Morris, C. (1987).

Enhancing calibration of comprehension. Journal of Experimental Psychology: General, 116, 119–136.

Graesser, A. C., & Bertus, E. (1998). The construction of causal inferences while reading expository texts on science and technology. Scientific Studies of Reading, 2(3), 247–269.

Gravetter, F. J., & Wallnau, L. B. (1999). Essentials of statistics for the behavioral sciences (3rd ed.). Pacific Grove, CA: Brooks/Cole Publish- ing Company.

Griffin, T. D., Wiley, J., & Thiede, K. W. (in press). Resource-availability and misdirected monitoring as sources of metacomprehension failure. Memory and Cognition.

Kintsch, W. (1988). The use of knowledge in discourse processing: A construction-integration model. Psychological Review, 95, 163–182.

Klare, G. R. (1974–1975). Assessing readability. Reading Research Quarterly, 10, 62–102.

Koriat, A. (1993). How do we know that we know? The accessibility model of the feeling of knowing. Psychological Review, 100, 609–639.

Koriat, A. (1997). Monitoring one’s own knowledge during study: A cue- utilization approach to judgments of learning. Journal of Experimental Psychology: General, 126, 349–370.

Maki, R. H. (1998a). Predicting performance on text: Delayed versus immediate predictions and tests. Memory and Cognition, 26, 959–964.

Maki, R. H. (1998b). Test predictions over text material. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 117–144). Mahwah, NJ: Erlbaum.

Maki, R. H., Jonas, D., & Kallod, M. (1994). The relationship between comprehension and metacomprehension ability. Psychonomic Bulletin and Review, 1, 126–129.

Maki, R. H., & Serra, M. (1992). The basis of test predictions for text material. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 116–126.

Maki, R. H., Shields, M., Wheeler, A. E., & Zacchilli, T. L. (2005). Individual differences in absolute and relative metacomprehension accuracy. Journal of Educational Psychology, 97, 723–731.

McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14, 1–43.

Metcalfe, J. (2002). Is study time allocated selectively to a region of proximal learning? Journal of Experimental Psychology: General, 131, 349–363.

Metcalfe, J., & Kornell, N. (2003). The dynamics of learning and allocation of study time to a region of proximal learning. Journal of Experimental Psychology: General, 132, 530–542.

Morris, C. C. (1990). Retrieval processes underlying confidence in comprehension judgments. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 223–232.

Nelson, T. O. (1984). A comparison of current measures of feeling-of- knowing accuracy. Psychological Bulletin, 95, 109–133.

Nelson, T. O., & Dunlosky, J. (1991). When people’s judgments of learning (JOLs) are extremely accurate at predicting subsequent recall: The ‘‘delayed-JOL effect’’. Psychological Science, 2, 267–270.

Nelson, T. O., & Narens, L. (1990). Metamemory: A theoretical framework and new findings. In G. H. Bower (Ed.), The psychology of learning and motivation (pp. 125–141). New York: Academic Press.

Rawson, K. A., & Dunlosky, J. (2007). Improving self-evaluation of learning for key concepts in expository texts. European Journal of Cognitive Psychology, 19, 559–579.

Rawson, K. A., Dunlosky, J., & Thiede, K. W. (2000). The rereading effect: Metacomprehension accuracy improves across reading trials. Memory and Cognition, 28, 1004–1010.

 

 

118 M.C.M. Anderson, K.W. Thiede / Acta Psychologica 128 (2008) 110–118

Royer, J. M., Carlo, M. S., Dufresne, R., & Mestre, J. (1996). The assessment of levels of domain expertise while reading. Cognition and Instruction, 14(3), 373–408.

Serra, M. J., & Dunlosky, J. (2005). Does retrieval fluency contribute to the underconfidence-with-practice effect? Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 1258–1266.

Shaughnessy, J. J., & Zechmeister, E. B. (1992). Memory monitoring accuracy as influenced by the distribution of retrieval practice. Bulletin of the Psychonomic Society, 30, 125–128.

Thiede, K. W., & Anderson, M. C. M. (2003). Summarizing can improve metacomprehension accuracy. Contemporary Educational Psychology, 28, 129–160.

Thiede, K. W., Anderson, M. C. M., & Therriault, D. (2003). Accuracy of metacognitive monitoring affects learning of texts. Journal of Educa- tional Psychology, 95, 66–73.

Thiede, K. W., & Dunlosky, J. (1999). Toward a general model of self- regulated study: An analysis of selection of items for study and self- paced study time. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 1024–1037.

Thiede, K. W., Dunlosky, J., Griffin, T. D., & Wiley, J. (2005). Understanding the delayed keyword effect on metacomprehension accuracy. Journal of Experiment Psychology: Learning, Memory and Cognition, 31, 1267–1280.

Weaver, C. A. (1990). Constraining factors in calibration of comprehen- sion. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 214–222.

Wiley, J. (2001). Supporting understanding through task and browser design. In Proceedings of the twenty-third annual conference of the cognitive science society. Hillsdale, NJ: Erlbaum.

Wiley, J., Griffin, T., & Thiede, K. W. (2005). Putting the comprehension in metacomprehension, and explanation in expository text. Journal of General Psychology, 132, 408–428.

Wiley, J., & Voss, J. F. (1999). Constructing arguments from multiple sources: Tasks that promote understanding and not just memory for text. Journal of Educational Psychology, 91, 301–311.

Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277–304). Mahwah, NJ: Erlbaum.