Impression formation of tests: retrospective judgments of performance are higher when easier questions come first

Four experiments are reported on the importance of retrospective judgments of performance (postdictions) on tests. Participants answered general knowledge questions and estimated how many questions they answered correctly. They gave higher postdictions when easy questions preceded difficult questions. This was true when time to answer each question was equalized and constrained, when participants were instructed not to write answers, and when questions were presented in a multiple-choice format. Results are consistent with the notion that first impressions predominate in overall perception of test difficulty.     

The effect of question order on evaluations of test performance: how does the bias evolve?

Weinstein and Roediger (Memory & Cognition 38:366-376, 2010) found that manipulating the order of questions on a general knowledge quiz resulted in differing evaluations of performance at the end of the quiz: Irrespective of their actual performance, participants were consistently more optimistic about their performance when questions were given in an easy-to-hard order. In the present experiment, the participants were stopped 10 times throughout a 100-item test and asked to evaluate their performance on the last 10 questions they had answered, as well as rating their impressions of the test so far and predicting their final performance. Arranging the questions from the easiest to the hardest produced more optimistic performance evaluations on each block than did an analogous hard-easy question order, even though performance on the two versions did not differ significantly as a function of question order. Furthermore, the ratings of item difficulty on each block of 10 questions were asymmetrical in the two conditions, with a higher sensitivity to increasing as compared to decreasing question difficulty. On the other hand, the item-by-item ratings and predictions remained unaffected by question order. Our findings are best explained by an anchoring interpretation, which suggests that students fail to adjust their evaluations of performance as the difficulty of the questions changes across the test.     

The effect of question order on evaluations of test performance: Can the bias dissolve?

Question difficulty order has been shown to affect students’ global postdictions of test performance. We attempted to eliminate the bias by letting participants experience the question order manipulation multiple times. In all three experiments, participants answered general knowledge questions and self-evaluated their performance. In Experiment 1, participants studied questions and answers in easy–hard or hard–easy question order prior to taking a test in the same order. In Experiment 2, participants took the same test twice in the opposite question order (easy–hard then hard–easy, or hard–easy then easy–hard). In Experiment 3, participants took two different tests in the opposite question order (easy–hard then hard–easy, or hard–easy then easy–hard). In all three experiments, we were unable to eliminate the bias, which suggests that repeated exposure is insufficient to overcome a strong initial anchor.     

Question order bias in retrospective evaluations of item and associative recognition

Previous research has demonstrated a retrospective memory bias in metacognitive judgments regarding performance on general knowledge questions: Test-takers rate their own performance more optimistically when tests begin with easy questions than when tests begin with hard questions. An anchoring heuristic has been proposed to explain this finding, in which experience with the early questions constrains global performance evaluations of the test. In the current study we report on two experiments using tasks of item recognition and associative recognition to investigate the generality of question order bias. As predicted by an anchoring explanation, participants’ estimates of performance were higher for item recognition tests beginning with easy items. However, the effect was reversed in the associative recognition task: Participants’ estimates of performance were higher for tests beginning with hard items. Specific recollections, if present, may have a greater impact on test performance perception than more general global impressions.     

Reducing stereotype threat by blurring intergroup boundaries

The authors aimed to establish whether interventions designed to reduce intergroup bias could be applied to the stereotype threat domain. In three experiments, the hypothesis was tested that blurring intergroup boundaries would reduce stereotype threat. In the first study, it was found that female participants who thought about characteristics shared between the genders tended to show less preference for stereotypical female careers than did participants in the baseline condition. In Experiment 2, participants who thought about overlapping characteristics answered more math questions correctly compared to a baseline group and participants who thought about differences between the genders. In Experiment 3, a specific threat manipulation was included. Participants who completed the overlapping characteristics task before receiving the threat completed significantly more math questions correctly than did participants in the baseline and threat conditions. The findings support the idea that interventions designed to reduce intergroup bias can be applied successfully in the reduction of stereotype threat.     

How accurately can older adults evaluate the quality of their text recall? The effect of providing standards on judgment accuracy

Adults have difficulties accurately judging how well they have learned text materials; unfortunately, such low levels of accuracy may obscure age‐related deficits. Higher levels of accuracy have been obtained when younger adults make postdictions about which test questions they answered correctly. Accordingly, we focus on the accuracy of postdictive judgments to evaluate whether age deficits would emerge with higher levels of accuracy and whether people's postdictive accuracy would benefit from providing an appropriate standard of evaluation. Participants read texts with definitions embedded in them, attempted to recall each definition, and then made a postdictive judgment about the quality of their recall. When making these judgments, participants either received no standard or were presented the correct definition as a standard for evaluation. Age‐related equivalence was found in the relative accuracy of these term‐specific judgments, and older adults' absolute accuracy benefited from providing standards to the same degree as did younger adults. Copyright © 2009 John Wiley & Sons, Ltd.     

Depressive cognition: a test of depressive realism versus negativity using general knowledge questions

This research is an examination of whether cognition in depressed individuals incorporates a realistic view of the world or a general tendency toward negativity. Participants provided two types of probability judgments of the likelihood that they correctly answered general knowledge questions: the probability that they correctly answered each of the individual questions and an aggregate judgment, after completing the questionnaire, of the percentage of all the questions they thought they had correctly answered. These tasks generally elicit overconfidence and accuracy in nondepressives. In accord with theories of both depressive realism and general negativity, in their item-by-item assessments of their answers to the individual questions, depressed participants demonstrated less overconfidence than nondepressed participants. In accord with the theory of general negativity but not with the theory of depressive realism, however, depressed participants demonstrated underconfidence in their aggregate judgments. The implications of these findings on theories of depressive cognition are discussed.     

The Effects of Expertise on the Hindsight Bias

We present data from three experiments examining the effects of objective and subjective expertise on the hindsight bias. In Experiment 1, participants read an essay about baseball or dogs and then answered questions about the baseball essay to the best of their ability, as if they had not read the essay, or to the best of their ability, although they read about dogs. Participants also completed a quiz about baseball rules and terminology, which was an objective measure of expertise. Results demonstrated that as participants' baseball expertise increased, their inability to act as if they never read the essay also increased; expertise exacerbated the hindsight bias. To test the effects of subjective expertise on hindsight bias and investigate factors underlying the relationship, participants in Experiment 2 ranked five topics in order of expertise and gave feeling‐of‐knowing (FOK) ratings for questions from these topics. Foresight participants then saw each question again and answered the questions; hindsight participants saw the questions and answers and gave the probability they would have known the answers had they not been provided. Hindsight bias increased with subjective expertise as did average FOK ratings. In Experiment 3, we experimentally manipulated perceived expertise but found that neither average FOK ratings nor hindsight bias was affected by experimentally induced expertise. Taken together, the results demonstrate that expertise exacerbates both objective and subjective hindsight bias but that an FOK, which likely exists only when expertise is naturally acquired, is necessary to engender the detrimental effect of expertise on the hindsight bias. Copyright © 2016 John Wiley & Sons, Ltd.     

The effects of study questions and grades on student test performance in a college course

Two experiments demonstrated the effects of study questions on student test performance in an introductory college course. Students in both experiments correctly answered study question items 20 to 30% more frequently than non-study question probes. Furthermore, mean performance on study question items was better than 90% during all phases of both experiments. The present experiments were also designed to study the effects of grades on test performance, and the relationship between long and short sets of study questions. The results of Experiment I clearly illustrated the importance of using grades to maintain high levels of student test performance. The results of Experiment II suggested that long sets of study questions may produce better performance on probe items than do short sets of study questions, but the effect was small.     

American and Chinese Students’ Calibration of Comprehension and Performance

In the present study we examined the ability of American and Chinese undergraduate students to calibrate their understanding of textbook passages translated into their native languages. Students read a series of texts and made predictions of their understanding of each text as well as the number of questions they would be able to answer correctly. Students also made postdictions of their test performance. Chinese students were significantly better than American students in calibrating their understanding of passages and predicting how many comprehension items they would answer correctly. Chinese students also outperformed American students on comprehension tests. All students were able to make more accurate postdictions of comprehension test scores than predictions. Results are related to possible instructional differences between American and Chinese students. Several possible directions for future research are discussed.     

尝试提取复杂信息失败的学习：吃一堑长一智？

摘 要：在学习中,提取失败能否产生对知识的有效学习是一个很有意义的研究课题。以事实和概念类知识为实验材料,三个实验采用经典范式探讨了复杂意义条件下的提取失败后的学习效果。结果显示仅在尝试提取事实类题目失败后,促进了知识的保持和迁移;在迁移成绩上延时反馈优于即时反馈的效果。表明尝试提取复杂材料的信息失败后,知识的类型和反馈的时机决定了学习效果。     

Memory for memory

Participants read short passages and 1 day later they answered questions via telephone about the passages (text facts) and about the experimental session (event facts). They were telephoned again 6 weeks later and answered the same questions about text and event facts. They also answered new questions about whether they remembered the answers they had given in the initial telephone interview (recall for prior memory performance). Although participants accurately remembered the majority of past memory successes, they were poor at remembering past memory failures. After being provided with the correct answer and tested again, the participants' performance improved somewhat, especially for memory failures. This suggests that some errors in recalling past forgetting might have been due to correctly remembering the answer previously given, but failing to realize that it had been wrong. These findings have implications for a variety of situations in which people are queried about past memory performance.     

PASSAGE INDEPENDENCY AND QUESTION CHARACTERISTICS: AN ANALYSIS OF THREE INFORMAL READING INVENTORIES

ABSTRACT

Three informal reading inventories were examined to identify passage independent test questions, questions which could be answered correctly without reading the corresponding passages. Fourth grade students of good and poor reading ability were administered the test questions orally without access to the passages. The percentage of questions answered correctly was calculated for each test. An analysis of variance procedure revealed that the Classroom Reading Inventory was the most passage independent followed by the Analytical Reading Inventory and Ginn 720 inventory respectively. However the two reader groups did not differ significantly in their performance across the three tests. An analysis of children's responses to the questions revealed that three question categories in particular tended to be passage independent in nature. These categories were: a) general information b) vocabulary meaning and c) affective. The investigation also examined the literal and inferential characteristics of the test questions. Suggestions were made for evaluating the validity of reading comprehension questions     

Depressive Cognition: A Test of Depressive Realism Versus Negativity Using General Knowledge Questions

This research is an examination of whether cognition in depressed individuals incorporates a realistic view of the world or a general tendency toward negativity. Participants provided two types of probability judgments of the likelihood that they correctly answered general knowledge questions: the probability that they correctly answered each of the individual questions and an aggregate judgment, after completing the questionnaire, of the percentage of all the questions they thought they had correctly answered. These tasks generally elicit overconfidence and accuracy in nondepressives. In accord with theories of both depressive realism and general negativity, in their item-by-item assessments of their answers to the individual questions, depressed participants demonstrated less overconfidence than nondepressed participants. In accord with the theory of general negativity but not with the theory of depressive realism, however, depressed participants demonstrated underconfidence in their aggregate judgments. The implications of these findings on theories of depressive cognition are discussed.     

Expecting to teach enhances learning and organization of knowledge in free recall of text passages

The present research assessed the potential effects of expecting to teach on learning. In two experiments, participants studied passages either in preparation for a later test or in preparation for teaching the passage to another student who would then be tested. In reality, all participants were tested, and no one actually engaged in teaching. Participants expecting to teach produced more complete and better organized free recall of the passage (Experiment 1) and, in general, correctly answered more questions about the passage than did participants expecting a test (Experiment 1), particularly questions covering main points (Experiment 2), consistent with their having engaged in more effective learning strategies. Instilling an expectation to teach thus seems to be a simple, inexpensive intervention with the potential to increase learning efficiency at home and in the classroom.     

An individual differences approach to the suggestibility of memory over time

We examined how certain personality traits might relate to the formation of suggestive memory over time. We hypothesised that compliance and trust relate to initial acceptance of misinformation as memory, whereas fantasy proneness might relate to integration of misinformation into memory after later intervals (relative to the time of exposure to misinformation). Participants watched an excerpt from a movie—the simulated eyewitness event. They next answered a recall test that included embedded misinformation about the movie. Participants then answered a yes/no recognition test. A week later, participants answered a second yes/no recognition test about the movie (each yes/no recognition test included different questions). Before both recognition tests, participants were warned about the misinformation shown during recall and were asked to base their answer on the movie excerpt only. After completing the second recognition test, participants answered questions from the Neuroticism Extroversion Openness Personality Inventory-3 (McCrae, Costa, & Martin, 2005) and Creative Experiences Questionnaire (Merckelbach, Horselenberg, & Muris, 2001). While compliance correlated with misinformation effects immediately after exposure to misinformation, fantasy-prone personality accounted for more of the variability in false recognition rates than compliance after a 1-week interval.     

Influence of Question Format and Text Availability on the Assessment of Expository Text Comprehension

This study examines how passage availability and reading comprehension question format (open-ended vs. multiple-choice) influence question answering. In two experiments, college undergraduates read an expository passage and answered open-ended and multiple-choice versions of text-based, local, and global bridging inference questions. Half the participants were allowed to refer to the passage when answering the questions and half were not. Participants' prior domain knowledge relating to the text contents was assessed using multiple-choice and open-ended questions. Correlation-based analyses in the two experiments indicated: (a) a decline in the relationship between prior domain knowledge and comprehension when the passage was available during question answering; and (b) a high correlation between multiple-choice and open-ended question answering performance when the passage was not available for reference. Overall the results indicate that the nature of the reading comprehension assessment is influenced by the specific task with which comprehension is assessed.     

The role of multiple-choice tests in increasing access to difficult-to-retrieve information

Answering multiple-choice questions improves access to otherwise difficult-to-retrieve knowledge tested by those questions. Here, I examine whether multiple-choice questions can also improve accessibility to related knowledge that is not explicitly tested. In two experiments, participants first answered challenging general knowledge (trivia) multiple-choice questions containing competitive incorrect alternatives and then took a final cued-recall test with those previously tested questions and new related questions for which a previously incorrect answer was the correct answer. In Experiment 1, participants correctly answered related questions more often and faster when they had taken a multiple-choice test than when they had not. In Experiment 2, I showed that the more accurate and faster responses were not simply a result of previous exposure to those alternatives. These findings have practical implications for potential benefits of multiple-choice testing and implications for the processes that occur when individuals answer multiple-choice questions.