The positive and negative consequences of multiple-choice testing

Multiple-choice tests are commonly used in educational settings but with unknown effects on students' knowledge. The authors examined the consequences of taking a multiple-choice test on a later general knowledge test in which students were warned not to guess. A large positive testing effect was obtained: Prior testing of facts aided final cued-recall performance. However, prior testing also had negative consequences. Prior reading of a greater number of multiple-choice lures decreased the positive testing effect and increased production of multiple-choice lures as incorrect answers on the final test. Multiple-choice testing may inadvertently lead to the creation of false knowledge.     

Memorial consequences of multiple-choice testing on immediate and delayed tests

Multiple-choice testing has both positive and negative consequences for performance on later tests. Prior testing increases the number of questions answered correctly on a later test but also increases the likelihood that questions will be answered with lures from the previous multiple-choice test (Roediger & Marsh, 2005). Prior research has shown that the positive effects of testing persist over a delay, but no one has examined the durability of the negative effects of testing. To address this, subjects took multiple-choice and cued recall tests (on subsets of questions) both immediately and a week after studying. Although delay reduced both the positive and negative testing effects, both still occurred after 1 week, especially if the multiple-choice test had also been delayed. These results are consistent with the argument that recollection underlies both the positive and negative testing effects.     

Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing

Multiple-choice tests are used frequently in higher education without much consideration of the impact this form of assessment has on learning. Multiple-choice testing enhances retention of the material tested (the testing effect); however, unlike other tests, multiple-choice can also be detrimental because it exposes students to misinformation in the form of lures. The selection of lures can lead students to acquire false knowledge (Roediger & Marsh, 2005). The present research investigated whether feedback could be used to boost the positive effects and reduce the negative effects of multiple-choice testing. Subjects studied passages and then received a multiple-choice test with immediate feedback, delayed feedback, or no feedback. In comparison with the no-feedback condition, both immediate and delayed feedback increased the proportion of correct responses and reduced the proportion of intrusions (i.e., lure responses from the initial multiple-choice test) on a delayed cued recall test. Educators should provide feedback when using multiple-choice tests.     

Using verification feedback to correct errors made on a multiple-choice test

A key educational challenge is how to correct students’ errors and misconceptions so that they do not persist. Simply labelling an answer as correct or incorrect on a short-answer test (verification feedback) does not improve performance on later tests; error correction requires receiving answer feedback. We explored the generality of this conclusion and whether the effectiveness of verification feedback depends on the type of test with which it is paired. We argue that, unlike for short-answer tests, learning whether one's multiple-choice selection is correct or incorrect should help participants narrow down the possible answers and identify specific lures as false. To test this proposition we asked participants to answer a series of general knowledge multiple-choice questions. They received no feedback, answer feedback, or verification feedback, and then took a short-answer test immediately and two days later. Verification feedback was just as effective as answer feedback for maintaining correct answers. Importantly, verification feedback allowed learners to correct more of their errors than did no feedback, although it was not as effective as answer feedback. Overall, verification feedback conveyed information to the learner, which has both practical and theoretical implications.     

Using verification feedback to correct errors made on a multiple-choice test

A key educational challenge is how to correct students' errors and misconceptions so that they do not persist. Simply labelling an answer as correct or incorrect on a short-answer test (verification feedback) does not improve performance on later tests; error correction requires receiving answer feedback. We explored the generality of this conclusion and whether the effectiveness of verification feedback depends on the type of test with which it is paired. We argue that, unlike for short-answer tests, learning whether one's multiple-choice selection is correct or incorrect should help participants narrow down the possible answers and identify specific lures as false. To test this proposition we asked participants to answer a series of general knowledge multiple-choice questions. They received no feedback, answer feedback, or verification feedback, and then took a short-answer test immediately and two days later. Verification feedback was just as effective as answer feedback for maintaining correct answers. Importantly, verification feedback allowed learners to correct more of their errors than did no feedback, although it was not as effective as answer feedback. Overall, verification feedback conveyed information to the learner, which has both practical and theoretical implications.     

When additional multiple‐choice lures aid versus hinder later memory

Three experiments were conducted to investigate whether increasing the number of lures on a multiple‐choice test helps, hinders or has no effect on later memory. All three patterns have been reported in the literature. In Experiment 1, the stimuli were unrelated word lists, and increasing the number of lures on an initial multiple‐choice test led to better performance on later free recall and cued recall tasks. In contrast, in Experiments 2 and 3, stimuli were facts from prose materials, and increasing the number of multiple‐choice lures led to robust costs in cued recall and smaller costs in free recall. The results indicate that performance on the initial multiple‐choice test is a critical factor. When initial multiple‐choice performance was near ceiling, testing with additional lures led to superior performance on subsequent tests. However, at lower levels of multiple‐choice performance, testing with additional lures produced costs on later tests. Copyright © 2006 John Wiley & Sons, Ltd.     

Regaining access to marginal knowledge in a classroom setting

Students learn large amounts of information, but not all of it is remembered after courses end – meaning that valuable class time is often spent reviewing background material. Crucially, laboratory research suggests different strategies will be effective when reactivating previously learned information (i.e. marginal knowledge), as opposed to learning new information. In two experiments, we evaluated whether these laboratory results translated to the classroom. Topics from prior courses were tested to document which information students could no longer retrieve. Half were assigned to a not-tested control and half to the intervention; for these topics, students answered multiple-choice questions (without feedback) that gave them the chance to recognize the information they had failed to retrieve. Weeks later, students completed a final assessment on all topics. Crucially, multiple-choice testing increased the retrieval of previously forgotten information, providing the first classroom demonstration of the reactivation of marginal knowledge.     

The role of multiple-choice tests in increasing access to difficult-to-retrieve information

Answering multiple-choice questions improves access to otherwise difficult-to-retrieve knowledge tested by those questions. Here, I examine whether multiple-choice questions can also improve accessibility to related knowledge that is not explicitly tested. In two experiments, participants first answered challenging general knowledge (trivia) multiple-choice questions containing competitive incorrect alternatives and then took a final cued-recall test with those previously tested questions and new related questions for which a previously incorrect answer was the correct answer. In Experiment 1, participants correctly answered related questions more often and faster when they had taken a multiple-choice test than when they had not. In Experiment 2, I showed that the more accurate and faster responses were not simply a result of previous exposure to those alternatives. These findings have practical implications for potential benefits of multiple-choice testing and implications for the processes that occur when individuals answer multiple-choice questions.     

Adapting to test structure: Letting testing teach what to learn

We propose that we encode and store information as a function of the particular ways we have used similar information in the past. More specifically, we contend that the experience of retrieval can serve as a powerful cue to the most effective ways to encode similar information in comparable future learning episodes. To explore these ideas, we did two studies in which all participants went through study–test cycles of single category lists while we manipulated the nature of the recognition tests. The recognition tests either included only same-category lures or only different-category lures. The experience of repeated testing leads participants to avoid conceptual-based strategies but only when conceptual knowledge was poorly diagnostic for recognition (i.e., in the same-category lures condition). In a second study with a similar manipulation, we showed that repeated testing with lures from the same category as study items improved performance in a final recall surprise test compared to conditions in which different-category lures were used. Such a difference is akin to the one obtained when encoding instructions focus on distinctive item features compared to cases in which the focus is on relational processing. We suggest that testing requirements lead to adaptive changes at encoding.     

Confidence judgments in real classroom settings: monitoring performance in different types of tests

During testing, students have a valuable opportunity to exercise and improve their self-regulatory skills. However, the extent to which they profit from those experiences may vary according to some personal, test-related, and environmental factors. This study investigated the effects of metacognitive skills and test types on students' test performances, confidence judgments, and on the accuracy of those judgments. A sample of 129 psychology undergraduate students (50 men and 79 women, mean age = 18.9 years) were categorized according to their metacognitive skills (high vs average vs low) and had their test performances and monitoring processes in two different types of tests (i.e., multiple-choice and short-answer tests) compared throughout one academic term. Their test preparation practices, along with their attributional and regulatory processes during test-taking, were also compared by using open-ended questions. The results showed that: (1) high-metacognitive students presented more effective test preparation practices, better test performances, and superior attributional, regulatory, and monitoring processes than their counterparts; (2) differences in performance and judgment accuracy were significantly larger in the short-answer tests than in the multiple-choice tests; and (3) over time, students' performances and confidence levels varied in specific patterns according to the type of test being taken. The results are discussed, focusing on the educational implications of the interactions observed and on how they may determine what students can learn from test-taking experiences. In addition, based on the results obtained, specific suggestions on how to increase the metacognitive awareness of university students through instruction and on how to improve their academic assessment are provided.     

Confabulation versus experimentally induced false memories in Korsakoff patients

The present study focuses on both the clinical symptom of confabulation and experimentally induced false memories in patients suffering from Korsakoff's syndrome. Despite the vast amount of case studies of confabulating patients and studies investigating false memories in the Deese-Roediger-McDermott (DRM) paradigm, the nature of Korsakoff patients' confabulatory behaviour and its association with DRM false memories have been rarely examined. Hence, the first aim of the present study was to evaluate confabulatory responses in a large sample of chronic Korsakoff patients and matched controls by means of the Dalla Barba Confabulation Battery. Second, the association between (provoked) confabulation and the patients' DRM false recognition performance was investigated. Korsakoff patients mainly confabulated in response to questions about episodic memory and questions to which the answer was unknown. A positive association was obtained between confabulation and the tendency to accept unstudied distractor words as being old in the DRM paradigm. On the other hand, there was a negative association between confabulation and false recognition of critical lures. The latter could be attributed to the importance of strategic retrieval at delayed memory testing.     

An item gains and losses analysis of false memories suggests critical items receive more item-specific processing than list items

In a repeated testing paradigm, list items receiving item-specific processing are more likely to be recovered across successive tests (item gains), whereas items receiving relational processing are likely to be forgotten progressively less on successive tests. Moreover, analysis of cumulative-recall curves has shown that item-specific processing produces a slower, but steadier rate of recall than relational processing. The authors relied on these findings to determine the type of processing that both list items and critical lures receive in the Deese-Roediger-McDermott false memory procedure. The first 2 experiments revealed that critical lures produced more item gains, but only the list items resulted in a decrease in item losses across successive tests. The critical lures also produced slower but steadier cumulative recall. In Experiments 3 and 4, the critical items were physically presented during study, which resulted in the lures producing progressively fewer losses across successive tests. The authors concluded that critical items receive more item-specific processing than list items but that unless they are presented in the list, they do not become part of participants' organized retrieval scheme.     

Ironic effects of drawing attention to story errors

Readers learn errors embedded in fictional stories and use them to answer later general knowledge questions (Marsh, Meade, & Roediger, 2003). Suggestibility is robust and occurs even when story errors contradict well-known facts. The current study evaluated whether suggestibility is linked to participants' inability to judge story content as correct versus incorrect. Specifically, participants read stories containing correct and misleading information about the world; some information was familiar (making error discovery possible), while some was more obscure. To improve participants' monitoring ability, we highlighted (in red font) a subset of story phrases requiring evaluation; readers no longer needed to find factual information. Rather, they simply needed to evaluate its correctness. Readers were more likely to answer questions with story errors if they were highlighted in red font, even if they contradicted well-known facts. Although highlighting to-be-evaluated information freed cognitive resources for monitoring, an ironic effect occurred: Drawing attention to specific errors increased rather than decreased later suggestibility. Failure to monitor for errors, not failure to identify the information requiring evaluation, leads to suggestibility.     

Suggestibility From Stories: Can Production Difficulties and Source Monitoring Explain a Developmental Reversal?

Children's memories improve throughout childhood, and this improvement is often accompanied by a reduction in suggestibility. In this context, it is surprising that older children learn and reproduce more factual errors from stories than do younger children (Fazio & Marsh, 2008 Fazio , L. K. , & Marsh , E. J. ( 2008 ). Older, not younger, children learn more false facts from stories . Cognition , 106 , 1081 – 1089 .[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]). The present study examined whether this developmental reversal is limited to production tests, or whether younger children are still less suggestible when the final test (multiple-choice) asks them to recognize the answer. A second goal was to explore the role of source monitoring in children's suggestibility by examining children's awareness of learning within, versus before, the experiment. Five-year-olds and 7-year-olds listened to stories containing correct, neutral, and misleading references and later took either a multiple-choice or short-answer general knowledge test. In addition, they judged whether each answer had appeared in the stories and whether they had known it before the experiment. Critically, a developmental reversal in suggestibility was observed on both tests; younger children were less suggestible even when faced with the story errors at test. Although older children showed superior source discriminability for whether their answers had appeared in the stories, they showed an illusion of prior knowledge, believing they had known their misinformation answers all along. To this effect, older children's increased suggestibility may be due not only to their superior memory capacity for specific story errors, but also to their ability and tendency to integrate story information into their knowledge base.     

Normative data for semantically associated Spanish word lists that create false memories

The present study provides norms for Spanish word lists that have been used to create false memories in native speakers of Spanish. The word lists reported are based on the Roediger and McDermott (1995) lists that have been used extensively to examine illusory memories. We employed Roediger and McDermott’s critical lures, translated them into Spanish, and created semantically associated Spanish word lists by testing native Spanish speakers. The resulting lists were then normed with additional native Spanish speakers. Overall, the participants recalled 53% of the list items and 32% of the critical lures with the word lists developed. In addition, 74% of the list items and 69% of the critical lures were recognized by the participants. The present study adds to the literature by providing a set of Spanish lists that can be used by researchers interested in evaluating false memories in individuals who speak Spanish. These norms may be downloaded from www.psychonomic.org/archive.     

Ironic effects of drawing attention to story errors

Readers learn errors embedded in fictional stories and use them to answer later general knowledge questions (Marsh, Meade, & Roediger, 2003). Suggestibility is robust and occurs even when story errors contradict well-known facts. The current study evaluated whether suggestibility is linked to participants' inability to judge story content as correct versus incorrect. Specifically, participants read stories containing correct and misleading information about the world; some information was familiar (making error discovery possible), while some was more obscure. To improve participants' monitoring ability, we highlighted (in red font) a subset of story phrases requiring evaluation; readers no longer needed to find factual information. Rather, they simply needed to evaluate its correctness. Readers were more likely to answer questions with story errors if they were highlighted in red font, even if they contradicted well-known facts. Although highlighting to-be-evaluated information freed cognitive resources for monitoring, an ironic effect occurred: Drawing attention to specific errors increased rather than decreased later suggestibility. Failure to monitor for errors, not failure to identify the information requiring evaluation, leads to suggestibility.     

TESTAN: An integrated modular system for personality assessment and test development on MS-DOS personal computers

TESTAN (TEST and ANalysis) is an authoring shell and integrated psychometric package for development of computerized multiple-choice questionnaires, tests, and personality scales on personal computers that use MS-DOS. TESTAN allows (1) writing and editing items, interpretation messages, keys, norms, a bank of profiles, and so on, (2) collecting data in on-line or paper-and-pencil modes, (3) selecting the most discriminating items by means of correlation and factor analysis for practical use, (4) validating test scales and items according to external criteria and expert ratings, and (5) making multifactor assessment decisions after testing. TESTAN can be used to test student conceptual knowledge in any area in the form of multiple-choice questions. This report describes essential functional properties and facilities of TESTAN for psychometrists and applied psychologists.     

Test-induced priming of false memories

Of interest was whether prior testing of related words primes false memories in the Deese/Roediger-McDermott (DRM) paradigm. After studying lists of related words, subjects made old-new judgments about zero, three, or six related items before being tested on critical nonpresented lures. When the recognition test was self-paced, prior testing of list items led to faster false recognition judgments, but did not increase the rate of false alarms to lures from studied lists. Critically, this pattern changed when decision making at test was speeded. When forced to respond quickly—presumably precluding the use of monitoring processes—clear test-induced priming effects were observed in the rate of false memories. The results are consistent with an activation-monitoring explanation of false memories and support that retrieving veridical memories can be a source of memory error.     

"John thinks that Mary feels. . ." False belief in children across affective and physical domains

Children aged 5-8 years (N = 64) were given 3 first- and 3 second-order tasks testing their ability to represent false beliefs about physical facts, positive emotions, and negative emotions. The children were also asked to justify their responses to the test questions. Older children were more successful than younger children at both answering the test questions correctly and justifying their responses. On the first-order problems, performance was better on the physical fact task than on the emotions tasks; the reverse was true for the second-order problems. Children primarily used situational explanations to explain correct judgments on the physical problems, whereas mentalistic explanations were more common than situational explanations on 3 of the 4 emotions tasks. The results extend knowledge of false belief beyond the simple forms studied at the preschool level.     

Proactive and retroactive effects of negative suggestion

The negative effects of false information presented either prior to (proactive interference; PI) or following (retroactive interference; RI) true information was examined with word definitions (Experiment 1) and trivia facts (Experiment 2). Participants were explicitly aware of which information was true and false when shown, and true-false discrimination was evaluated via multiple-choice tests. Negative suggestion, defined as poorer performance on interference items than noninterference (control) items, consistently occurred when the wrong information followed the correct information (RI) but not when it preceded the correct information (PI). These effects did not change as a function of retention interval (immediate, 1 week, or 3 weeks) or number of incorrect alternatives (1 or 3). Implications of this outcome for experiencing incorrect information in both academic and nonacademic situations are considered.