Feedback-related brain activity predicts learning from feedback in multiple-choice testing

Different event-related potentials (ERPs) have been shown to correlate with learning from feedback in decision-making tasks and with learning in explicit memory tasks. In the present study, we investigated which ERPs predict learning from corrective feedback in a multiple-choice test, which combines elements from both paradigms. Participants worked through sets of multiple-choice items of a Swahili–German vocabulary task. Whereas the initial presentation of an item required the participants to guess the answer, corrective feedback could be used to learn the correct response. Initial analyses revealed that corrective feedback elicited components related to reinforcement learning (FRN), as well as to explicit memory processing (P300) and attention (early frontal positivity). However, only the P300 and early frontal positivity were positively correlated with successful learning from corrective feedback, whereas the FRN was even larger when learning failed. These results suggest that learning from corrective feedback crucially relies on explicit memory processing and attentional orienting to corrective feedback, rather than on reinforcement learning.     

The effect of feedback delay and feedback type on perceptual category learning: the limits of multiple systems

Evidence that learning rule-based (RB) and information-integration (II) category structures can be dissociated across different experimental variables has been used to support the view that such learning is supported by multiple learning systems. Across 4 experiments, we examined the effects of 2 variables, the delay between response and feedback and the informativeness of feedback, which had previously been shown to dissociate learning of the 2 types of category structure. Our aim was twofold: first, to determine whether these dissociations meet the more stringent inferential criteria of state-trace analysis and, second, to determine the conditions under which they can be observed. Experiment 1 confirmed that a mask-filled feedback delay dissociated the learning of RB and II category structures with minimally informative (yes/no) feedback and also met the state-trace criteria for the involvement of multiple latent variables. Experiment 2 showed that this effect is eliminated when a less similar, fixed pattern mask is presented in the interval between response and feedback. Experiment 3 showed that the selective effect of feedback delay on II learning is reduced with fully informative feedback (in which the correct category is specified after an incorrect response) and that feedback type did not dissociate RB and II learning. Experiment 4 extended the results of Experiment 2, showing that the differential effect of feedback delay is eliminated when a fixed pattern mask is used. These results pose important challenges to models of category learning, and we discuss their implications for multiple learning system models and their alternatives.     

Bias and error in multiple-choice tests

A formula for estimating real scores on a multiple-choice test from a knowledge of raw scores is derived. This formula does not involve the assumption of a binomial distribution of real scores as does the Calandra formula. Other important formulas derived show: the variance of real scores in terms of the variance of raw scores and the correlation between real scores and raw scores. If the variance of real scores (or of raw scores also) is binomial, the regression of real scores on raw scores is linear; but, otherwise the regression is curvilinear. Yet the linear estimating formula is a close approximation to the curvilinear relationship. Factors affecting the regression of real scores on raw scores and the correlation coefficient are: (1) the number of choices per question; (2) the number of questions answered; (3) the ratio of the average group raw score to the variance of raw scores.     

The maximum expected correlation between two multiple-choice tests

A formula is derived which gives the maximum expected correlation between two multiple-choice tests as a function of the distributions of proportions correct for the items in the two tests and the probability of chance success. The formula is similar to one derived by Carroll based on true item difficulties. A numerical example is provided.     

The effect of real-time auditory feedback on learning new characters

The present study investigated the effect of handwriting sonification on graphomotor learning. Thirty-two adults, distributed in two groups, learned four new characters with their non-dominant hand. The experimental design included a pre-test, a training session, and two post-tests, one just after the training sessions and another 24 h later. Two characters were learned with and two without real-time auditory feedback (FB). The first group first learned the two non-sonified characters and then the two sonified characters whereas the reverse order was adopted for the second group. Results revealed that auditory FB improved the speed and fluency of handwriting movements but reduced, in the short-term only, the spatial accuracy of the trace. Transforming kinematic variables into sounds allows the writer to perceive his/her movement in addition to the written trace and this might facilitate handwriting learning. However, there were no differential effects of auditory FB, neither long-term nor short-term for the subjects who first learned the characters with auditory FB. We hypothesize that the positive effect on the handwriting kinematics was transferred to characters learned without FB. This transfer effect of the auditory FB is discussed in light of the Theory of Event Coding.     

Memorial consequences of multiple-choice testing on immediate and delayed tests

Multiple-choice testing has both positive and negative consequences for performance on later tests. Prior testing increases the number of questions answered correctly on a later test but also increases the likelihood that questions will be answered with lures from the previous multiple-choice test (Roediger & Marsh, 2005). Prior research has shown that the positive effects of testing persist over a delay, but no one has examined the durability of the negative effects of testing. To address this, subjects took multiple-choice and cued recall tests (on subsets of questions) both immediately and a week after studying. Although delay reduced both the positive and negative testing effects, both still occurred after 1 week, especially if the multiple-choice test had also been delayed. These results are consistent with the argument that recollection underlies both the positive and negative testing effects.     

The effect of feedback on timing in children and adults: the temporal generalization task.

Children aged 5 and 8 years and adults were tested on a temporal generalization task with a standard duration of 600 ms in a condition with or without corrective feedback. In all conditions, the participants produced orderly temporal generalization gradients, although these were flatter in the younger children, especially in the no-feedback condition. Nevertheless, the results show that the feedback increased the steepness of the generalization gradient in all age groups and in a greater extent in the younger children. Our clock-based model suggested that feedback reduces the variability of the memory representation of the standard duration but also the probability of random responses in the 5-year-olds.     

The effect of aural feedback in second language vocabulary learning

We examined instructional procedures in the teaching of second language vocabulary, using a paired-associate paradigm. Four presentation methods were investigated, including simultaneous standard, simultaneous reversal, aural feedback, and visual feedback. The study involved 16 grade 7 school children as subjects. Subjects were tested on 28 second language words during the post-test stage. Orthogonal comparisons were carried out for the obtained data. The two feedback conditions proved to be the most beneficial and the more favourable out of these was the aural feedback condition. Also, reversing the two inputs proved to be better than the conventional paired-associate (e.g. presenting the second language word first with the first language word second). Based on the results of this study, it is advocated that the feedback procedures, in particular, be used in second language learning with aural feedback replacing visual feedback.     

The role of multiple-choice tests in increasing access to difficult-to-retrieve information

Answering multiple-choice questions improves access to otherwise difficult-to-retrieve knowledge tested by those questions. Here, I examine whether multiple-choice questions can also improve accessibility to related knowledge that is not explicitly tested. In two experiments, participants first answered challenging general knowledge (trivia) multiple-choice questions containing competitive incorrect alternatives and then took a final cued-recall test with those previously tested questions and new related questions for which a previously incorrect answer was the correct answer. In Experiment 1, participants correctly answered related questions more often and faster when they had taken a multiple-choice test than when they had not. In Experiment 2, I showed that the more accurate and faster responses were not simply a result of previous exposure to those alternatives. These findings have practical implications for potential benefits of multiple-choice testing and implications for the processes that occur when individuals answer multiple-choice questions.     

Using verification feedback to correct errors made on a multiple-choice test

A key educational challenge is how to correct students' errors and misconceptions so that they do not persist. Simply labelling an answer as correct or incorrect on a short-answer test (verification feedback) does not improve performance on later tests; error correction requires receiving answer feedback. We explored the generality of this conclusion and whether the effectiveness of verification feedback depends on the type of test with which it is paired. We argue that, unlike for short-answer tests, learning whether one's multiple-choice selection is correct or incorrect should help participants narrow down the possible answers and identify specific lures as false. To test this proposition we asked participants to answer a series of general knowledge multiple-choice questions. They received no feedback, answer feedback, or verification feedback, and then took a short-answer test immediately and two days later. Verification feedback was just as effective as answer feedback for maintaining correct answers. Importantly, verification feedback allowed learners to correct more of their errors than did no feedback, although it was not as effective as answer feedback. Overall, verification feedback conveyed information to the learner, which has both practical and theoretical implications.     

Using verification feedback to correct errors made on a multiple-choice test

A key educational challenge is how to correct students’ errors and misconceptions so that they do not persist. Simply labelling an answer as correct or incorrect on a short-answer test (verification feedback) does not improve performance on later tests; error correction requires receiving answer feedback. We explored the generality of this conclusion and whether the effectiveness of verification feedback depends on the type of test with which it is paired. We argue that, unlike for short-answer tests, learning whether one's multiple-choice selection is correct or incorrect should help participants narrow down the possible answers and identify specific lures as false. To test this proposition we asked participants to answer a series of general knowledge multiple-choice questions. They received no feedback, answer feedback, or verification feedback, and then took a short-answer test immediately and two days later. Verification feedback was just as effective as answer feedback for maintaining correct answers. Importantly, verification feedback allowed learners to correct more of their errors than did no feedback, although it was not as effective as answer feedback. Overall, verification feedback conveyed information to the learner, which has both practical and theoretical implications.     

The predicted and observed effect of chance success on multiple-choice test validity

Assuming chance to be fully operative, the predicted effect of chance success on test validity when answer options are supplied depends on the number of options, the difficulty of the test and the variance of test scores. Predicted validity values are compared with empirical validity values in an experiment which used the same mathematics test items with and without answer options.Part of a paper presented at the 1952 meetings of the APA. The author acknowledges with appreciation the helpful advice of Professor Harold Gulliksen and the joint support of the project by the College Entrance Examination Board and the Educational Testing Service.     

The effects of tests on learning and forgetting

In three experiments, we investigated whether memory tests enhance learning and reduce forgetting more than additional study opportunities do. Subjects learned obscure facts (Experiments 1 and 2) or Swahili-English word pairs (Experiment 3) by either completing a test with feedback (test/study) or receiving an additional study opportunity (study). Recall was tested after 5 min or 1, 2, 7, 14, or 42 days. We explored forgetting by means of an ANOVA and also by fitting a power function to the data. In all three experiments, testing enhanced overall recall more than restudying did. According to the power function, in two out of three experiments, testing also reduced forgetting more than restudying did, although this was not always the case according to the ANOVA. We discuss the implications of these results both for approaches to measuring forgetting and for the use of tests in promoting long-term retention. The stimuli used in these experiments may be found at www.psychonomic.org/archive.     

The role of instructions in the variability of sex-related differences in multiple-choice tests

When both experts and lay people interpret data on sex-related differences, they usually forget that the instruments for data collection might be provoking such differences. This experiment, carried out on 240 participants, focused on the effects of four instruction/scoring conditions on sex effect size in two computerized tests — vocabulary and mental rotation, for which sex-related differences had been shown to be, respectively, small (favoring females) and large (favoring males). Given the caution which seems to characterize female performance, our general hypothesis predicted that, under instructions encouraging guessing, effect sizes favoring males would augment and effect sizes favoring females would diminish. The opposite results were expected under instructions discouraging guessing. Some supporting evidence was found.     

The effect of type of practice on motor learning in children

The present investigation was concerned with the variability-of-practice hypothesis of Schmidt's schema theory of motor learning; i. e. it was sought to determine if there is an optimal way to structure the variable practice session with regard to schema formation. Furthermore, children's motor recognition (movement evaluation) was examined, in addition to recall (movement production), since apparently no such studies had been done within the context of schema theory before. Eighty-eight girls and boys with a mean age of 11.3 years were tested on a throwing task. Three groups of subjects practised under variable conditions (different target distances and weights) that differed with regard to contextual interference and sequential organization of task variations; another group received constant throwing practice. Following practice trials with knowledge of results, the four treatment groups and a control group without prior practice were required to produce and evaluate a novel response (novel target distance and weight) without vision and knowledge of results. The results supported the variability-of-practice hypothesis, in that variable practice in general facilitated recall and recognition on the novel task. Furthermore, variable practice with the highest level of contextual interference (random order of tasks) produced most effective transfer performance, suggesting that the structure of the variable-practice session in fact plays a role with regard to the effectiveness of motor schema development. Contrary to the ‘organization hypothesis’ (Gentile and Nacson, 1976) a hierarchical organization of movement variations was not more beneficial to schema formation than random practice.     

Meta-analysis of the reduced relative feedback frequency effect on motor learning and performance

A fundamental motor learning principle conveyed in textbooks is that augmented terminal feedback frequency differentially affects motor learning and performance. The guidance hypothesis predicts that relative to a reduced frequency of feedback, providing learners with feedback following every practice trial enhances practice performance but degrades subsequent motor learning. This change in effectiveness for each relative feedback frequency is called a reversal effect, and because it is thought that practice variables can have distinct impacts on learning and performance, delayed retention tests are considered the gold standard in motor learning research. The objectives of this meta-analysis were to a) synthesize the available evidence regarding feedback frequency, performance, and motor learning to test whether there are significant changes in effectiveness from acquisition and immediate retention to delayed retention, b) evaluate potential moderators of these effects, and c) investigate the potential influence of publication bias on this literature. We screened 1662 articles found in PubMed and PsycINFO databases as well as with reference tracing and a targeted author search. A final sample of 61 eligible papers were included in the primary analysis (k = 75, N = 2228). Results revealed substantial heterogeneity but no significant moderators, high levels of uncertainty, and no significant effect of reduced feedback frequency at any time point. Further, multilevel analyses revealed no evidence of a significant change in effect from acquisition or immediate retention to delayed retention. Z-curve analysis suggested the included studies were severely underpowered. These results suggest that robust evidence regarding feedback frequency and motor learning is lacking.