The effect of normative models on individual and group choice

Studied the effect of a normative model (maximization of expected value) upon group and individual choice. 109 MBA students in a Lain square research design chose between two alternatives differing in expected value and in range of outcomes. Group choices were significantly (p < .05) closer to those predicted by the normative model than were individual choices. This difference was not due only to information about the presence and applicability of the normative model but rather it was due to the persuasiveness of the model in a group as a cogent and correct solution to the choice dilemma. Task instructions emphasizing the rewards from risk taking produced significantly more choices (p < .05) of the riskier alternative, particularly by individuals as opposed to groups, than did instructions emphasizing the penalties of risk taking. ‘Risky’ and ‘conservative’ shifts in choice between groups and individuals were explicable through knowledge of the influence of the normative model in individual and group choice.     

Receiving right/wrong feedback: Consequences for learning

Prior work suggests that receiving feedback that one's response was correct or incorrect (right/wrong feedback) does not help learners, as compared to not receiving any feedback at all (Pashler, Cepeda, Wixted, & Rohrer, 2005). In three experiments we examined the generality of this conclusion. Right/wrong feedback did not aid error correction, regardless of whether participants learned facts embedded in prose (Experiment 1) or translations of foreign vocabulary (Experiment 2). While right/wrong feedback did not improve the overall retention of correct answers (Experiments 1 and 2), it facilitated retention of low-confidence correct answers (Experiment 3). Reviewing the original materials was very useful to learners, but this benefit was similar after receiving either right/wrong feedback or no feedback (Experiments 1 and 2). Overall, right/wrong feedback conveys some information to the learner, but is not nearly as useful as being told the correct answer or having the chance to review the to-be-learned materials.     

The DIAGNOSER project: Combining assessment and learning

DIAGNOSER is an Internet-based tool for classroom instruction. It delivers continuous formative assessment and feedback to high school physics students and their teachers about the correct and incorrect concepts and ideas the students may hold regarding physical situations. That is, it diagnoses misconceptions that underlie wrong answers of students, such as a confusion of velocity with acceleration. We use data about patterns of student responses, particularly consistency of errors from question to question, to improve the system’s understanding of student concepts.     

Patterns of Response Times and Response Choices to Science Questions: The Influence of Relative Processing Time

We report on five experiments investigating response choices and response times to simple science questions that evoke student “misconceptions,” and we construct a simple model to explain the patterns of response choices. Physics students were asked to compare a physical quantity represented by the slope, such as speed, on simple physics graphs. We found that response times of incorrect answers, resulting from comparing heights, were faster than response times of correct answers comparing slopes. This result alone might be explained by the fact that height was typically processed faster than slope for this kind of task, which we confirmed in a separate experiment. However, we hypothesize that the difference in response time is an indicator of the cause (rather than the result) of the response choice. To support this, we found that imposing a 3‐s delay in responding increased the number of students comparing slopes (answering correctly) on the task. Additionally a significant proportion of students recognized the correct written rule (compare slope), but on the graph task they incorrectly compared heights. Finally, training either with repetitive examples or providing a general rule both improved scores, but only repetitive examples had a large effect on response times, thus providing evidence of dual paths or processes to a solution. Considering models of heuristics, information accumulation models, and models relevant to the Stroop effect, we construct a simple relative processing time model that could be viewed as a kind of fluency heuristic. The results suggest that misconception‐like patterns of answers to some science questions commonly found on tests may be explained in part by automatic processes that involve the relative processing time of considered dimensions and a priority to answer quickly.     

青少年的心理弹性及自我概念：一年跟踪的交叉滞后分析

为考察青少年心理弹性的发展与内在作用机制,促进学生的心理健康发展。研究选取369名青少年（高中生260名,大学生109名）,分别在学生入学一月及一年后两次跟踪测查青少年的心理弹性以及包括青少年对自我言语能力、数学能力、学校一般情况、体能、外貌、与异性关系、与同性关系、与父母关系、诚实可信赖度、情绪稳定性和一般自我概念等因子的自我评价概念状况。结果表明,青少年的心理弹性总体状况一年后显著增强（F(1, 366) = 16.35, p < .01, ηp2 = .04）。心理弹性与自我概念各因子交叉滞后分析表明,自我概念总分及青少年对自身外貌、体能、与同性关系、与异性关系的评价等四个因子能够分别预测一年后的心理弹性。研究为促进青少年心理弹性及积极心理健康提供了理论参考。     

Assessing Mathematics Misunderstandings via Bayesian Inverse Planning

Online educational technologies offer opportunities for providing individualized feedback and detailed profiles of students' skills. Yet many technologies for mathematics education assess students based only on the correctness of either their final answers or responses to individual steps. In contrast, examining the choices students make for how to solve the equation and the ways in which they might answer incorrectly offers the opportunity to obtain a more nuanced perspective of their algebra skills. To automatically make sense of step-by-step solutions, we propose a Bayesian inverse planning model for equation solving that computes an assessment of a learner's skills based on her pattern of errors in individual steps and her choices about what sequence of problem-solving steps to take. Bayesian inverse planning builds on existing machine learning tools to create a generative model relating (mis)-understandings to equation solving choices. Two behavioral experiments demonstrate that the model can interpret people's equation solving and that its assessments are consistent with those of experienced teachers. A third experiment uses this model to tailor guidance for learners based on individual differences in misunderstandings, closing the loop between assessing understanding, and using that assessment within an educational technology. Finally, because the bottleneck in applying inverse planning to a new domain is in creating the model of possible student misunderstandings, we show how to combine inverse planning with an existing production rule model to make inferences about student misunderstandings of fraction arithmetic.     

Subjective confidence in one's answers: the consensuality principle

In answering general-information questions, a within-person confidence-accuracy (C-A) correlation is typically observed, suggesting that people can monitor the correctness of their knowledge. However, because the correct answer is generally the consensual answer--the one endorsed by most participants--confidence judgment may actually monitor the consensuality of the answer rather than its correctness. Indeed, the C-A correlation was positive for items with a consensually correct answer but negative for items with a consensually wrong answer. Results suggest that the consensuality-confidence correlation may be mediated by 2 internal mnemonic cues that are correlated with consensuality: Consensual answers are reached faster and are selected more consistently by the same person on different occasions than nonconsensual answers. The results argue against a direct-access view of confidence judgments and suggest that such judgments will be accurate only as long as people's responses are by and large correct across the sampled items, thus stressing the criticality of a representative design.     

Increasing the Parent—Child Communication Through Counselor—Parent Conferences

This study was designed to measure what effect the establishment of a parent-counselor relationship prior to the child's entrance into junior high school would have upon the child's adjustment to school and parent-child communication. Subjects were students and parents from Lealman Junior High School in St. Petersburg, Florida. Parents of the experimental group had a one-hour individual conference with the school counselor prior to the child's entrance into junior high school. Experimental and control groups were followed over a three-year period. The experimental group showed significantly greater (p < .001) parental contact with the school, significantly better student attendance (p < .001), and significantly higher grade point averages (p < .001) than the control group. This was accompanied by a lower dropout rate (p < .05) and fewer disciplinary referrals (p < .001). An overall more effective use of school counselors and other school personnel resulted.     

People's hypercorrection of high-confidence errors: did they know it all along?

This study investigated the "knew it all along" explanation of the hypercorrection effect. The hypercorrection effect refers to the finding that when people are given corrective feedback, errors that are committed with high confidence are easier to correct than low-confidence errors. Experiment 1 showed that people were more likely to claim that they knew it all along when they were given the answers to high-confidence errors as compared with low-confidence errors. Experiments 2 and 3 investigated whether people really did know the correct answers before being told or whether the claim in Experiment 1 was mere hindsight bias. Experiment 2 showed that (a) participants were more likely to choose the correct answer in a 2nd guess multiple-choice test when they had expressed an error with high rather than low confidence and (b) that they were more likely to generate the correct answers to high-confidence as compared with low-confidence errors after being told they were wrong and to try again. Experiment 3 showed that (c) people were more likely to produce the correct answer when given a 2-letter cue to high- rather than low-confidence errors and that (d) when feedback was scaffolded by presenting the target letters 1 by 1, people needed fewer such letter prompts to reach the correct answers when they had committed high- rather than low-confidence errors. These results converge on the conclusion that when people said that they knew it all along, they were right. This knowledge, no doubt, contributes to why they are able to correct those high-confidence errors so easily.     

问题表征过程的一项研究

问题表征是解决问题时理解问题的方式．本研究选用智力数学题为实验作业,通过详细分析３４名大学生被试的问题表征环节及他们问题解决的结果,探讨问题表征中的信息加工过程及其对问题解决结果的影响．实验结果表明：（１）正确的问题表征是解决问题的必要前提,在错误的或者不完整的问题空间中进行搜索不可能求得问题的正确解．（２）问题表征是对问题信息的提取和理解的过程,问题规则在问题表征中起重要作用．（３）在问题表征过程中,导致建构出错误的或者不完整的问题空间的因素包括：信息遗漏（未能将问题的有关信息全部提取出来）,信息误解（对某些问题信息做了错误的分析和理解）,隐喻干扰（问题信息中潜在的歧义性使被试困惑或误导被试的解题思路）等．     

Responding to Feedback after Multiple-choice Answers: The Influence of Response Confidence

A model is proposed that describes how people choose multiple-choice answers and deal with subsequent feedback. Central to the model is the construct of likelihood, a signal that: (1) indexes the match between task demands and what is known, (2) gives a basis for decisions, and (3) is estimated by response confidence ratings. In two experiments, subjects answered an item, rated confidence, and studied feedback (for 100 items), and then answered the items again. All predictions of the model were confirmed. Specifically, for right/wrong feedback messages, feedback study intervals were longer after correct than incorrect choices (for correct choices, these intervals decreased as confidence increased). Also, there was no relation between the probability of correcting errors and level of confidence. For feedback messages that consisted of an item with correct alternative marked, there was a positive relation between probability of correcting errors and level of confidence.     

Can People Identify “Deceptive” or “Misleading” Items that Tend to Produce Mostly Wrong Answers?

In many domains, two‐alternative forced‐choice questions produce more correct responses than wrong responses across participants. However, some items, dubbed “deceptive” or “misleading”, produce mostly wrong answers. These items yield poor calibration and poor resolution because the dominant, erroneous response tends to be endorsed with great confidence, even greater than that of the correct response. In addition, for deceptive items, group discussion amplifies rather than mitigates error while enhancing confidence in the erroneous response. Can participants identify deceptive items when they are warned about their existence? It is argued that people's ability to discriminate between deceptive and non‐deceptive items is poor when the erroneous responses are based on the same process assumed to underlie correct responses. Indeed, participants failed to discriminate between deceptive and non‐deceptive perceptual items when they were warned that some of the items (Experiment 1) or exactly half of the items (Experiment 2) were deceptive. A similar failure was observed for general‐knowledge questions (Experiment 3) except when participants were informed about the correct answer (Experiment 4). Possibly, for these tasks, people cannot escape the dangers lurking in deceptive items. In contrast, the results suggest that participants can identify deceptive problems for which the wrong answer stems from reliance on a fast, intuitive process that differs from the analytic mode that is likely to yield correct answers (Experiment 5). The practical and theoretical implications of the results were discussed. Copyright © 2017 John Wiley & Sons, Ltd.     

Optimal correction for guessing in multiple-choice tests

Subjects’ decisions in multiple-choice tests are an interesting domain for the analysis of decision making under uncertainty. When the test is graded using a rule that penalizes wrong answers, each item can be viewed as a lottery where a rational examinee would choose whether to omit (sure reward) or answer (take the lottery) depending on risk aversion and level of knowledge. We formalize students as heterogeneous decision makers with different risk attitudes and levels of knowledge. Building on IRT, we compute the optimal penalty given students’ optimal behavior and the trade-off between bias and measurement error. Although MCQ examinations are frequently used, there is no consensus as to whether a penalty for wrong answers should be used or not. For example, examinations for medical licensing in some countries include MCQ sections with penalty while in others there is no penalty for wrong answers. We contribute to this discussion with a formal analysis of the effects of penalties; our simulations indicate that the optimal penalty is relatively high for perfectly rational students but also when they are not fully rational: even though penalty discriminates against risk averse students, this effect is small compared with the measurement error that it prevents.     

The self-consistency model of subjective confidence

How do people monitor the correctness of their answers? A self-consistency model is proposed for the process underlying confidence judgments and their accuracy. In answering a 2-alternative question, participants are assumed to retrieve a sample of representations of the question and base their confidence on the consistency with which the chosen answer is supported across representations. Confidence is modeled by analogy to the calculation of statistical level of confidence (SLC) in testing hypotheses about a population and represents the participant's assessment of the likelihood that a new sample will yield the same choice. Assuming that participants draw representations from a commonly shared item-specific population of representations, predictions were derived regarding the function relating confidence to inter-participant consensus and intra-participant consistency for the more preferred (majority) and the less preferred (minority) choices. The predicted pattern was confirmed for several different tasks. The confidence-accuracy relationship was shown to be a by-product of the consistency-correctness relationship: It is positive because the answers that are consistently chosen are generally correct, but negative when the wrong answers tend to be favored. The overconfidence bias stems from the reliability-validity discrepancy: Confidence monitors reliability (or self-consistency), but its accuracy is evaluated in calibration studies against correctness. Simulation and empirical results suggest that response speed is a frugal cue for self-consistency, and its validity depends on the validity of self-consistency in predicting performance. Another mnemonic cue-accessibility, which is the overall amount of information that comes to mind-makes an added, independent contribution. Self-consistency and accessibility may correspond to the 2 parameters that affect SLC: sample variance and sample size.     

Using verification feedback to correct errors made on a multiple-choice test

A key educational challenge is how to correct students’ errors and misconceptions so that they do not persist. Simply labelling an answer as correct or incorrect on a short-answer test (verification feedback) does not improve performance on later tests; error correction requires receiving answer feedback. We explored the generality of this conclusion and whether the effectiveness of verification feedback depends on the type of test with which it is paired. We argue that, unlike for short-answer tests, learning whether one's multiple-choice selection is correct or incorrect should help participants narrow down the possible answers and identify specific lures as false. To test this proposition we asked participants to answer a series of general knowledge multiple-choice questions. They received no feedback, answer feedback, or verification feedback, and then took a short-answer test immediately and two days later. Verification feedback was just as effective as answer feedback for maintaining correct answers. Importantly, verification feedback allowed learners to correct more of their errors than did no feedback, although it was not as effective as answer feedback. Overall, verification feedback conveyed information to the learner, which has both practical and theoretical implications.     

Groups perform better than the best individuals on Letters-to-Numbers problems

Eighty-two four-person cooperative groups and 328 independent individuals solved a random coding of the letters A–J to the numbers 0–9. On each trial the group or individual proposed an equation in letters (e.g., A + D=?), received the answer in letters (e.g., A + D=B), proposed one specific mapping (e.g., A=3), received the answer (e.g., True, A=3), and proposed the full mapping of the 10 letters to the 10 numbers. As predicted, the groups had fewer trials to solution, proposed more complex equations, and identified more letters per equation than each of the best, second-best, third-best, and fourth-best individuals. We interpret the group superiority as due to the highly intellective nature of Letters-to-Numbers problems, which entail demonstrable recognition of correct answers, demonstrable rejection of erroneous answers, and multiple insights into effective collective information processing strategies.     

Associative responding versus analogical reasoning by children

Following upon work which showed that college and high school students' analogy solutions were predictable from ratings of the relatedness of the words comprising the analogy choices, this study extended those results to children's analogy solutions. Achenbach's Children's Associative Responding Test was used as the criterion to determine (1) whether analogy solutions by children were as related to word relatedness ratings as analogy solutions were in previous subject samples, (2) whether individual differences in using associative responding versus analogical reasoning processes could enhance the predictability estimates, and (3) whether associative priming can affect analogy solutions. Two correlational studies were done, one on an American and another on a Nigerian sample. A third study was an experimental manipulation of choices on analogy items on an American sample. The results suggest that the answer to questions (1) and (3) is yes, but the answer to question (2) is no.     

Retention of problem solutions: the re-solution effect

Four experiments compared the re-solution performance of prior solvers with that of prior nonsolvers given the correct solutions. Experiments 1 and 2 challenged Weisberg and Alba's (1981) contention that solving a problem and being shown the solution yield equivalent problem knowledge. In both experiments, students who initially solved problems showed near-perfect recall of the solutions after a 1-week delay, far superior to recall by students who had been shown the correct answers. In Experiment 3, solves showed poor solution retention when the connection between the problem and the solution was not meaningful. Experiment 4 showed that with meaningful problems, solvers and those merely provided with solutions have qualitatively different problem representations. The findings can be explained in terms of differential understanding of problems and their solutions.     

Modification of deficits in reading for comprehension

Subjects were asked to read a passage orally and then to answer questions about the passage. When praise and pennies were given for correct answers, the percentage of correct answers increased in two sixth-grade subjects whose reading for comprehension was tested to be 2 yr below grade level. The behavior of these subjects was compared to that of two subjects whose reading for comprehension was tested to be on grade level. Although no evidence for changes in the accuracy of answering comprehension questions is found in the previous literature, the percentage of correct answers in the children with deficits increased to approximately the same level as the children tested to be on grade level.