Automatic grading and hinting in open-ended text questions

Open-ended text questions provide better assessment of learner’s knowledge, but analysing answers for this kind of questions, checking their correctness, and generating of detailed formative feedback about errors for the learner are more difficult and complex tasks than for closed-ended questions like multiple-choice.The analysis of answers for open-ended questions can be performed on different levels. Analysis on character level allows to find errors in characters’ placement inside a word or a token; it is typically used to detect and correct typos, allowing to differ typos from actual errors in the learner’s answer. The word-level or token-level analysis allows finding misplaced, extraneous, or missing words in the sentence. The semantic-level analysis is used to capture formally the meaning of the learner’s answer and compare it with the meaning of the correct answer that can be provided in a natural or formal language. Some systems and approaches use analysis on several levels.The variability of the answers for open-ended questions significantly increases the complexity of the error search and formative feedback generation tasks. Different types of patterns including regular expressions and their use in questions with patterned answers are discussed. The types of formative feedback and modern approaches and their capabilities to generate feedback on different levels are discussed too.Statistical approaches or loosely defined template rules are inclined to false-positive grading. They are generally lowering the workload of creating questions, but provide low feedback. Approaches based on strictly-defined sets of correct answers perform better in providing hinting and answer-until-correct feedback. They are characterised by a higher workload of creating questions because of the need to account for every possible correct answer by the teacher and fewer types of detected errors.The optimal choice for creating automatised e-learning courses are template-based open-ended question systems like OntoPeFeGe, Preg, METEOR, and CorrectWriting which allows answer-until-correct feedback and are able to find and report various types of errors. This approach requires more time to create questions, but less time to manage the learning process in the courses once they are run.     

Consistency measures for pairwise comparison matrices

We propose new measures of consistency of additive and multiplicative pairwise comparison matrices. These measures, the relative consistency and relative error, are easy to compute and have clear and simple algebraic and geometric meaning, interpretation and properties. The correspondence between these measures in the additive and multiplicative cases reflects the same correspondence which underpins the algebraic structure of the problem and relates naturally to the corresponding optimization models and axiom systems. The relative consistency and relative error are related to one another by the theorem of Pythagoras through the decomposition of comparison matrices into their consistent and error components. One of the conclusions of our analysis is that inconsistency is not a sufficient reason for revision of judgements. © 1998 John Wiley & Sons, Ltd.     

Prolonged exposure in patients with chronic PTSD: predictors of treatment outcome and dropout

The present study investigated predictors of treatment outcome and dropout in two samples of PTSD-patients with mixed traumas treated using prolonged imaginal exposure. Possible predictors were analysed in both samples separately, in order to replicate in one sample findings found in the other. The only stable finding across the two groups was that patients who showed more PTSD-symptoms at pre-treatment, showed more PTSD-symptoms at post-treatment and follow-up. Indications were found that benzodiazepine use was related to both treatment outcome and dropout, and alcohol use to dropout. Demographic variables, depression and general anxiety, personality, trauma characteristics, feelings of anger, guilt, and shame and nonspecific variables regarding therapy were not related to either treatment outcome or dropout, disconfirming generally held beliefs about these factors as contra-indications for exposure therapy. It is concluded that it is difficult to use pre-treatment variables as a powerful and reliable tool for predicting treatment outcome or dropout. Clinically seen, it is therefore argued that exclusion of PTSD-patients from prolonged exposure treatment on the basis of pre-treatment characteristics is not justified.     

A computational model of event segmentation from perceptual prediction

People tend to perceive ongoing continuous activity as series of discrete events. This partitioning of continuous activity may occur, in part, because events correspond to dynamic patterns that have recurred across different contexts. Recurring patterns may lead to reliable sequential dependencies in observers' experiences, which then can be used to guide perception. The current set of simulations investigated whether this statistical structure within events can be used 1) to develop stable internal representations that facilitate perception and 2) to learn when to update such representations in a self-organizing manner. These simulations demonstrate that experience with recurring patterns enables a system to accurately predict upcoming stimuli within an event, to identify boundaries between such events based on transient increases in prediction error, and to use such boundaries to improve prediction about subsequent activities.     

汉语双字词的心理词典存储—来自发展性阅读障碍个案的证据

研究探讨了发展性阅读障碍个案S汉字命名过程选择性错误的产生机制。汉字命名、词汇判断和组词实验有三个主要发现,个案S在汉字命名过程中产生大量的选择性错误,选择性错误在复合词中发生的前后位置没有显著差异;S在词汇判断过程中,混淆词汇中两个词素的先后顺序;S对产生选择性错误的汉字组词的数量显著少于控制组,且其产生选择性错误的复合词的频率显著高于他组出的其他词汇频率。研究以Zhou&Marslen-Wilson(2000)的复合词的多水平平行结构心理词典模型为基础讨论了这些发现的理论意义,并着重分析了汉字命名过程中词素频率和词汇意义结构之间的交互作用。     

Using automation to combat the replication crisis: A case study from controlled-rearing studies of newborn chicks

The accuracy of science depends on the precision of its methods. When fields produce precise measurements, the scientific method can generate remarkable gains in knowledge. When fields produce noisy measurements, however, the scientific method is not guaranteed to work – in fact, noisy measurements are now regarded as a leading cause of the replication crisis in psychology. Scientists should therefore strive to improve the precision of their methods, especially in fields with noisy measurements. Here, we show that automation can reduce measurement error by ∼60% in one domain of developmental psychology: controlled-rearing studies of newborn chicks. Automated studies produce measurements that are 3–4 times more precise than non-automated studies and produce effect sizes that are 3–4 times larger than non-automated studies. Automation also eliminates experimenter bias and allows replications to be performed quickly and easily. We suggest that automation can be a powerful tool for improving measurement precision, producing high powered experiments, and combating the replication crisis.     

On Being Happy and Mistaken on a Good Day: Revisiting Forgas’s (1998) Mood-Bias Result

Forgas (1998) reported “on being happy and mistaken,” finding that a happy mood increased the fundamental attribution error (FAE) compared to a sad mood. However, the standard attitude-attribution paradigm used by Forgas might contain demand characteristics, to which happy people might be especially susceptible. In addition, Goldenberg and Forgas (2012) showed that a happy mood decreased a form of the FAE. Using the questioner-contestant paradigm, we replicated Forgas’s result, but only when participants were already having a “good day.”     

The relationship between cognitive errors,coping strategies,and clients' experiences in session: An exploratory study

Aim: Cognitive errors (CE) and coping strategies (CS) can bear weight on how individuals relate to others and perceive interpersonal relationships. However, there is little research into how clients' erroneous beliefs and maladaptive coping strategies can interfere with the therapeutic process. This study utilised a sample of healthy clients to explore the relationship between their CEs and CSs and their evaluation of therapy. Method: Therapy sessions of undergraduate student clients (N =26) were rated using the Cognitive Error Rating Scale (CERS – 3rd edition), the Coping Patterns Rating Scale (CPRS;), the Session Evaluation Questionnaire (SEQ) and the Session Impact Scale (SIS). Results: Clients who engaged in dichotomous thinking endorsed problem solving less and were more likely to feel unsupported and misunderstood by the therapist. Clients who discounted the positive tended to feel more pressured and judged by therapists. Conversely, those who engaged in problem solving were more likely to find sessions deeper and more valuable as compared to those who reacted to stressful events by submission, escape, or opposition. Implications: Better understanding how and when a client's cognitive errors and coping mechanisms are at play during therapy can help clinicians to address them and intervene appropriately.