Using Automated Essay Scores as an Anchor When Equating Constructed Response Writing Tests

Assessments consisting of only a few extended constructed response items (essays) are not typically equated using anchor test designs as there are typically too few essay prompts in each form to allow for meaningful equating. This article explores the idea that output from an automated scoring program designed to measure writing fluency (a common objective of many writing prompts) can be used in place of a more traditional anchor. The linear-logistic equating method used in this article is a variant of the Tucker linear equating method appropriate for the limited score range typical of essays. The procedure is applied to historical data. Although the procedure only results in small improvements over identity equating (not equating prompts), it does produce a viable alternative, and a mechanism for checking that the identity equating is appropriate. This may be particularly useful for measuring rater drift or equating mixed format tests.     

Consequences of erudite vernacular utilized irrespective of necessity: problems with using long words needlessly

Most texts on writing style encourage authors to avoid overly‐complex words. However, a majority of undergraduates admit to deliberately increasing the complexity of their vocabulary so as to give the impression of intelligence. This paper explores the extent to which this strategy is effective. Experiments 1–3 manipulate complexity of texts and find a negative relationship between complexity and judged intelligence. This relationship held regardless of the quality of the original essay, and irrespective of the participants' prior expectations of essay quality. The negative impact of complexity was mediated by processing fluency. Experiment 4 directly manipulated fluency and found that texts in hard to read fonts are judged to come from less intelligent authors. Experiment 5 investigated discounting of fluency. When obvious causes for low fluency exist that are not relevant to the judgement at hand, people reduce their reliance on fluency as a cue; in fact, in an effort not to be influenced by the irrelevant source of fluency, they over‐compensate and are biased in the opposite direction. Implications and applications are discussed. Copyright © 2005 John Wiley & Sons, Ltd.     

Motor Imagery Shapes Abstract Concepts

The concepts of “good” and “bad” are associated with right and left space. Individuals tend to associate good things with the side of their dominant hand, where they experience greater motor fluency, and bad things with their nondominant side. This mapping has been shown to be flexible: Changing the relative fluency of the hands, or even observing a change in someone else's motor fluency, results in a reversal of the conceptual mapping, such that good things become associated with the side of the nondominant hand. Yet, based on prior studies, it is unclear whether space–valence associations were determined by the experience of fluent versus disfluent actions, or by the mere expectation of fluency. Here, we tested the role of expected fluency by removing motor execution and perceptual feedback altogether. Participants were asked to imagine themselves performing a psychomotor task with one of their hands impaired, after which their implicit space–valence mapping was measured. After imagining that their right hand was impaired, right‐handed participants showed the “good is left” association typical of left‐handers. Motor imagery can change people's implicit associations between space and emotional valence. Although asymmetric motor experience may be necessary to establish body‐specific associations between space and valence initially, neither motoric nor perceptual experience is needed to change these associations subsequently. The mere expectation of fluent versus disfluenct actions can drive fluency‐based effects on people's implicit spatialization of “good” and “bad.” These results suggest a reconsideration of the mechanisms and boundary conditions of fluency effects.     

The bit in the middle and why it’s important: a computational analysis of the linguistic features of body paragraphs

This study examines the relationship between the linguistic characteristics of body paragraphs of student essays and the total number of paragraphs in the essays. Results indicate a significant relationship between the total number of paragraphs and a variety of linguistic characteristics known to affect student essay scores. These linguistic characteristics (e.g., semantic overlap, syntactic complexity) contribute to two underlying factors (i.e., textual cohesion and difficulty) that are used as dependent variables in mixed-effect models. Results suggest that student essays with 5-8 paragraphs tend to be more linguistically consistent than student essays with 3, 4, and 9 paragraphs. Essays with totals of 5-8 paragraphs, considered by many educators to contain an optimal number of paragraphs, may include functionally and structurally similar paragraphs. These findings could aid writing researchers and educators in obtaining a clearer view of the relationship between the total number of paragraphs comprising an essay and the linguistic characteristics that affect essay evaluation. Consequently, writing interventions may become better equipped to pinpoint student difficulties and facilitate student writing skills by providing more detailed and informed feedback.     

阅读范文对小学生作文成绩的影响

为考察阅读范文是否能够提高三年级小学生记叙文写作的成绩,在建立低年级小学生记叙文写作评价指标的基础上,选择1篇完整范文并设计8篇不同类型的不完整范文,选取300名三年级小学生为被试进行了实验研究。实验结果表明:(1)阅读完整范文和联结内容式、补充中间式、补充开头式、补充结尾式等不完整范文能够明显提高小学生记叙文写作结构分项的成绩,阅读其他4种类型的不完整范文作用不明显。(2)阅读完整的和8种不完整范文均能明显提高小学生在记叙文写作内容分项的成绩。(3)除了补充开头式、补充中间和结尾式两种不完整范文之外,阅读其他7种范文均能明显提高作文词句分项的成绩。(4)阅读范文对避免写作中出现错别字、错用标点符号和病句无显著作用。     

Steps Toward an Ecology of Writing in Digital Environments: A 21st-Century Walk Through van Gogh's Bedroom

This study examined how exposure to a computer-generated, avatar-based environment influences student composition of an essay in response to a writing prompt. Drawing on the distinctions between firsthand and secondhand experiences, this article sketches an ecological model of writing that attempts to explain why students exposed to an avatar-based environment wrote first-person essays in the present tense, whereas students exposed to a comparable web page-based environment wrote third-person essays in the past tense.     

Measuring Processing Fluency: One versus Five Items

While there is an ample amount of consumer behavior research that recruits processing fluency as an explanatory construct, the question how to best measure the fluency experience has received little attention. Therefore, there is a lack of consistency in measuring the construct, particularly with regard to the use of single‐item versus multi‐item measures. The current research, thus, aims to investigate how processing fluency can be consistently measured across different experimental fluency manipulations and which type of measure has the highest validity. Based on classic scale development procedures, we propose a reliable and valid multi‐item measure and compare this measure against a single‐item measure in terms of predictive validity. We show that both measures mediate the effect of five established fluency manipulations and that the single‐item measure is sufficient. In addition to providing a measure for future research that can be adapted to different empirical settings, we provide empirical evidence on the replicability of fluency effects and on the theoretical conjecture that people have a uniform fluency experience across different manipulations of fluency.     

The influence of thinking dispositions on integration and recall of multiple texts

We investigated the association between thinking dispositions and two outcomes of multiple-texts comprehension: integration of conflicting information in argumentative essays; and recall of inferential information as an index of deep comprehension. We focused on two thinking dispositions, need for cognition (NFC) and actively open-minded thinking (AOT), as relevant individual differences in the processes involved in multiple-texts comprehension. NFC is the tendency to engage in and enjoy cognitively demanding activities, whereas AOT is the tendency to rationally evaluate arguments and supporting evidence, without being influenced by biases from one's own prior beliefs and prior knowledge. 73 university students completed perceived topic knowledge, perceived exposure to argumentative writing, and perceived competence in argumentative writing, NFC and AOT questionnaires, read two contradictory texts, wrote an argumentative essay, and recalled the information read 1 month later. Argumentative essays were assessed by length and level of integration of conflicting perspective. Text recalls were assessed by number of valid inferences included. Research questions were investigated through a path analysis model. The path analysis model had a good fit. NFC was indirectly associated with argumentation quality of the essay via the essay length. AOT was directly associated with the inferences included in the recall task. The present study contributes to the literature on multiple-texts comprehension by emphasizing the role of thinking dispositions.     

MY MOTHER,MY STORY

This piece returns to the writer’s memoir essays about her mother’s chronic lung disease to examine the relationship between the act of caregiving and the act of writing. In arguing for important differences between the clinical, healing imperatives of narrative medicine and the primacy for the writer of self-reflection, personal need and career, the essay demonstrates how writing remains in many ways at odds with the obligations and the hopes of caregiving. At the same time, the essay argues that writing her mother’s stories of illness holds the potential for both honor and mutuality—and can, in fact, constitute a form of caregiving.     

目标设置对写作困难儿童议论文说服力的促进效应研究

将写作困难儿童随机分入三个不同的目标设置组,总目标设置条件要求儿童根据话题写议论文,目标为文章应能说服读者赞同他们的观点;分目标设置条件对儿童有同样的要求,但给儿童提供了以议论文的基本构成为基础的分目标;辩论加分目标设置除了与分目标设置组相同的条件外,还要求儿童在写作前进行辩论。研究结果表明,分目标设置组儿童的议论文比总目标设置组和辩论加分目标设置组儿童的议论文更具说服力度。在文章构成成分上,分目标设置组和辩论加分目标设置组要好于总目标设置组。这三种目标设置中,分目标对写作困难儿童的议论文说服力有很大的促进作用。     

THE EFFECT OF MOOD ON THE RELIABILITY OF ESSAY ASSESSMENT

Summary . One potential source of bias that has not been examined in relation to the grading of essays is that of the mood of the marker. Several lines of research in social psychology can be generalised to predict different assessment outcomes as a function of mood. This study examined the effect of mood on student grading of children's essays. Markers first watched a film designed to induce either a positive or a negative mood state. They then graded nine essays, including a number of target essays. Mood affected only the first essay graded. The results do not support a mood effect in essay assessment.     

Instructor Feedback for College Writing Assignments in Introductory Classes

We compared three instructor feedback protocols to determine whether minimal instructor feedback could improve undergraduate students' writing skills in large introductory classes. The three feedback conditions were specific feedback, nonspecific feedback, and grade only. Sixty community college students participated in the study. Sections of an Introduction to Human Services course were randomly assigned to the three feedback conditions. Students wrote 9 timed essays during the semester. Pretest data were collected from the first essay at the beginning of the semester. Posttest data were taken from the final essay. Data for each student's essays consisted of the number of words written, the number of spelling errors, and the number of syntax grammar errors. Between the pretest and the posttest, students wrote 7 timed essays during class periods. Students in the specific feedback condition received a tally of spelling and grammar errors on grade criteria sheets and the location of the errors marked on their essays. In the nonspecific feedback condition, students received a tally of the errors on their grade criteria sheets but the locations of the errors were not marked on the essays. In the grade only condition, students received feedback on the number of points they had earned for each essay. Significant differences were found for pretest–posttest difference (gain) scores for the proportion of errors to the number of words written between the specific feedback condition and the grade only condition. Specific feedback decreased the proportion of errors to words written. The grade only condition maintained the same proportion of errors to words written from pretest to posttest. Significant differences were also found between difference (gain) scores for total errors between the specific feedback condition and the grade only condition and between the nonspecific feedback condition and the grade only condition.     

Affective-cognitive consistency and thought-induced attitude polarization

Subjects whose preexperimental attitudes toward either capital punishment or censorship were high or low in affective-cognitive consistency were identified. These four groups thought about their attitudes by writing two essays, one on the topic for which consistency had been assessed (relevant essay) and one on the unassessed topic (distractor essay). In accord with the hypothesis that thought-induced attitude polarization requires the presence of a well-developed knowledge structure, high-consistency subjects evidenced greater polarization than low-consistency subjects only on the relevant topic after writing the relevant essay. Content analyses of subjects' relevant essays yielded additional data confirming Tesser's ideas regarding mediation: High (vs. low) consistency subjects expressed a greater proportion of cognitions that were evaluatively consistent with their prior affect toward the attitude object and a smaller proportion of evaluatively inconsistent and neutral cognitions. Moreover, although high-and low-consistency subjects did not differ in the amount of attitudinally relevant information they possessed or their awareness of inconsistent cognitions, their method of dealing with discrepant information diverged: High-consistency subjects evidenced a greater tendency to assimilate discrepant information by generating refutational thoughts that discredited or minimized the importance of inconsistent information.     

Does using a computer disturb the organization of children's writing?

Children's speed and fluency of writing has elsewhere been shown to correlate with the quality of their composition. Here, we compared speed and fluency of text production when children aged between 6 and 11 used either a pen or a computer keyboard. Younger children were reliably slower and less fluent when writing at a keyboard. All children were slower when the text was more demanding. These impediments to smooth writing were shown to be associated with the different patterns of visual attention required by the two writing tools. It is argued that writing is usefully approached as a tool‐mediated activity whose coordination presents developmental challenges. Moreover, the results suggest grounds for investing more in helping children towards greater confidence in visual–manual control of the keyboard.     

Towards a better understanding of the legibility bias in performance assessments: The case of gender‐based inferences

Background. Handwriting legibility systematically biases evaluations in that highly legible handwriting results in more positive evaluations than less legible handwriting. Because performance assessments in educational contexts are not only based on computerized or multiple choice tests but often include the evaluation of handwritten work samples, understanding the causes of this bias is critical. Aims. This research was designed to replicate and extend the legibility bias in two tightly controlled experiments and to explore whether gender‐based inferences contribute to its occurrence. Sample(s). A total of 132 students from a German university participated in one pre‐test and two independent experiments. Method. Participants were asked to read and evaluate several handwritten essays varying in content quality. Each essay was presented to some participants in highly legible handwriting and to other participants in less legible handwriting. In addition, the assignment of legibility to participant group was reversed from essay to essay, resulting in a mixed‐factor design. Results. The legibility bias was replicated in both experiments. Results suggest that gender‐based inferences do not account for its occurrence. Rather it appears that fluency from legibility exerts a biasing impact on evaluations of content and author abilities. Conclusions. The legibility bias was shown to be genuine and strong. By refuting a series of alternative explanations, this research contributes to a better understanding of what underlies the legibility bias. The present research may inform those who grade on what to focus and thus help to better allocate cognitive resources when trying to reduce this important source of error.     

Planning and Revising Written Arguments: the Effects of Two Text Structure-Based Interventions on Persuasiveness of 8th-Grade Students’ Essays

The purpose of the study was to investigate the effects of two comprehensive argumentative writing interventions—Text Structure Instruction (TSI) and Text Structure Revision Instruction (TSRI)—on the eighth-grade students' ability to compose convincing essays that include structural elements of argumentative discourse. Both treatment groups received explicit text structure instruction in revising argumentative essays. The students in TSRI treatment group were further supported by implementing a free-writing approach during the planning stage and by providing procedural facilitation during the revision stage, whereas the students in TSI treatment group were provided with a traditional planning strategy and a checklist for revising essay. The results indicated that across the conditions, significant increases in the participants. performance from pre-test to post-test were observed on structural elements of argumentative discourse and overall persuasiveness, indicating the effectiveness of both interventions.     

Does self-affirmation, cognitive processing, or discovery of meaning explain cancer-related health benefits of expressive writing?

Although expressive writing has positive effects on health, little is known about the underlying psychological mechanisms for these effects. The present study assessed self-affirmation, cognitive processing, and discovery of meaning as potential mediators of the effects of expressive writing on physical health in early-stage breast cancer survivors. A content analysis of the essays showed that self-affirmation writing was associated with fewer physical symptoms at a 3-month follow-up assessment, with self-affirmation writing fully mediating the effects of the emotional expression and benefit-finding writing conditions on reduced physical symptoms. Cognitive processing and discovery of meaning writing were not associated with any physical health outcomes. Consistent with evidence showing that self-affirmation plays an important role in buffering stress, the present study provides the first evidence for self-affirmation as a viable mechanism underlying the health benefits of expressive writing.     

John Webster on Dogmatics and Ethics

From his earliest published writing Webster identified the moral life as the horizon of dogmatic enquiry, and his last‐published work offers a subtle clarification of how dogmatics relates to ethics. The principle ‘action follows being’ governs both the relation of the two sub‐disciplines and the internal logic of each. As theology is a knowledge acquired not only speculatively but also practically, dogmatics must look forward and practical theology look back. Their interaction covers the whole range of theological topics. It is a fittingly Chalcedonian conception stressing the duality of dogmatics and ethics alongside their unity. While specialized dogmatics and ethics are legitimate, the two remain proximate, in a definite order of origin and derivation. The temptation of a de‐moralized theology equalled by that of a theology wrongly constructed on a moral a priori. In his late essays Webster presents worked examples of how dogmatics may handle moral questions without venturing into casuistic exploration of dilemmas. One essay, however, goes further, troublingly describing ‘mortification and vivification’ as the primary human acts of the regenerate state, a function that the tradition has always assigned to love.