Examiner error in curriculum-based measurement of oral reading

Although curriculum based measures of oral reading (CBM-R) have strong technical adequacy, there is still a reason to believe that student performance may be influenced by factors of the testing situation, such as errors examiners make in administering and scoring the test. This study examined the construct-irrelevant variance introduced by examiners using a cross-classified multilevel model. We sought to determine the extent of variance in student CBM-R scores attributable to examiners and, if present, the extent to which it was moderated by students' grade level and English learner (EL) status. Fit indices indicated that a cross-classified random effects model (CCREM) best fits the data with measures nested within students, students nested within schools, and examiners crossing schools. Intraclass correlations of the CCREM revealed that roughly 16% of the variance in student CBM-R scores was associated between examiners. The remaining variance was associated with the measurement level, 3.59%; between students, 75.23%; and between schools, 5.21%. Results were moderated by grade level but not by EL status. The discussion addresses the implications of this error for low-stakes and high-stakes decisions about students, teacher evaluation systems, and hypothesis testing in reading intervention research.     

Client as expert: A Delphi poll of clients’ subjective experience of therapeutic alliance formation variables

This study extends prior research that identified client-derived variables believed to contribute to therapeutic alliance development. Forty-two clients participated in a three-round-modified Delphi poll in which the subjective importance of 74 client-derived, common alliance formation variables was rated using a six-point scale. Participants consensually identified 23 variables as highly important, five as moderately important and three as low in importance to the formation of a strong alliance. Variables related to validation and asking about parts of the client’s life other than the presenting concern were considered as most important by at least one-third of the sample. Limitations of the study are presented and implications for theory, measurement, practice and future research are briefly discussed.     

Exploring Differential Subgroup Functioning on SAT Writing Items: What Happens When English Is Not a Test Taker's Best Language?

The purpose of this study is to explore patterns in model-data fit related to subgroups of test takers from a large-scale writing assessment. Using data from the SAT, a calibration group was randomly selected to represent test takers who reported that English was their best language from the total population of test takers (N = 322,011). A reference scale for the items was constructed based on EBL responses. Response behaviors of test takers who reported that English was not their best language (ENBL) were examined in relationship to this reference scale. This study illustrates the use of differential subgroup analyses to identify patterns related to person misfit within subgroups, as well as subsets of items, that may affect the validity of writing scores for ENBL test takers. The methodology described here offers an approach that can be used to explore, understand, and improve the validity of scores obtained from ENBL test takers in large-scale writing assessments.     

判断与决策研究纵观:基于模糊痕迹理论的视角(英文)

模糊痕迹理论是用于解释记忆、判断与决策的综合性理论,该理论的提出和发展主要基于对信息存储、表征、提取和加工过程的研究。本文首先介绍了模糊痕迹理论的基本原则,在此基础上重点讨论了其要义(gist)如何发挥核心作用,使得模糊痕迹理论有别于其他传统的决策模型。该理论将高级直觉与原始冲动性进行了区分,并且预测决策误差来源于判断与决策的各种不同成分,如背景知识、信息表征、提取和加工过程等。模糊痕迹理论不仅可以解释诸如框架效应、合取谬误等传统决策与判断文献中常讨论的误差现象,同时基于该理论的研究还得到了一些与传统决策理论相悖的新发现。此外,对脑与行为如何发育性变化的研究为我们了解成人的认知过程提供了至关重要的新视角,这些对脑与行为的发育性研究和对特殊人群的研究结果也都支持了模糊痕迹理论对要义加工依赖的预测。     

基于结构方程模型的多重中介效应分析

多重中介模型是指存在多个中介变量的模型。多重中介模型可以分析特定中介效应、总的中介效应和对比中介效应。指出了目前多重中介模型分析普遍存在的问题,包括分析不完整、使用Sobel检验带来的局限。建议通过增加辅助变量的方法进行完整的多重中介效应分析,使用偏差校正的Bootstrap方法进行中介检验。总结出一个多重中介SEM分析流程,并有示例和相应的MPLUS程序。随后展望了辅助变量和中介效应检验方法的发展方向。     

Proactive control of proactive interference using the method of loci

Proactive interferencebuilds up with exposure to multiple lists of similar items with a resulting reduction in recall. This study examined the effectiveness of using a proactive strategy of the method of loci to reduce proactive interference in a list recall paradigm of categorically similar words. While all participants reported using some form of strategy to recall list words, this study demonstrated that young adults were able to proactively use the method of loci after 25 min of instruction to reduce proactive interference as compared with other personal spontaneous strategies. The implications of this study are that top-down proactive strategies such as the method of loci can significantly reduce proactive interference, and that the use of image and sequence or location are especially useful in this regard.     

Validation and Standardization of the Dutch Trauma Symptom Checklist for Young Children in a Normative and Clinical Sample

The study aimed to describe the psychometric properties of the Dutch version of the Trauma Symptom Checklist for Young Children (TSCYC) in normative and clinical populations in the Netherlands. Caregivers’ ratings on the TSCYC were obtained for 1,802 children from the normal population, and for 515 children from a clinical population of traumatized children. In the clinical sample, additional measures were taken. The internal consistency and test-retest reliability of TSCYC scales were adequate. Confirmatory factor analysis showed acceptable fit on the putative scale structure. Regarding criterion validity, the clinical sample scored significantly higher on all clinical scales when compared to the normal population sample. Within the clinical sample, significant associations were found between TSCYC scales and convergent scales of other instruments. The Posttraumatic Stress-Total subscale demonstrated excellent discriminative ability between traumatized children and children from the normal population. The Dutch version of the TSCYC proved a valid and reliable instrument to measure trauma symptoms in young children through caregiver report, similar to the original American version. Further comparisons with diagnostic interviews are warranted.     

An Evaluation of the Word Triad Method for Monitoring Spelling Progress

Although spelling skill progress has typically been studied within the context of students' responses to written story starters (Deno, Marsten, & Mirkin, 1982; Fuchs & Fuchs, 2011; Hosp, Hosp, & Howell, 2007; Shinn & Shinn, 2002), there has been little research conducted within a curriculum-based measurement framework that has studied spelling progress monitoring on a weekly basis throughout an entire school year. The authors evaluated spelling progress using word triads, defined as a group of 3 words (with the same number of letters) that have a similar phonetic or morphological structure. Successful spelling of word triads as the unit of analysis has the potential to be more useful diagnostically to teachers than successful spelling of single words as the unit of analysis. An alternating series of three 15-word spelling tests were administered weekly to 10 third- and fourth-grade classrooms in the same school district throughout the school year for 8 rounds (1 round = 3 weeks of different but phonetically and/or morphologically similar word lists). Results indicate steady progress throughout the school year in the correct spelling of word triads (despite teacher reporting of no explicit spelling instruction in classrooms using in the words employed in the study). Correlations between the number of correctly spelled word triads with standard scores from a group-administered communication arts test were significant and comparable to alternate spelling test scoring methods. The authors conclude with limitations of the study and implications for practitioners.