Invariance Properties for General Diagnostic Classification Models

In item response theory (IRT), the invariance property states that item parameter estimates are independent of the examinee sample, and examinee ability estimates are independent of the test items. While this property has long been established and understood by the measurement community for IRT models, the same cannot be said for diagnostic classification models (DCMs). DCMs are a newer class of psychometric models that are designed to classify examinees according to levels of categorical latent traits. We examined the invariance property for general DCMs using the log-linear cognitive diagnosis model (LCDM) framework. We conducted a simulation study to examine the degree to which theoretical invariance of LCDM classifications and item parameter estimates can be observed under various sample and test characteristics. Results illustrated that LCDM classifications and item parameter estimates show clear invariance when adequate model data fit is present. To demonstrate the implications of this important property, we conducted additional analyses to show that using pre-calibrated tests to classify examinees provided consistent classifications across calibration samples with varying mastery profile distributions and across tests with varying difficulties.     

构思策略教学改善小学写困生故事文质量的实验研究

采用实验组、控制组前后测实验设计,运用SRSD策略教学模式,对23名小学写困生进行构思策略的训练。实验结果表明,在训练之前,实验组与控制组学生在故事文的长度、6项分质量指标及总体质量上的得分差异均不显著;但在接受策略教学后,实验组学生所写的后测故事文在情节吸引力、条理清晰度和总体质量上的平均得分均显著优于控制组学生,其总体质量的改善效应为0.80。因此,构思策略的教学能显著改善小学写困生的故事文写作质量。     

Programming for the Generalization of Oral Reading Fluency: Repeated Readings of Entire Text versus Multiple Exemplars

This study compared a multiple exemplar approach involving the training of key words and sentence structures to a typical repeated readings procedure for their effects on students’ generalized oral reading fluency. The two training approaches were also compared in terms of their relative learning rates (i.e., fluency gain per minute of instruction time). Participants were 111 first- and second-grade students from three public schools in a large urban school district. This study utilized an across students pre-test/post-test experimental design and one-way analyses of variance on the gain scores from pre-test to post-test to assess the effectiveness of multiple exemplar instruction, listening passage preview/repeated readings, and a time and attention control condition. Results showed significantly higher gain scores for students in both the multiple exemplar and listening passage preview/repeated readings groups versus controls on the trained and generalization passages. Multiple exemplar instruction also resulted in significantly larger learning rates than listening passage preview/repeated readings. Implications, limitations, and directions for future research are discussed.     

一种基于多阶认知诊断模型测评科学素养的方法

科学素养是指作为一名有反思意识的公民所具有的解决科学问题和运用科学理念的能力。为实现在认知诊断中对科学素养的测评, 本文基于PISA 2015科学素养测评框架首次提出科学素养包含的三阶潜在结构, 使用新提出的多阶认知诊断模型对PISA 2015科学测评数据进行分析, 并通过模拟研究探究新模型的心理测量学性能。结果表明：(1)新模型能够较好地分析包含三阶潜在结构的科学素养; (2)科学知识对科学素养的影响最大, 科学背景次之, 科学能力的影响最小; (3)全贝叶斯MCMC算法能够为新模型提供较精准的参数估计。     

A New Concurrent Calibration Method for Nonequivalent Group Design under Nonrandom Assignment

This study proposes a new item parameter linking method for the common-item nonequivalent groups design in item response theory (IRT). Previous studies assumed that examinees are randomly assigned to either test form. However, examinees can frequently select their own test forms and tests often differ according to examinees’ abilities. In such cases, concurrent calibration or multiple group IRT modeling without modeling test form selection behavior can yield severely biased results. We proposed a model wherein test form selection behavior depends on test scores and used a Monte Carlo expectation maximization (MCEM) algorithm. This method provided adequate estimates of testing parameters.     

Item Response Modeling With BILOG-MG and MULTILOG for Windows

Item response theory (IRT) has become one of the most popular scoring frameworks for measurement data. IRT models are used frequently in computerized adaptive testing, cognitively diagnostic assessment, and test equating. This article reviews two of the most popular software packages for IRT model estimation, BILOG-MG (Zimowski, Muraki, Mislevy, & Bock, 1996) and MULTILOG (Thissen, 1991), which are for the first time available on a single CD-ROM with new features. Most prominently, the number of items to be calibrated and examinees to be scored is now limited only by memory capacities of the hardware, MULTILOG has an interactive Windows-oriented process for creating basic command file syntax, and both BILOG-MG and MULTILOG come with a new graphics interface that displays numerous curves relevant to IRT analyses in a professional format. This article reviews the models that are and are not estimable with these programs and describes the fundamental ideas of the underlying estimation algorithms without providing detailed derivations. Moreover, the user-friendliness of both programs is assessed with a user in mind who is interested in easy-to-use IRT estimation programs within a Windows point-and-click environment. Both programs fulfill such an expectation to a large degree; yet, this review also points out some obstacles that someone relatively unfamiliar to IRT or syntax programming might have to overcome to obtain meaningful results.     

Specifying Ability Growth Models Using a Multidimensional Item Response Model for Repeated Measures Categorical Ordinal Item Response Data

When categorical ordinal item response data are collected over multiple timepoints from a repeated measures design, an item response theory (IRT) modeling approach whose unit of analysis is an item response is suitable. This study proposes a few longitudinal IRT models and illustrates how a popular compensatory multidimensional IRT model can be utilized to formulate such longitudinal IRT models, which permits an investigation of ability growth at both individual and population levels. The equivalence of an existing multidimensional IRT model and those longitudinal IRT models is also elaborated so that one can make use of an existing multidimensional IRT model to implement the longitudinal IRT models.     

The effect of attentional instructions during modeling on gaze behavior and throwing accuracy in 7 to 10 year-old children

Research has shown the effectiveness of attentional instructions and observation on the performance and learning in children. However, there is little research on the effects of instruction that manipulates attentional focus during observation. The purpose of the study was to investigate performance following instructions that directed an individual to focus internally or externally during the viewing of a model. Eye tracking was also used to investigate the visual search strategy under different instructional groups. 24 children between the ages of 7 and 10 years were randomly divided into an internal (IFM) or external (EFM) modeling group. Children performed a 10-trial pre-test to measure throwing accuracy and quiet eye duration (QED). Following pre-test, children observed a model performing the correct overarm throw 10 times while eye movement was recorded. They then performed a 10-trial post-test. Results revealed that both groups improved accuracy from pre-test to post-test (p < .001) and increased QED from pre- to post-test (p < .001). It was also observed that EFM had greater accuracy and (p = .01) and longer QED (p < .001) than IFM during post-testing. Evidence was also observed for adherence to the attentional focus strategy as EFM and IFM looked more frequently at the specific cue provided (p < .001). Directing visual attention to the movement effects while observing a model significantly benefits motor performance and learning in children. Changing the focus of instruction to an external focus improved both throwing accuracy and increased QED.     

高校毕业生择业焦虑的心理教育实验研究

通过组织择业心理辅导小组对毕业生择业焦虑进行干预,考察其心理教育效果.采用实验组、对照组前后测实验设计,以《高校毕业生择业焦虑问卷》和《状态-特质焦虑量表》为测量工具.结果表明,实验组(n=18)与对照组(n=34)的后测结果差异显著,实验组与对照组的前后测结果差量之间有显著差异,对前测结果进行控制的协方差分析也表明实验组与对照组的组间差异显著.说明研究所设计的小组心理辅导与行为治疗相结合的心理教育方案是帮助毕业生克服择业焦虑的有效方法,可用于毕业生择业心理教育.     

Efficacy of a Self-Administered Home-Based Parent Intervention on Parenting Behaviors for Preventing Adolescent Substance Use

A growing body of literature suggests that parenting practices characterized by careful monitoring, firm and consistent limit setting, and nurturing communication patterns with children are protective against adolescent substance use and other problem behaviors. Family-based prevention programs that promote these behaviors can be an effective way to prevent adolescent substance use. However, low participation rates remain problematic for many such programs, particularly programs that require parents to attend scheduled meetings outside the home. The purpose of this study was to determine the efficacy of a newly developed substance use prevention program when self-administered at home by parents of middle school students. As part of a randomized trial, 338 parents of middle school students either received the parent prevention program or served as control group participants. Parents completed self-report surveys at home that assessed parenting behaviors at pre-test, post-test, and one-year follow-up time points. A series of mixed model ANCOVAs were conducted, examining the post-test and one-year follow-up means for the parent outcomes, controlling for pre-test levels on these outcomes. Analyses showed that at the post-test assessment, intervention parents reported significant increases relative to controls in appropriate role modeling, disciplinary practices, family communication, and parental monitoring skills. At the one-year follow-up assessment, intervention effects on family communication skills remained significant and effects on parental role modeling were marginally significant. This study shows that a theoretically rich prevention program can be effectively self-administered by parents at home and improve key parenting skills that have been shown to prevent adolescent substance use.     

心理干预对轻中度脑瘫儿童综合功能和生活质量的影响*

采用对照组和干预组前测、后测设计,对80名脑瘫儿童的综合功能和生活质量进行了测量,分析了心理干预对脑瘫儿童康复疗效的影响。结果表明:(1)四组脑瘫儿童后测综合功能评定效果存在显著差异。(2)生活质量问卷中,在使用特殊器具感觉维度上,四个组别间的追踪水平逐级显著递增;在对疼痛和困扰感觉维度上,四个组别间的后测水平、追踪水平均逐级显著递减;在其它四个维度上,四个组别间的后测水平、追踪水平均逐级显著递增。(3)在使用特殊器具感觉维度上,三个干预组内的追踪水平显著高于前测和后测水平;在对疼痛和困扰感觉维度上,三个干预组内的前测、后测和追踪水平均呈逐级显著递减趋势;在其它四个维度上,三个干预组内的前测、后测和追踪水平均逐级显著递增。干预组比对照组能显著提升脑瘫患儿的综合功能和生活质量。     

数学教学中培养学生学习自我效能感的实验研究

通过构建的渗透培育模式,以初中二年级数学教学为依托,运用教学实验方法探索数学教学中渗透学生学习自我效能感培养的有效性。采用实验组、对照组前后测实验设计,以《自我效能感自陈式问卷》、《数学自我效能问卷》和《中学生心理健康问卷》为测量工具。结果表明:实验后,实验班自我效能感高于前测,高于对照班,差异显著;在学习压力、适应不良、情绪不平衡、心理不平衡及总分上,实验班学生低于对照班,差异显著;数学学业成绩实验班高于对照班,高于实验前,但无显著差异。说明该模式能够显著提高学生学习自我效能感和心理健康水平,促进其学业进步。     

Intelligence and individual differences in becoming neurally efficient

Physiological approaches to human psychometric intelligence have shown a higher neural efficiency (i.e. less cortical activation) during cognitive performance in brighter subjects. The main aim of this study was to explore the relationship between intelligence and cortical activation patterns in the framework of the learning test concept. In 27 participants we assessed the topography and extent of cortical activation by means of event-related desynchronization (ERD) during reasoning tests in a pre-test--training--post-test design and related it to psychometric intelligence (measured by the German Leistungs-Prüf-System, LPS). Significant associations between intelligence and cortical activation patterns were exclusively found at anterior (frontal) recording sites, which corroborates the central role of the frontal lobe for higher-order cognitive functions. The hypothesized negative intelligence-activation correlation was observed only after the training, i.e. in the post-test, but not in the pre-test. More important, the decrease in cortical investment from pre-test to post-test correlated negatively with intelligence, indicating that the higher the subjects' general mental ability the larger the decrease in the amount of cortical activation. These findings suggest intelligence-related individual differences in becoming neurally efficient.     

正念训练对康复期精神分裂症患者抑制控制的影响

探究正念训练对康复期精神分裂症患者正念觉知水平和抑制控制能力的影响。选取54名康复期精神分裂症住院患者随机分为正念组和对照组。对照组予以常规康复治疗,正念组在常规康复治疗基础上进行8周正念训练。结果发现：（1）在正念觉知方面,正念组后测得分显著高于前测,对照组前后测得分无显著差异;正念组后测得分显著高于对照组。（2）在抑制控制方面,正念组后测Stroop任务反应时显著低于前测,对照组前后测反应时无显著差异;正念组后测反应时显著低于对照组。结果表明,正念训练有助于提高康复期精神分裂症患者正念觉知水平和改善其抑制控制能力。     

Effectiveness of Rational Emotive Behavior Therapy on Psychological Well-Being of People with Late Blindness

The aim of the present study was to investigate the effectiveness of Rational Emotive Behavior Therapy (REBT) on improving the psychological wellbeing of people with late blindness. The design of this study was organized into pre-test, post-test and follow-up with two groups; experimental and control. After choosing qualified individuals, they were replaced into experimental and control groups at random (30 clients in each group). Pre-test was conducted for both groups. Thereafter, the experimental group received REBT training. Both groups, took part in post-test at the end of the training course. After 1 month, follow-up test was taken. Tools such as demographic particulars, IBT questionnaire, depression, anxiety and stress 21 inventory, Eysenck questionnaire were used for gathering information of the study. All the analytical process has been done by SPSS18 program. The scores of both groups (experimental and control groups) were not different significantly in pre-test. The scores gained by the experimental group in post-test showed a significant increase compared with the pre-test result. It proved the effectiveness of REBT with experimental group. And also, in comparing post-test with follow-up experimental group no significant difference were yield. So, as a result the effectiveness interference through group training of REBT with experimental group can be evaluated as constant. In control group there were no significant difference among the scores gained in pre-test, post-test and follow up. The findings of this research indicate the effectiveness of group coaching of REBT with enhancing the indices of psychological well being of people with late blindness.     

Combining Item Response Theory and Diagnostic Classification Models: A Psychometric Model for Scaling Ability and Diagnosing Misconceptions

Traditional testing procedures typically utilize unidimensional item response theory (IRT) models to provide a single, continuous estimate of a student’s overall ability. Advances in psychometrics have focused on measuring multiple dimensions of ability to provide more detailed feedback for students, teachers, and other stakeholders. Diagnostic classification models (DCMs) provide multidimensional feedback by using categorical latent variables that represent distinct skills underlying a test that students may or may not have mastered. The Scaling Individuals and Classifying Misconceptions (SICM) model is presented as a combination of a unidimensional IRT model and a DCM where the categorical latent variables represent misconceptions instead of skills. In addition to an estimate of ability along a latent continuum, the SICM model provides multidimensional, diagnostic feedback in the form of statistical estimates of probabilities that students have certain misconceptions. Through an empirical data analysis, we show how this additional feedback can be used by stakeholders to tailor instruction for students’ needs. We also provide results from a simulation study that demonstrate that the SICM MCMC estimation algorithm yields reasonably accurate estimates under large-scale testing conditions.     

让自适应测验更知人善选——基于推荐系统的选题策略

基于推荐系统中协同过滤推荐的思想, 提出两种可以利用已有答题者数据的CAT选题策略：直接基于答题者推荐(DEBR)和间接基于答题者推荐(IEBR)。通过两个模拟研究, 在不同题库和不同长度的测验中, 比较了两种推荐选题策略与两种传统选题策略(FMI和BAS)在测量精度和对题目曝光率控制上的表现, 以及影响推荐选题策略表现的因素。结果发现：两种推荐选题策略对题目曝光率的控制优于两种传统选题策略, 测量精度不亚于BAS方法, 其中DEBR侧重选题精度, IEBR对题目曝光率控制最好。已有答题者数据的特点和质量是影响推荐选题策略表现的主要因素。     

Item response theory and the measurement of clinical change

An instrument's sensitivity to detect individual-level change is an important consideration for both psychometric and clinical researchers. In this article, we develop a cognitive problems measure and evaluate its sensitivity to detect change from an item response theory (IRT) perspective. After illustrating assumption checking and model fit assessment, we detail 4 features of IRT modeling: (a) the scale information curve and its relation to the bandwidth of measurement precision, (b) the scale response curve and how it is used to link the latent trait metric with the raw score metric, (c) content-based versus norm-based score referencing, and (d) the level of measurement of the latent trait scale. We conclude that IRT offers an informative, alternative framework for understanding an instrument's psychometric properties and recommend that IRT analyses be considered prior to investigations of change, growth, or the effectiveness of clinical interventions.     

Clinical utility of new cut-off scores for the world health organization ADHD self-report scale among adolescents in psychiatric outpatient care

Previous analyses of the proposed cut-off score for the 18-item World Health Organization ADHD Self-Report Scale (ASRS-18) among psychiatrically referred adolescents have shown limited clinical utility. This prospective study examined the diagnostic accuracy and clinical utility of new cut-off scores of the ASRS-18 in a consecutive sample of 111 Swedish adolescent psychiatric outpatients. Using the Kiddie Schedule of Affective Disorders and Schizophrenia (K-SADS) as the reference standard and based on predefined sensitivity and specificity criteria, multiple new general and sex-specific cut-off scores were evaluated. Results showed that clinical utility was greater for sex-specific cut-off scores than for general cut-off scores. The greatest change in pre-test to post-test probability of ADHD diagnosis was observed with a balanced, high-specificity diagnostic cut-off score for girls, where the probability of ADHD increased from 40% pre-test to 82% post-test. The proposed new cut-off levels for the ASRS-18 are useful for the detection and identification of ADHD among adolescents in general psychiatric outpatient settings.     

Non‐ignorable missingness item response theory models for choice effects in examinee‐selected items

Examinee‐selected item (ESI) design, in which examinees are required to respond to a fixed number of items in a given set, always yields incomplete data (i.e., when only the selected items are answered, data are missing for the others) that are likely non‐ignorable in likelihood inference. Standard item response theory (IRT) models become infeasible when ESI data are missing not at random (MNAR). To solve this problem, the authors propose a two‐dimensional IRT model that posits one unidimensional IRT model for observed data and another for nominal selection patterns. The two latent variables are assumed to follow a bivariate normal distribution. In this study, the mirt freeware package was adopted to estimate parameters. The authors conduct an experiment to demonstrate that ESI data are often non‐ignorable and to determine how to apply the new model to the data collected. Two follow‐up simulation studies are conducted to assess the parameter recovery of the new model and the consequences for parameter estimation of ignoring MNAR data. The results of the two simulation studies indicate good parameter recovery of the new model and poor parameter recovery when non‐ignorable missing data were mistakenly treated as ignorable.