Adding Value to Second-Language Listening and Reading Subscores: Using a Score Augmentation Approach

This study examined whether reporting subscores for groups of items within a test section assessing a second-language modality (specifically reading or listening comprehension) added value from a measurement perspective to the information already provided by the section scores. We analyzed the responses of 116,489 test takers to reading and listening items from operational administrations of two large-scale international tests of English as a foreign language. To “strengthen” the reliability of the subscores, and thus improve their added value, we applied a score augmentation method (Haberman, 2008). In doing so, our aim was to examine whether reporting augmented subscores for specific groups of reading and listening items could improve the added value of these subscores and consequently justify providing more fine-grained information about test taker performance. Our analysis indicated that in general, there was lack of support for reporting subscores from a psychometric perspective, and that score augmentation marginally improved the added value of the subscores. We discuss several implications of our findings for test developers wishing to report more fine-grained information about test performance. We conclude by arguing that research on how to best report such refined feedback should remain the focus of future efforts related to second-language proficiency tests.     

A Propensity Score Analysis of Homework Adherence-Outcome Relations in Cognitive Behavioral Therapy for Depression

Little is known about whether or not a consistently high level of homework adherence over the course of therapy benefits patients. This question was examined in two samples of patients who were receiving individual Cognitive Behavioral Therapy (CBT) for depression (Ns = 128 [Sequenced Treatment Alternatives to Relieve Depression: STAR-D] and 183 [Continuation Phase Cognitive Therapy Relapse Prevention: C-CT-RP]). Logistic and linear regression and propensity score models were used to identify whether or not clinician assessments of homework adherence differentiated symptom reduction and remission, as assessed by the Hamilton Depression Rating Scale-17 (HDRS-17), the Quick Inventory of Depressive Symptomatology–Self-Reported Scale (QIDS-SR), and the QIDS–Clinician Scale (QIDS-C). CBT-related response and remission were equally likely between both high and low homework adherers in both studies and in all models. But in propensity adjusted models that adjusted for session attendance, for both the STAR-D and C-CT-RP samples, greater homework adherence was significantly associated with greater response and remission from depression in the first and last 8 sessions of CBT. Our results suggest that homework adherence can account for response and remission early and late in treatment, with adequate session attendence.     

Score blending: How scale response grouping biases perceived standing

Numerical values—from test scores to credit scores—inform us of our relative standing and can shape our decisions. The values are usually presented in a continuous format (which places scores on a single line) or a grouped format (which separates scores into several score groups). We investigate whether and how the presentation format affects one's perception of their standing in a distribution and subsequent decision making. We show that when values are grouped, individuals merge the values within each group, a phenomenon we term score blending, which leads to an underweighting of intragroup score shifts and an overweighting of intergroup score shifts. Across six studies (N = 2,108) and a variety of paradigms (e.g., test taking and financial decision making), we observe robust effects of score blending, even when score groups are explicitly described as meaningless and generated randomly. The effect persists when the score is merely recalled without the scale present and can bias key decision parameters with behavioral consequences.     

用信息函数法对标准参照测验作质量分析

根据项目反应理论的信息函数法,以自主开发的“现代教育和心理测量通用分析系统”为工具,对两个大规模实施的标准参照测验进行了测验质量分析。结果说明,信息函数分析法比经典真分数理论分析法,可以获得更丰富、深刻、有针对性的资料,能对项目筛选修订、测验编制开发给出有力的指导,值得大力引进、推广。     

追踪研究在组织行为学中的应用

追踪研究是一种通过引入时滞, 以探索变量动态变化趋势和变量之间相互关系的调查研究方法。近年来, 在组织行为学中, 追踪研究越来越受到学者们的关注, 追踪研究成果也在逐渐增加, 但是大多数研究者尤其是国内研究者, 对追踪研究尚缺乏全面正确的认识。追踪研究设计可以依照重复测量之间的时间间隔长短进行分类, 不同的类型对应不同的样本量、调查方式和统计分析方法, 未来可以拓展追踪研究在团队和领导力、组织文化和变革以及员工激励等方面的应用, 或开展跨文化的追踪研究。     

The Spanish validation of the Short Health Anxiety Inventory: Psychometric properties and clinical utility

Background/ObjectiveThe Short Health Anxiety Inventory (SHAI) is a widely used self-report instrument to evaluate health anxiety. To assess the SHAI's factor structure, psychometric properties, and accuracy in differentiating Spanish non-clinical individuals from patients with severe health anxiety or hypochondriasis.MethodA total of 342 community participants (61.6% women; M_age = 34.60, SD = 14.91) and 31 hypochondriacal patients (51.6% women; M_age = 32.74, SD = 9.69) completed the SHAI and other self-reports assessing symptoms of hypochondriasis, depression, anxiety sensitivity, worry, and obsessive-compulsive.ResultsThe original two-factor structure was selected as the best structure, based on its parsimony and empirical support (Factor 1: Illness likelihood; Factor 2: Negative consequences of illness). Moreover, the Spanish version of the SHAI demonstrated good construct and concurrent and discriminant validity, and internal consistency. A cutoff of 40.5 (total score) accurately distinguished non-clinical individuals from patients with severe health anxiety or hypochondriasis.ConclusionsThe SHAI is an adequate screening instrument to measure health anxiety in Spanish-speaking community adults.     

Replies to comments on "Which method delivers greater signal-to-noise ratio: Structural equation modelling or regression analysis with weighted composites?" by Yuan and Fang (2023)

Observational data typically contain measurement errors. Covariance-based structural equation modelling (CB-SEM) is capable of modelling measurement errors and yields consistent parameter estimates. In contrast, methods of regression analysis using weighted composites as well as a partial least squares approach to SEM facilitate the prediction and diagnosis of individuals/participants. But regression analysis with weighted composites has been known to yield attenuated regression coefficients when predictors contain errors. Contrary to the common belief that CB-SEM is the preferred method for the analysis of observational data, this article shows that regression analysis via weighted composites yields parameter estimates with much smaller standard errors, and thus corresponds to greater values of the signal-to-noise ratio (SNR). In particular, the SNR for the regression coefficient via the least squares (LS) method with equally weighted composites is mathematically greater than that by CB-SEM if the items for each factor are parallel, even when the SEM model is correctly specified and estimated by an efficient method. Analytical, numerical and empirical results also show that LS regression using weighted composites performs as well as or better than the normal maximum likelihood method for CB-SEM under many conditions even when the population distribution is multivariate normal. Results also show that the LS regression coefficients become more efficient when considering the sampling errors in the weights of composites than those that are conditional on weights.     

Measuring inaccuracy of uncertain doxastic states in many-valued logical systems

I will propose an alternative philosophical approach to the representation of uncertain doxastic states. I will argue that the current account of measuring inaccuracy of uncertain doxastic states is inadequate for Belnap's four-valued logic. Specifically, a situation can be found in which either an inaccuracy measure returns a completely wrong result or an agent's inaccuracy score is inadequate relative to the mistake in her doxastic attitude. This will motivate an alternative representation of uncertain doxastic states based on ordered pairs. I will describe a possible inaccuracy measure that is suitable for ordered pairs, and I will show that it has all the qualities that are required for an inaccuracy measure to be legitimate. Finally, I will introduce conditions of rationality for uncertain doxastic states represented by ordered pairs.