Rater Model Using Signal Detection Theory for Latent Differential Rater Functioning

Differential rater functioning (DRF) occurs when raters show evidence of exercising differential severity or leniency when scoring examinees within different subgroups. Previous studies of DRF have examined rater bias using manifest variables (e.g., use of covariates) to determine the subgroups. These manifest variables include gender and the ethnicity of the examinee. For example, a rater may score males more severely. Ideally, each rater’s severity should be invariant across subgroups. This study examines DRF in the context of latent subgroups that classify possible sources of DRF based on raters’ scoring behavior rather than manifest factors. An extension of the latent class signal detection theory (LC-SDT) model for identifying DRF is proposed and examined using real-world data and simulations. Results from real-world data show that the signal detection approach leads to an effective method to identify latent DRF. Simulations with varying sample sizes and conditions of rater precision were shown to recover parameters at an adequate level, supporting its use to identify latent DRF in large-scale data. These findings suggest that the DRF extension of the LC-SDT can be a useful model to examine characteristics of raters and add information that can aid rater training.     

Fast-dm: A free program for efficient diffusion model analysis

In the present article, a flexible and fast computer program, called fast-dm, for diffusion model data analysis is introduced. Fast-dm is free software that can be downloaded from the authors' websites. The program allows estimating all parameters of Ratcliff's (1978) diffusion model from the empirical response time distributions of any binary classification task. Fast-dm is easy to use: it reads input data from simple text files, while program settings are specified by commands in a control file. With fast-dm, complex models can be fitted, where some parameters may vary between experimental conditions, while other parameters are constrained to be equal across conditions. Detailed directions for use of fast-dm are presented, as well as results from three short simulation studies exemplifying the utility of fast-dm.     

Detecting Curvilinear Relationships: A Comparison of Scoring Approaches Based on Different Item Response Models

There is a growing use of noncognitive assessments around the world, and recent research has posited an ideal point response process underlying such measures. A critical issue is whether the typical use of dominance approaches (e.g., average scores, factor analysis, and the Samejima's graded response model) in scoring such measures is adequate. This study examined the performance of an ideal point scoring approach (e.g., the generalized graded unfolding model) as compared to the typical dominance scoring approaches in detecting curvilinear relationships between scored trait and external variable. Simulation results showed that when data followed the ideal point model, the ideal point approach generally exhibited more power and provided more accurate estimates of curvilinear effects than the dominance approaches. No substantial difference was found between ideal point and dominance scoring approaches in terms of Type I error rate and bias across different sample sizes and scale lengths, although skewness in the distribution of trait and external variable can potentially reduce statistical power. For dominance data, the ideal point scoring approach exhibited convergence problems in most conditions and failed to perform as well as the dominance scoring approaches. Practical implications for scoring responses to Likert-type surveys to examine curvilinear effects are discussed.     

Automated respiratory sinus arrhythmia measurement: Demonstration using executive function assessment

Respiratory sinus arrhythmia (RSA) is a quantitative metric that reflects autonomic nervous system regulation and provides a physiological marker of attentional engagement that supports cognitive and affective regulatory processes. RSA can be added to executive function (EF) assessments with minimal participant burden because of the commercial availability of lightweight, wearable electrocardiogram (ECG) sensors. However, the inclusion of RSA data in large data collection efforts has been hindered by the time-intensive processing of RSA. In this study we evaluated the performance of an automated RSA-scoring method in the context of an EF study in preschool-aged children. The absolute differences in RSA across both scoring methods were small (mean RSA differences = –0.02–0.10), with little to no evidence of bias for the automated relative to the hand-scoring approach. Moreover, the relative rank-ordering of RSA across both scoring methods was strong (rs = .96–.99). Reliable changes in RSA from baseline to the EF task were highly similar across both scoring methods (96%–100% absolute agreement; Kappa = .83–1.0). On the basis of these findings, the automated RSA algorithm appears to be a suitable substitute for hand-scoring in the context of EF assessment.     

认知重评和表达抑制情绪调节策略的脑网络分析：来自EEG和ERP的证据

本文旨在对认知重评和表达抑制两种常用情绪调节策略的自发脑网络特征及认知神经活动进行深入探讨。研究采集36名在校大学生的静息态和任务态脑电数据, 经过源定位和图论分析发现节点效率与两种情绪调节显著相关的脑区, 以及脑区之间的功能连接。研究结果表明, 在使用认知重评进行情绪调节时会激活前额叶皮质、前扣带回、顶叶、海马旁回和枕叶等多个脑区, 在使用表达抑制进行情绪调节时会激活前额叶皮质、顶叶、海马旁回、枕叶、颞叶和脑岛等多个脑区。因此, 这些脑区的节点效率或功能连接强度可能成为评估个体使用认知重评和表达抑制调节情绪效果的指标。     

Equivalence of Narcissistic Personality Inventory constructs and correlates across scoring approaches and response formats

The prevalent scoring practice for the Narcissistic Personality Inventory (NPI) ignores the forced-choice nature of the items. The aim of this study was to investigate whether findings based on NPI scores reported in previous research can be confirmed when the forced-choice nature of the NPI’s original response format is appropriately modeled, and when NPI items are presented in different response formats (true/false or rating scale). The relationships between NPI facets and various criteria were robust across scoring approaches (mean score vs. model-based), but were only partly robust across response formats. In addition, the scoring approaches and response formats achieved equivalent measurements of the vanity facet and in part of the leadership facet, but differed with respect to the entitlement facet.     

IVR Test & Survey: A computer program to collect data via computerized telephonic applications

Several studies have demonstrated the advantages of using interactive voice response (IVR) technology to collect self-report data from research participants and recipients of psychological/medical services. IVR allows participants to phone a computer and respond to recorded questions by pressing the appropriate touch-tone keys on their telephone. Because this technology offers substantial benefits in terms of cost and efficiency, it is surprising that it has not been more widely utilized by researchers and practitioners. Along with the automation of the administration and scoring of tests or surveys, IVR provides for questioning to be adapted to the participants’ responses. One possible explanation for the failure to exploit this technology is the absence of easy-to-use software that does not require programming knowledge. This article describes IVR Test & Survey, a program to facilitate the administration, scoring, and analysis of information collected with the use of IVR technology.     

等级反应多水平侧面模型及其在主观题评分中的应用

探讨了康春花,孙小坚和曾平飞(2016)提出的等级反应多水平侧面模型(GR-MLFM)在包含被试及评分者层面预测变量(完整模型)下的返真性和适用性。结果表明:(1)GR-MLFM完整模型具有逻辑上和数理上的合理性,可用于主观题的评分情境,能较好地检测出评分者效应、影响因素及其影响程度;(2)在数学问题解决的评分实践中,评分员存在两种类型的评分倾向(宽松和严格效应),但绝大多数评分员的宽严度不明显;评分者的责任心可正向预测其严格程度,自信心可正向预测其宽松程度,而情绪稳定性和评分经验的预测作用不显著。     

Stress applied during primary immunization affects the secondary humoral immune response in the rat: involvement of opioid peptides

The effect of unpredictable, inescapable and uncontrollable electric tail shocks (ES) on the humoral immune response to bovine serum albumin (BSA) was investigated in the rat. Contributions of the procedures that accompany shock delivery, such as witnessing the ES procedure (stress witnessing, SW) and exposure to the apparatus for shock delivery (apparatus control, AC) to the changes in specific immunity induced by ES were also tested. All procedures were applied during primary and/or secondary immunization. It was demonstrated that exposure to ES during primary immunization with BSA significantly suppressed specific anti-BSA antibody production after secondary and tertiary immunization with the same antigen. Exposure to the SW procedure during primary immunization with BSA enhanced the specific antibody level after secondary immunization, while exposure to the apparatus alone did not influence the development of either the primary or secondary humoral immune response to BSA. Both ES-induced suppression and SW-induced potentiation of the humoral immune response were partially inhibited by prior treatment with the opioid receptor antagonist naloxone. Additionally, treatments with the opioid peptides methionine- and leucine-enkephalin decreased anti-BSA antibody level, mimicking to some extent the effects of ES. It is suggested that ES and endogenous opioid peptides had long-term effects on humoral immunity through mechanisms involving immunologic memory.     

The treatment of muscle tics with dissimilar competing response practice

Prior research has shown that muscle tics can be suppressed by the performance of a competing response contingent on the occurrence of the muscle tics. In an effort to determine whether the topography of the competing response was important to the muscle tic suppressing effects of contingent competing response practice, we evaluated the effects of a competing response that was topographically dissimilar to the muscle tic. Three subjects engaged in dissimilar competing responses contingent on the occurrence of a muscle tic; 2 of these subjects subsequently engaged in similar competing response practice. The results showed a decrease in objective measures of muscle tic frequency with the introduction of dissimilar competing response practice for each subject; subsequent exposure to similar competing response practice for 2 subjects resulted in no additional decrement in the level of muscle tics. These results suggest that the topography of the competing response may not be crucial for the suppression of muscle tics. Discrepancies between the objective measures of muscle tics and self-recorded measures are noted and discussed.     

The role of the response-punishment contingency in the suppression of a positively-reinforced operant

Two experiments tested the prediction from current theories of punishment that a response-contingent procedure is superior to a comparable noncontingent procedure for the administration of punishment. Results demonstrated that the two procedures produced comparable contiguity effects and did not contain artifacts present in previous research. Three different measures of suppression corroborated the findings that more response suppression occurred in the contingent procedure. Results were shown to be stable across eight days of shock administration (Experiment 1), and three levels of shock intensity (Experiment 2). In both experiments, response suppression occurred during contingent shocks associated with shock at offset, as well as after conditioned stimulus (CS) offset. The contingent procedure was associated with more suppression during both of these periods and also longer runs of CSs with complete suppression. The relevance of the present findings to current theories of punishment was discussed.     

Information continuity across the response selection bottleneck: Early parallel Task 2 response activation contributes to overt Task 2 performance

Several studies of dual-task performance have demonstrated Task 2 to Task 1 response priming (backward compatibility effects), indicating some degree of parallel response computation for concurrent tasks and suggesting that the well-established response selection bottleneck (RSB) model may be incomplete. However, the RSB might be considered to remain informationally intact if this early parallel Task 2 response information does not persist across the attentional shift between tasks to contribute to overt Task 2 performance. We used an adapted psychological refractory period paradigm with an additional early transient Task 2 stimulus to examine whether response information generated for Task 2 in parallel with overt Task 1 response selection could persist across the bottleneck to influence eventual overt Task 2 performance. After controlling for potential indirect effects of Task 1 processing stage variability propagating onto Task 2 reaction time via locus of slack effects, we observed reliable and consistent effects of early Task 2 response information facilitating Task 2 reaction times. These effects were observed only when the responses to both tasks of the dual-task pair were compatible, under both univalent and bivalent response mappings across tasks. These findings may represent evidence of a variably sensitive response gating or suppression mechanism in dual-task performance and support the idea that backward response compatibility effects represent transient informational influences on central response codes, rather than later postbottleneck response execution processes.     

Distinct neural correlates for two types of inhibition in bilinguals: response inhibition versus interference suppression

To examine the effects of bilingualism on cognitive control, we studied monolingual and bilingual young adults performing a flanker task with functional MRI. The trial types of primary interest for this report were incongruent and no-go trials, representing interference suppression and response inhibition, respectively. Response times were similar between groups. Brain data were analyzed using partial least squares (PLS) to identify brain regions where activity covaried across conditions. Monolinguals and bilinguals activated different sets of brain regions for congruent and incongruent trials, but showed activation in the same regions for no-go trials. During the incongruent trials, monolinguals activated the left temporal pole and left superior parietal regions. In contrast, an extensive network including bilateral frontal, temporal and subcortical regions was active in bilinguals during the incongruent trials and in both groups for the no-go trials. Correlations between brain activity and reaction time difference relative to neutral trials revealed that monolinguals and bilinguals showed increased activation in different brain regions to achieve less interference from incongruent flankers. Results indicate that bilingualism selectively affects neural correlates for suppressing interference, but not response inhibition. Moreover, the neural correlates associated with more efficient suppression of interference were different in bilinguals than in monolinguals, suggesting a bilingual-specific network for cognitive control.     

2PLM下缺失数据处理方法及其比较

项目反应理论(IRT)是用于客观测量的现代教育与心理测量理论之一,广泛用于缺失数据十分常见的大尺度测验分析。IRT中两参数逻辑斯蒂克模型(2PLM)下仅有完全随机缺失机制下缺失反应和缺失能力处理的EM算法。本研究推导2PLM下缺失反应忽略的EM 算法,并提出随机缺失机制下缺失反应和缺失能力处理的EM算法和考虑能力估计和作答反应不确定性的多重借补法。研究显示：在各种缺失机制、缺失比例和测验设计下,缺失反应忽略的EM算法和多重借补法表现理想。     

Maximum likelihood additivity analysis

A maximum likelihood estimation procedure was developed to fit unweighted and weighted additive models to conjoint data obtained by the categorical rating, the pair comparison or the directional ranking method. The scoring algorithm used to fit the models was found to be both reliable and efficient, and the program MAXADD is capable of handling up to 300 parameters to be estimated. Practical uses of the procedure are reported to demonstrate various advantages of the procedure as a statistical method.The research reported here was supported by Grant A6394 to the author from the Natural Sciences and Engineering Research Council of Canada. Portions of this research were presented at the Psychometric Society meeting in Iowa City, Iowa, in May, 1980.Thanks are due to Jim Ramsay, Justine Sergent and anonymous reviewers for their helpful comments.Two MAXADD programs which perform the computations discussed in this paper may be obtained from the author.     

GPCM模型项目参数估计程序的开发与研究

该文对GPCM的项目参数估计的方法进行较为深入的探讨,特别对容易混淆的参数初值计算以及项目参数分两步进行估计的原因进行了阐述。并且基于MMLE／EM算法,开发了相应的项目参数估计程序。通过大量的蒙特卡洛模拟,与国外专业软件PARSCALE比较,本程序的步骤参数估计值更好,区分度参数估计值相当,具有较好的应用价值。     

Emotions in cognitive conflicts are not aversive but are task specific

It has been suggested that cognitive conflicts require effortful processing and, therefore, are aversive (Botvinick, 2007). In the present study, we compared conflicts emerging from the inhibition of a predominant response tendency in a go/no-go task with those between incompatible response activations in a Simon task in a within-subjects design, using the same type of stimuli. Whereas no-go trials elicited reduced skin conductance and pupillometric responses, but prolonged corrugator muscle activity, as compared with go trials, incompatible and compatible Simon trials were indistinguishable with respect to these parameters. Furthermore, the conflictsensitive N2 components of the event-related brain potential were similar in amplitude, but showed significantly different scalp distributions, indicating dissociable neural generator systems. The present findings suggest the involvement of different emotional and cognitive processes in both types of cognitive conflicts—none being aversive, however. In addition, the N2 findings call into question claims of common monitoring systems for all kinds of cognitive conflicts.     

Actigraph data are reliable, with functional reliability increasing with aggregation

Motion sensor devices such as actigraphs are increasingly used in studies that seek to obtain an objective assessment of activity level. They have many advantages, and are useful additions to research in fields such as sleep assessment, drug efficacy, behavior genetics, and obesity. However, questions still remain over the reliability of data collected using actigraphic assessment. We aimed to apply generalizability theory to actigraph data collected on a large, general-population sample in middle childhood, during 8 cognitive tasks across two body loci, and to examine reliability coefficients on actigraph data aggregated across different numbers of tasks and different numbers of attachment loci. Our analyses show that aggregation greatly increases actigraph data reliability, with reliability coefficients on data collected at one body locus during 1 task (.29) being much lower than that aggregated across data collected on two body loci and during 8 tasks (.66). Further increases in reliability coefficients by aggregating across four loci and 12 tasks were estimated to be modest in prospective analyses, indicating an optimum trade-off between data collection and reliability estimates. We also examined possible instrumental effects on actigraph data and found these to be nonsignificant, further supporting the reliability and validity of actigraph data as a method of activity level assessment.     

CowLog: Open-source software for coding behaviors from digital video

We have developed CowLog, which is open-source software for recording behaviors from digital video and is easy to use and modify. CowLog tracks the time code from digital video files. The program is suitable for coding any digital video, but the authors have used it in animal research. The program has two main windows: a coding window, which is a graphical user interface used for choosing video files and defining output files that also has buttons for scoring behaviors, and a video window, which displays the video used for coding. The windows can be used in separate displays. The user types the key codes for the predefined behavioral categories, and CowLog transcribes their timing from the video time code to a data file. CowLog comes with an additional feature, an R package called Animal, for elementary analyses of the data files. With the analysis package, the user can calculate the frequencies, bout durations, and total durations of the coded behaviors and produce summary plots from the data.     

Analysis of Multiple Partially Ordered Responses to Belief Items with Don’t Know Option

Understanding beliefs, values, and preferences of patients is a tenet of contemporary health sciences. This application was motivated by the analysis of multiple partially ordered set (poset) responses from an inventory on layman beliefs about diabetes. The partially ordered set arises because of two features in the data—first, the response options contain a Don’t Know (DK) option, and second, there were two consecutive occasions of measurement. As predicted by the common sense model of illness, beliefs about diabetes were not necessarily stable across the two measurement occasions. Instead of analyzing the two occasions separately, we studied the joint responses across the occasions as a poset response. Few analytic methods exist for data structures other than ordered or nominal categories. Poset responses are routinely collapsed and then analyzed as either rank ordered or nominal data, leading to the loss of nuanced information that might be present within poset categories. In this paper we developed a general class of item response models for analyzing the poset data collected from the Common Sense Model of Diabetes Inventory. The inferential object of interest is the latent trait that indicates congruence of belief with the biomedical model. To apply an item response model to the poset diabetes inventory, we proved that a simple coding algorithm circumvents the requirement of writing new codes such that standard IRT software could be directly used for the purpose of item estimation and individual scoring. Simulation experiments were used to examine parameter recovery for the proposed poset model.