Simple Structure Detection Through Bayesian Exploratory Multidimensional IRT Models

In modern validity theory, a major concern is the construct validity of a test, which is commonly assessed through confirmatory or exploratory factor analysis. In the framework of Bayesian exploratory Multidimensional Item Response Theory (MIRT) models, we discuss two methods aimed at investigating the underlying structure of a test, in order to verify if the latent model adheres to a chosen simple factorial structure. This purpose is achieved without imposing hard constraints on the discrimination parameter matrix to address the rotational indeterminacy. The first approach prescribes a 2-step procedure. The parameter estimates are obtained through an unconstrained MCMC sampler. The simple structure is, then, inspected with a post-processing step based on the Consensus Simple Target Rotation technique. In the second approach, both rotational invariance and simple structure retrieval are addressed within the MCMC sampling scheme, by introducing a sparsity-inducing prior on the discrimination parameters. Through simulation as well as real-world studies, we demonstrate that the proposed methods are able to correctly infer the underlying sparse structure and to retrieve interpretable solutions.     

Construct validity for the verbal associated pairs screen

The present study introduces the Verbal Associated Pairs Screen (VAPS) as a new measure for assessing performance validity in pediatric populations. This study presents initial data on psychometric properties and establishes construct validity for the VAPS in a sample of 30 adolescent healthy controls and 206 youths with traumatic brain injury (TBI: moderate/severe, N = 30; mild, N = 176). The control group’s age (M = 14.93, SD = 1.8) was significantly higher than the moderate/severe TBI (M = 13.9, SD = 2.8), t(68.508) = ?3.038, p = .003, and mild TBI (mTBI) groups (M = 14, SD = 2.8), t(54.147) = 2.038, p = .046. The TBI groups were administered the VAPS in accord with other established performance validity tests (PVTs) and well-established memory tests as part of routine clinical evaluations. The healthy control group was administered the VAPS only. VAPS score distributions for the control group were negatively skewed and highly kurtotic. VAPS scores from the moderate/severe TBI and control groups were indistinguishable for Trial 2 (U = 274, p < .01) and the Delay (U = 396, p = .218). In the mTBI group, convergent and divergent validity was established with other well-validated PVTs and memory tests, respectively. ROC curve analyses identified optimal cutoff scores for the VAPS Total Score, with acceptable sensitivity (55%) and excellent specificity (100%), as well as strong detectability (AUC = .829, 95% CI: 0.731 – 0.928, p < .001). Clinical applications, limitations, and directions for future research with the VAPS are discussed.     

Concordance of teacher-rated and performance-based measures of executive functioning in preschoolers

The most common method used to evaluate child behavior and functioning is rating scales completed by parents and/or teachers. Given that executive functioning (EF) plays a fundamental role in the developing child’s cognitive, behavioral, and social-emotional development, it would be ideal if ratings of EF and performance-based EF measures assess the same construct. However, most studies report a small to negligible association between performance-based measures and ratings of EF. There are few studies investigating this association for preschoolers, and most only include parent ratings. Teachers may be more reliable reporters of EF behaviors due to the higher demand for EF skills in the preschool setting than at home and because teachers may have a better sense of what behaviors are normative. In this study, we reviewed the associations between three EF rating scales completed by teachers on 243 preschool children. Results showed small to moderate correlations with EF measures of inhibition and cognitive flexibility/switching for all three scales, with the strongest associations observed between Child Behavior Rating Scale (CBRS) Behavioral Regulation subscale and child EF measures. Exploratory multivariate path analyses showed that, after controlling for age, sex, and socioeconomic status (SES), Behavioral Regulation significantly predicted performance-based measures of EF and accounted for incrementally more variance in the models. We conclude that in ideal situations, it is best to measure EF using both rating scales and performance-based measures of EF. The CBRS seems to be a sensitive measure of EF in preschoolers and may be a helpful brief screening tool for use with teachers.     

儿童青少年正念量表(CAMM)在中国青少年群体中的信效度检验

目的:检验儿童青少年正念量表(CAMM)在中国青少年群体中应用的信效度,为青少年正念研究提供科学工具。方法:选取310名中学生(样本1)进行条目分析和探索性因素分析,再选取309名中学生(样本2)进行验证性因素分析、效标效度、增量效度和信度分析。结果:中文版CAMM抽取2个因素最为合适,并具有良好的结构效度、效标效度、增量效度及较高的信度。结论:中文版CAMM具有良好的心理测量学指标,可以在中国青少年群体中使用。     

The assessment of autistic traits with the Autism Spectrum Quotient: Contribution of the French version to its construct validity

IntroductionThe Autism Spectrum Questionnaire (AQ, Baron-Cohen et al., 2001) is a self-report assessment tool aiming at screening autistic traits in normal intelligence adults. While numerous versions in other languages than English now exist, few factorial evidence do sustain the valid use of this instrument as it was conceived, based upon five distinct dimensions (Social skills, Communication, Attention to detail, Attention switching, Imagination); no such study exists with a French version of the AQ. The aim of our study is therefore to present the French version of the scale and to explore its factorial validity with confirmatory factorial analyses and, possibly, its invariance across men and women.MethodSeveral confirmatory factorial analyses, with the robust WLSMV estimator for categorical response format, were run on the questionnaire data from 788 French-speaking students (17–25 years old) at university faculties or schools for higher education in Belgium. The original five-factor measurement model of the AQ was assessed as well as alternative models. An exploratory factorial analysis was also applied to get more insight as to possible sources of misfit.ResultsNo measurement model – neither the original five-factor one nor any of the six other models tested – did produce statistics or fit indices close to significant values: there was no fit to the data. The internal consistency of the subscales was weak; the exploratory factorial analysis further confirmed that as much as ten factors were needed to explain 45% of the data variance.ConclusionOur results, with a French version of the scale, add to many other ones which suggest that the AQ is a too heterogeneous questionnaire with somewhat ill-defined dimensions and non specific/ambiguous items. The questionnaire should probably be shortened and its content realigned to core features of the autism spectrum.     

Evaluating the Impact of Careless Responding on Aggregated-Scores: To Filter Unmotivated Examinees or Not?

When examinees' test-taking motivation is questionable, practitioners must determine whether careless responding is of practical concern and if so, decide on the best approach to filter such responses. As there has been insufficient research on these topics, the objectives of this study were to: a) evaluate the degree of underestimation in the true mean when careless responses are present, and b) compare the effectiveness of two filtering procedures in purifying biased aggregated-scores. Results demonstrated that: a) the true mean was underestimated by around 0.20 SDs if the total amount of careless responses exceeded 6.25%, 12.5%, and 12.5% for easy, moderately difficult, and difficult tests, respectively, and b) listwise deleting data from unmotivated examinees artificially inflated the true mean by as much as .42 SDs when ability was related to careless responding. Findings from this study have implications for when and how practitioners should handle careless responses for group-based low-stakes assessments.     

Estimating Incremental Validity Under Missing Data

A common form of missing data is caused by selection on an observed variable (e.g., Z). If the selection variable was measured and is available, the data are regarded as missing at random (MAR). Selection biases correlation, reliability, and effect size estimates when these estimates are computed on listwise deleted (LD) data sets. On the other hand, maximum likelihood (ML) estimates are generally unbiased and outperform LD in most situations, at least when the data are MAR. The exception is when we estimate the partial correlation. In this situation, LD estimates are unbiased when the cause of missingness is partialled out. In other words, there is no advantage of ML estimates over LD estimates in this situation. We demonstrate that under a MAR condition, even ML estimates may become biased, depending on how partial correlations are computed. Finally, we conclude with recommendations about how future researchers might estimate partial correlations even when the cause of missingness is unknown and, perhaps, unknowable.     

Introducing a forced-choice recognition task to the California Verbal Learning Test – Children’s Version

The importance of performance validity tests (PVTs) is increasingly recognized in pediatric neuropsychology. To date, research has focused on investigating whether PVTs designed for adults function similarly in children. The downward extension of adult cutoffs is counter-intuitive considering the robust effect of age-related changes in basic cognitive skills in children and adolescents. The purpose of this study was to examine the signal detection properties of a forced-choice recognition trial (FCR-C) for the California Verbal Learning Test – Children’s Version. A total of 72 children aged 6–15 years (M = 11.1 , SD = 2.6) completed the FCR-C as part of a larger neuropsychological assessment battery. Cross-validation analyses revealed that the FCR-C had good signal detection performance against reference PVTs. The first level of failure (≤14/15) produced the best combination of overall sensitivity (.31) and specificity (.87). A more conservative FCR-C cutoff (≤13) resulted in a predictable trade-off between sensitivity (.15) and specificity (.94), but also a net loss in discriminant power. Lowering the cutoff to ≤12 resulted in a slight improvement in specificity (.97) but further deterioration in sensitivity (.14). These preliminary findings suggest that the FCR-C has the potential to become the newest addition to a growing arsenal of pediatric PVTs.     

组织变革前非正式信息的作用机制

非正式信息是变革前信息沟通的重要组成部分, 对变革决策和执行有潜在的影响。围绕组织变革前非正式信息开展以下研究：(1)探究组织变革前非正式信息的内涵和结构; (2)基于心理因应和认知重构机制, 探索员工获取非正式信息之后, 其变革认知和情感的动态调整过程; (3)研究潜在的组织变革决策→非正式信息反馈→再决策过程, 即变革领导者通过获得员工变革认知和情感的反馈, 进行渐进决策优化的过程。研究结果能丰富变革沟通理论和指导我国变革实践。     

A PRAGMATIC CRITIQUE OF PLURALISM IN TEXT INTERPRETATION

Abstract: I take a pragmatic approach to what interpreters do when they interpret and argue that critical pluralists have focused almost exclusively on one aspect of interpretation: the fact that it is an event taking place in a historical and cultural milieu that influences the many ways interpreters approach a given text. However, there is also in interpretation a pragmatic aspect: the fact that it is an act performed by individuals who, through the utterance of their statements, implicitly make claims, for example, regarding the truth of their interpretation. In the first part I explain what critical pluralists overlook and the consequences of such oversight. In the second part I indicate how the validity of interpretation can be assessed.