评分者内部一致性的研究和应用

在行为科学的研究和实践中,研究者常常需要将个体层次的评价,整合到群体层次的评价,对于这种自下而上整合模式的一致性问题,国内一些学者常常错误地使用评分者内部信度作为评分者内部一致性的指标。评分者内部一致性和评分者内部信度不仅在理论基础上存在差异,而且在实践中也存在前者很高(或很低),而后者却很低(或很高)的不一致情况。文章阐述了学术界对评分一致性这一问题的提出、争论和取得一致观点的发展脉络,以期学者们对这一问题能够有深入的思索,避免在今后的研究中出现类似的错误。     

Agreement of adolescent ratings with mother ratings and teacher ratings of ADHD symptom groups: A correlated trait-correlated method minus one analysis

This study examined the level of agreement of adolescent ratings with mother ratings, and adolescent ratings with teacher ratings of the inattention (IA) and hyperactivity/impulsivity (HI) symptom groups of ADHD. A total of 214 adolescents provided self-ratings of IA and HI, and their IA and HI were also rated by their mothers and teachers. The correlated trait-correlated method minus one model was applied, with adolescent ratings as the reference method, and the other two ratings as the non-reference methods. The findings indicated no additional variance in adolescent ratings for IA and HI that could not be accounted by mother ratings of IA and HI, respectively. In contrast, there was additional variance in adolescent ratings for IA and HI that could was not accounted by teacher ratings of IA and HI, respectively. The findings suggest that when diagnosing ADHD in adolescents, their reports of their own ADHD behaviors are not needed when mother reports of such behaviors are used.     

Agreement between Parents and Children on Ratings of Post-Concussive Symptoms Following Mild Traumatic Brain Injury

The level of parent-child agreement on post-concussive symptoms (PCS) was examined in children following mild traumatic brain injuries (TBI). As part of a larger longitudinal study, 186 children with mild TBI and 99 with orthopedic injuries (OI), from 8 to 15 years of age, were recruited prospectively. Parents and children completed the PCS Interview (PCS-I) and the Health and Behavior Inventory (HBI) at 2 weeks, 1 month, 3 months, and 12 months postinjury. Item-level correlations between child and parent ratings on both measures of PCS were significant but modest in both groups. Parent-child correlations for composite scales on the HBI and the total score on the PCS-I were significant in both groups, but somewhat higher in the OI group than in the mild TBI group. Mean symptom ratings tended to be significantly higher for children as compared to parents, especially for somatic symptoms. Parents and children display modest agreement when reporting PCS; their ratings correlate significantly, but children report higher mean levels of symptoms than parents.     

Processing biological gender and number information during Chinese pronoun resolution: ERP evidence for functional differentiation

There have been a number of behavioral and neural studies on the processing of syntactic gender and number agreement information, marked by different morpho-syntactic features during sentence comprehension. By using the event-related potential (ERP) technique, the present study investigated whether the processing of semantic gender information and the processing of notional number information can be differentiated and to what extent they might interact during Chinese pronoun resolution. The pronoun (with singular form in Experiment 1 and with plural form in Experiment 2) in a sentence matched its antecedent or mismatched it with respect to either biological gender or notional number or both. While the number mismatch elicited a P600 effect starting from 550 ms (for singular pronoun) or 400 ms (for plural pronoun) post-onset of the pronoun, the gender mismatch elicited an earlier (for singular) and larger (for both singular and plural) P600 effect. More importantly, the double mismatch produced a P600 effect identical to the effect elicited by the single gender mismatch. These results demonstrate that biological gender information and notional number information are processed differentially and have different processing priorities during Chinese pronoun resolution.     

A Comparison of Imputation Strategies for Ordinal Missing Data on Likert Scale Variables

This article compares a variety of imputation strategies for ordinal missing data on Likert scale variables (number of categories = 2, 3, 5, or 7) in recovering reliability coefficients, mean scale scores, and regression coefficients of predicting one scale score from another. The examined strategies include imputing using normal data models with naïve rounding/without rounding, using latent variable models, and using categorical data models such as discriminant analysis and binary logistic regression (for dichotomous data only), multinomial and proportional odds logistic regression (for polytomous data only). The result suggests that both the normal model approach without rounding and the latent variable model approach perform well for either dichotomous or polytomous data regardless of sample size, missing data proportion, and asymmetry of item distributions. The discriminant analysis approach also performs well for dichotomous data. Naïvely rounding normal imputations or using logistic regression models to impute ordinal data are not recommended as they can potentially lead to substantial bias in all or some of the parameters.     

题目因素分析：基于SEM和基于IRT的两类方法

心理和教育测量一般只能达到顺序量表的水平,其测量数据与被测因子间并非简单线性关系。题目因素分析是用来描述测量题目与因子间非线性关系的统计模型。题目因素分析主要有基于结构方程模型和基于项目反应理论两类方法,两类方法之间存在紧密的联系,甚至可以看作是同一模型的两种表现形式。本文详细阐述了该关系,同时对两类方法在参数估计、模型拟合指标、测量一致性检验和支撑软件等方面的特点进行了分析和比较,以便研究者选择最为适合其研究的方法。     

Information quality in personality judgment: The value of personal disclosure

We examined the relative quality of different types of information and its effects on accuracy in personality judgment. Three hundred and forty-four unacquainted undergraduates participated in a small group round-robin rating scheme. In one condition, participants were asked to disclose three things that were very important to them in life. In a second condition, participants were asked to disclose three distinctive personal facts: things that differentiated them from others. We found that (a) people believed values information was more personality-relevant than facts information and (b) although there was no clear advantage for one condition over the other in terms of generalized accuracy, there were some differences across conditions for specific traits. Implications for the study of information quality in personality judgment are discussed.     

Acquaintanceship and self/observer agreement in personality judgment

Self/observer agreement on HEXACO-PI-R scale scores was examined as a function of observers’ subjective ratings of acquaintanceship. For each participant (N = 2199), personality self-reports were obtained along with observer reports from a friend. Each factor-level scale displayed a different pattern of upward accuracy (agreement) trends in personality judgment. Self/observer agreement for Extraversion, Emotionality, and Openness was noticeably stronger at lower acquaintanceship than that for Conscientiousness, Agreeableness, and Honesty-Humility. Conscientiousness showed a steep upward accuracy trend across acquaintanceship levels, reaching a level of accuracy comparable to that of Extraversion and Emotionality. Self/observer agreement for Honesty-Humility and Agreeableness showed slower upward trends than that of Conscientiousness. In several cases, facet-level traits within the same broad factor differed in their accuracy trends.     

A tutorial on how to do a Mokken scale analysis on your test and questionnaire data

Over the past decade, Mokken scale analysis (MSA) has rapidly grown in popularity among researchers from many different research areas. This tutorial provides researchers with a set of techniques and a procedure for their application, such that the construction of scales that have superior measurement properties is further optimized, taking full advantage of the properties of MSA. First, we define the conceptual context of MSA, discuss the two item response theory (IRT) models that constitute the basis of MSA, and discuss how these models differ from other IRT models. Second, we discuss dos and don'ts for MSA; the don'ts include misunderstandings we have frequently encountered with researchers in our three decades of experience with real‐data MSA. Third, we discuss a methodology for MSA on real data that consist of a sample of persons who have provided scores on a set of items that, depending on the composition of the item set, constitute the basis for one or more scales, and we use the methodology to analyse an example real‐data set.