A Discussion of Modern Versus Traditional Psychometrics As Applied to Personality Assessment Scales

Item response theory (IRT) methods are used by large testing firms, state agencies, and school districts to construct, analyze, and score most major aptitude, achievement, proficiency, entrance, and professional licensure exams. Personality assessment, in contrast, has not generally adopted these more powerful, modern psychometric techniques. We evaluate the possible role of IRT in the personality domain by highlighting key areas in which IRT and traditional methods differ. Although we conclude that IRT has a significant role to play in future personality measurement, there are many systemic and technical barriers to its routine application.     

Advances in Clinical Personality Measurement: An Item Response Theory Analysis of the MMPI-2 PSY-5 Scales

Item response theory (IRT) provides valuable methods for the analysis of the psychometric properties of a psychological measure. To date, however, these methods have not been used frequently by personality assessment researchers, in part because many researchers have not been introduced to the methods and in part because most of the development of IRT has taken place in applied education assessment settings, resulting in terminology that is ability focused rather than trait focused. The purpose of this article is twofold. First, an overview of IRT is presented, highlighting the concepts of the three-parameter IRT model, item and test information, and conditional standard error of measurement. Second, the psychometric properties of the (MMPI-2) PSY-5 scales are examined to demonstrate IRT's value.     

Capturing Abnormal Personality With Normal Personality Inventories: An Item Response Theory Approach

ABSTRACT Correlational and factor‐analytic methods indicate that abnormal and normal personality constructs may be tapping the same underlying latent trait. However, they do not systematically demonstrate that measures of abnormal personality capture more extreme ranges of the latent trait than measures of normal range personality. Item Response Theory (IRT) methods, in contrast, do provide this information. In the present study, we use IRT methods to evaluate the range of the latent trait assessed with a normal personality measure and a measure of psychopathy as one example of an abnormal personality construct. Contrary to the expectation that the measure of psychopathy would be more extreme than the measure of normal personality traits, the measures overlapped substantially in terms of the regions of the latent trait for which they provide information. Moreover, both types of inventories were limited in terms of measurement bandwidth, such that they did not provide information across the entire latent trait continuum. Implications and future directions are discussed.     

人格测验中作假的控制方法

被试很容易对人格测验作假,这严重影响了人格测验的有效性。目前测评专家已经提出了一些应对作假的方法,它们可被分为事前控制技术和事后识别技术两大类。前者包括迫选式量表,警告及假渠道技术等,后者包括作假识别量表,IRT及反应时识别技术等。目前,在人格测验中嵌套使用作假识别量表,以及在测验指导语中加入警告是比较有效的两种方法,迫选式量表的发展也值得期待。由于研究者对作假的内部发生机制了解较少,这制约了IRT与反应时识别技术的发展。     

CTT与IRT方法对人格测验结果处理的比较研究

为了说明使用经典测量理论(CTT)方法和项目反应理论(IRT)方法计算出的人格测验结果的差异,本研究使用IRT和CTT这两种方法分别计算出模拟人格测验和实际人格测验的测验结果,并对此进行比较。研究表明,两种不同的方法得到的测验结果之间平均有0．11个标准差以上的差异。进一步研究发现,在对测验结果进行分析时,IRT方法比CTT方法更为有效。     

Constructing personality scales under the assumptions of an ideal point response process: toward increasing the flexibility of personality measures

The main aim of this article is to explicate why a transition to ideal point methods of scale construction is needed to advance the field of personality assessment. The study empirically demonstrated the substantive benefits of ideal point methodology as compared with the dominance framework underlying traditional methods of scale construction. Specifically, using a large, heterogeneous pool of order items, the authors constructed scales using traditional classical test theory, dominance item response theory (IRT), and ideal point IRT methods. The merits of each method were examined in terms of item pool utilization, model-data fit, measurement precision, and construct and criterion-related validity. Results show that adoption of the ideal point approach provided a more flexible platform for creating future personality measures, and this transition did not adversely affect the validity of personality test scores.     

A Comparison of the Psychometric Properties of the Forced Choice and Likert Scale Versions of a Personality Instrument

The present research investigated if an item response theory (IRT)‐scored forced‐choice personality questionnaire has the same normative data structures as a similar version that uses a 5‐point Likert scale instead. The study was conducted using a sample of 349 training delegates who completed both an IRT‐scored forced‐choice and a normative single‐stimulus version of the questionnaire. Results largely supported the scaling properties, measurement precision, and equivalence of the data structures of the two scoring methods.     

Computerization and adaptive administration of the NEO PI-R

This study asks, how well does an item response theory (IRT) based computerized adaptive NEO PI-R work? To explore this question, real-data simulations (N = 1,059) were used to evaluate a maximum information item selection computerized adaptive test (CAT) algorithm. Findings indicated satisfactory recovery of full-scale facet scores with the administration of around four items per facet scale. Thus, the NEO PI-R could be reduced in half with little loss in precision by CAT administration. However, results also indicated that the CAT algorithm was not necessary. We found that for many scales, administering the "best" four items per facet scale would have produced similar results. In the conclusion, we discuss the future of computerized personality assessment and describe the role IRT methods might play in such assessments.     

应用项目反应理论对《中国士兵人格问卷》的项目分析

采用项目反应理论（IRT）对《中国士兵人格问卷》进行项目分析。计算机呈现中国士兵人格问卷（CSPQ）对100,523名适龄男性青年进行测验,随机抽取2676名任一维度标准分均低于70的定为合格组;将任一维度大于70分并经专业人员访谈不合格的274名定为不合格组;从精神病院抽取男性年龄相当的221名缓解期精神分裂症患者定为精神病组,并完成CSPQ测验。运用基于IRT的双参数Logistic模型进行分析;结果发现,区分度参数超过区间(0.30,4.00)的条目删除前后,被试的能力值与标准分均存在显著相关;精神病组的测验分数经IRT分析,图形曲线与不合格组有高度吻合。研究结果说明,在测验精度基本相同的条件下,应用IRT可以减少施测条目,提高测验效率,可在一定程度上更精确地区分被试的特质水平     

Factor analytic approaches to personality item-level data

Factor analysis models have played a central role in formulating conceptual models in personality and personality assessment, as well as in empirical examinations of personality measurement instruments. Yet, the use of item-level data presents special problems for factor analysis, applications. In this article, we review recent developments in factor analysis that are appropriate for the type of item-level data often collected in personality. Included in this review are discussions of how these developments have been addressed in the context of two different (but formally related) statistical models item response theory (IRT: Hambleton, Swaminathan, & Rogers, 1991) and structural, equation modeling (Bollen 1989) for item-level data. We also discuss the relevance of item scaling in the context of these models. Using the restandardization data for the Minnesota Multiphasic Personality Inventory-2 Scale (cf. Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989), we show brief examples of the utility of these approaches to address basic questions about responses to personality scale items regarding: (a) scale, dimensionality and general item properties, (b) the "appropriateness" of the observed responses, and (c) differential item functioning across subsamples. implications for analyses of personality item-level data in the IRT and factor analytic traditions are discussed.     

Analyzing psychopathology items: a case for nonparametric item response theory modeling

The authors discuss the applicability of nonparametric item response theory (IRT) models to the construction and psychometric analysis of personality and psychopathology scales, and they contrast these models with parametric IRT models. They describe the fit of nonparametric IRT to the Depression content scale of the Minnesota Multiphasic Personality Inventory--2 (J. N. Butcher, W. G. Dahlstrom, J. R. Graham, A. Tellegen, & B. Kaemmer, 1989). They also show how nonparametric IRT models can easily be applied and how misleading results from parametric IRT models can be avoided. They recommend the use of nonparametric IRT modeling prior to using parametric logistic models when investigating personality data.     

Using SAS PROC NLMIXED to fit item response theory models

Researchers routinely construct tests or questionnaires containing a set of items that measure personality traits, cognitive abilities, political attitudes, and so forth. Typically, responses to these items are scored in discrete categories, such as points on a Likert scale or a choice out of several mutually exclusive alternatives. Item response theory (IRT) explains observed responses to items on a test (questionnaire) by a person’s unobserved trait, ability, or attitude. Although applications of IRT modeling have increased considerably because of its utility in developing and assessing measuring instruments, IRT modeling has not been fully integrated into the curriculum of colleges and universities, mainly because existing general purpose statistical packages do not provide built-in routines with which to perform IRT modeling. Recent advances in statistical theory and the incorporation of those advances into general purpose statistical software such as the Statistical Analysis System (SAS) allow researchers to analyze measurement data by using a class of models known as generalized linear mixed effects models (McCulloch & Searle, 2001), which include IRT models as special cases. The purpose of this article is to demonstrate the generality and flexibility of using SAS to estimate IRT model parameters. With real data examples, we illustrate the implementations of a variety of IRT models for dichotomous, polytomous, and nominal responses. Since SAS is widely available in educational institutions, it is hoped that this article will contribute to the spread of IRT modeling in quantitative courses.     

Two item response theory models for analysing normative forced‐choice personality items

This paper proposes two unidimensional item response theory (IRT) models for analysing normative forced‐choice personality items. Both models are derived from a common theoretical framework and arise as a result of different assumptions regarding the mechanism of choice. The simplest mechanism gives rise to the one‐parameter normal‐ogive model. The second mechanism gives rise to a new IRT model, which is closely related to the Coombs–Zinnes probabilistic unfolding model. The second model is compared theoretically to the normal‐ogive model in terms of item characteristic curves and amount of item information. Next, procedures for estimating the respondent and the item parameters in the second model are described. Finally, both models are empirically compared by using two well‐known personality measures.     

迫选式人格测验的传统计分与IRT计分模型

迫选测验的传统计分方式会产生自模式数据, 不能进行传统的信效度检验、因素分析和方差分析等。近年来研究者提出了一些基于项目反应理论的计分模型, 如瑟斯顿IRT模型和MUPP模型等, 它们可以规避自模式数据的弊端。瑟斯顿IRT模型方便进行参数估计, 模型定义灵活; 而MUPP模型的拓展性较差, 参数估计的方法有待提高。另一方面, 已有研究者基于MUPP模型开发了一些抗作假的迫选测验, 而瑟斯顿IRT模型距离这种应用还比较远。此外, 两个模型的适用性和有效性都有待更多的实证研究来检验。     

Item Characteristic Curve Estimation of Signal Detection Theory-Based Personality Data: A Two-Stage Approach to Item Response Modeling

Signal Detection Theory (SDT; MacMillan & Creelman, 1991) is a method of data collection that has been used for several years, which describes the decision-making strategies of individuals. However, its use has been largely restricted to experiments involving sensation and perception. The Overclaiming Questionnaire (OCQ; Paulhus & Bruce, 1990) is a scale that has been developed to measure intellectual ability and personality, using SDT as a guideline. Although the scale has been successful in measuring human characteristics such as narcissism and intelligence, it is still unclear how to measure the characteristics of the various stimuli used (e.g., item difficulty, item discrimination, etc.). In some ways, this is a direct consequence of the general lack of research involved in item parameter estimation in the field of SDT. Using the OCQ, this article presents a graphical and nonparametric form of item response modeling to address this issue. In many ways, the approach is influenced by and structured around item response theory (IRT; Hambleton, Swaminathan, & Rogers, 1991). The general features of both SDT and IRT are described. Results suggest that this method is indeed a reasonable approach to describing item functioning, and there are several advantages to using this method over traditional IRT methods. Furthermore, SDT appears to be a fruitful approach to assessing intelligence, ability, and other psychological constructs, with advantages over traditional approaches. Overall, the results provide interesting implications for item selection and test development in several scientific and academic fields.     

Item response theory in personality assessment: a demonstration using the MMPI-2 depression scale

Item response theory (IRT) analyses have, over the past 3 decades, added much to our understanding of the relationships among and characteristics of test items, as revealed in examinees response patterns. Assessment instruments used outside the educational context have only infrequently been analyzed using IRT, however. This study demonstrates the relevance of IRT to personality data through analyses of Scale 2 (the Depression Scale) on the revised Minnesota Multiphasic Personality Inventory (MMPI-2). A rich set of hypotheses regarding the items on this scale, including contrasts among the Harris-Lingoes and Wiener-Harmon subscales and differences in the items measurement characteristics for men and women, are investigated through the IRT analyses.     

Personality Trait Differences Between Young and Middle‐Aged Adults: Measurement Artifacts or Actual Trends?

A growing body of research demonstrates that older individuals tend to score differently on personality measures than younger adults. However, recent research using item response theory (IRT) has questioned these findings, suggesting that apparent age differences in personality traits merely reflect artifacts of the response process rather than true differences in the latent constructs. Conversely, other studies have found the opposite—age differences appear to be true differences rather than response artifacts. Given these contradictory findings, the goal of the present study was to examine the measurement equivalence of personality ratings drawn from large groups of young and middle‐aged adults (a) to examine whether age differences in personality traits could be completely explained by measurement nonequivalence and (b) to illustrate the comparability of IRT and confirmatory factor analysis approaches to testing equivalence in this context. Self‐ratings of personality traits were analyzed in two groups of Internet respondents aged 20 and 50 (n = 15,726 in each age group). Measurement nonequivalence across these groups was negligible. The effect sizes of the mean differences due to nonequivalence ranged from –.16 to .15. Results indicate that personality trait differences across age groups reflect actual differences rather than merely response artifacts.     

IRT展开模型及对非累积反应机制的检测

被试回答人格测验题目时并不是特质水平越高其得分率越高,这称为非累积反应机制。广义等级展开模型GGUM就是针对这一机制提出来的。使用EPQ和五因素人格问卷发现GGUM比累积IRT模型有更好的模型拟合度和测量精度。研究结果表明GGUM有其合理性,且有助于反应心理过程机制的深入探讨。     

Personality assessment: Does the medium matter? No

The equivalence of an Internet administration of personality tests with two other administration formats was assessed using Item Response Theory (IRT) and various other statistical methods. The analyses were conducted on measures of Neuroticism, Extroversion, Agreeableness, and Conscientiousness. A total of 728 participants took part in the study. Participants were randomly assigned to one of three administrative conditions: paper-and-pencil, proctored computer lab, and unproctored Internet. Analyses with IRT, factor analysis, criterion-related validity, and mean differences supported the equivalence of Internet and traditional paper-and-pencil administrations of personality tests.     

计算机自适应测验在特质焦虑量表中的运用

将基于项目反应理论的计算机自适应测验运用于特质焦虑量表,考察这一测验形式在人格测量中所具有的特性.收集特质焦虑量表真实纸笔作答数据,选用合适的心理测量模型,模拟计算机自适应测验.结果表明：相对纸笔测验而言,计算机自适应测验的测试效率更高、对被试的分辨力更强、结果更直观.计算机自适应测验在人格测量中的实践值得进一步探索.