Tailored Interviewing: An Application of Item Response Theory for Personality Measurement

Despite recent technological advances in mass survey administration, surveys seeking to measure personality traits still employ procedures identical to the traditional paper-and-pencil scales, in which every respondent is asked exactly the same items, regardless of his or her trait level. We present an empirical application of personality measurement in which the number and sequence of scale items are tailored to the trait level of each respondent and show that the total number of questions asked of each respondent could be reduced substantially, with a measurable and controllable increase in the standard error of measurement.     

Psychometric and structural analysis of the MMPI-2 Personality Psychopathology Five (PSY-5) facet subscales

The Personality Psychopathology Five (PSY-5) is a model of personality psychopathology assessed in adult populations with a set of Minnesota Multiphasic Personality Inventory-2 (MMPI-2) scales. The authors examine the reliability and validity of recently developed lower-order facet subscales for each of these five domains, with an emphasis on structural invariance, using both confirmatory factor analysis (CFA) and exploratory factor analysis (EFA). MMPI-2 protocols completed by psychiatric patients (N = 693) served as the data source. The reliability and discriminant validity of the subscales were mostly inadequate. Results from the CFAs reveal universally poor statistical fits. Subsequent EFAs extracted alternate latent structures, which also demonstrate mostly inadequate reliability and validity. Overall, results suggest that the item pool that forms the MMPI-2 PSY-5 domain scales may not be able to sustain meaningful facet subscales.     

应用项目反应理论对《中国士兵人格问卷》的项目分析

采用项目反应理论（IRT）对《中国士兵人格问卷》进行项目分析。计算机呈现中国士兵人格问卷（CSPQ）对100,523名适龄男性青年进行测验,随机抽取2676名任一维度标准分均低于70的定为合格组;将任一维度大于70分并经专业人员访谈不合格的274名定为不合格组;从精神病院抽取男性年龄相当的221名缓解期精神分裂症患者定为精神病组,并完成CSPQ测验。运用基于IRT的双参数Logistic模型进行分析;结果发现,区分度参数超过区间(0.30,4.00)的条目删除前后,被试的能力值与标准分均存在显著相关;精神病组的测验分数经IRT分析,图形曲线与不合格组有高度吻合。研究结果说明,在测验精度基本相同的条件下,应用IRT可以减少施测条目,提高测验效率,可在一定程度上更精确地区分被试的特质水平     

Capturing Abnormal Personality With Normal Personality Inventories: An Item Response Theory Approach

ABSTRACT Correlational and factor‐analytic methods indicate that abnormal and normal personality constructs may be tapping the same underlying latent trait. However, they do not systematically demonstrate that measures of abnormal personality capture more extreme ranges of the latent trait than measures of normal range personality. Item Response Theory (IRT) methods, in contrast, do provide this information. In the present study, we use IRT methods to evaluate the range of the latent trait assessed with a normal personality measure and a measure of psychopathy as one example of an abnormal personality construct. Contrary to the expectation that the measure of psychopathy would be more extreme than the measure of normal personality traits, the measures overlapped substantially in terms of the regions of the latent trait for which they provide information. Moreover, both types of inventories were limited in terms of measurement bandwidth, such that they did not provide information across the entire latent trait continuum. Implications and future directions are discussed.     

Correlates of the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) Personality Psychopathology Five (PSY-5) scales in a forensic assessment setting

Empirical correlates of the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) Personality Psychopathology Five (PSY-5) scales in a forensic assessment setting were identified. Archival extratest data, including demographics, psychosocial history, criminal behavior history, and current mental status and psychodiagnosis, were extracted from the case files of 593 men and women referred to a forensic assessment clinic for criminal court-ordered evaluations. Zero-order and multiple correlations were calculated between the MMPI-2 PSY-5 scales and relevant criterion variables. Findings indicated that the PSY-5 scales' empirical correlates in a forensic setting are similar to and consistent with those found in general mental health settings. Linear combinations of MMPI-2 PSY-5 Scale scores accounted for moderate proportions of variance in the collateral indicators.     

Internal Consistencies and Structural Analysis of the MMPI-2 PSY-5 Facet Subscales in Chinese Population

Using the normative data set of the Minnesota Multiphasic Personality Inventory-2 of the People’s Republic of China, this study examined the psychometric properties of the Personality Psychopathology Five facet subscales recently developed by Arnau et al. (2005). The internal consistency reliability, convergent and discriminant validity of the Psy-5 were found to be unsatisfactory. Results of confirmatory factor analysis (CFA) indicated that both item-to-facet factor structure (except the PSYC domain facets subscales) and facet-to-domain factor structure fit the data poorly. These findings suggested that the PSY-5 facet subscales may not be reliable and valid in population independent of those used to derive them. Thus, the utility of these facet subscales in clinical and research settings is premature at least for Chinese people at the present time.     

Principal components analyses of the MMPI-2 PSY-5 scales: identification of facet subscales

The Personality Psychopathology Five (PSY-5) is a five-factor personality trait model designed for assessing personality pathology using quantitative dimensions. Harkness, McNulty, and Ben-Porath developed Minnesota Multiphasic Personality Inventory-2 (MMPI-2) scales based on the PSY-5 model, and these scales were recently added to the standard MMPI-2 profile. Although the PSY-5 constructs are multidimensional in definition, explicit subscales for the broader PSY-5 scales have not been developed. The primary goals of this study were to empirically derive subscales for the MMPI-2 PSY-5 scales using principal components analysis (PCA) and to replicate these subscales with an independent sample. Individual PSY-5 scales were analyzed using PCA with an initial sample of 4,325 MMPI-2 protocols, and the component structure was replicated with a second sample of 4,277 MMPI-2 protocols. A third sample of 4,327 protocols was used to further evaluate the internal consistency reliabilities of the resulting facet subscales. Overall, replicable facet subscales were identified with content areas that are largely congruent with Harkness and McNulty's model.     

应用项目反应理论对瑞文测验联合型的分析

使用BILOG-MG3.0软件,边际极大似然估计,3参数Logistic模型对354名不同能力水平的男性青年的瑞文测验联合型数据进行了分析。结果显示:大多数瑞文测验联合型的题目都适合3参数Logistic模型(有6道题不适合)。整个测验的信息函数峰值的位置在难度量表的-3到-2之间,其值为16.82。共有18道题的信息函数峰值在0.2以下。从区分度来看,72道题目的区分度均大于0.5,比较理想。难度参数显示所有题目均较低,绝大部分都在0以下,最高的只有1.01。题目的难度主要由所需的操作水平决定。伪猜测参数在0.07-0.24之间。综合分析表明瑞文测验联合型对正常青年的智力评价精度较差。     

Incremental validity of the MMPI-2 PSY-5 scales in assessing self-reported personality disorder criteria

The Minnesota Multiphasic Personality Inventory-2 (MMPI-2) Personality Psychopathology-Five (PSY-5) scales were developed to measure abnormal personality symptomatology. The present study examines the incremental validity of the PSY-5 scales beyond the clinical and content scales in assessing criteria associated with personality disorders. The current sample includes 240 male and 407 female clients from private practice settings who completed the MMPI-2 and the Multiaxial Diagnostic Inventory (MDI), a self-report checklist of Diagnostic and Statistical Manual of Mental Disorders (3rd ed., revised) symptoms. Six of the MDI personality disorder scales, conceptually related to the PSY-5 scales, are used as criteria. Hierarchical regression analyses determine the incremental validity of each PSY-5 scale. In most analyses, PSY-5 scales add a significant increment of variance to the clinical and content scales. Implications of the results are discussed.     

The Spiritual Transcendence Index: An Item Response Theory Analysis

ABSTRACT

In an attempt to measure understudied dimensions of spirituality, recent efforts have focused on the transcendent dimension of spirituality. The Spiritual Transcendence Index (STI) was developed to assess a perceived experience of the sacred that affects one’s ability to transcend life’s difficulties. The main focus of the current study was to investigate the psychometric properties of the STI by utilizing the microscopic item-level examination tools unique in item response theory (IRT), as well as its scale-level exploration devices for psychometric properties of an assessment measure. IRT analyses were conducted to investigate the STI’s psychometric properties across samples (N = 712) including how well the measure assesses the latent construct, spiritual transcendence, from the low to high range of the construct. The findings confirm that the 8-item index is a single factor that assesses the latent construct, spiritual transcendence. Instead of the original 6-category version, these findings support a 4-category response version; the 3 categories of disagreement may be collapsed into a single category. These findings not only inform the refinement of the STI but also highlight an important psychometric approach for the refinement of spirituality/religiousness measures, especially those with ceiling effects.     

An Item Response Theory Analysis of the Spiritual Assessment Inventory

Item response theory (IRT) was applied to evaluate the psychometric properties of the Spiritual Assessment Inventory (SAI; Hall & Edwards, 1996 Hall, T. W. and Edwards, K. J. 1996. The initial development and factor analysis of the spiritual assessment inventory. Journal of Psychology and Theology, 24: 233–246. [Crossref], [Web of Science ®] , [Google Scholar], 2002 Hall, T. W. and Edwards, K. J. 2002. The spiritual assessment inventory: A theistic model and measure for assessing spiritual development. Journal for the Scientific Study of Religion, 41: 341–357. [Crossref], [Web of Science ®] , [Google Scholar]). The SAI is a 49-item self-report questionnaire designed to assess five aspects of spirituality: Awareness of God, Disappointment (with God), Grandiosity (excessive self-importance), Realistic Acceptance (of God), and Instability (in one's relationship to God). IRT analysis revealed that for several scales: (a) two or three items per scale carry the psychometric workload and (b) measurement precision is peaked for all five scales, such that one end of the scale, and not the other, is measured precisely. We considered how sample homogeneity and the possible quasi-continuous nature of the SAI constructs may have affected our results and, in light of this, made suggestions for SAI revisions, as well as for measuring spirituality, in general.     

正念注意觉知量表IRT分析研究*

正念注意觉知量表(MAAS)是测量正念注意水平最常用的量表之一,以中小学教师为被试,以项目反应理论用方法与技术对量表各项目的区分度、域值和信息函数峰值4个参数做了分析探讨。结果显示MMAS支持单维性假设,具有良好的心理测量学指标,对正念注意水平的测量具有较高的精准性。量表存在6个信息量很高的项目,其信息量之和接近量表总信息量的70%,提示这几个项目可以构成一个简式量表。对新量表做验证性因素分析,各项指标达到要求。     

Personality and psychopathology: mapping the MMPI-2 Restructured Clinical (RC) Scales onto the Five Factor Model of personality

The MMPI-2 Restructured Clinical (RC) Scales (Tellegen et al., 2003) reflect a recent shift for this instrument toward the measurement of contemporary conceptualizations of psychopathology. The current investigation aimed to replicate and extend the theoretical and empirical linkage between the RC scales and dimensional models of personality and to investigate how well the RC scales conform to a higher-order structure of psychopathology. Participants were 271 psychiatric patients who had been administered the MMPI-2 and revised NEO Personality Inventory (Costa & McCrae, 1992) as part of a routine psychological evaluation. The results indicated that the RC scales map onto the Five Factor Model of personality as hypothesized and in congruence with previous findings in personality and psychopathology. The RC scales conformed to a higher-order structure of internalizing, externalizing, and thought disturbance, replicating and extending previous work concerning hierarchical structures of psychopathology.     

Measurement Equivalence Between Applicant and Incumbent Groups: An IRT Analysis of Personality Scales

Incumbents are often used in the development and validation of a wide variety of personnel selection instruments, including noncognitive instruments such as personality tests. However, the degree to which assumed motivational factors impact the measurement equivalence and validity of tests developed using incumbents has not been adequately addressed. This study addressed this issue by examining the measurement equivalence of 6 personality scales between a group applying for jobs as sales managers in a large retail organization (N = 999) and a group of sales managers currently employed in that organization (N = 796). A graded item response theory model (Samejima, 1969) was fit to the personality scales in each group. Results indicated that moderately large differences existed in personality scale scores (approximately 1/2 standard deviation units) but only one of the six scales contained any items that evidenced differential item functioning and no scales evidenced differential test functioning. In addition, person-level analyses showed no apparent differences across groups in aberrant responding. The results suggest that personality measures used for selection retain similar psychometric properties to those used in incumbent validation studies.     

Bridging the divide: comments on the Restructured Clinical Scales of the MMPI-2

In this comment, I discuss the target articles written by Nichols (2006/this issue) and Rogers, Sewell, Harrison, and Jordan (2006/this issue) regarding the Restructured Clinical (RC) Scales (Tellegen et al., 2003) of the MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989). Both articles provide thoughtful discussion of the extent to which the RC Scales achieved the primary goals set forth by Tellegen et al., but they set different tones and come to somewhat different conclusions. I argue that the initial studies conducted on the RC Scales suggest a promising future for them. However, further studies are needed to explore the construct validity of the scales, assess their incremental validity with respect to other more established MMPI-2 scale sets, and determine their interpretive meaning in clinical contexts.     

Using reliability generalization methods to explore measurement error: an illustration using the MMPI-2 PSY-5 scales

Reliability generalization (RG) is a meta-analytic technique that allows for the systematic examination of variation in score reliability for different samples of test takers; this procedure is based on the recognition that reliability is not a stable property of a test but is sample dependent. As a demonstration of an RG analysis, I obtained 63 reliability coefficients for each of the MMPI-2 (Butcher et al., 2001) Personality Psychopathology 5 (Harkness, McNulty, & Ben-Porath, 1995) scales. The overall variability of alpha coefficients supports the argument that reliability is sample dependent and underscores the need for researchers to calculate reliability estimates based on their research samples rather than simply citing published alpha coefficients as evidence of score reliability. I observed statistically significant mean reliability differences for scores across the 5 scales, with the highest level of reliability observed for scores on the measure of Negative Emotionality and the lowest levels of reliability observed for scores on the measures of Aggression and Disconstraint. There was no evidence that the sex-composition of a sample was systematically related to score reliability, and there were no statistically significant differences in reliability between scores obtained with the English version of the test and those obtained with translated forms. However, reliability was consistently lower for scores on some scales when the data were obtained in nonclinical settings as opposed to clinical ones. Sample size was not significantly correlated with reliability estimates. RG methods have the potential for deepening the level of understanding about the role of reliability in the evaluation and use of personality tests.     

Commitment to Islam in Kazakhstan and Kyrgyzstan: An Item Response Theory Analysis

Worthington conceptualized a model of religiosity assessment. The dimensions of the model include Religious Norms, Religious Doctrine, and Authority of Leaders. A 10-item scale for Islamic religious assessment was constructed and administered in Kazakhstan and Kyrgyzstan. First-order factor analysis conducted on the 10 items of the religiosity scale revealed factorial structure corresponding to Worthington’s model. A second-order factor analysis assured 1 underlying latent trait. Two-parameter logistic item response theory models were fit to responses collected in Kazakhstan and Kyrgyzstan. Results supported psychometric soundness of the instrument. The items on the scale revealed excellent discrimination properties between the populations of high and low religious commitment. The study offers a short, practical scale for assessment of commitment to Islam in Central Asian countries.     

Adult Attachment Ratings (AAR): An Item Response Theory Analysis

The Adult Attachment Ratings (AAR) include 3 scales for anxious, ambivalent attachment (excessive dependency, interpersonal ambivalence, and compulsive care-giving), 3 for avoidant attachment (rigid self-control, defensive separation, and emotional detachment), and 1 for secure attachment. The scales include items (ranging from 6–16 in their original form) scored by raters using a 3-point format (0 = absent, 1 = present, and 2 = strongly present) and summed to produce a total score. Item response theory (IRT) analyses were conducted with data from 414 participants recruited from psychiatric outpatient, medical, and community settings to identify the most informative items from each scale. The IRT results allowed us to shorten the scales to 5-item versions that are more precise and easier to rate because of their brevity. In general, the effective range of measurement for the scales was 0 to +2 SDs for each of the attachment constructs; that is, from average to high levels of attachment problems. Evidence for convergent and discriminant validity of the scales was investigated by comparing them with the Experiences of Close Relationships–Revised (ECR–R) scale and the Kobak Attachment Q-sort. The best consensus among self-reports on the ECR–R, informant ratings on the ECR–R, and expert judgments on the Q-sort and the AAR emerged for anxious, ambivalent attachment. Given the good psychometric characteristics of the scale for secure attachment, however, this measure alone might provide a simple alternative to more elaborate procedures for some measurement purposes. Conversion tables are provided for the 7 scales to facilitate transformation from raw scores to IRT-calibrated (theta) scores.     

The MMPI-2 Restructured Clinical Scales: a paradigmatic shift in scale development

Tellegen et al. (2003) proposed fundamental changes in MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) scale development by discarding empirical scale development in favor of construct validation via Jackson's (1970) sequential system of scale development. As a result of their efforts, a general distress factor (Demoralization) was identified and 8 Restructured Clinical (RC) Scales were developed. Using 7,330 clinical cases from Caldwell's (1997) data set, in this study, we sought to cross-validate the MMPI-2 RC Scales. Scale homogeneity was confirmed with high alpha coefficients and interitem correlations in the expected range. We also achieved a major objective of reducing interscale correlations. In replicating Tellegen et al.'s principal components analysis, we achieved a high concordance for 6 of the 8 RC Scales. We critically examine these results in light of Jackson's construct validation. We discuss the clinical usefulness of the MMPI-2 RC Scales within the context of current and future research.