An item response theory analysis of the narcissistic personality inventory

This research uses item response theory methods to evaluate the Narcissistic Personality Inventory (NPI; Raskin & Terry, 1988). Analyses using the 2-parameter logistic model were conducted on the total score and the Corry, Merritt, Mrug, and Pamp (2008) and Ackerman et?al. (2011) subscales for the NPI. In addition to offering precise information about the psychometric properties of the NPI item pool, these analyses generated insights that can be used to develop new measures of the personality constructs embedded within this frequently used inventory.     

Addiction Severity Index Recent and Lifetime summary indexes based on nonparametric item response theory methods

Baseline Addiction Severity Index (5th ed.; ASI-5) data of 2,142 substance abuse patients were analyzed with two nonparametric item response theory (NIRT) methods: Mokken scaling and conditional covariance techniques. Nine reliable and dimensionally homogeneous Recent Problem indexes emerged in the ASI-5's seven areas, including two each in the Employment/Support and Family/Social Relationships areas. Lifetime Problem indexes were derived for five of the areas--Medical, Drug, Alcohol, Legal, and Psychiatric--but not for the Employment/Support and Family/Social Relationships areas. Correlational analyses conducted on a subsample of 586 patients revealed the indexes for the seven areas to be largely independent. At least moderate correlations were obtained between the Recent and Lifetime indexes within each area where both existed. Concurrent validity analyses conducted on this same subsample found meaningful relationships, except for the Employment/Support area. NIRT-based methods were able to add to findings produced previously by classical psychometric methods and appear to offer promise for the psychometric analysis of complex, mixed-format instruments such as the ASI-5.     

An item response theory analysis of the Impulsive Behaviors Checklist for Adolescents

This study used item response theory (IRT) to examine the Impulsive Behaviors Checklist for Adolescents (IBCL-A) among 6,276 (67.7% girls) Chinese secondary school students. The IBCL-A included 15 maladaptive impulsive behaviors adapted from the Revised Diagnostic Interview for Borderlines. The authors obtained the severity and discrimination parameters for each item in the IBCL-A, examined differential item functioning across gender and age groups, and tested reliability and concurrent validity of the IBCL-A IRT-scaled score. Most items in the IBCL-A were the most accurate in assessing moderate to high levels of impulsivity and discriminated well among adolescents with varied levels of impulsivity. Differential item functioning emerged in several items across gender. The IRT-scaled score showed good construct validity and incremental predictive validity. Findings demonstrate the sound psychometric properties of the IBCL-A and support the clinical utility of this scale.     

An item response theory analysis of self-report measures of adult attachment

Self-report measures of adult attachment are typically scored in ways (e.g., averaging or summing items) that can lead to erroneous inferences about important theoretical issues, such as the degree of continuity in attachment security and the differential stability of insecure attachment patterns. To determine whether existing attachment scales suffer from scaling problems, the authors conducted an item response theory (IRT) analysis of 4 commonly used self-report inventories: Experiences in Close Relationships scales (K. A. Brennan, C. L. Clark, & P. R. Shaver, 1998), Adult Attachment Scales (N. L. Collins & S. J. Read, 1990), Relationship Styles Questionnaire (D. W. Griffin & K. Bartholomew, 1994) and J. Simpson's (1990) attachment scales. Data from 1,085 individuals were analyzed using F. Samejima's (1969) graded response model. The authors' findings indicate that commonly used attachment scales can be improved in a number of important ways. Accordingly, the authors show how IRT techniques can be used to develop new attachment scales with desirable psychometric properties.     

Bayesian item fit analysis for unidimensional item response theory models

Assessing item fit for unidimensional item response theory models for dichotomous items has always been an issue of enormous interest, but there exists no unanimously agreed item fit diagnostic for these models, and hence there is room for further investigation of the area. This paper employs the posterior predictive model‐checking method, a popular Bayesian model‐checking tool, to examine item fit for the above‐mentioned models. An item fit plot, comparing the observed and predicted proportion‐correct scores of examinees with different raw scores, is suggested. This paper also suggests how to obtain posterior predictive p‐values (which are natural Bayesian p‐values) for the item fit statistics of Orlando and Thissen that summarize numerically the information in the above‐mentioned item fit plots. A number of simulation studies and a real data application demonstrate the effectiveness of the suggested item fit diagnostics. The suggested techniques seem to have adequate power and reasonable Type I error rate, and psychometricians will find them promising.     

The impact of social desirability bias on the EPQ-R item scores: An item response theory analysis

This study assessed the potential influence of social desirability (SD) response bias on the E, N, and P EPQ-R scores at the level of individual items. The study was based on a bidimensional IRT model which was fitted in a large sample. This allowed a detailed analysis of both the internal validities of the items and the content of the items which were most affected by SD. The E items were least affected by SD, but the direction of the impact depended on the type of item. As expected, in the N and P cases the relations obtained were consistently negative, but the strength of the SD impact also depended considerably on the type of item. The P scale was the most problematic in terms of convergent and discriminant validity.     

An item response theory/confirmatory factor analysis of the Autobiographical Memory Test

The Autobiographical Memory Test (AMT) is used to assess the degree of specificity of autobiographical memory. The AMT usually contains cue words of both positive and negative valence, but it is unclear whether these valences form separate factors or not. Accordingly, confirmatory factor analysis assessed whether the AMT measures one overall factor, or whether different cue types are related to different factors. Results were consistent across three datasets (N = 333, N = 405, and N = 336). A one-factor model fitted each dataset well, which suggests that responses to positive and negative cues are related to the one construct. In addition, item response theory analyses showed that the AMT is most precise for people who score low on memory specificity. Implications for using the AMT with high-functioning samples are discussed.     

An introduction to mixture item response theory models

Mixture item response theory (IRT) allows one to address situations that involve a mixture of latent subpopulations that are qualitatively different but within which a measurement model based on a continuous latent variable holds. In this modeling framework, one can characterize students by both their location on a continuous latent variable as well as by their latent class membership. For example, in a study of risky youth behavior this approach would make it possible to estimate an individual's propensity to engage in risky youth behavior (i.e., on a continuous scale) and to use these estimates to identify youth who might be at the greatest risk given their class membership. Mixture IRT can be used with binary response data (e.g., true/false, agree/disagree, endorsement/not endorsement, correct/incorrect, presence/absence of a behavior), Likert response scales, partial correct scoring, nominal scales, or rating scales. In the following, we present mixture IRT modeling and two examples of its use. Data needed to reproduce analyses in this article are available as supplemental online materials at http://dx.doi.org/10.1016/j.jsp.2016.01.002.     

Introduction to bifactor polytomous item response theory analysis

A bifactor item response theory model can be used to aid in the interpretation of the dimensionality of a multifaceted questionnaire that assumes continuous latent variables underlying the propensity to respond to items. This model can be used to describe the locations of people on a general continuous latent variable as well as on continuous orthogonal specific traits that characterize responses to groups of items. The bifactor graded response (bifac-GR) model is presented in contrast to a correlated traits (or multidimensional GR model) and unidimensional GR model. Bifac-GR model specification, assumptions, estimation, and interpretation are demonstrated with a reanalysis of data (Campbell, 2008) on the Shared Activities Questionnaire. We also show the importance of marginalizing the slopes for interpretation purposes and we extend the concept to the interpretation of the information function. To go along with the illustrative example analyses, we have made available supplementary files that include command file (syntax) examples and outputs from flexMIRT, IRTPRO, R, Mplus, and STATA. Supplementary data to this article can be found online at http://dx.doi.org/10.1016/j.jsp.2016.11.001. Data needed to reproduce analyses in this article are available as supplemental materials (online only) in the Appendix of this article.     

A multigroup item response theory analysis of the psychopathy checklist--revised

Item response theory was used to investigate the functioning of the Psychopathy Checklist-Revised (PCL-R; R. D. Hare, 1991, 2003) in several offender populations. With male criminal offenders (N=3,847) as a reference group, differential item functioning analyses were performed for 3 comparison groups: female criminal offenders (N=1,219), male forensic psychiatric patients (N=1,246), and male criminal offenders scored from file reviews (N=2,626). Results are discussed in the context of the 2-factor, 4-facet model for the PCL-R (R. D. Hare, 2003; J. Parker, G. Sitarenios, & R. D. Hare, 2003). Application of a multigroup graded response model to all 4 groups suggests scalar equivalence may hold at least approximately for each population, although the PCL-R provided slightly greater information about the latent trait of psychopathy for male criminal offenders scored from the standard procedure.     

An investigation of hierarchical Bayes procedures in item response theory

Hierarchical Bayes procedures for the two-parameter logistic item response model were compared for estimating item and ability parameters. Simulated data sets were analyzed via two joint and two marginal Bayesian estimation procedures. The marginal Bayesian estimation procedures yielded consistently smaller root mean square differences than the joint Bayesian estimation procedures for item and ability estimates. As the sample size and test length increased, the four Bayes procedures yielded essentially the same result.The authors wish to thank the Editor and anonymous reviewers for their insightful comments and suggestions.     

Conjunctive item response theory kernels

Conjunctive item response models are introduced such that (a) sufficient statistics for latent traits are not necessarily additive in item scores; (b) items are not necessarily locally independent; and (c) existing compensatory (additive) item response models including the binomial, Rasch, logistic, and general locally independent model are special cases. Simple estimates and hypothesis tests for conjunctive models are introduced and evaluated as well. Conjunctive models are also identified with cognitive models that assume the existence of several individually necessary component processes for a global ability. It is concluded that conjunctive models and methods may show promise for constructing improved tests and uncovering conjunctive cognitive structure. It is also concluded that conjunctive item response theory may help to clarify the relationships between local dependence, multidimensionality, and item response function form.I appreciate the many helpful suggestions that were given by the reviewers and Ivo Molenaar.     

An item factor analysis and item response theory-based revision of the Everyday Discrimination Scale

The Everyday Discrimination Scale (EDS), a widely used measure of daily perceived discrimination, is purported to be unidimensional, to function well among African Americans, and to have adequate construct validity. Two separate studies and data sources were used to examine and cross-validate the psychometric properties of the EDS. In Study 1, an exploratory factor analysis was conducted on a sample of African American law students (N = 589), providing strong evidence of local dependence, or nuisance multidimensionality within the EDS. In Study 2, a separate nationally representative community sample (N = 3,527) was used to model the identified local dependence in an item factor analysis (i.e., bifactor model). Next, item response theory (IRT) calibrations were conducted to obtain item parameters. A five-item, revised-EDS was then tested for gender differential item functioning (in an IRT framework). Based on these analyses, a summed score to IRT-scaled score translation table is provided for the revised-EDS. Our results indicate that the revised-EDS is unidimensional, with minimal differential item functioning, and retains predictive validity consistent with the original scale.     

Solving the measurement invariance anchor item problem in item response theory

The efficacy of tests of differential item functioning (measurement invariance) has been well established. It is clear that when properly implemented, these tests can successfully identify differentially functioning (DF) items when they exist. However, an assumption of these analyses is that the metric for different groups is linked using anchor items that are invariant. In practice, however, it is impossible to be certain which items are DF and which are invariant. This problem of anchor items, or referent indicators, has long plagued invariance research, and a multitude of suggested approaches have been put forth. Unfortunately, the relative efficacy of these approaches has not been tested. This study compares 11 variations on 5 qualitatively different approaches from recent literature for selecting optimal anchor items. A large-scale simulation study indicates that for nearly all conditions, an easily implemented 2-stage procedure recently put forth by Lopez Rivas, Stark, and Chernyshenko (2009) provided optimal power while maintaining nominal Type I error. With this approach, appropriate anchor items can be easily and quickly located, resulting in more efficacious invariance tests. Recommendations for invariance testing are illustrated using a pedagogical example of employee responses to an organizational culture measure. (PsycINFO Database Record (c) 2012 APA, all rights reserved).     

On the sampling theory roundations of item response theory models

Item response theory (IT) models are now in common use for the analysis of dichotomous item responses. This paper examines the sampling theory foundations for statistical inference in these models. The discussion includes: some history on the stochastic subject versus the random sampling interpretations of the probability in IRT models; the relationship between three versions of maximum likelihood estimation for IRT models; estimating versus estimating -predictors; IRT models and loglinear models; the identifiability of IRT models; and the role of robustness and Bayesian statistics from the sampling theory perspective.A presidential address can serve many different functions. This one is a report of investigations I started at least ten years ago to understand what IRT was all about. It is a decidedly one-sided view, but I hope it stimulates controversy and further research. I have profited from discussions of this material with many people including: Brian Junker, Charles Lewis, Nicholas Longford, Robert Mislevy, Ivo Molenaar, Donald Rock, Donald Rubin, Lynne Steinberg, Martha Stocking, William Stout, Dorothy Thayer, David Thissen, Wim van der Linden, Howard Wainer, and Marilyn Wingersky. Of course, none of them is responsible for any errors or misstatements in this paper. The research was supported in part by the Cognitive Science Program, Office of Naval Research under Contract No. Nooo14-87-K-0730 and by the Program Statistics Research Project of Educational Testing Service.     

An item response theory analysis of DSM-IV personality disorder criteria across younger and older age groups

Many of the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV; American Psychiatric Association, 1994) personality disorder (PD) diagnostic criteria focus on a younger social and occupational context. The absence of age-appropriate criteria for older adults forces researchers and clinicians to draw conclusions based on existing criteria, which are likely inadequate. To explore which DSM-IV PD criteria contain age group measurement bias, the authors report 2 analyses of data on nearly 37,000 participants, ages 18-98 years, taken from a public data set that includes 7 of the 10 PDs (antisocial, avoidant, dependent, histrionic, obsessive-compulsive, paranoid, and schizoid). The 1st analysis revealed that older age groups tend to endorse fewer PD criteria than younger age groups. The 2nd analysis revealed that 29% of the criteria contain measurement bias. Although the latent variable structure for each PD was quite similar across younger and older age groups, some individual criteria were differentially endorsed by younger and older adults with equivalent PD pathology. The presence of measurement bias for these criteria raises questions concerning the assessment of PDs in older adults and the interpretation of existing data.     

Dissecting the expanded cognitive reflection test: an item response theory analysis

This study presents a psychometric evaluation of the Expanded Cognitive Reflection Test (CRT7) based on item response theory. The participants (N?=?1204) completed the CRT7 and provided self-reported information about their cognitive styles through the Preference for Intuition and Deliberation Scale (PID). A two-parameter logistic model was fitted to the data to obtain the item difficulty and discrimination parameters of the CRT7. The results showed that the items had good discriminatory power (αs?=?.80 ? 2.92), but the range of difficulty was restricted (βs ranged from ?.60 to .32). Moreover, the CRT7 showed a pattern of correlations with the PID which was similar to that of the original CRT. When taken together, these results are evidence of the adequacy of the CRT7 as an expanded tool for measuring cognitive reflection; however, one of the newer items (the pig item) was consistently problematic across analyses, and so it is recommended that in future studies it should be removed from the CRT7.     

A factor analysis and item response theory analysis of an employee honesty test

Although paper and pencil tests of employee honesty are becoming increasingly widespread in industry, a paucity of research exists regarding them. In a recent review of this literature, Sackett and Harris (1984) noted that scant psychometric evidence is available as to their merits or weaknesses. The aim of this paper is to report on the factor and item analysis of one such test. A principal axis solution and item response theory model (1-parameter) were used to examine the data. The factor analysis revealed four readily interpretable factors. With regard to the item analysis, the results indicated that on the whole most of the 40 items showed a reasonable fit to the model. The implications of this research are addressed.