An application of item response theory to the DSM-III-R criteria for borderline personality disorder

This paper summarizes results from analyses of the DSM criteria for borderline personality disorder (BPD) using models from item response theory (IRT). The study sample consisted of 353 participants, the majority of whom were psychiatric patients. Confirmatory factor analysis showed that a one-factor model provided the best fit to the data. All the DSM BPD criteria had moderate or higher item discrimination parameters, indicating that all items contributed meaningful information in assessing BPD. Item information functions revealed that the BPD criteria as a whole were useful for capturing BPD traits in the moderately severe to severe range, but that they performed less well in the less severe range. The general conclusion is that the criteria do represent a coherent syndrome and that further research on the informational value of the individual criteria would be useful.     

An item response theory analysis of DSM-IV personality disorder criteria across younger and older age groups

Many of the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV; American Psychiatric Association, 1994) personality disorder (PD) diagnostic criteria focus on a younger social and occupational context. The absence of age-appropriate criteria for older adults forces researchers and clinicians to draw conclusions based on existing criteria, which are likely inadequate. To explore which DSM-IV PD criteria contain age group measurement bias, the authors report 2 analyses of data on nearly 37,000 participants, ages 18-98 years, taken from a public data set that includes 7 of the 10 PDs (antisocial, avoidant, dependent, histrionic, obsessive-compulsive, paranoid, and schizoid). The 1st analysis revealed that older age groups tend to endorse fewer PD criteria than younger age groups. The 2nd analysis revealed that 29% of the criteria contain measurement bias. Although the latent variable structure for each PD was quite similar across younger and older age groups, some individual criteria were differentially endorsed by younger and older adults with equivalent PD pathology. The presence of measurement bias for these criteria raises questions concerning the assessment of PDs in older adults and the interpretation of existing data.     

Selecting the most informative items in the IIP scales for personality disorders: an application of item response theory.

The first goal of the present analyses was to shorten the five scales (Pilkonis, P. A., Kim, Y., Proietti, J. M., & Barkham, M. [1996]. Journal of Personality Disorders, 10, 355-369) for personality disorders (PDs) developed from the Inventory of Interpersonal Problems (IIP), thereby increasing their attractiveness for screening purposes. The second goal was to illustrate, for more general purposes, the utility of item response theory (IRT) for such scale refinement. IRT analyses were performed using data collected from six different samples (N = 1149) at five sites and a two-parameter (2P) graded model designed for multiple response items like those on the IIP. The five most informative items from each scale were identified, based on the magnitude of item discrimination parameters and the range and elevation of individual item information functions. Preliminary analyses of the reliability and validity of the short forms of the scales (totaling 25 items) supported their value as alternatives to the longer forms (consisting of 47 items), although definitive tests of their psychometric properties await crossvalidation in independent samples. Analyses of the quality receiver operating characteristics (QROC) of the long and short forms showed that both versions can be useful in predicting the presence versus absence of any PD diagnosis arrived at by using either a "best estimate" clinical consensus method or a structured Axis II interview.     

An item response theory analysis of the narcissistic personality inventory

This research uses item response theory methods to evaluate the Narcissistic Personality Inventory (NPI; Raskin & Terry, 1988). Analyses using the 2-parameter logistic model were conducted on the total score and the Corry, Merritt, Mrug, and Pamp (2008) and Ackerman et?al. (2011) subscales for the NPI. In addition to offering precise information about the psychometric properties of the NPI item pool, these analyses generated insights that can be used to develop new measures of the personality constructs embedded within this frequently used inventory.     

Item response theory analysis of diagnostic criteria for alcohol and cannabis use disorders in adolescents: implications for DSM-V

Item response theory (IRT) has advantages over classical test theory in evaluating diagnostic criteria. In this study, the authors used IRT to characterize the psychometric properties of Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV; American Psychiatric Association, 1994) alcohol and cannabis use disorder symptoms among 472 clinical adolescents. For both substances, DSM-IV symptoms fit a model specifying a unidimensional latent trait of problem severity. Threshold (severity) parameters did not distinguish abuse and dependence symptoms. Abuse symptoms of legal problems and hazardous use, and dependence symptoms of tolerance, unsuccessful attempts to quit, and physical-psychological problems, showed relatively poor discrimination of problem severity. There were gender differences in thresholds for hazardous use, legal problems, and physical-psychological problems. The results illustrate limitations of DSM-IV criteria for alcohol and cannabis use disorders when applied to adolescents. The development process for the fifth edition (DSM-V) should be informed by statistical models such as those used in this study.     

Differential impairment as an indicator of sex bias in DSM-IV criteria for four personality disorders

The aim of the present study was to investigate the possibility of sex bias in the diagnostic criteria for borderline, schizotypal, avoidant, and obsessive-compulsive personality disorders. A clinical sample of 668 individuals was evaluated for personality disorder criteria using a semistructured interview, and areas of functional impairment were assessed with both self-report and semistructured interview. The authors used a regression model of bias to identify bias as differences in slopes or intercepts between men and women in the relationship between each diagnostic criterion and level of impairment. The results suggest that most of the diagnostic criteria examined do not seem to display sex bias. However, those criteria that displayed evidence of bias came largely from the borderline diagnosis.     

The impact of social desirability bias on the EPQ-R item scores: An item response theory analysis

This study assessed the potential influence of social desirability (SD) response bias on the E, N, and P EPQ-R scores at the level of individual items. The study was based on a bidimensional IRT model which was fitted in a large sample. This allowed a detailed analysis of both the internal validities of the items and the content of the items which were most affected by SD. The E items were least affected by SD, but the direction of the impact depended on the type of item. As expected, in the N and P cases the relations obtained were consistently negative, but the strength of the SD impact also depended considerably on the type of item. The P scale was the most problematic in terms of convergent and discriminant validity.     

Sex bias in the diagnosis of personality disorders: an evaluation of the DSM-IV criteria.

This study considered whether the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; American Psychiatric Association, 1994) is biased against women by requiring less dysfunction for the personality disorders that are more commonly diagnosed in women (e.g., histrionic). Clinicians estimated the extent of social dysfunction, occupational dysfunction, and personal distress suggested by each of the diagnostic criteria for 6 personality disorders. The results failed to suggest a bias against women, as there was no difference in the overall level of dysfunction associated with the female-typed personality disorder diagnostic criteria (fewer criteria are also required for the male-typed diagnoses). However, the considerable variation in dysfunction across disorders and criteria, and the minimal degree of impairment implied by some of the diagnostic criteria, also raise more general issues that should perhaps be addressed in future editions of the diagnostic manual.     

Bayesian item fit analysis for unidimensional item response theory models

Assessing item fit for unidimensional item response theory models for dichotomous items has always been an issue of enormous interest, but there exists no unanimously agreed item fit diagnostic for these models, and hence there is room for further investigation of the area. This paper employs the posterior predictive model‐checking method, a popular Bayesian model‐checking tool, to examine item fit for the above‐mentioned models. An item fit plot, comparing the observed and predicted proportion‐correct scores of examinees with different raw scores, is suggested. This paper also suggests how to obtain posterior predictive p‐values (which are natural Bayesian p‐values) for the item fit statistics of Orlando and Thissen that summarize numerically the information in the above‐mentioned item fit plots. A number of simulation studies and a real data application demonstrate the effectiveness of the suggested item fit diagnostics. The suggested techniques seem to have adequate power and reasonable Type I error rate, and psychometricians will find them promising.     

Normative warrant in diagnostic criteria: the case of DSM-IV-TR personality disorders

This article focuses on the kinds of evaluative judgments made when applying DSM-IV-TR diagnostic criteria within the diagnostic interview between clinician and patient. The authors name these kinds of value judgments in diagnosis "normative warrant" because they involve one or more justifications (warrants) for standard-bearing (normative) elements involved in applying diagnostic criteria to actual patients. Seven types of normative warrant judgments are described (Type 1, Semantic-Phenomenal Matching; Type 2, Solicitation Choice; Type 3, Sociocultural Context; Type 4, Performance-Context Matching; Type 5, Deviance Threshold; Type 6, Threshold Characterization; Type 7, Disvalue characterization) and the typology is illustrated by applying it to various DSM-IV-TR personality disorder criteria. A research and clinical understanding of normative warrant may well contribute to the refinement of criteria sets as well as the refinement of the clinical use of criteria sets.     

Dissecting the expanded cognitive reflection test: an item response theory analysis

This study presents a psychometric evaluation of the Expanded Cognitive Reflection Test (CRT7) based on item response theory. The participants (N?=?1204) completed the CRT7 and provided self-reported information about their cognitive styles through the Preference for Intuition and Deliberation Scale (PID). A two-parameter logistic model was fitted to the data to obtain the item difficulty and discrimination parameters of the CRT7. The results showed that the items had good discriminatory power (αs?=?.80 ? 2.92), but the range of difficulty was restricted (βs ranged from ?.60 to .32). Moreover, the CRT7 showed a pattern of correlations with the PID which was similar to that of the original CRT. When taken together, these results are evidence of the adequacy of the CRT7 as an expanded tool for measuring cognitive reflection; however, one of the newer items (the pig item) was consistently problematic across analyses, and so it is recommended that in future studies it should be removed from the CRT7.     

Challenges associated with a polythetic diagnostic system: criteria combinations in the personality disorders

Converging research on the diagnostic criteria for personality disorders (PDs) reveals that most criteria have different psychometric properties. This finding is inconsistent with the PD diagnostic system according to the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; American Psychiatric Association, 1994), which weights each criterion equally. The purpose of the current study was to examine the potential effects of using equal weights for differentially functioning criteria. Data from over 2,100 outpatients were used to analyze and score response patterns to the diagnostic criteria for 9 PDs within an item response theory framework. Results indicated that combinations that included the same number of endorsed criteria yielded differing estimates of PD traits, depending on which criteria were met. Moreover, trait estimates from subthreshold criteria combinations often overlapped with diagnostic (at-threshold or higher) combinations, indicating that there were subthreshold combinations of criteria that indicated as much or more PD than did some combinations at the diagnostic threshold. These results suggest that counting the number of criteria an individual meets provides only a coarse estimation of his or her PD trait level. Implications for the assessment of polythetically defined mental disorders and for the PD proposal for the 5th edition of the Diagnostic and Statistical Manual of Mental Disorders are discussed.     

Gender bias in self-report personality disorder inventories

There has been considerable controversy and research regarding gender biases in the diagnosis of personality disorders, but few studies have explored whether personality disorder self-report inventories might contain gender biases. The current study investigated whether items from three commonly used inventories evidence a potential for gender bias. Subjects were from outpatient mental health clinics. Items were considered gender biased if they exhibited gender differences and failed to correlate with or, more importantly, correlated negatively with dysfunction. Thirty-eight items evidenced potential bias, the majority of which were from Narcissistic scales. The implications of the results for the clinical assessment of purportedly maladaptive personality traits and for the construction of personality disorder scales are discussed.     

A factor analysis and item response theory analysis of an employee honesty test

Although paper and pencil tests of employee honesty are becoming increasingly widespread in industry, a paucity of research exists regarding them. In a recent review of this literature, Sackett and Harris (1984) noted that scant psychometric evidence is available as to their merits or weaknesses. The aim of this paper is to report on the factor and item analysis of one such test. A principal axis solution and item response theory model (1-parameter) were used to examine the data. The factor analysis revealed four readily interpretable factors. With regard to the item analysis, the results indicated that on the whole most of the 40 items showed a reasonable fit to the model. The implications of this research are addressed.     

Content and form characteristics that predict discriminating power in personality items: an item response theory-based analysis

In two personality item pools that measure extraversion and neuroticism, the relations between three groups of itemmetric properties and the estimated discrimination indexes according to the two-parameter model are assessed. The itemmetric properties, which are conceptualized as dimensions, are: verbal complexity, ambiguity, and trait indicativity, and the hypotheses concerning their impact on the discriminating power are derived from a processing-stage model. The dimension of trait indicativity shows the clearest relations with the discriminating power in the expected direction. The implications of the results in the construction of personality measures are discussed.     

Two item response theory models for analysing normative forced‐choice personality items

This paper proposes two unidimensional item response theory (IRT) models for analysing normative forced‐choice personality items. Both models are derived from a common theoretical framework and arise as a result of different assumptions regarding the mechanism of choice. The simplest mechanism gives rise to the one‐parameter normal‐ogive model. The second mechanism gives rise to a new IRT model, which is closely related to the Coombs–Zinnes probabilistic unfolding model. The second model is compared theoretically to the normal‐ogive model in terms of item characteristic curves and amount of item information. Next, procedures for estimating the respondent and the item parameters in the second model are described. Finally, both models are empirically compared by using two well‐known personality measures.     

An application of item response theory analysis to alcohol, cannabis, and cocaine criteria in DSM-IV

Item response theory (IRT) is supplanting classical test theory as the basis for measures development. This study demonstrated the utility of IRT for evaluating DSM-IV diagnostic criteria. Data on alcohol, cannabis, and cocaine symptoms from 372 adult clinical participants interviewed with the Composite International Diagnostic Interview--Expanded Substance Abuse Module (CIDI-SAM) were analyzed with Mplus (B. Muthen & L. Muthen, 1998) and MULTILOG (D. Thissen, 1991) software. Tolerance and legal problems criteria were dropped because of poor fit with a unidimensional model. Item response curves, test information curves, and testing of variously constrained models suggested that DSM-IV criteria in the CIDI-SAM discriminate between only impaired and less impaired cases and may not be useful to scale case severity. IRT can be used to study the construct validity of DSM-IV diagnoses and to identify diagnostic criteria with poor performance.     

Gender bias in the diagnosis of personality disorders: the roles of base rates and social stereotypes

In all three studies described in this article, novices were trained to associate traits from the DSM-IV Cluster B personality disorder categories with four letters of the alphabet. Novices were also taught various gender associations (i.e., "base rate" information) with the categories (none, stereotype-consistent, or stereotype-inconsistent). Results showed that when no gender associations were taught (Study I), case gender did not affect ratings of case vignettes. When associations were consistent with social stereotypes (Study II), case gender influenced ratings in the expected direction. When associations were the opposite of social stereotypes (Study III), only the ratings for narcissistic and histrionic were affected. Across the three studies, these results suggested that subjects rated cases in accordance with known "base rates," but that the correspondence between base rates and stereotypic associations affected the consistency and magnitude of this base rate effect.