An investigation of hierarchical Bayes procedures in item response theory

Hierarchical Bayes procedures for the two-parameter logistic item response model were compared for estimating item and ability parameters. Simulated data sets were analyzed via two joint and two marginal Bayesian estimation procedures. The marginal Bayesian estimation procedures yielded consistently smaller root mean square differences than the joint Bayesian estimation procedures for item and ability estimates. As the sample size and test length increased, the four Bayes procedures yielded essentially the same result.The authors wish to thank the Editor and anonymous reviewers for their insightful comments and suggestions.     

An item factor analysis and item response theory-based revision of the Everyday Discrimination Scale

The Everyday Discrimination Scale (EDS), a widely used measure of daily perceived discrimination, is purported to be unidimensional, to function well among African Americans, and to have adequate construct validity. Two separate studies and data sources were used to examine and cross-validate the psychometric properties of the EDS. In Study 1, an exploratory factor analysis was conducted on a sample of African American law students (N = 589), providing strong evidence of local dependence, or nuisance multidimensionality within the EDS. In Study 2, a separate nationally representative community sample (N = 3,527) was used to model the identified local dependence in an item factor analysis (i.e., bifactor model). Next, item response theory (IRT) calibrations were conducted to obtain item parameters. A five-item, revised-EDS was then tested for gender differential item functioning (in an IRT framework). Based on these analyses, a summed score to IRT-scaled score translation table is provided for the revised-EDS. Our results indicate that the revised-EDS is unidimensional, with minimal differential item functioning, and retains predictive validity consistent with the original scale.     

An item response theory analysis of the narcissistic personality inventory

This research uses item response theory methods to evaluate the Narcissistic Personality Inventory (NPI; Raskin & Terry, 1988). Analyses using the 2-parameter logistic model were conducted on the total score and the Corry, Merritt, Mrug, and Pamp (2008) and Ackerman et?al. (2011) subscales for the NPI. In addition to offering precise information about the psychometric properties of the NPI item pool, these analyses generated insights that can be used to develop new measures of the personality constructs embedded within this frequently used inventory.     

An item response theory analysis of the Problem Gambling Severity Index

Increases in the availability of gambling heighten the need for a short screening measure of problem gambling. The Problem Gambling Severity Index (PGSI) is a brief measure that allows for the assessment of characteristics of gambling behavior and severity and its consequences. The authors evaluate the psychometric properties of the PGSI using item response theory methods in a representative sample of the urban adult population in South Africa (N = 3,000). The PGSI items were evaluated for differential item functioning (DIF) due to language translation. DIF was not detected. The PGSI was found to be unidimensional, and use of the nominal categories model provided additional information at higher values of the underlying construct relative to a simpler binary model. This study contributes to the growing literature supporting the PGSI as the screen of choice for assessing gambling problems in the general population.     

Aggregate item response analysis

A stochastic postulate is given for the multiple-item, successive-intervals scaling of populations. The logistic equivalent of this postulate provides an aggregate item response model in which a unidimensional submodel may be nested. This reduction provides a subtractive conjoint measurement of several items and stimuli on the same latent scale. Generalized-least-squares methods are used to estimate and test the multiple-item model, and its unidimensional reduction, on aggregate survey responses. The entire procedure is illustrated with an analysis of semantic-differential attitude data. This analysis exhibits an item selection procedure that is applicable to various social constructs.The authors dedicate this paper to the memory and contributions of Clyde Coombs.The programming and data analyses for the present paper were carried out by José Ventura of the Department of Industrial and Systems Engineering, and Jerry Meiten of the Department of Statistics, University of Florida.The study was also supported by the College of Business Administration, University of Florida, and the Faculty of Social Sciences, Hebrew University of Jerusalem.     

An introduction to mixture item response theory models

Mixture item response theory (IRT) allows one to address situations that involve a mixture of latent subpopulations that are qualitatively different but within which a measurement model based on a continuous latent variable holds. In this modeling framework, one can characterize students by both their location on a continuous latent variable as well as by their latent class membership. For example, in a study of risky youth behavior this approach would make it possible to estimate an individual's propensity to engage in risky youth behavior (i.e., on a continuous scale) and to use these estimates to identify youth who might be at the greatest risk given their class membership. Mixture IRT can be used with binary response data (e.g., true/false, agree/disagree, endorsement/not endorsement, correct/incorrect, presence/absence of a behavior), Likert response scales, partial correct scoring, nominal scales, or rating scales. In the following, we present mixture IRT modeling and two examples of its use. Data needed to reproduce analyses in this article are available as supplemental online materials at http://dx.doi.org/10.1016/j.jsp.2016.01.002.     

The impact of social desirability bias on the EPQ-R item scores: An item response theory analysis

This study assessed the potential influence of social desirability (SD) response bias on the E, N, and P EPQ-R scores at the level of individual items. The study was based on a bidimensional IRT model which was fitted in a large sample. This allowed a detailed analysis of both the internal validities of the items and the content of the items which were most affected by SD. The E items were least affected by SD, but the direction of the impact depended on the type of item. As expected, in the N and P cases the relations obtained were consistently negative, but the strength of the SD impact also depended considerably on the type of item. The P scale was the most problematic in terms of convergent and discriminant validity.     

An item response theory analysis of the Impulsive Behaviors Checklist for Adolescents

This study used item response theory (IRT) to examine the Impulsive Behaviors Checklist for Adolescents (IBCL-A) among 6,276 (67.7% girls) Chinese secondary school students. The IBCL-A included 15 maladaptive impulsive behaviors adapted from the Revised Diagnostic Interview for Borderlines. The authors obtained the severity and discrimination parameters for each item in the IBCL-A, examined differential item functioning across gender and age groups, and tested reliability and concurrent validity of the IBCL-A IRT-scaled score. Most items in the IBCL-A were the most accurate in assessing moderate to high levels of impulsivity and discriminated well among adolescents with varied levels of impulsivity. Differential item functioning emerged in several items across gender. The IRT-scaled score showed good construct validity and incremental predictive validity. Findings demonstrate the sound psychometric properties of the IBCL-A and support the clinical utility of this scale.     

An analysis of procedures that affect response variability

Response variability is sensitive to antecedent and consequent manipulations. Researchers have investigated inducement, direct production through reinforcement, and stimulus control of response variability. Recently, researchers have shown that lag reinforcement schedules reliably increase variability but may also produce higher‐order stereotypy. There has been limited investigation of appropriate variability levels and alternation between repetition and variation. In a three‐part study, we evaluated levels of variability across a group of children, the effects of various procedures on producing response variability and novelty, and the use of schedule‐correlated stimuli for producing rapid alternation between repetition and variation. In Study 1, there was a nearly bimodal distribution of children emitting either low or high variability. In Study 2, for most children, fixed lag 4 and variable lag 4 schedules produced the highest levels of variability and novelty. In Study 3, responding was brought under control of schedule‐correlated stimuli, allowing for rapid alternation between repetition and variation.     

Introduction to bifactor polytomous item response theory analysis

A bifactor item response theory model can be used to aid in the interpretation of the dimensionality of a multifaceted questionnaire that assumes continuous latent variables underlying the propensity to respond to items. This model can be used to describe the locations of people on a general continuous latent variable as well as on continuous orthogonal specific traits that characterize responses to groups of items. The bifactor graded response (bifac-GR) model is presented in contrast to a correlated traits (or multidimensional GR model) and unidimensional GR model. Bifac-GR model specification, assumptions, estimation, and interpretation are demonstrated with a reanalysis of data (Campbell, 2008) on the Shared Activities Questionnaire. We also show the importance of marginalizing the slopes for interpretation purposes and we extend the concept to the interpretation of the information function. To go along with the illustrative example analyses, we have made available supplementary files that include command file (syntax) examples and outputs from flexMIRT, IRTPRO, R, Mplus, and STATA. Supplementary data to this article can be found online at http://dx.doi.org/10.1016/j.jsp.2016.11.001. Data needed to reproduce analyses in this article are available as supplemental materials (online only) in the Appendix of this article.     

An item response theory analysis of self-report measures of adult attachment

Self-report measures of adult attachment are typically scored in ways (e.g., averaging or summing items) that can lead to erroneous inferences about important theoretical issues, such as the degree of continuity in attachment security and the differential stability of insecure attachment patterns. To determine whether existing attachment scales suffer from scaling problems, the authors conducted an item response theory (IRT) analysis of 4 commonly used self-report inventories: Experiences in Close Relationships scales (K. A. Brennan, C. L. Clark, & P. R. Shaver, 1998), Adult Attachment Scales (N. L. Collins & S. J. Read, 1990), Relationship Styles Questionnaire (D. W. Griffin & K. Bartholomew, 1994) and J. Simpson's (1990) attachment scales. Data from 1,085 individuals were analyzed using F. Samejima's (1969) graded response model. The authors' findings indicate that commonly used attachment scales can be improved in a number of important ways. Accordingly, the authors show how IRT techniques can be used to develop new attachment scales with desirable psychometric properties.     

An item response model with internal restrictions on item difficulty

An IRT model based on the Rasch model is proposed for composite tasks, that is, tasks that are decomposed into subtasks of different kinds. There is one subtask for each component that is discerned in the composite tasks. A component is a generic kind of subtask of which the subtasks resulting from the decomposition are specific instantiations with respect to the particular composite tasks under study. The proposed model constrains the difficulties of the composite tasks to be linear combinations of the difficulties of the corresponding subtask items, which are estimated together with the weights used in the linear combinations, one weight for each kind of subtask. Although the model does not belong to the exponential family, its parameters can be estimated using conditional maximum likelihood estimation. The approach is demonstrated with an application to spelling tasks. We thank Eric Maris for his helpful comments.     

An item response theory/confirmatory factor analysis of the Autobiographical Memory Test

The Autobiographical Memory Test (AMT) is used to assess the degree of specificity of autobiographical memory. The AMT usually contains cue words of both positive and negative valence, but it is unclear whether these valences form separate factors or not. Accordingly, confirmatory factor analysis assessed whether the AMT measures one overall factor, or whether different cue types are related to different factors. Results were consistent across three datasets (N = 333, N = 405, and N = 336). A one-factor model fitted each dataset well, which suggests that responses to positive and negative cues are related to the one construct. In addition, item response theory analyses showed that the AMT is most precise for people who score low on memory specificity. Implications for using the AMT with high-functioning samples are discussed.     

Bayesian item fit analysis for unidimensional item response theory models

Assessing item fit for unidimensional item response theory models for dichotomous items has always been an issue of enormous interest, but there exists no unanimously agreed item fit diagnostic for these models, and hence there is room for further investigation of the area. This paper employs the posterior predictive model‐checking method, a popular Bayesian model‐checking tool, to examine item fit for the above‐mentioned models. An item fit plot, comparing the observed and predicted proportion‐correct scores of examinees with different raw scores, is suggested. This paper also suggests how to obtain posterior predictive p‐values (which are natural Bayesian p‐values) for the item fit statistics of Orlando and Thissen that summarize numerically the information in the above‐mentioned item fit plots. A number of simulation studies and a real data application demonstrate the effectiveness of the suggested item fit diagnostics. The suggested techniques seem to have adequate power and reasonable Type I error rate, and psychometricians will find them promising.     

An application of item response theory analysis to alcohol, cannabis, and cocaine criteria in DSM-IV

Item response theory (IRT) is supplanting classical test theory as the basis for measures development. This study demonstrated the utility of IRT for evaluating DSM-IV diagnostic criteria. Data on alcohol, cannabis, and cocaine symptoms from 372 adult clinical participants interviewed with the Composite International Diagnostic Interview--Expanded Substance Abuse Module (CIDI-SAM) were analyzed with Mplus (B. Muthen & L. Muthen, 1998) and MULTILOG (D. Thissen, 1991) software. Tolerance and legal problems criteria were dropped because of poor fit with a unidimensional model. Item response curves, test information curves, and testing of variously constrained models suggested that DSM-IV criteria in the CIDI-SAM discriminate between only impaired and less impaired cases and may not be useful to scale case severity. IRT can be used to study the construct validity of DSM-IV diagnoses and to identify diagnostic criteria with poor performance.     

An application of item response theory to the DSM-III-R criteria for borderline personality disorder

This paper summarizes results from analyses of the DSM criteria for borderline personality disorder (BPD) using models from item response theory (IRT). The study sample consisted of 353 participants, the majority of whom were psychiatric patients. Confirmatory factor analysis showed that a one-factor model provided the best fit to the data. All the DSM BPD criteria had moderate or higher item discrimination parameters, indicating that all items contributed meaningful information in assessing BPD. Item information functions revealed that the BPD criteria as a whole were useful for capturing BPD traits in the moderately severe to severe range, but that they performed less well in the less severe range. The general conclusion is that the criteria do represent a coherent syndrome and that further research on the informational value of the individual criteria would be useful.     

An extension of commonly used toilet-training procedures to children with autism spectrum disorder

The current study evaluated a toilet-training treatment package described by Greer et al. (2016) with children diagnosed with autism spectrum disorder (ASD). Most of the current research on toilet-training interventions for children with ASD are replications and modifications of Azrin and Foxx (1971) or (more recently) LeBlanc et al. (2005). However, these procedures are composed of components that are not included in studies with typically developing (TD) children. For example, Greer et al. evaluated the effectiveness of three typical components within a toilet-training package, mostly with TD participants: a 30-min sit schedule, placing participants in underwear, and differential reinforcement. The primary purpose of the current study was to replicate and extend the treatment package described by Greer et al. to children with ASD. A secondary purpose was to evaluate modifications necessary for individualized toilet training when the commonly used components were ineffective. The results of Greer et al. were replicated for 11 participants with ASD in the current study, suggesting that intensive toileting interventions (e.g., interventions requiring overcorrection, reprimands, and dense sit schedules) may only be necessary for a subset of individuals with ASD.     

The unique correspondence of the item response function and item category response functions in polytomously scored item response models

The item response function (IRF) for a polytomously scored item is defined as a weighted sum of the item category response functions (ICRF, the probability of getting a particular score for a randomly sampled examinee of ability ). This paper establishes the correspondence between an IRF and a unique set of ICRFs for two of the most commonly used polytomous IRT models (the partial credit models and the graded response model). Specifically, a proof of the following assertion is provided for these models: If two items have the same IRF, then they must have the same number of categories; moreover, they must consist of the same ICRFs. As a corollary, for the Rasch dichotomous model, if two tests have the same test characteristic function (TCF), then they must have the same number of items. Moreover, for each item in one of the tests, an item in the other test with an identical IRF must exist. Theoretical as well as practical implications of these results are discussed.This research was supported by Educational Testing Service Allocation Projects No. 79409 and No. 79413. The authors wish to thank John Donoghue, Ming-Mei Wang, Rebecca Zwick, and Zhiliang Ying for their useful comments and discussions. The authors also wish to thank three anonymous reviewers for their comments.