The Short Mood and Feelings Questionnaire (SMFQ): A Unidimensional Item Response Theory and Categorical Data Factor Analysis of Self-Report Ratings from a Community Sample of 7-through 11-Year-Old Children

Item response theory (IRT) and categorical data factor analysis (CDFA) are complementary methods for the analysis of the psychometric properties of psychiatric measures that purport to measure latent constructs. These methods have been applied to relatively few child and adolescent measures. We provide the first combined IRT and CDFA analysis of a clinical measure (the Short Mood and Feelings Questionnaire—SMFQ) in a community sample of 7-through 11-year-old children. Both latent variable models supported the internal construct validity of a single underlying continuum of severity of depressive symptoms. SMFQ items discriminated well at the more severe end of the depressive latent trait. Item performance was not affected by age, although age correlated significantly with latent SMFQ scores suggesting that symptom severity increased within the age period of 7–11. These results extend existing psychometric studies of the SMFQ and confirm its scaling properties as a potential dimensional measure of symptom severity of childhood depression in community samples.     

The Spiritual Transcendence Index: An Item Response Theory Analysis

ABSTRACT

In an attempt to measure understudied dimensions of spirituality, recent efforts have focused on the transcendent dimension of spirituality. The Spiritual Transcendence Index (STI) was developed to assess a perceived experience of the sacred that affects one’s ability to transcend life’s difficulties. The main focus of the current study was to investigate the psychometric properties of the STI by utilizing the microscopic item-level examination tools unique in item response theory (IRT), as well as its scale-level exploration devices for psychometric properties of an assessment measure. IRT analyses were conducted to investigate the STI’s psychometric properties across samples (N = 712) including how well the measure assesses the latent construct, spiritual transcendence, from the low to high range of the construct. The findings confirm that the 8-item index is a single factor that assesses the latent construct, spiritual transcendence. Instead of the original 6-category version, these findings support a 4-category response version; the 3 categories of disagreement may be collapsed into a single category. These findings not only inform the refinement of the STI but also highlight an important psychometric approach for the refinement of spirituality/religiousness measures, especially those with ceiling effects.     

Item response theory and the measurement of clinical change

An instrument's sensitivity to detect individual-level change is an important consideration for both psychometric and clinical researchers. In this article, we develop a cognitive problems measure and evaluate its sensitivity to detect change from an item response theory (IRT) perspective. After illustrating assumption checking and model fit assessment, we detail 4 features of IRT modeling: (a) the scale information curve and its relation to the bandwidth of measurement precision, (b) the scale response curve and how it is used to link the latent trait metric with the raw score metric, (c) content-based versus norm-based score referencing, and (d) the level of measurement of the latent trait scale. We conclude that IRT offers an informative, alternative framework for understanding an instrument's psychometric properties and recommend that IRT analyses be considered prior to investigations of change, growth, or the effectiveness of clinical interventions.     

混合IRT潜在模型及其应用轨迹

项目反应理论是测量被试潜在特质的现代测量理论, 潜在类别分析是基于模型的潜在特质分类技术。混合项目反应理论将项目反应理论与潜在类别分析相结合, 能够同时对被试分类并量化其潜在特质。在阐述混合项目反应理论概念、原理的基础上, 介绍了MRM、mNRM和mPCM等几种常见混合模型及其参数估计方法, 并从心理与行为特征分类、项目功能差异检测、测验效度评价等方面评述了其在心理测验中的应用发展轨迹。     

Advances in Clinical Personality Measurement: An Item Response Theory Analysis of the MMPI-2 PSY-5 Scales

Item response theory (IRT) provides valuable methods for the analysis of the psychometric properties of a psychological measure. To date, however, these methods have not been used frequently by personality assessment researchers, in part because many researchers have not been introduced to the methods and in part because most of the development of IRT has taken place in applied education assessment settings, resulting in terminology that is ability focused rather than trait focused. The purpose of this article is twofold. First, an overview of IRT is presented, highlighting the concepts of the three-parameter IRT model, item and test information, and conditional standard error of measurement. Second, the psychometric properties of the (MMPI-2) PSY-5 scales are examined to demonstrate IRT's value.     

Item response theory analysis of diagnostic criteria for alcohol and cannabis use disorders in adolescents: implications for DSM-V

Item response theory (IRT) has advantages over classical test theory in evaluating diagnostic criteria. In this study, the authors used IRT to characterize the psychometric properties of Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV; American Psychiatric Association, 1994) alcohol and cannabis use disorder symptoms among 472 clinical adolescents. For both substances, DSM-IV symptoms fit a model specifying a unidimensional latent trait of problem severity. Threshold (severity) parameters did not distinguish abuse and dependence symptoms. Abuse symptoms of legal problems and hazardous use, and dependence symptoms of tolerance, unsuccessful attempts to quit, and physical-psychological problems, showed relatively poor discrimination of problem severity. There were gender differences in thresholds for hazardous use, legal problems, and physical-psychological problems. The results illustrate limitations of DSM-IV criteria for alcohol and cannabis use disorders when applied to adolescents. The development process for the fifth edition (DSM-V) should be informed by statistical models such as those used in this study.     

Looking Closer at the Effects of Framing on Risky Choice: An Item Response Theory Analysis

Item response theory (IRT) methodology allowed an in-depth examination of several issues that would be difficult to explore using traditional methodology. IRT models were estimated for 4 risky-choice items, answered by students under either a gain or loss frame. Results supported the typical framing finding of risk-aversion for gains and risk-seeking for losses but also suggested that a latent construct we label preference for risk was influential in predicting risky choice. Also, the Asian Disease item, most often used in framing research, was found to have anomalous statistical properties when compared to other framing items.     

The Accuracy of the Life Orientation Test–Revised (LOT–R) in Measuring Dispositional Optimism: Evidence From Item Response Theory Analyses

The accuracy of the Life Orientation Test–Revised (LOT–R) in measuring dispositional optimism was investigated applying item response theory (IRT). The study was conducted on a sample of 484 university students (62% males, M age = 22.79 years, SD = 5.63). After testing the 1-factor structure of the scale, IRT was applied to evaluate the functioning of the LOT–R along the pessimism–optimism continuum. Item parameter estimates and the test information function showed that each item and the global scale satisfactorily measured the latent trait. Referring to the IRT estimated trait levels, the validity of the LOT–R was studied examining the relationships between dispositional optimism and psychological well-being, sense of mastery, and sense of coherence. Overall findings based on IRT analyses provide evidence of the accuracy of the LOT–R and suggest possible modifications of the scale to improve the assessment of dispositional optimism.     

A discussion of modern versus traditional psychometrics as applied to personality assessment scales

Item response theory (IRT) methods are used by large testing firms, state agencies, and school districts to construct, analyze, and score most major aptitude, achievement, proficiency, entrance, and professional licensure exams. Personality assessment, in contrast, has not generally adopted these more powerful, modern psychometric techniques. We evaluate the possible role of IRT in the personality domain by highlighting key areas in which IRT and traditional methods differ. Although we conclude that IRT has a significant role to play in future personality measurement, there are many systemic and technical barriers to its routine application.     

A Doubly Latent Space Joint Model for Local Item and Person Dependence in the Analysis of Item Response Data

Item response theory (IRT) is one of the most widely utilized tools for item response analysis; however, local item and person independence, which is a critical assumption for IRT, is often violated in real testing situations. In this article, we propose a new type of analytical approach for item response data that does not require standard local independence assumptions. By adapting a latent space joint modeling approach, our proposed model can estimate pairwise distances to represent the item and person dependence structures, from which item and person clusters in latent spaces can be identified. We provide an empirical data analysis to illustrate an application of the proposed method. A simulation study is provided to evaluate the performance of the proposed method in comparison with existing methods.

     

A Discussion of Modern Versus Traditional Psychometrics As Applied to Personality Assessment Scales

Item response theory (IRT) methods are used by large testing firms, state agencies, and school districts to construct, analyze, and score most major aptitude, achievement, proficiency, entrance, and professional licensure exams. Personality assessment, in contrast, has not generally adopted these more powerful, modern psychometric techniques. We evaluate the possible role of IRT in the personality domain by highlighting key areas in which IRT and traditional methods differ. Although we conclude that IRT has a significant role to play in future personality measurement, there are many systemic and technical barriers to its routine application.     

A Psychometric Analysis of the Trait Emotional Intelligence Questionnaire–Short Form (TEIQue–SF) Using Item Response Theory

Trait emotional intelligence refers to a constellation of emotional self-perceptions located at the lower levels of personality hierarchies. In 2 studies, we sought to examine the psychometric properties of the Trait Emotional Intelligence Questionnaire–Short Form (TEIQue–SF; Petrides, 2009) using item response theory (IRT). Study 1 (N= 1,119, 455 men) showed that most items had good discrimination and threshold parameters and high item information values. At the global level, the TEIQue–SF showed very good precision across most of the latent trait range. Study 2 (N= 866, 432 men) used similar IRT techniques in a new sample based on the latest version of the TEIQue–SF (version 1.50). Results replicated Study 1, with the instrument showing good psychometric properties at the item and global level. Overall, the 2 studies suggest the TEIQue-SF can be recommended when a rapid assessment of trait emotional intelligence is required.     

An application of item response theory analysis to alcohol, cannabis, and cocaine criteria in DSM-IV

Item response theory (IRT) is supplanting classical test theory as the basis for measures development. This study demonstrated the utility of IRT for evaluating DSM-IV diagnostic criteria. Data on alcohol, cannabis, and cocaine symptoms from 372 adult clinical participants interviewed with the Composite International Diagnostic Interview--Expanded Substance Abuse Module (CIDI-SAM) were analyzed with Mplus (B. Muthen & L. Muthen, 1998) and MULTILOG (D. Thissen, 1991) software. Tolerance and legal problems criteria were dropped because of poor fit with a unidimensional model. Item response curves, test information curves, and testing of variously constrained models suggested that DSM-IV criteria in the CIDI-SAM discriminate between only impaired and less impaired cases and may not be useful to scale case severity. IRT can be used to study the construct validity of DSM-IV diagnoses and to identify diagnostic criteria with poor performance.     

An Introduction to Network Psychometrics: Relating Ising Network Models to Item Response Theory Models

In recent years, network models have been proposed as an alternative representation of psychometric constructs such as depression. In such models, the covariance between observables (e.g., symptoms like depressed mood, feelings of worthlessness, and guilt) is explained in terms of a pattern of causal interactions between these observables, which contrasts with classical interpretations in which the observables are conceptualized as the effects of a reflective latent variable. However, few investigations have been directed at the question how these different models relate to each other. To shed light on this issue, the current paper explores the relation between one of the most important network models—the Ising model from physics—and one of the most important latent variable models—the Item Response Theory (IRT) model from psychometrics. The Ising model describes the interaction between states of particles that are connected in a network, whereas the IRT model describes the probability distribution associated with item responses in a psychometric test as a function of a latent variable. Despite the divergent backgrounds of the models, we show a broad equivalence between them and also illustrate several opportunities that arise from this connection.     

Stress assessment during adolescence: Psychometric properties and measurement invariance of the Student Stress Inventory-Stress Manifestations across gender and age

The main purpose of this study was to analyze the psychometric properties and measurement invariance across gender and age of the Student Stress Inventory-Stress Manifestations (SSI-SM) scores in a large sample of adolescents. The sample was comprised by a total of 1108 students (482 were male), with a mean age of 14.61 years (SD = 1.71). The results indicated that the SSI-SM scores presented adequate psychometric properties from both classical test theory and Item Response Theory (IRT). Confirmatory factorial analysis (CFA), showed that both the bifactor model and a three-factor model (emotional, physiological, and behavioural) were adequate. Multi-group CFA showed that the three-factor model had strong measurement invariance across gender and age. Statistically significant differences in gender were found between latent means as well as raw scores of SSI-SM. Ordinal alpha was .78 for Physiological, .90 for the Emotional, and .79 for the Behavioural subscales. Using IRT, the SSI-SM provides more accuracy information at the medium level of the latent trait. SSI-SM subscales were associated with emotional and behavioural problems. These results provided new sources of validity evidence of the SSI-SM scores in adolescents from general population. The SSI-SM appears to be a useful, brief, and easy to administrate self-report instrument for the screening of stress manifestations at school and educational settings.     

Item Response Theory Analyses of the Parent and Teacher Ratings of the DSM-IV ADHD Rating Scale

The graded response model (GRM), which is based on item response theory (IRT), was used to evaluate the psychometric properties of the inattention and hyperactivity/impulsivity symptoms in an ADHD rating scale. To accomplish this, parents and teachers completed the DSM-IV ADHD Rating Scale (DARS; Gomez et al., Journal of Child Psychology and Psychiatry, 40, 265–274, 1999) for a group of 1,475 primary school-aged children. The results for the discrimination parameters showed that all symptoms for both groups of respondents were generally good for discriminating their respective latent traits. For virtually all symptoms, their threshold values showed moderate to large increases in the level of the latent trait at each subsequent response dichotomy, with the symptoms being especially good at representing the appropriate traits from mean to moderately high trait levels. The item information function values for most symptoms indicated reasonable reliability from, approximately, the mean trait levels to moderately high trait levels. These findings indicate good psychometric properties for the parent and teacher ratings of the DARS. The implications of the findings for the use of the DARS and other similar scales are discussed.     

Robust Measurement via A Fused Latent and Graphical Item Response Theory Model

Item response theory (IRT) plays an important role in psychological and educational measurement. Unlike the classical testing theory, IRT models aggregate the item level information, yielding more accurate measurements. Most IRT models assume local independence, an assumption not likely to be satisfied in practice, especially when the number of items is large. Results in the literature and simulation studies in this paper reveal that misspecifying the local independence assumption may result in inaccurate measurements and differential item functioning. To provide more robust measurements, we propose an integrated approach by adding a graphical component to a multidimensional IRT model that can offset the effect of unknown local dependence. The new model contains a confirmatory latent variable component, which measures the targeted latent traits, and a graphical component, which captures the local dependence. An efficient proximal algorithm is proposed for the parameter estimation and structure learning of the local dependence. This approach can substantially improve the measurement, given no prior information on the local dependence structure. The model can be applied to measure both a unidimensional latent trait and multidimensional latent traits.     

Analysis of the Reliability of the Leadership Practices Inventory in the Item Response Theory Framework

The paper examines the psychometric properties of the leadership practices inventory (LPI) in the framework of item response theory (IRT). The LPI assesses five dimensions (i.e. leadership practices) of transformational leadership and consists of 30 items. IRT is a model‐based theory that relates the characteristics of questionnaire items (item parameters) and characteristics of individuals (latent variables) to the probability of choosing each of the response categories. The theory does not assume that the instrument is equally reliable for all levels of the latent variable examined. Samejima's graded response model was used to estimate LPI item characteristics, such as item difficulty and item discrimination power. The results show that some items are redundant in the sense they contribute little to the overall precision of the instrument. Moreover, the LPI seems to be most precise and reliable for respondents with low to medium leadership competence, whereas it becomes increasingly unreliable for high‐quality leaders. These findings suggest that the LPI is best used for training and development purposes, but not for leader selection purposes.     

The Loneliness Questionnaire-Short Version: an evaluation of reverse-worded and non-reverse-worded items via item response theory

Although reverse-worded items have often been incorporated in scale construction to minimize the effects of acquiescent reporting biases, some researchers have more recently begun questioning this approach and wondering whether the advantages associated with incorporating reverse-worded items is worth the complexities that they bring to measures (e.g., Brown, 2003 ; Marsh, 1996 ). In this study, we used item response theory (IRT) to determine whether there is statistical justification to eliminate the reverse-worded items (e.g., "I have lots of friends") from the Loneliness Questionnaire (LQ; Asher, Hymel, & Renshaw, 1984) and retain only the non-reverse-worded items (e.g., "I'm lonely") to inform the provision of a shortened LQ version. Using a large sample of children (Grades 2-7; n = 6,784) and adolescents (Grades 8-12; n = 4,941), we examined the psychometric properties of the 24-item LQ and found support for retaining the 9 non-reverse-worded LQ items to make up a shortened measure of loneliness in youth. We found that the non-reverse-worded items were associated with superior psychometric properties relative to the reverse-worded items with respect to reliability and IRT parameters (e.g., discrimination and item information). A 3-point Likert-type scale was also found to be more suitable for measuring loneliness across both children and adolescents compared to the original 5-point scale. The relative contributions of reverse-worded and non-reverse-worded items in scale development for youth instruments are also discussed.