Empirical validation and psychometric evaluation of the Brief Fear of Negative Evaluation Scale in patients with social anxiety disorder

The Brief Fear of Negative Evaluation Scale (BFNE; M. R. Leary, 1983a) is often used to assess fear of negative evaluation, the core feature of social anxiety disorder. However, few studies have examined its psychometric properties in large samples of socially anxious patients. Although the BFNE yields a single total score, confirmatory factor analysis indicated a 2-factor solution to be more appropriate, with the 1st factor consisting of all straightforwardly worded items (BFNE-S) and the 2nd of all reverse-scored items (BFNE-R). Support was obtained for the convergent and discriminant validity of the BFNE and BFNE-S, but not the BFNE-R. These results suggest that standard scoring of the BFNE may not be optimal for patients with social anxiety disorder.     

Brief version of the Fear of Negative Evaluation Scale - Straightforward Items (BFNE-S): psychometric properties in a Spanish population

The aim of this study was to examine the psychometric properties of the Brief version of the Fear of Negative Evaluation Scale - Straightforward Items (BFNE-S) in a non-clinical Spanish population. Rodebaugh et al. (2004) recommend the use of this scale composed of 8 straightforwardly-worded items, instead of the 12-item version of the BFNE. The sample consisted of 542 undergraduate students, 71.3% of whom were women and 28.7% were men; the mean age was 21.71 (4.78) years. Exploratory factor analysis produced one factor which accounted for 51.28% of variance. The internal consistency of the scale was alpha = .89. The BFNE-S correlated with the Social Avoidance and Distress Scale (r = .44), the Personal Report of Confidence as Speaker Modified (r = .44), the Public Speaking Self-Efficacy Questionnaire (r = -.38) and both subscales of the Self-Statements during Public Speaking (SSPS-P r = -.22; SSPS-N r = .53). ANOVAs revealed significant differences in the BFNE-S amongst a non-clinical population, persons suffering from specific social phobia, non-generalized social phobia and generalized social phobia.     

The Short Mood and Feelings Questionnaire (SMFQ): A Unidimensional Item Response Theory and Categorical Data Factor Analysis of Self-Report Ratings from a Community Sample of 7-through 11-Year-Old Children

Item response theory (IRT) and categorical data factor analysis (CDFA) are complementary methods for the analysis of the psychometric properties of psychiatric measures that purport to measure latent constructs. These methods have been applied to relatively few child and adolescent measures. We provide the first combined IRT and CDFA analysis of a clinical measure (the Short Mood and Feelings Questionnaire—SMFQ) in a community sample of 7-through 11-year-old children. Both latent variable models supported the internal construct validity of a single underlying continuum of severity of depressive symptoms. SMFQ items discriminated well at the more severe end of the depressive latent trait. Item performance was not affected by age, although age correlated significantly with latent SMFQ scores suggesting that symptom severity increased within the age period of 7–11. These results extend existing psychometric studies of the SMFQ and confirm its scaling properties as a potential dimensional measure of symptom severity of childhood depression in community samples.     

Unidimensionality and bandwidth in the Center for Epidemiologic Studies Depression (CES-D) Scale

In this study, we compared classical test theory (CTT) and item response theory (IRT) approaches in analyzing the Center for Epidemiological Studies Depression (CES-D) Scale (Radloff, 1977). Standard item analyses, as well as Rasch (1960) analyses, both revealed item departures from unidimensionality in a sample of 2,455 older persons responding to the CES-D. Positive affect items in the scale performed poorly overall, their removal reducing the scale's bandwidth only slightly. Modeling depression scores derived from Rasch measures and raw totals showed subtle but important differences for statistical inference. The assessment of depressive risk was slightly enhanced by using 16-item scale measures obtained from the results of the Rasch analysis as the dependent variable. Confirmatory factor analysis and parallel analysis verified the advantages of removing positively worded items. IRT and CTT techniques proved to be complementary in this study and can be usefully combined to improve measuring depression.     

Item response theory analyses of the Delis-Kaplan Executive Function System card sorting subtest

In the current study, we examined the dimensionality of the 16-item Card Sorting subtest of the Delis-Kaplan Executive Functioning System assessment in a sample of 264 native English-speaking children between the ages of 9 and 15 years. We also tested for measurement invariance for these items across age and gender groups using item response theory (IRT). Results of the exploratory factor analysis indicated that a two-factor model that distinguished between verbal and perceptual items provided the best fit to the data. Although the items demonstrated measurement invariance across age groups, measurement invariance was violated for gender groups, with two items demonstrating differential item functioning for males and females. Multigroup analysis using all 16 items indicated that the items were more effective for individuals whose IRT scale scores were relatively high. A single-group explanatory IRT model using 14 non-differential item functioning items showed that for perceptual ability, females scored higher than males and that scores increased with age for both males and females; for verbal ability, the observed increase in scores across age differed for males and females. The implications of these findings are discussed.     

Selecting the most informative items in the IIP scales for personality disorders: an application of item response theory.

The first goal of the present analyses was to shorten the five scales (Pilkonis, P. A., Kim, Y., Proietti, J. M., & Barkham, M. [1996]. Journal of Personality Disorders, 10, 355-369) for personality disorders (PDs) developed from the Inventory of Interpersonal Problems (IIP), thereby increasing their attractiveness for screening purposes. The second goal was to illustrate, for more general purposes, the utility of item response theory (IRT) for such scale refinement. IRT analyses were performed using data collected from six different samples (N = 1149) at five sites and a two-parameter (2P) graded model designed for multiple response items like those on the IIP. The five most informative items from each scale were identified, based on the magnitude of item discrimination parameters and the range and elevation of individual item information functions. Preliminary analyses of the reliability and validity of the short forms of the scales (totaling 25 items) supported their value as alternatives to the longer forms (consisting of 47 items), although definitive tests of their psychometric properties await crossvalidation in independent samples. Analyses of the quality receiver operating characteristics (QROC) of the long and short forms showed that both versions can be useful in predicting the presence versus absence of any PD diagnosis arrived at by using either a "best estimate" clinical consensus method or a structured Axis II interview.     

Item response theory in personality assessment: a demonstration using the MMPI-2 depression scale

Item response theory (IRT) analyses have, over the past 3 decades, added much to our understanding of the relationships among and characteristics of test items, as revealed in examinees response patterns. Assessment instruments used outside the educational context have only infrequently been analyzed using IRT, however. This study demonstrates the relevance of IRT to personality data through analyses of Scale 2 (the Depression Scale) on the revised Minnesota Multiphasic Personality Inventory (MMPI-2). A rich set of hypotheses regarding the items on this scale, including contrasts among the Harris-Lingoes and Wiener-Harmon subscales and differences in the items measurement characteristics for men and women, are investigated through the IRT analyses.     

The Psychometric Properties of an Internet-Administered Version of the Depression Anxiety and Stress Scales (DASS) in a Sample of Dutch Adults

Psychometric work on the widely used Depression Anxiety and Stress Scales (DASS) has mostly used classical psychometrics and ignored common internet-administered versions. Therefore, the present study used not only classical, but also modern psychometrics based on item response theory (IRT) to evaluate an internet-administered version of the DASS (Dutch translation). Internet-administered DASS data were collected as part of a large internet-based study in the Dutch adult population (n = 7972). Initially, external correlates (i.e. demographics other measures) and some classical psychometrics (internal consistency, convergent/divergent validity) of the DASS scales were evaluated. Next, IRT was used to investigate the scales’ dimensionality, discrimination and item-functioning. Finally, the DASS depression scale was further investigated by linking it to the more clinically-oriented Quick Inventory of Depressive Symptomatology (QIDS) using item response theory (IRT). Initial classical psychometric analyses supported the scales’ internal consistency (alpha = 0.94–0.98) and convergent/divergent validity. IRT analyses showed that each of the DASS scales was only suitable to measure variations in a very narrow and rather mild severity range. Linking the DASS depression scale with the QIDS also showed that the DASS depression scale discriminated best in the mild-moderate severity range, but not at higher severity levels that were covered by the QIDS. In conclusion, the scales of the internet-administered DASS show good internal consistency and validity. However, users should be aware that the scales discriminate best at mild-moderate severity ranges in the general population.     

Is it prudent to administer all items for each Child Behavior Checklist cross-informant syndrome? Evaluating the psychometric properties of the Youth Self-Report dimensions with confirmatory factor analysis and item response theory

Through surveying of children in 10 nations with parent, teacher, and Youth Self-Report (YSR) forms of the Child Behavior Checklist (CBCL), cross-informant syndromes (CISs) were derived and cross-validated by sample-dependent methodology. Generalizing CBCL syndromes and norms to nations excluded from its normative sample is problematic. This study used confirmatory factor analyses (CFAs) to test factor model fit for CISs on the YSR responses of 625 Jamaican children ages 11 to 18 years. Item response theory (IRT), a sample-independent methodology, was used to estimate the psychometric properties of individual items on each dimension. CFAs indicated poor to moderate model-to-data fit. Across all syndromes, IRT analyses revealed that more than 3/4 of the cross-informant items yielded little information. Eliminating such items could be cost effective in terms of administration time yet improve the measures discrimination across syndrome severity levels.     

分类数据测量等价性检验方法及其比较：项目阈值(难度)参数的组间差异性检验

测量工具满足等价性是进行多组比较的前提, 测量等价性的检验方法主要有基于CFA的多组比较法和基于IRT的DIF检验两类方法。文章比较了单维测验情境下基于CCFA的DIFFTEST检验方法和基于IRT模型的IRT-LR检验方法, 以及多维测验情境下DIFFTEST和基于MIRT的卡方检验方法的差异。通过模拟研究的方法, 比较了几种方法的检验力和第一类错误, 并考虑了样本总量、样本量的组间均衡性、测验长度、阈值差异大小以及维度间相关程度的影响。研究结果表明：(1)在单维测验下, IRT-LR是比DIFFTEST更为严格的检验方法; 多维测验下, 在测验较长、测验维度之间相关较高时, MIRT-MG比DIFFTEST更容易检验出项目阈值的差异, 而在测验长度较短、维度之间相关较小时, DIFFTEST的检验力反而略高于MIRT-MG方法。(2)随着阈值差值增加, DIFFTEST、IRT-LR和MIRT-MG三种方法的检验力均在增加, 当阈值差异达到中等或较大时, 三种方法都可以有效检验出测验阈值的不等价性。(3)随着样本总量增加, DIFFTEST、IRT-LR和MIRT-MG方法的检验力均在增加; 在总样本量不变, 两组样本均衡情况下三种方法的检验力均高于不均衡的情况。(4)违背等价性题目个数不变时, 测验越长DIFFTEST的检验力会下降, 而IRT-LR和MIRT-MG检验力则上升。(5) DIFFTEST方法的一类错误率平均值接近名义值0.05; 而IRT-LR和MIRT-MG方法的一类错误率平均值远低于0.05。     

Toward a hierarchical model of criminal thinking: evidence from item response theory and confirmatory factor analysis

Item response theory (IRT) methods were applied to items from the 80-item Psychological Inventory of Criminal Thinking Styles (PICTS; G. D. Walters, 1995) to determine how well they measure the latent trait of criminal thinking in a group of 2,872 male medium security prison inmates. Preliminary analyses revealed that the 64 PICTS thinking style items, 32 PICTS proactive criminal thinking items, and 24 PICTS reactive criminal thinking items were sufficiently unidimensional to meet the local independence requirements of IRT. The PICTS was fitted to a 2-parameter logistic-graded response IRT model, the results of which showed that the 8 items measuring denial of harm (Sentimentality) displayed weak discrimination (a < 0.5), whereas most of the proactive and reactive items displayed moderate to good discrimination (a > 1.0). Information function analysis revealed that all 3 components of a hierarchical model of criminal thinking--PICTS total scale, PICTS proactive factor, and PICTS reactive factor--displayed greater precision at higher rather than lower levels of the trait dimension. The study findings indicate that items from the PICTS Sentimentality scale do a poor job of measuring general criminal thinking, whereas items from the other 7 PICTS thinking style scales provide their most precise estimates at the upper end of the trait dimension.     

The factor structures of the STEM and the STEU

The factor structures of two recently developed measures of emotional intelligence, the Situational Test of Emotional Understanding and Situational Test of Emotion Management (STEU, STEM; MacCann & Roberts, 2008) were examined. The results did not support a factor structure of either measure’s subscales indicated by the approach used in developing the test items, and examination of the factors obtained using parallel analysis to determine the number of factors to extract did not yield interpretable factors. These findings suggest that only total scale scores should be used for these tests, although the general factor extracted from the items was not strong for either test; further development work on these tests is indicated.     

Evaluating the Psychometric and Measurement Characteristics of a Measure of Sexual Orientation Harassment

We use classical test theory (CTT) and item response theory (IRT) methodologies to examine the psychometric and measurement properties of an instrument designed to assess sexual orientation harassment among military personnel (N?=?71,989). CTT analyses indicated that items were unidimensional and exhibited adequate levels of reliability. IRT analyses demonstrated that the items functioned similarly and exhibited appropriate levels of item discrimination. However, the analyses also suggested that the sensitivity of the items may be limited. Differential test functioning analyses provided evidence of the measurement equivalence of the instrument across male and female respondents. The findings provide support for the psychometric properties and measurement equivalence of the instrument for measuring sexual orientation harassment among male and female military personnel. We discuss the implications of our findings for future research on sexual orientation harassment in the workplace.     

Bifactor models and rotations: exploring the extent to which multidimensional data yield univocal scale scores

The application of psychological measures often results in item response data that arguably are consistent with both unidimensional (a single common factor) and multidimensional latent structures (typically caused by parcels of items that tap similar content domains). As such, structural ambiguity leads to seemingly endless "confirmatory" factor analytic studies in which the research question is whether scale scores can be interpreted as reflecting variation on a single trait. An alternative to the more commonly observed unidimensional, correlated traits, or second-order representations of a measure's latent structure is a bifactor model. Bifactor structures, however, are not well understood in the personality assessment community and thus rarely are applied. To address this, herein we (a) describe issues that arise in conceptualizing and modeling multidimensionality, (b) describe exploratory (including Schmid-Leiman [Schmid & Leiman, 1957] and target bifactor rotations) and confirmatory bifactor modeling, (c) differentiate between bifactor and second-order models, and (d) suggest contexts where bifactor analysis is particularly valuable (e.g., for evaluating the plausibility of subscales, determining the extent to which scores reflect a single variable even when the data are multidimensional, and evaluating the feasibility of applying a unidimensional item response theory (IRT) measurement model). We emphasize that the determination of dimensionality is a related but distinct question from either determining the extent to which scores reflect a single individual difference variable or determining the effect of multidimensionality on IRT item parameter estimates. Indeed, we suggest that in many contexts, multidimensional data can yield interpretable scale scores and be appropriately fitted to unidimensional IRT models.     

The Loneliness Questionnaire-Short Version: an evaluation of reverse-worded and non-reverse-worded items via item response theory

Although reverse-worded items have often been incorporated in scale construction to minimize the effects of acquiescent reporting biases, some researchers have more recently begun questioning this approach and wondering whether the advantages associated with incorporating reverse-worded items is worth the complexities that they bring to measures (e.g., Brown, 2003 ; Marsh, 1996 ). In this study, we used item response theory (IRT) to determine whether there is statistical justification to eliminate the reverse-worded items (e.g., "I have lots of friends") from the Loneliness Questionnaire (LQ; Asher, Hymel, & Renshaw, 1984) and retain only the non-reverse-worded items (e.g., "I'm lonely") to inform the provision of a shortened LQ version. Using a large sample of children (Grades 2-7; n = 6,784) and adolescents (Grades 8-12; n = 4,941), we examined the psychometric properties of the 24-item LQ and found support for retaining the 9 non-reverse-worded LQ items to make up a shortened measure of loneliness in youth. We found that the non-reverse-worded items were associated with superior psychometric properties relative to the reverse-worded items with respect to reliability and IRT parameters (e.g., discrimination and item information). A 3-point Likert-type scale was also found to be more suitable for measuring loneliness across both children and adolescents compared to the original 5-point scale. The relative contributions of reverse-worded and non-reverse-worded items in scale development for youth instruments are also discussed.     

Differential functioning of the Beck depression inventory in late-life patients: use of item response theory

The present analyses examined age-related measurement bias in responses to items on the revised Beck Depression Inventory (BDI) in depressed late-life patients versus midlife patients. Item response theory (IRT) models were used to equate the scale and to differentiate true-group differences from bias in measurement in the 2 samples. Baseline BDI data (218 late life and 613 midlife) were used for the present analysis. IRT results indicated that late-life patients tended to report fewer cognitive symptoms, especially at low to average levels of depression. Conversely, they tended to report more somatic symptoms, especially at higher levels of depression. Adjusted cutoff scores in the late-life group are provided, and possible reasons for age-related differences in the performance of the BDI are discussed.     

Factor analytic approaches to personality item-level data

Factor analysis models have played a central role in formulating conceptual models in personality and personality assessment, as well as in empirical examinations of personality measurement instruments. Yet, the use of item-level data presents special problems for factor analysis, applications. In this article, we review recent developments in factor analysis that are appropriate for the type of item-level data often collected in personality. Included in this review are discussions of how these developments have been addressed in the context of two different (but formally related) statistical models item response theory (IRT: Hambleton, Swaminathan, & Rogers, 1991) and structural, equation modeling (Bollen 1989) for item-level data. We also discuss the relevance of item scaling in the context of these models. Using the restandardization data for the Minnesota Multiphasic Personality Inventory-2 Scale (cf. Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989), we show brief examples of the utility of these approaches to address basic questions about responses to personality scale items regarding: (a) scale, dimensionality and general item properties, (b) the "appropriateness" of the observed responses, and (c) differential item functioning across subsamples. implications for analyses of personality item-level data in the IRT and factor analytic traditions are discussed.     

Does the Test of Self-Conscious Affect (TOSCA) measure maladaptive aspects of guilt and adaptive aspects of shame? An empirical investigation

The purpose of this study was to examine whether the Test of Self-Conscious Affect (TOSCA; Tangney, J. P., Wagner, P. E., & Gramzow, R. (1989). The Test of Self-Concious Affect. Fairfax, VA: George Mason University) measures maladaptive forms or aspects of guilt and adaptive aspects of shame that have been described in the literature. First, a judgmental and logical analysis showed that the TOSCA primarily measures mild and adaptive forms and aspects of guilt and maladaptive aspects of shame. Next, principal components analyses (PCAs) in a student (N=328) and adult (N=542) sample showed that items that had a high loading on the guilt factor primarily were items that referred to reparative behavior, while items that had high loadings on the shame factor consisted primarily of items that referred to low self-esteem. To investigate to which extent these items were responsible for correlations found with the TOSCA, we constructed a revised guilt scale containing only items that referred to reparative behavior and a revised shame scale consisting of items that only referred to negative self-esteem, and related these to indices of interpersonal and intrapersonal functioning. The revised TOSCA scales reproduced both the pattern and magnitude of correlations obtained with the original TOSCA scales. Thus, taken together, the results of this study support the interpretation of the TOSCA guilt scale as a measure of mild and adaptive forms of guilt and the TOSCA shame scale as a measure of maladaptive aspects associated with shame. Implications of these findings for further research on the nature of guilt and shame are discussed.     

Computerization and adaptive administration of the NEO PI-R

This study asks, how well does an item response theory (IRT) based computerized adaptive NEO PI-R work? To explore this question, real-data simulations (N = 1,059) were used to evaluate a maximum information item selection computerized adaptive test (CAT) algorithm. Findings indicated satisfactory recovery of full-scale facet scores with the administration of around four items per facet scale. Thus, the NEO PI-R could be reduced in half with little loss in precision by CAT administration. However, results also indicated that the CAT algorithm was not necessary. We found that for many scales, administering the "best" four items per facet scale would have produced similar results. In the conclusion, we discuss the future of computerized personality assessment and describe the role IRT methods might play in such assessments.