Ipsative and Normative Scales in Adjectival Measurement of Personality: Problems of Bias and Discrepancy

887 respondents completed ipsative and normative versions of the PAL-TOPAS personality questionnaire. Data were analysed to test for (1) systematic bias in scores associated with the two response formats and (2) predictors of the magnitude of the discrepancy in the individual's ipsative and normative scores. Discrepancy was assessed for both item responses and scale scores. Sources of biases investigated included ipsative scaling artifact, extremeness of scores on the normative scales and response variability. Results showed that systematic bias in scale scores and magnitude of discrepancy were predicted by different factors. One source of systematic bias was associated with ipsative scaling artifact: the ipsative scales measure both the scale itself and rejection of other alternatives. A second source of systematic bias was acquiescence in response to normative items. A confirmatory factor analysis showed that a good but imperfect fit to the data may be obtained by constructing a structural model of the inter-relationship between normative and ipsative scores which accommodates both sources of bias. The strongest influence on discrepancy in scale scores was extremeness of normative scoring, associated with a bias towards either general acceptance or rejection of trait adjectives. It is concluded that both normative and ipsative response formats have limitations, and it may often be desirable to assess both.     

迫选式人格测验的传统计分与IRT计分模型

迫选测验的传统计分方式会产生自模式数据, 不能进行传统的信效度检验、因素分析和方差分析等。近年来研究者提出了一些基于项目反应理论的计分模型, 如瑟斯顿IRT模型和MUPP模型等, 它们可以规避自模式数据的弊端。瑟斯顿IRT模型方便进行参数估计, 模型定义灵活; 而MUPP模型的拓展性较差, 参数估计的方法有待提高。另一方面, 已有研究者基于MUPP模型开发了一些抗作假的迫选测验, 而瑟斯顿IRT模型距离这种应用还比较远。此外, 两个模型的适用性和有效性都有待更多的实证研究来检验。     

适用于多维迫选测验的IRT计分模型

迫选(forced-choice, FC)测验由于可以控制传统李克特方法带来的反应偏差, 被广泛应用于非认知测验中, 而迫选测验的传统计分方式会产生自模式数据, 这种数据由于不适合于个体间的比较, 一直备受批评。近年来, 多种迫选IRT模型的发展使研究者能够从迫选测验中获得接近常模性的数据, 再次引起了研究者与实践人员对迫选IRT模型的兴趣。首先, 依据所采纳的决策模型和题目反应模型对6种较为主流的迫选IRT模型进行分类和介绍。然后, 从模型构建思路、参数估计方法两个角度对各模型进行比较与总结。其次, 从参数不变性检验、计算机化自适应测验(computerized adaptive testing, CAT)和效度研究3个应用研究方面进行述评。最后提出未来研究可以在模型拓展、参数不变性检验、迫选CAT测验和效度研究4个方向深入。     

Psychometric critique of acculturation psychology: the case of Iranian migrants in Norway

The presumptions, terminology, psychometrics, statistical analyses, and ethics of the fourfold acculturation paradigm are criticized in detail. Illustrative data came from Iranian refugees in Norway (N = 80) answering: 1) the Satisfaction with Life Scale (SWLS), 2) Zung's Self-Rating Depression Scale (ZSRDS), 3) ipsative fourfold scales of Integration, Assimilation, Separation, and Marginalization, 4) orthogonal scales of attitudes towards Norwegian and Iranian cultures, measured independently and using balanced reverse-keying, and 5) ipsative forced-choice preferences for cultural practices of Norway, Iran, both, or from other societies as well. Iranians in Norway favored global multiculturalism and, as a group. did not show distress. The SWLS and ZSRDS were correlated, but the measures of acculturation failed to replicate one another. As unconstrained ipsative measures, the fourfold scales showed acquiescence response bias contamination and doubtful operationalization of scale constructs. Recommendations are discussed for improving acculturation research.     

Equivalence of Narcissistic Personality Inventory constructs and correlates across scoring approaches and response formats

The prevalent scoring practice for the Narcissistic Personality Inventory (NPI) ignores the forced-choice nature of the items. The aim of this study was to investigate whether findings based on NPI scores reported in previous research can be confirmed when the forced-choice nature of the NPI’s original response format is appropriately modeled, and when NPI items are presented in different response formats (true/false or rating scale). The relationships between NPI facets and various criteria were robust across scoring approaches (mean score vs. model-based), but were only partly robust across response formats. In addition, the scoring approaches and response formats achieved equivalent measurements of the vanity facet and in part of the leadership facet, but differed with respect to the entitlement facet.     

Análisis factorial de ítems de respuesta forzada: una revisión y un ejemplo

Forced-choice tests are widely used in order to reduce the impact of different response set biases typically associated to psychological tests (e.g. acquiescence or social desirability). However, these tests produce ipsative data which have undesirable properties, thereby making an inappropriate application of classical factor analysis techniques for psychometric evaluation commonly used by researchers. This paper explains the analytical properties of forced-choice tests, along with an example that illustrates how these properties have an impact on the application of conventional statistical techniques and produce improper results. Additionally, one of the current proposals is presented in order to analyze these data based on the comparative judgment model by Thurstone, along with the results of a simulation study which illustrates its implementation and effectiveness in recovering the original factor structure.     

An assessment of the Edwards Personal Preference Schedule from the perspective of the five-factor model.

We examined the validity of need scales of the Edwards Personal Preference Schedule (EPPS) by correlating them with a measure of the five basic factors of personality; we also considered test format as a possible source of invalidity. Three hundred thirty (223 women, 107 men) undergraduate students completed both the NEO Personality Inventory (NEO-PI)--a measure of the five factors--and one of two versions of the EPPS. Results show that both ipsative and normative versions of the EPPS could be meaningfully interpreted within the five-factor model, although the ipsative, forced-choice format of the standard EPPS apparently lowered validity coefficients and decreased convergent and discriminant validity. We argue that the five-factor model can provide a useful interpretive context for evaluating many clinical measures.     

A dual process item response theory model for polytomous multidimensional forced-choice items

The use of multidimensional forced-choice (MFC) items to assess non-cognitive traits such as personality, interests and values in psychological tests has a long history, because MFC items show strengths in preventing response bias. Recently, there has been a surge of interest in developing item response theory (IRT) models for MFC items. However, nearly all of the existing IRT models have been developed for MFC items with binary scores. Real tests use MFC items with more than two categories; such items are more informative than their binary counterparts. This study developed a new IRT model for polytomous MFC items based on the cognitive model of choice, which describes the cognitive processes underlying humans' preferential choice behaviours. The new model is unique in its ability to account for the ipsative nature of polytomous MFC items, to assess individual psychological differentiation in interests, values and emotions, and to compare the differentiation levels of latent traits between individuals. Simulation studies were conducted to examine the parameter recovery of the new model with existing computer programs. The results showed that both statement parameters and person parameters were well recovered when the sample size was sufficient. The more complete the linking of the statements was, the more accurate the parameter estimation was. This paper provides an empirical example of a career interest test using four-category MFC items. Although some aspects of the model (e.g., the nature of the person parameters) require additional validation, our approach appears promising.     

A text-stimuli presentation manager for the IBM PC with ipsatization correction for response sets and reaction times

IPSAPRO, an ipsative scoring program written for the IBM PC, aids in the detection and transformation of response sets that often contaminate rating scale and reaction time experiments. Response sets such as the tendency to use only extreme points of a rating scale or to work for speed over accuracy in reaction time experiments are removed in IPSAPRO by standardizing each subject’s ratings or times against their own means and standard deviations. Ipsatization can be applied to existing data sets or take place automatically at the data collection stage in a text-stimuli presentation manager that is provided with the program.     

How do applicants fake? A response process model of faking on multidimensional forced-choice personality assessments

Faking on personality assessments remains an unsolved issue, raising major concerns regarding their validity and fairness. Although there is a large body of quantitative research investigating the response process of faking on personality assessments, for both rating scales (RS) and multidimensional forced choice (MFC), only a few studies have yet qualitatively investigated the faking cognitions when responding to MFC in a high-stakes context (e.g., Sass et al., 2020). Yet, it could be argued that only when we have a process model that adequately describes the response decisions in high stakes, can we begin to extract valid and useful information from assessments. Thus, this qualitative study investigated the faking cognitions when responding to MFC personality assessment in a high-stakes context. Through cognitive interviews with N = 32 participants, we explored and identified factors influencing the test-takers' decisions regarding specific items and blocks, and factors influencing the willingness to engage in faking in general. Based on these findings, we propose a new response process model of faking forced-choice items, the Activate-Rank-Edit-Submit (A-R-E-S) model. We also make four recommendations for practice of high-stakes assessments using MFC.     

The equivalence of target and nontarget processing in visual search

A comparison of a forced-choice visual search task with an item recognition task did not support Neisser’s (1967) hypothesis of a preattentive stage that processes targets and nontargets differentially. In the forced-choice condition, Ss indicated which of two items in a visual display was a target; in item recognition, Ss determined whether or not the single item in the visual display was a target. The size of the memorized set of possible targets was varied from one to six items for both tasks. Latencies increased linearly with memory set size in both conditions; the slopes for forced choice and item recognition were 41.8 and 27.9 msec per item, respectively. The ratio of 1.38 between the two slopes was well fit by Sternberg’s (1967) item recognition model, which predicts a ratio of 1.50.     

Interactive versus Ipsative Measurement of Career Interest

Measurement of interests via the self-report type of inventory takes either of two forms—the interactive or “free-response” variety, or the ipsative or “forced-choice” variety. There are many differences between the two forms but little empirical evidence concerning the validity of one as compared to the other. This study is an investigation of the difference in concurrent validity between an interactive and an ipsative form of a short career preference inventory for college males, using several of the scales on the Strong Vocational Interest Blank as criteria. The results indicate that despite the theoretical and practical differences in the two forms, neither holds any substantial advantage in statistical validity.     

Measurement Invariance of the Adolescent Quality of Life-Mental Health Scale (AQOL-MHS) across Gender,Age and Treatment Context

The Adolescent Quality of Life-Mental Health Scale (AQOL-MHS) was designed to measure quality of life in clinical samples of Latino adolescents aged 12–18 years, but has also been used in community samples. The original measure included three factors: Emotional Regulation (ER), Self-Concept (SC) and Social Context (SoC). The goals of this study are to replicate the factor structure using confirmatory factor analysis (CFA), shorten the instrument and test the degree of measurement invariance across gender, age, and type of sample. Participants for the analyses (N?=?354) came from two populations in the San Juan Metropolitan Area: (1) adolescents from randomly selected households, using a multi-stage probability sampling design (n?=?295), and (2) adolescents receiving treatment at mental health clinics (n?=?59). We first carried out a conceptual item analysis for item reduction purposes and then assessed dimensional, configural, metric and scalar invariance for each factor using the Mplus software system. The original 3-factor structure was replicated with comparable model fit in each treatment context. Metric invariance was attained for all three scales across groups. Either full or partial scalar invariance was also observed with DIF in a total of 6 items. Invariance testing supports the use of the abridged 21 item version of the AQOL-MHS to compare diverse individuals with little bias using observed scores, but for refined estimates the ideal scoring will be from a latent variable model.     

Normative versus ipsative configural frequency analysis in personality research—their use discussed in a reanalysis of data on situation-bound anxiety

Configural frequency analysis (CFA) tests whether certain individual patterns in different variables are observed more frequently in a sample than expected by chance. In normative CFA, these patterns are derived from the subject's specific position in relation to sample characteristics such as the median or the mean. In ipsative CFA, patterns are defined within an individual reference system, e.g. relative to the subject's median of different variable scores. Normative CFA examines dimensionality of scales and is comparable to factor analysis in this respect. Ipsative CFA rather yields information about location of scores in different variables, in a similar way to ANOVA or Friedman testing. However, both normative and ipsative CFA may supply information not obtainable by means of the aforementioned methods. This is illustrated in a reanalysis of data in four scales of an anxiety inventory. © 1997 John Wiley & Sons, Ltd.     

神经质人格迫选量表的开发及其抗作假效果研究

传统的迫选量表得分是自模式数据,最近提出的Thurstone IRT模型建构了被试对迫选量表反应的数学模型,能够更精确地度量被试的特质水平.研究自编了神经质人格迫选量表,与常用的测量神经质的Likert量表一起,在无压力、模拟应聘和实际应聘三种情境下进行施测.结果发现,迫选量表的实测数据能够较好地拟合Thurstone IRT模型,该模型估计的特质得分不具有自模式数据的性质,比传统计分更能够抵抗作假.无论采用哪种计分方式,迫选量表都比Likert量表更能够抵抗作假.     

Expectancy-value models of attitudes: A note on the relationship between theory and methodology

Concern has been expressed in the literature regarding the method of scoring ‘beliefs’ within expectancy-value models of attitudes. This paper reviews the major issues and focuses upon some hitherto largely neglected problems with scoring methods. Empirical findings from a series of studies concerned with ‘the theory of reasoned action’ are examined: with a multiplicative Combination of beliefs and evaluations, it is found that bipolar scoring of belief items leads to higher correlations of the summed products of beliefs and evaluations with attitudes than are achieved with unipolar scoring. These findings contrast markedly with recently reported research and indicate the important role played by contextual factors (such as belief content and the response scales presented to subjects). It is concluded that more attention needs to be paid to the relationship between conceptual and methodological issues.     

Minimizing gender differences in children's interest assessment: Development of the Inventory of Children's Activities-3 (ICA-3)

The focus of this study was on revising the Inventory of Children's Activities–Revised (ICA-R; Tracey & Ward, 1998) to enhance its psychometric properties while minimizing gender differences in scale scores. The original 30 ICA-R items and an additional 30 items were administered to 70,280 fifth-eighth grades students. The original scoring was compared to a revised scoring method based solely on the empirically best items and a scoring method balancing empirical scoring with minimizing gender differences. All three item sets (original, empirical, and combined empirical/gender balancing) resulted in strong internal consistency estimates and adequate fit to the circular structure, yet the combined empirical/gender method had much lower gender differences especially for the scales measuring Investigative and Social interests. The implications of using the revised scale with children is discussed.     

Empirical derivation of SVIB-Holland scales: A brief report

The Strong Vocational Interest Blank responses of 93 students were used to construct six empirical scales similar to the scales of Holland's Vocational Preference Inventory. Scores on the empirical scales were correlated with actual VPI scores. The resulting correlations were compared to coefficients obtained from correlating the intuitive scales designed by Campbell with actual VPI scores. It was concluded that (1) meaningful estimates of VPI profiles can be obtained by scoring selected items from the SVIB and (2) further work with the empirical scales is needed prior to settling on a SVIB scoring procedure for estimating VPI profiles.     

Internal Validity and Reliability of Kolb’s Learning Style Inventory Version 3 (1999)

This study explores the internal validity and reliability of Kolb’s revised Learning Style Inventory (LSI-2A and LSI-3) in a sample of 221 graduate and undergraduate business students. Research on the LSI is also reviewed and the implications of conducting factor analysis using ipsative data are explored. Experiential Learning Theory is presented and the concept of learning styles explained. This study largely supports prior research supporting the internal reliability of scales. Principle Component Analysis provides evidence for a 2 factor structure as hypothesized by Kolb.