A CONCEPTUAL AND EMPIRICAL REEXAMINATION OF THE MEASUREMENT OF THE SOCIAL DESIRABILITY OF ITEMS: IMPLICATIONS FOR DETECTING DESIRABLE RESPONSE STYLE AND SCALE DEVELOPMENT

This paper calls into question traditional methods of measuring the social desirability of items and their use in scale construction. First, we make explicit that the proper focus for desirability studies of items and traits are the rated desirabilities of the alternative item responses indicating different trait levels. Second, the results from our first study show that the relation between degree of endorsement of an item and its judged desirability level is often nonlinear and varies across items such that no general model of item desirability can be adopted that will accurately represent the relations across all items, traits, and trait levels. In addition, the nature of these relationships can vary depending on whether desirability is considered in a work or general context. Third, results from a second study indicate specifically that people when instructed to self-present in a maximally desirable manner will choose for some attributes a moderate level of endorsement (e.g., "agree") rather than a more extreme response option (e.g., "strongly agree"). Subjects offer several different reasons for viewing the less extreme response options, which yield more moderate trait level scores, as more desirable. These reasons are linked to perceptions of the more extreme response option as being associated with negative behaviors and concerns about how others will view a more extreme response to the item. Both studies indicate that desirable responding to personality items is more complex than previously believed.     

Corrected item-test correlations

Previous papers on this subject derive the correlation between an item and the remainder of the test. This correlation is unsatisfactory because the reliability of the remainder varies inversely with the reliability of the item omitted. The present paper derives the correlation between an item and the total test, with that item replaced by a rationally equivalent item. The general formula is then modified, for dichotomus items, to give the corrected point-biserial, biserial, and Brogden biserial correlations. The results apply strictly only to factorially homogeneous tests: those in which the same trait or combination of traits is measured (apart from error) by every item.     

Use of item ratings to examine personality test item cognitive response processes

This research examines the processes respondents use to answer personality test items. A total of 158 true/false items from four scales of the Personality Research Form and the California Psychological Inventory were used as stimuli. University students (N = 120) responded to each item and indicated one of nine strategies used in deciding on a response. Obtained response strategy ratings for items were reliable and their frequencies corresponded closely to previous findings with other items. Subsequently, the associations between item response strategy frequencies and item-total correlations were computed. Congruent with previous research, better items avoided behaviours or experiences and evoked responding based on traits and on referring to the statements of others. The associations between item response strategies and other indices of item quality are discussed and implications regarding scale development are offered.     

RELATIONSHIPS BETWEEN RESPONSE STABILITY AND ITEM ENDORSEMENT,SOCIAL DESIRABILITY,AND AMBIGUITY IN THE MMPI AND THE CPI

Relationships between two indices of response stability and item endorsement, social desirability, and seven ambiguity indices were investigated separately within the MMPI, the unique items of the CPI, and two subpools moderate in endorsement and social desirability. Within the two original pools, zero-order correlations and multiple regression analyses revealed that only extremeness of endorsement and social desirability were substantially related to response stability; within the moderate subpools, however, indices of ambiguity-especially item length and ratings of global ambiguity, behavioral reference, and estimated stability-accounted for important degrees of variance individually as well as in combination. Reasons for the moderating effects of endorsement and social desirability are discussed, as are the implications for scale construction.     

Item placement on a personality measure: effects on faking behavior and test measurement properties

Although personality tests are widely used to select applicants for a variety of jobs, there is concern that such measures are fakable. One procedure used to minimize faking has been to disguise the true intent of personality tests by randomizing items such that items measuring similar constructs are dispersed throughout the test. In this study, we examined if item placement does influence the fakability and psychometric properties of a personality measure. Study participants responded to 1 of 2 formats (random vs. grouped items) of a personality test honestly and also under instructions to fake or to behave like an applicant. Results indicate that the grouped item placement format was more fakable for the Neuroticism and Conscientiousness scales. The test with items randomly placed fit the data better within the honest and applicant conditions. These findings demonstrate that the issue of item placement should be seriously considered before administering personality measures because different item presentations may affect the incidence of faking and the psychometric properties of the measure.     

非言语五因素人格问卷(FF-NPQ):一种半投射式人格测验评介

非言语五因素人格问卷(FF-NPQ)由Paunonen等人于2001年开发,用于测查人格五因素。它是一种半投射式人格测验,由60幅黑白图片组成,被试用7点李克特量表评价图中中心人物的行为。FF-NPQ多用于跨文化研究,也可用于文盲、老人或有语言、阅读障碍人群的人格研究。多个国家研究表明,FF-NPQ的内部一致性信度、与多个言语式五因素人格问卷的会聚效度及对行为的预测效度,均达到了心理测量学要求。在中国,该测验尚未使用,建议引进并根据使用情况修订。     

A successive approximation method of maximizing test validity

The ratio of item validity to item-total correlation can be used to select items which will tend to yield the maximum correlation with a criterion. Items to be retained are identified by comparing the ratio for each item with the validity of the original test. Further improvement of the validity in the experimental sample can be obtained by adding items to or removing items from the selected nucleus, according to recomputed ratios involving the correlations of the items with the nucleus and evaluated by means of a revised cut-off point. With slight variations, the method may be used for interest and personality tests as well as for aptitude material. The principal advantage over previous methods is that for any cycle of the analysis an exact cut-off point is provided.     

Evidence for Context Switching in the Effects of Average Item Length and Item-Length Variability on Internal Consistency

This investigation examined the effects of 3 item characteristics—the average number of words per item, within-scale variability in item length, and item “direction”—on internal consistency reliability and interitem correlation. In Study 1, we examined the effects of these variables on overall scale-level reliability using 444 subscales from 9 personality scales. In Study 2, we examined interitem correlation at the paired-item level using 477 nonredundant item pairs from 14 personality scales. Lower scale reliability was associated with more average words per item, greater within-scale variability in item length, and a greater percentage of reverse-keyed items. Similarly, smaller interitem correlations were associated with a greater degree of mismatch in item length between the paired items and with a mismatch (vs. match) in the items' respective “directions.” The pattern of results across both studies supports our notion that lower internal consistency results from increased context switching; that is, from the confusion that occurs when respondents must switch back and forth between the interpretive frames pertaining to short versus long items, or between items pertaining to one pole of a personality dimension and its “opposite” pole. Suggestions for maximizing the internal consistency of personality scales are proposed.     

Probing item social desirability by correlating personality items with Balanced Inventory of Desirable Responding (BIDR): A validity examination

Researchers often include a social desirability measure in personality measures, commonly the Balanced Inventory of Desirable Responding (BIDR), and correlate it with personality items to probe for social desirability of the items. A strong correlation between BIDR scores and a personality item would indicate high item social desirability. The current research assesses the validity of this practice. Results showed that these correlations have high validity only when BIDR scores are calculated as a continuous variable rather than as dichotomized item scores. In addition, self-deception scores have higher validity for detecting item social desirability than do impression management scores. The current research supported the use of the self-deception scores, in particular, to detect highly desirable or undesirable items.     

Social desirability in personality inventories: Symptoms,diagnosis and prescribed cure

An analysis of social desirability in personality assessment is presented. Starting with the symptoms, Study 1 showed that mean ratings of graded personality items are moderately to strongly linearly related to social desirability (Self Deception, Impression formation, and the first Principal Component), suggesting that item popularity may be a useful heuristic tool for identifying items which elicit socially desirable responding. We diagnose the cause of socially desirable responding as an interaction between the evaluative content of the item and enhancement motivation in the rater. Study 2 introduced a possible cure; evaluative neutralization of items. To test the feasibility of the method lay psychometricians (undergraduates) reformulated existing personality test items according to written instructions. The new items were indeed lower in social desirability while essentially retaining the five factor structure and reliability of the inventory. We conclude that although neutralization is no miracle cure, it is simple and has beneficial effects.     

Factor analytic approaches to personality item-level data

Factor analysis models have played a central role in formulating conceptual models in personality and personality assessment, as well as in empirical examinations of personality measurement instruments. Yet, the use of item-level data presents special problems for factor analysis, applications. In this article, we review recent developments in factor analysis that are appropriate for the type of item-level data often collected in personality. Included in this review are discussions of how these developments have been addressed in the context of two different (but formally related) statistical models item response theory (IRT: Hambleton, Swaminathan, & Rogers, 1991) and structural, equation modeling (Bollen 1989) for item-level data. We also discuss the relevance of item scaling in the context of these models. Using the restandardization data for the Minnesota Multiphasic Personality Inventory-2 Scale (cf. Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989), we show brief examples of the utility of these approaches to address basic questions about responses to personality scale items regarding: (a) scale, dimensionality and general item properties, (b) the "appropriateness" of the observed responses, and (c) differential item functioning across subsamples. implications for analyses of personality item-level data in the IRT and factor analytic traditions are discussed.     

Scale validation via quantifying item validity using the Dm index

While most validity indices are based on total test scores, this paper describes a method for quantifying the construct validity of items. The approach is based on the item selection technique originally described by Piazza in 1980. Unfortunately, Piazza's P2 index suffers from some substantial limitations. The Dm coefficient provides an alternative which can be used for item selection and provides a validity index for a set of items. The index is similar to that of traditional criterion-related validity indices. Criterion-related validity is used to demonstrate the accuracy of hypothesized relations of the measure with outcome variables of interest in research and practice. This method may be useful when the sample of items or persons is small, rendering more traditional approaches such as factor analysis or item response theory inappropriate. An example of how to use the technique is provided.     

BRIDGING THE GAP BETWEEN OVERT AND PERSONALITY-BASED INTEGRITY TESTS

Overt and personality-based integrity tests are used for the same purposes, but the relationship between the two kinds of measures is unclear. Moreover, although the construct validity of personality-based integrity measures is well understood, the psychological meaning of overt integrity measures is unclear. A sample of applicants ( N = 2,168) for driver, warehouse, and clerical jobs completed an overt integrity test (Reid Report), a personality-based integrity test (Employee Reliability Index) and a measure of normal personality (Hogan Personality Inventory). A principal components analysis of the intercorrelations between the overt and the personality-based integrity item responses revealed four themes: (a) punitive attitudes, (b) admissions of illegal drug use, (c) reliability, and (d) theft admissions. A model testing for a general conscientiousness factor provided a good fit for the overt and personality-based integrity test variables, although item overlap between the two test types was minimal. Finally, the punitive attitudes and theft admissions components of the Reid item pool are most closely related to the Big Five personality factors of conscientiousness and emotional stability; the Reid component of illegal drug use was unrelated to personality measures.     

随机截距因子分析模型在控制条目表述效应中的应用

本研究用中文修订版罗森博格自尊量表(RSES-R)考察随机截距因子分析模型在控制条目表述效应时的表现。用RSES-R和过分宣称问卷组成的量表调查621名中学生。结果表明,随机截距模型在建模时,拟合指数良好、因子方差与负荷合理,自尊因子分与RSES-R总分有极高相关,表明该模型能有效分离RSES-R得分的特质与表述效应。分离的表述效应因子分与受测者的自我提升水平具有显著但较弱的相关,表明表述效应与自受测者的社会赞许性有共同的成分。     

认知诊断评价中测验结构的优化设计

Q矩阵是认知诊断评价的基础和核心要素, 它反映了测验的构念和内容设计, 直接影响着测验诊断分类的效果。本文采用Monte Carlo模拟, 研究了6种属性层级关系下, 不同的Q矩阵设计对于认知诊断效果的影响。用模式判准率的均值和标准差分别从分类准确性和稳定性的角度来评价诊断效果。实验结果表明：(1) 不同属性层级关系下, 分类准确性会随着测验长度的增加而提高, 但当测验长度增加到一定程度时, 会出现“天花板效应”; (2) Q矩阵中R*的个数(NR*)会影响测验的分类准确性及稳定性：NR*越大, 测验的分类稳定性越高, 当测验长度为属性个数的整数倍, 且NR*为测验长度相对属性个数的最大奇数倍时分类准确性最高; (3) Q矩阵中除R*以外的项目考察的属性个数会随着属性层级关系的不同对测验的分类准确性和稳定性产生不同的影响。根据实验结果, 本研究提出了进行诊断评价时Q矩阵优化设计的一些建议。     

分类视角下认知诊断测验项目区分度指标及应用

在认知诊断中还没有指标能在无作答数据情况下直接评价项目的属性分类准确率或属性判准率。项目水平上的属性分类准确率,与项目属性向量、项目参数、先验分布和作答反应等有关。综合各个影响因素定义了项目水平上的属性期望分类准确率指标,并将其用于组卷。模拟研究显示：新指标可十分准确地评价项目的属性判准率,新指标对于项目筛选十分重要;以模式分类准确率为评价指标,基于新指标的组卷方法与经典的组卷方法表现相当。     

Gender bias in diagnostic criteria for personality disorders: an item response theory analysis

The authors examined gender bias in the diagnostic criteria for Diagnostic and Statistical Manual of Mental Disorders (4th ed., text revision; American Psychiatric Association, 2000) personality disorders. Participants (N=599) were selected from 2 large, nonclinical samples on the basis of information from self-report questionnaires and peer nominations that suggested the presence of personality pathology. All were interviewed with the Structured Interview for DSM-IV Personality (B. Pfohl, N. Blum, & M. Zimmerman, 1997). Using item response theory methods, the authors compared data from 315 men and 284 women, searching for evidence of differential item functioning in the diagnostic features of 10 personality disorder categories. Results indicated significant but moderate measurement bias pertaining to gender for 6 specific criteria. In other words, men and women with equivalent levels of pathology endorsed the items at different rates. For 1 paranoid personality disorder criterion and 3 antisocial criteria, men were more likely to endorse the biased items. For 2 schizoid personality disorder criteria, women were more likely to endorse the biased items.     

Random Item IRT Models

It is common practice in IRT to consider items as fixed and persons as random. Both, continuous and categorical person parameters are most often random variables, whereas for items only continuous parameters are used and they are commonly of the fixed type, although exceptions occur. It is shown in the present article that random item parameters make sense theoretically, and that in practice the random item approach is promising to handle several issues, such as the measurement of persons, the explanation of item difficulties, and trouble shooting with respect to DIF. In correspondence with these issues, three parts are included. All three rely on the Rasch model as the simplest model to study, and the same data set is used for all applications. First, it is shown that the Rasch model with fixed persons and random items is an interesting measurement model, both, in theory, and for its goodness of fit. Second, the linear logistic test model with an error term is introduced, so that the explanation of the item difficulties based on the item properties does not need to be perfect. Finally, two more models are presented: the random item profile model (RIP) and the random item mixture model (RIM). In the RIP, DIF is not considered a discrete phenomenon, and when a robust regression approach based on the RIP difficulties is applied, quite good DIF identification results are obtained. In the RIM, no prior anchor sets are defined, but instead a latent DIF class of items is used, so that posterior anchoring is realized (anchoring based on the item mixture). It is shown that both approaches are promising for the identification of DIF.