Common items and reliability in personality measurement

Most psychological tests and inventories, particularly in personality and attitude measurement, contain common items, that is, various subscales of the test or inventory may share one or more items in common. Surprisingly, the role of common items in such a fundamental personality test attribute as reliability is not known. The present report specifically considered common-item contributions to test internal consistency reliability using extant data from the California Psychological Inventory. No negative contribution of item overlap was found. To the contrary, it was concluded that the use of common items is an economical technique for increasing the total number of items in the scales, thus increasing scale internal reliabilities without increasing the overall number of items in the inventory.     

Examining the item response process to personality measures in high-stakes situations: Issues of measurement validity and predictive validity

We conducted two experimental studies with between-subjects and within-subjects designs to investigate the item response process for personality measures administered in high- versus low-stakes situations. Apart from assessing measurement validity of the item response process, we examined predictive validity; that is, whether or not different response models entail differential selection outcomes. We found that ideal point response models fit slightly better than dominance response models across high- versus low-stakes situations in both studies. Additionally, fitting ideal point models to the data led to fewer items displaying differential item functioning compared to fitting dominance models. We also identified several items that functioned as intermediate items in both the faking and honest conditions when ideal point models were fitted, suggesting that ideal point model is “theoretically” more suitable across these contexts for personality inventories. However, the use of different response models (dominance vs. ideal point) did not have any substantial impact on the validity of personality measures in high-stakes situations, or the effectiveness of selection decisions such as mean performance or percent of fakers selected. These findings are significant in that although prior research supports the importance and use of ideal point models for measuring personality, we find that in the case of personality faking, though ideal point models seem to have slightly better measurement validity, the use of dominance models may be adequate with no loss to predictive validity.     

Examining Faking on Personality Inventories Using Unfolding Item Response Theory Models

A concern about personality inventories in diagnostic and decision-making contexts is that individuals will fake. Although there is extensive research on faking, little research has focused on how perceptions of personality items change when individuals are faking or responding honestly. This research demonstrates how the delta parameter from the generalized graded unfolding item response theory model can be used to examine how individuals’ perceptions about personality items might change when responding honestly or when faking. The results indicate that perceptions changed from honest to faking conditions for several neuroticism items. The direction of the change varied, indicating that faking can operate to increase or decrease scores within a personality factor.     

The process of responding to personality items: Inconsistent responses to repeated presentation of identical items

A promising approach to understanding the processes involved when subjects respond to personality items is provided by the investigation of the causes of inconsistent responses when subjects answer the same item on two occasions. Among these causes are the properties of the item. Previous item research focused almost exclusively on properties which are not highly specific to the item, such as endorsement rate (ER) and social desirability scale value (SDSV). Although past studies found that items with ‘extreme’ SDSVs and/or ERs elicit fewer inconsistencies, these studies ignored more item-specific properties such as item content and item ambiguity. The present study demonstrates that contrary results regarding consistency may be obtained when more item-specific properties are taken into consideration. These results are interpreted as evidence that certain kinds of item content can increase the indecision and conflict that characterize some subjects' response processes.     

The content of personality inventory items

Four aspects of the item content of personality inventories were studied: the area of psychological functioning addressed (cognitive, preferences, feelings, behavioral), degree of reference to situational factors, degree of reference to response frequency, and the nature of reference to time. Three judges rated items of the California Psychological Inventory, Eysenck Personality Inventory, Maudsley Personality Inventory, Minnesota Multiphasic Personality Inventory, Jackson Personality Research Form, and Sixteen Personality Factor Questionnaire. Behavioral acts were found infrequently addressed in inventory items, and the other areas of functioning were approximately equally represented. More than half of the items referred to situation characteristics, 39% included the notion of frequency of behavior or experience, and most referred to the present. Rating reliabilities were also presented, and results for individual inventories were compared. Implications for controversies in the field of personality were discussed.     

Factor analytic approaches to personality item-level data

Factor analysis models have played a central role in formulating conceptual models in personality and personality assessment, as well as in empirical examinations of personality measurement instruments. Yet, the use of item-level data presents special problems for factor analysis, applications. In this article, we review recent developments in factor analysis that are appropriate for the type of item-level data often collected in personality. Included in this review are discussions of how these developments have been addressed in the context of two different (but formally related) statistical models item response theory (IRT: Hambleton, Swaminathan, & Rogers, 1991) and structural, equation modeling (Bollen 1989) for item-level data. We also discuss the relevance of item scaling in the context of these models. Using the restandardization data for the Minnesota Multiphasic Personality Inventory-2 Scale (cf. Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989), we show brief examples of the utility of these approaches to address basic questions about responses to personality scale items regarding: (a) scale, dimensionality and general item properties, (b) the "appropriateness" of the observed responses, and (c) differential item functioning across subsamples. implications for analyses of personality item-level data in the IRT and factor analytic traditions are discussed.     

Stuck in the Middle: The Use and Interpretation of Mid-Points in Items on Questionnaires

Likert-type scales are commonly used when assessing attitudes, personality characteristics, and other psychological variables. This study examined the effect of varying the number of response options on the same set of 28 attitudinal items. Participants answered items using either a 4-point scale (forced choice), a 5-point scale that included a “neither” mid-point, or a 4-point scale with an option of “no opinion” presented after the item. The questionnaire also included an item asking participants what they believe the midpoint in a scale indicated. As predicted, participants’ interpretations of the midpoint varied widely with the most common responses being: “no opinion,” “don't care,” “unsure,” “neutral,” “equal/both,” and “neither.” The quantitative results showed that participants’ levels of item endorsement varied based on the response options offered. For example, “neither” was chosen more often than “no opinion” on all of the items.     

Gender bias in self-report personality disorder inventories

There has been considerable controversy and research regarding gender biases in the diagnosis of personality disorders, but few studies have explored whether personality disorder self-report inventories might contain gender biases. The current study investigated whether items from three commonly used inventories evidence a potential for gender bias. Subjects were from outpatient mental health clinics. Items were considered gender biased if they exhibited gender differences and failed to correlate with or, more importantly, correlated negatively with dysfunction. Thirty-eight items evidenced potential bias, the majority of which were from Narcissistic scales. The implications of the results for the clinical assessment of purportedly maladaptive personality traits and for the construction of personality disorder scales are discussed.     

Introduction to the Special Issue: Personality and the Prediction of Job Performance: More Than the Big Five

The effects of motivated distortion on forced-choice (FC) and normative inventories were examined in three studies. Study 1 examined the effects of distortion on the construct validity of the two item formats in terms of convergent and discriminant validity. The results showed that both types of measures were susceptible to motivated distortion, however the FC items were better indicators of personality and less related to socially desirable responding when participants were asked to respond as if applying for a job. Study 2 considered the criterion-related validity of the inventories in terms of predicting supervisors' ratings of job performance, finding that distortion had a more deleterious effect on the validity of the normative inventory with some enhancement of the validity of the FC inventory being observed. Study 3 investigated whether additional constructs are introduced into the measurement process when motivated respondents attempt to increase scores on FC items. Results of Study 3 indicated that individuals higher in cognitive ability tend to have more accurate theories about which traits are job-related and therefore are more successful at improving scores on FC inventories. Implications for using personality inventories in personnel selection are discussed.     

Computerized adaptive personality assessment: an illustration with the Absorption scale

This article introduces the theory behind and applications of adaptive personality assessment based on the item response theory. Two adaptive testing strategies were compared: (a) fixed test length and (b) clinical decision. Real-data simulations, based on the item responses from 1,000 subjects who had previously taken the 34-item Absorption scale (Tellegen, 1982) by means of paper-and-pencil format, were used to illustrate these strategies. Results suggest that computerized adaptive personality assessment works impressively well. With the fixed-test-length strategy, a 50% savings in administered items was achieved with little loss of measurement precision. In the clinical-decision testing strategy, individuals who were extreme on the Absorption trait were identified with perfect accuracy using, on average, 25% of the available items. The implications of these results for personality research and assessment are discussed.     

Do personality scale items function differently in people with high and low IQ?

Intelligence differences might contribute to true differences in personality traits. It is also possible that intelligence might contribute to differences in understanding and interpreting personality items. Previous studies have not distinguished clearly between these possibilities. Before it can be accepted that scale score differences actually reflect personality differences, personality items should show measurement invariance. The authors used item response theory to test measurement invariance in the five-factor model scales of the International Personality Item Pool (IPIP) and NEO-Five-Factor Inventory (NEO-FFI) across two groups of participants from the Lothian Birth Cohort 1936 with relatively low and high cognitive abilities. Each group consisted of 320 individuals, with equal numbers of men and women. The mean IQ difference of the groups was 21 points. It was found that the IPIP and NEO-FFI items were measurement invariant across all five scales, making it possible to conclude that any differences in IPIP and NEO-FFI scores between people with low and high cognitive abilities reflected personality trait differences.     

Item grouping and item randomization in personality measurement

This paper describes a study examining the impact of item order in personality measurement on reliability, measurement equivalence and scale-level correlations. A large sample of university students completed one of three forms of the International Personality Item Pool version of the Big Five personality inventory: items sorted at random, items sorted by factor, and items cycled through factors. Results showed that the underlying measurement model and the internal consistency of the IPIP-Big Five scale was unaffected by differences in item order. Also, most of the scale-level correlations among factors were not significantly different across forms. Implications for the administration of tests and interpretation of test scores are discussed, and future research directions are offered.     

A CONCEPTUAL AND EMPIRICAL REEXAMINATION OF THE MEASUREMENT OF THE SOCIAL DESIRABILITY OF ITEMS: IMPLICATIONS FOR DETECTING DESIRABLE RESPONSE STYLE AND SCALE DEVELOPMENT

This paper calls into question traditional methods of measuring the social desirability of items and their use in scale construction. First, we make explicit that the proper focus for desirability studies of items and traits are the rated desirabilities of the alternative item responses indicating different trait levels. Second, the results from our first study show that the relation between degree of endorsement of an item and its judged desirability level is often nonlinear and varies across items such that no general model of item desirability can be adopted that will accurately represent the relations across all items, traits, and trait levels. In addition, the nature of these relationships can vary depending on whether desirability is considered in a work or general context. Third, results from a second study indicate specifically that people when instructed to self-present in a maximally desirable manner will choose for some attributes a moderate level of endorsement (e.g., "agree") rather than a more extreme response option (e.g., "strongly agree"). Subjects offer several different reasons for viewing the less extreme response options, which yield more moderate trait level scores, as more desirable. These reasons are linked to perceptions of the more extreme response option as being associated with negative behaviors and concerns about how others will view a more extreme response to the item. Both studies indicate that desirable responding to personality items is more complex than previously believed.     

The growing evidence for personality change in adulthood: Findings from research with personality inventories

Theories of adult development all agree that adulthood is a time of important changes in goals, resources, and coping. Yet, impressed with the rank-order stability of individual differences in personality, many researchers interested in personality traits and personality assessment doubt that personality changes in meaningful and systematic ways during adulthood. This article reviews large studies of mean-level change in personality characteristics measured with broad-band personality inventories, and includes both cross-sectional and cross-cohort longitudinal research. The results show considerable generalizability across samples, cohorts, and studies. In particular, people score higher with age on characteristics such as conscientiousness, agreeableness, and norm-adherence, and they score lower with age on social vitality. These findings provide evidence that personality does change during adulthood and that these changes are non-negligible in size, systematic, not necessarily linear, and theoretically important.     

Using available signals on LinkedIn for personality assessment

LinkedIn is considered the most effective social network website for job seekers and recruiters. Although LinkedIn profiles are regularly accessed to evaluate candidates, we know very little about the type of information conveyed. The aim of this study is to determine if LinkedIn profiles convey accurate information about individuals’ personality traits. Drawing from signaling theory, we expect that individuals portray themselves in a manner that will reflect their personality. To examine this assertion, 607 LinkedIn profiles were coded on 33 indicators. Regression analyses and classification statistics demonstrate that Linkedin profiles contain accurate signals of personality traits. Potential use and limitations of LinkedIn as a source of accurate information about personality are discussed.     

Response Mixture Modeling: Accounting for Heterogeneity in Item Characteristics across Response Times

In item response theory modeling of responses and response times, it is commonly assumed that the item responses have the same characteristics across the response times. However, heterogeneity might arise in the data if subjects resort to different response processes when solving the test items. These differences may be within-subject effects, that is, a subject might use a certain process on some of the items and a different process with different item characteristics on the other items. If the probability of using one process over the other process depends on the subject’s response time, within-subject heterogeneity of the item characteristics across the response times arises. In this paper, the method of response mixture modeling is presented to account for such heterogeneity. Contrary to traditional mixture modeling where the full response vectors are classified, response mixture modeling involves classification of the individual elements in the response vector. In a simulation study, the response mixture model is shown to be viable in terms of parameter recovery. In addition, the response mixture model is applied to a real dataset to illustrate its use in investigating within-subject heterogeneity in the item characteristics across response times.     

The consequences of pairing questions: context effects in personality measurement

The effect of context on responses to questions has been a prominent focus in social and political survey research. However, little investigation of context effects has been done for the measurement of psychological constructs. A measure of anger experience and expression in development uses vignettes describing interpersonal situations that provoke varying degrees of anger and require respondents to indicate their affective and/or expressive response to the situation. In this study, the consequences of pairing the two questions for each vignette are investigated. Pairing the anger-experience and likelihood-of-expression questions changes the item's context. Item response theory analysis similar to that used to detect differential item functioning was performed. For some of the items, responding to a single or paired question affected the extremity of responses. The findings are discussed in terms of their implications for personality measurement.