Specificity Matters: Criterion-Related Validity of Contextualized and Facet Measures of Conscientiousness in Predicting College Student Performance

To enhance the predictive validity of self-report personality measures, 2 distinct ways of increasing specificity of personality measures have been proposed in the literature—contextual specificity (i.e., providing a contextual referent) and content specificity (i.e., focusing on more specific constructs such as the Big Five facets). This study extends this line of research by examining whether there is an optimal way to configure, align, or integrate contextual and content specificity using measures of conscientiousness to predict college student success. A sample of 478 undergraduate students completed 4 measures of conscientiousness that varied in the level of content and contextual specificity. These forms of specificity were crossed to yield 4 distinct measures of conscientiousness. We then evaluated and compared the relative importance and the incremental importance of these different measures in the prediction of academic success. Superior predictive validity was found for both contextualized and facet measures of conscientiousness compared to a measure of global conscientiousness in predicting grade-point average and a broader behavioral criterion of student performance. When contextual and content specificity approaches were compared and combined, we observed the strongest predictive validity when the level of specificity is appropriately matched between predictor and criterion.     

Forced-choice and conventional personality assessment: Each may have unique value in pre-employment testing

Forced-choice (FC) personality scales are purported to predict criteria such as counterproductive work behavior (CWB) incrementally beyond conventional personality measures in pre-employment testing situations. Unfortunately, the research upon which FC tests’ claims of superiority are founded, relies upon undergraduate participants in highly artificial scenarios. Moreover, the possible predictive advantages of FC personality scales with regards to important contextual performance (CP; Borman & Motowidlo, 1993) criteria have not been investigated. We used a sample of call center employees to investigate the incremental validity of a FC personality scale over a conventional personality scale (and vice versa) with CP measures, and CWB, as criteria. Our FC and conventional personality measures were highly correlated, but each still offered unique predictive value with regards to CP or CWB. Thus, contrary to past research with student samples, in field settings FC personality scales may not be clearly superior to conventional personality scales. Moreover, contrary to widespread concern that FC measures’ predictiveness is attributable to their overlap with general mental ability, the unique predictive value of our FC measure remained even when variance due to general mental ability was statistically controlled.     

Measuring motivation multidimensionally: development of the Assessment of Individual Motives-Questionnaire (AIM-Q)

We report the development of the Assessment of Individual Motives-Questionnaire (AIM-Q), a new instrument based on an evolutionary psychology theory of human motivation. It provides multitrait-multimethod (MTMM) assessment of individual differences on 15 motive scales. A total heterogeneous sample of N = 1,251 participated in eight studies that evaluated the homogeneity, internal consistency, test-retest reliability, and MTMM convergent and discriminant validities of the AIM-Q's three methods. These studies generally support the overall strategy of assessing individual differences in multiple evolutionary-based motives with multiple methods. Additional validity studies are underway and, when validated further, the AIM-Q may offer a promising option for evolutionary psychologists and behavioral geneticists wanting to incorporate individual differences into their research but have had to use existing self-report measures of personality, which were not designed for such a purpose. It may also offer clinical and counseling psychologists an additional approach to personality measures for the prediction of behavior.     

The Eysenckian personality factors and their correlations with academic performance

Background. The relationship between personality and academic performance has long been explored, and a recent meta‐analysis established that measures of the five‐factor model (FFM) dimension of Conscientiousness have similar validity to intelligence measures. Although currently dominant, the FFM is only one of the currently accepted models of personality, and has limited theoretical support. In contrast, the Eysenckian personality model was developed to assess a specific theoretical model and is still commonly used in educational settings and research. Aims. This meta‐analysis assessed the validity of the Eysenckian personality measures for predicting academic performance. Sample. Statistics were obtained for correlations with Psychoticism, Extraversion, and Neuroticism (20–23 samples; N from 8,013 to 9,191), with smaller aggregates for the Lie scale (7 samples; N= 3,910). Methods. The Hunter–Schmidt random effects method was used to estimate population correlations between the Eysenckian personality measures and academic performance. Moderating effects were tested using weighted least squares regression. Results. Significant but modest validities were reported for each scale. Neuroticism and Extraversion had relationships with academic performance that were consistent with previous findings, while Psychoticism appears to be linked to academic performance because of its association with FFM Conscientiousness. Age and educational level moderated correlations with Neuroticism and Extraversion, and gender had no moderating effect. Correlations varied significantly based on the measurement instrument used. Conclusions. The Eysenckian scales do not add to the prediction of academic performance beyond that provided by FFM scales. Several measurement problems afflict the Eysenckian scales, including low to poor internal reliability and complex factor structures. In particular, the measurement and validity problems of Psychoticism mean its continued use in academic settings is unjustified.     

Using Indirect Measurement Tasks to Assess the Self‐concept of Personality: A Systematic Review and Meta‐analyses

This systematic review presents the current state of research investigating the implicit self‐concept of personality. First, we present results on meta‐analyses estimating internal consistency, reliability coefficients, the implicit–explicit consistency and the single association predictive effect of implicit self‐concept of personality measures. To do this, studies were aggregated over personality domains. Second, for each of the Five Factor personality domains, different aspects of construct validity and predictive validity are reviewed in a narrative way. Results show that implicit self‐concept of personality measures are reliable, and there is evidence for the construct and predictive validity of these implicit measures, especially in the extraversion and agreeableness domains of personality. However, it must be kept in mind that clear evidence for publication bias was found for studies examining the single association predictive pattern. Finally, this systematic review identifies some achievable improvements that are needed in future research. Large cross‐lab efforts are important in this respect. Moreover, the implicit self‐concept of personality field must move from an ‘ad hoc’ to a ‘validation’ approach in developing new indirect measurement tasks. By adopting these research objectives, the information processing account of personality will increase its potential to become integrated into mainstream personality theory and research. Copyright © 2017 European Association of Personality Psychology     

Two promising shame and guilt scales: a construct validity comparison

This study compared the validity of two promising measures of shame and guilt proneness: revisions of the Harder Personal Feelings Questionnaire (PFQ2; Harder & Lewis, 1987) and the Hoblitzelle Adapted Shame and Guilt Scale (ASGS; Hoblitzelle, 1982). Internal consistency, test-retest stability, factor structure, and construct validity with convergent and discriminant personality dimensions were examined for both scales. In addition to the shame and guilt measures, 63 (37 male, 26 female) mostly freshman college students completed a randomly ordered battery of personality scales theoretically relevant to shame and guilt proneness. Results support the reliability and shame/guilt factor structure of each scale. ASGS Shame correlations appeared marginally more valid with 11 external construct variables than PFQ2 Shame, whereas PFQ2 Guilt was clearly more valid than its corresponding ASGS subscale. New, potentially improved scales were constructed from the factor analyses and from item analyses. However, the resulting scales did not show improved validity.     

A dynamic assessment of offender risk, needs, and strengths in a sample of pre-release general offenders

Static, dynamic, and protective factors have been identified as three focal domains significantly related to offender recidivism. However, few measures include comprehensive and inclusive assessment of these variables. The Inventory of Offender Risk, Needs, and Strengths (IORNS) was developed to fill a void in the assessment of risk and needs for offenders. The current study examines the reliability and initial validity of the IORNS in a sample of pre-release offenders assessed for risk and treatment need. Results indicate moderate to high levels of internal consistency and identical IORNS scale results across race, and that the IORNS indexes, scales, and subscales display good convergent validity with self-report and interview measures of static risk, dynamic risk, antisocial behavior, psychopathy, personality pathology, substance abuse, depression, and anxiety. Initial predictive validity examination of the IORNS indicates that several of the indexes, scales, and subscales were able to differentiate offenders who were sent back to prison for half-way house rule violations from those who did not violate rules.     

Intra- and intercultural comparisons of the personality profiles of medical students in Argentina and the United States

This study was designed to compare the personality profiles of medical students in Argentina and the United States. The ultimate purpose of the research was to study the value of personality measures in predicting academic and professional performances. Participants were 421 medical students in Argentina (254 women, 167 men) and 623 medical students in the United States (207 women, 416 men). Eight personality measures were administered: Perception of Stressful Life Events, Test Anxiety, General Anxiety, Loneliness, Self-Esteem, Locus of Control, Extraversion, and Neuroticism. Intracultural comparisons showed some minor gender differences in personality profiles within each culture (e.g., in the United States, women scored higher than men on the Perception of Stressful Life Events and General Anxiety scales, and in Argentina, women scored higher on the Test Anxiety scale). Intercultural comparisons of personality profiles showed that Argentine medical students obtained higher average scores than did their American counterparts on the Perception of Stressful Life Events, Test Anxiety, General Anxiety, External Locus of Control, Extraversion, and Neuroticism scales. Argentine students scored lower on the Loneliness scale than did their American counterparts. Psychometric findings supported the measurement properties of the personality measures in the two cultures (e.g., construct validity, and internal consistency aspect of reliability). Further study of the implications of the findings in predicting academic attainment in medical school and to physician performance is recommended.     

On the validity of narrow and broad personality traits: A response to

Salgado, Moscoso, and Berges (2013) have recently commented on the issue of the comparative validity of broad factor-level personality traits and narrow facet-level personality traits, providing arguments and data suggesting superior validity for the former. In response, we first clarify some of our previous points that were discussed by Salgado et al. and comment on some other aspects of their article. We then provide an empirical example in which a facet-level personality scale does better in predicting a delinquency criterion than does the factor-level personality scale that contains that facet. We also use CFA to demonstrate that it was the specific trait variance in our narrow predictor that added significantly to predictive validity, above and beyond the validity achieved by the common factor variance contained within the trait measure. These results support our previous conclusion that exclusive reliance on broad factor measures can be counterproductive for understanding and predicting behavior.     

Personality and symptom change in treatment-refractory inpatients: evaluation of the phase model of change using Rorschach,TAT, and DSM-IV Axis V

In this study, we examined global treatment outcomes during 16 months of intensive, psychodynamic treatment for 77 inpatients suffering from treatment-refractory disorders. Hypotheses based on the phase model of treatment change (Howard, Lueger, Maling, & Martinovich, 1993; Howard, Moras, Brill, Martinovich, & Lutz, 1996) were supported in the study results. Diagnostic and Statistical Manual of Mental Disorders (4th ed.; American Psychiatric Association, 1994) Axis V scales assessing behavioral functioning demonstrated large and medium effect size change, whereas stable, enduring personality functioning assessed by psychoanalytic Rorschach scales and the Social Cognition and Object Relations Scale (Westen, 1995) for the Thematic Apperception Test (Murray, 1943) demonstrated small and medium effect size change. We also report assessment of reliable change index and clinical significance. The ecological validity of Rorschach measures is supported by significant validity coefficients (in the hypothesized directions) between implicit measures of personality functioning and behavioral ratings.     

Construct-oriented Biodata: Capturing Change-related and Contextually Relevant Future Performance

This study highlights an approach to the development of performance and predictor constructs that has the potential to permit context and culture relevant selection. Task, contextual and adaptive measures of performance were used as criterion data in a selection study involving 325 staff in the hotel industry. Construct-oriented biodata were developed to predict adaptive and contextual performance and the validities were contrasted with measures of cognitive ability and personality. The results provided construct support for the separation of adaptive performance from task and contextual performance and for the predictive validity of change-related biodata after controlling for cognitive ability. The pattern of correlations among the personality measures, biodata scales and cognitive tests supported the construct validity of the biodata scales. Results are discussed in relation to the context in which the data were collected, and the general relevance of the procedure across countries or cultures.     

Accuracy of fear inventories and self-efficacy scales in predicting agoraphobic behavior

Self-rating fear inventories are widely used as primary dependent measures of therapeutic change in agoraphobia, but objective behavioral evidence of their validity has been largely unavailable. In contrast, self-efficacy scales are rarely used in agoraphobia research despite evidence that they correlate highly with agoraphobic behavior. In the present study, 37 agoraphobics completed the Marks and Mathews Fear Questionnaire, the Mobility Inventory for Agoraphobia, a set of agoraphobia-related self-efficacy scales, and a set of agoraphobia-related behavioral avoidance tests in the natural community environment. The Fear Questionnaire and Mobility Inventory items proved to be only modestly predictive of agoraphobic dysfunction, whereas the self-efficacy scales were highly predictive of agoraphobic dys-function. Although self-efficacy scales are not intended to be substitute measures of behavior, the findings indicate that they estimate agoraphobia much more accurately than do the standard fear inventories.     

Personality and absenteeism: a meta‐analysis of integrity tests

Until recently, research focus has been on a variety of demographic, attitudinal, and organizational variables in predicting and explaining absenteeism. If personality traits predict absenteeism, then it may be possible to use measures of these traits to identify and select job applicants and thereby reduce absenteeism rates. In this research, our goal was to examine whether integrity tests could be used to predict absenteeism. Meta‐analysis was applied to studies of the validity of pre‐employment integrity tests for predicting voluntary absenteeism. Twenty‐eight studies based on a total sample of 13 972 were meta‐analysed. The estimated mean predictive validity of personality‐based integrity tests was 0.33. This operational validity generalized across various predictor scales, organizations, settings, and jobs (SD_ρ = 0.00). Overt integrity tests, however, showed much lower predictive validity for absenteeism and greater variability than personality‐based tests (ρ = 0.09; SD_ρ = 0.16). The results indicate that a personnel selection approach to reducing absenteeism in organizations may be a useful strategy, particularly if personality‐based integrity tests are utilized. Potential explanations for differences between these results and those found for Big Five measures of personality are offered. Future research investigating models of absenteeism should incorporate the personality constructs assessed by integrity tests. Copyright © 2003 John Wiley & Sons, Ltd.     

Predicting Performance with Contextualized Inventories,No Frame‐of‐reference Effect?

A recent meta‐analysis showed that contextualized personality inventories have incremental predictive validity over generic personality inventories when predicting job performance. This study aimed to investigate the differences between two types of contextualization of items: Adding an ‘at work’ tag versus completely modifying items. One hundred thirty‐nine pharmacy assistants from 29 pharmacies filled out a generic, a tagged, and a completely modified personality inventory. The assistants also provided participant reactions for each of the personality inventories. Performance ratings were collected from the supervising pharmacists. We expected to find incremental criterion validity for both the tagged inventory and the completely modified inventory for predicting job performance. However, the results showed an unexpected decrease in predictive validity for the contextualized inventories. Contextualized inventories were liked less than the generic inventory, but were considered somewhat more face valid and predictive by the participants.     

Five scales in search of a construct: exploring continued attachment to parents in college students

The multitude of measures and differences across self-report indices of continued parental attachment raise questions regarding the validity and meaning of these scales. The purpose of this study was to examine the convergent and construct validity of 5 measures of continued parental attachment. Five attachment scales and 6 personality scales were administered to 216 undergraduate students. Factor analyses and correlational analyses indicate that the 5 attachment measures differentiate healthy from pathological bonding with parents, the construct being assessed has multiple dimensions, and scores on these attachment measures are correlated with personify variables as would be expected. These attachment scales appear to assess a construct that is related to attachment, although they may be more illustrative of the general affective quality of relationships. Subsequent assessment efforts should employ more specific ratings and measure concrete behavioral manifestations of attachment to most effectively examine the construct.     

MMPI scales for DSM-III personality disorders: their derivation and correlates

Scales tapping the dimensions of personality disorder as represented in the DSM-III were derived using a combined rational/empirical strategy. The final version of the scales demonstrated both content validity as well as internal consistency. Correlations between the derived scales and between these scales and the original MMPI clinical scales were generally found to be in the expected direction providing preliminary evidence of criterion related validity. It is suggested that the derived scales may represent an advance toward the reliable assessment of DSM-III personality constructs.     

A Five-Factor Measure of Dependent Personality Traits

This study provides convergent, discriminant, and incremental validity data for a new measure of dependent personality traits from the perspective of the five-factor model (FFM). Dependent personality trait scales were constructed as maladaptive variants of FFM facets (e.g., Gullibility as a maladaptive variant of FFM trust). Based on responses from 383 undergraduates, the convergent validity of the Five-Factor Dependency Inventory (FFDI) scales was tested with respect to 2 measures of the FFM, 6 dependency trait scales, and 4 measures of dependent personality disorder. Discriminant validity was tested with respect to FFM facets from alternative domains. Incremental validity was tested with respect to the ability of the FFM dependent personality trait scales to account for variance in 2 established measures of dependency, after variance accounted for by respective FFM facet scales and other measures of DPD was first removed. The results of this study provided support for the validity of the FFDI assessment of dependency from the perspective of the FFM.     

Difficulties in Interpersonal Emotion Regulation: Initial Development and Validation of a Self-Report Measure

Recent attention has focused on the role of interpersonal emotion regulation (IER) in the development and maintenance of a range of forms of psychopathology, including anxiety, depression, and borderline personality disorder (BPD). Despite the relevance of IER in psychopathology, few measures exist to characterize patterns of maladaptive IER. Our aim was to (1) develop a measure of maladaptive IER, (2) begin to explore the factor structure of this new measure, the difficulties in interpersonal regulation of emotions (DIRE), and (3) examine its association with symptoms of psychopathology. In Study 1, 853 Mechanical Turk workers completed the DIRE and measures of psychopathology symptoms. We identified two factors each in the IER and intrapersonal emotion regulation scales. In Study 2, 142 undergraduate students completed the DIRE and daily measures of emotion regulation and coping for 14 days. Preliminary findings suggest that the DIRE has adequate internal consistency and construct and predictive validity. This measure has the potential to supplement future efforts in assessing IER in psychopathology.     

A five-factor measure of dependent personality traits

This study provides convergent, discriminant, and incremental validity data for a new measure of dependent personality traits from the perspective of the five-factor model (FFM). Dependent personality trait scales were constructed as maladaptive variants of FFM facets (e.g., Gullibility as a maladaptive variant of FFM trust). Based on responses from 383 undergraduates, the convergent validity of the Five-Factor Dependency Inventory (FFDI) scales was tested with respect to 2 measures of the FFM, 6 dependency trait scales, and 4 measures of dependent personality disorder. Discriminant validity was tested with respect to FFM facets from alternative domains. Incremental validity was tested with respect to the ability of the FFM dependent personality trait scales to account for variance in 2 established measures of dependency, after variance accounted for by respective FFM facet scales and other measures of DPD was first removed. The results of this study provided support for the validity of the FFDI assessment of dependency from the perspective of the FFM.