Personality assessment: Does the medium matter? No

The equivalence of an Internet administration of personality tests with two other administration formats was assessed using Item Response Theory (IRT) and various other statistical methods. The analyses were conducted on measures of Neuroticism, Extroversion, Agreeableness, and Conscientiousness. A total of 728 participants took part in the study. Participants were randomly assigned to one of three administrative conditions: paper-and-pencil, proctored computer lab, and unproctored Internet. Analyses with IRT, factor analysis, criterion-related validity, and mean differences supported the equivalence of Internet and traditional paper-and-pencil administrations of personality tests.     

Paper-and-pencil and web-based testing: the measurement invariance of the Big Five personality tests in applied settings

This study investigates the measurement equivalence of a five-factor measure of personality across two groups applying for jobs, who completed the same questionnaire using either a paper-and-pencil (n = 429) or a web online answer format (n = 651). The data were collected using the Big Five Questionnaire-2 (BFQ-2; which is a measure of the Five Factor Model) of personality traits. Multiple-group confirmatory factor analysis was used to test for the equivalence of factor covariance and mean structures of the BFQ-2. Findings suggested that the Five Factor Model scales have the same measurement unit and origin across applicants using different administration modes. However, latent means were slightly higher for applicants who responded in a web and unproctored condition than for applicants who completed a paper-and-pencil version of the same test. Practical implications for personality assessment are discussed.     

The Magnitude and Extent of Cheating and Response Distortion Effects on Unproctored Internet‐Based Tests of Cognitive Ability and Personality

The use of unproctored internet‐based testing (UIT) for employee selection is quite widespread. Although this mode of testing has advantages over onsite testing, researchers and practitioners continue to be concerned about potential malfeasance (e.g., cheating and response distortion) under high‐stakes conditions. Therefore, the primary objective of the present study was to investigate the magnitude and extent of high‐ and low‐stakes retest effects on the scores of a UIT speeded cognitive ability test and two UIT personality measures. These data permitted inferences about the magnitude and extent of malfeasant responding. The study objectives were accomplished by implementing two within‐subjects design studies ( Study 1 N=296; Study 2 N=318) in which test takers first completed the tests as job applicants (high‐stakes) or incumbents (low‐stakes) then as research participants (low‐stakes). For the speeded cognitive ability measure, the pattern of test score differences was more consonant with a psychometric practice effect than a malfeasance explanation. This result is likely due to the speeded nature of the test. And for the UIT personality measures, the pattern of higher high‐stakes scores compared with lower low‐stakes scores is similar to those reported for proctored tests in the extant literature. Thus, our results indicate that the use of a UIT administration does not uniquely threaten personality measures in terms of elevated scores under high‐stakes testing that are higher than those observed for proctored tests in the extant literature.     

Proctored Versus Unproctored Internet Tests: Are unproctored noncognitive tests as predictive of job performance?

As the usage of unproctored Internet testing (UIT) increases in selection settings, concerns about the validity of such practices must be addressed. While recent examinations of the issue have focused primarily on the practicality, ethics, and potential legal ramifications of UIT, this paper provides an examination of the criterion‐related validity of unproctored assessments. Using a database of validity evidence, we examine the predictive validity of several noncognitive assessments (i.e., personality and biodata) with respect to a number of subjective and objective job performance metrics. Results generally indicate that assessments administered in proctored and unproctored settings have similar validities. Limitations of this research and implications for practice are discussed.     

Using Invariance to Examine Cheating in Unproctored Ability Tests

Despite their widespread use in personnel selection, there is concern that cheating could undermine the validity of unproctored Internet‐based tests. This study examined the presence of cheating in a speeded ability test used for personnel selection. The same test was administered to applicants in either proctored or unproctored conditions. Item response theory differential functioning analyses were used to evaluate the equivalence of the psychometric properties of test items across proctored and unproctored conditions. A few items displayed different psychometric properties, and the nature of these differences was not uniform. Theta scores were not reflective of widespread cheating among unproctored examinees. Thus, results were not consistent with what would be expected if cheating on unproctored tests was pervasive.     

Selection Testing via the Internet: Practical Considerations and Exploratory Empirical Findings*

Despite a growing body of applied research on using the Internet for some human resource management practices, few studies have provided equivalence information or practical lessons concerning selection testing via the Internet. We identify several issues associated with measurement and validity, the role of several individual characteristics, respondents' reactions and behaviors, and other considerations concerning Internet test administration. We also report results from an exploratory study of the correlation between paper‐and‐pencil and Internet‐administered cognitively oriented selection tests (including timed and untimed, proctored tests). Our empirical results suggest modest degrees of cross‐mode equivalence for an untimed situational judgment test (r= .84) and for a timed cognitive ability test (r= .60). Further, some types of items (math, verbal, spatial) in the timed cognitive ability test seem to play a differential role in the reduced cross‐mode equivalence. New issues regarding the perception of, and reaction to, items presented via the Internet are presented, and a variety of practical issues are derived and discussed.     

The Use of Mobile Devices in High‐stakes Remotely Delivered Assessments and Testing

With Internet access no longer restricted to desktop and laptop computers, job applicants now have the opportunity to complete remotely delivered assessments on mobile, handheld small screen devices such as smartphones, and personal digital assistants. In this study, a large dataset is used to investigate demographic and score differences between job applicants who completed a remotely delivered high‐stakes assessment on a mobile device and those who completed it on a nonmobile device. Based on a sample of 3,575,207 job applicants who completed an unproctored Internet‐based assessment between January 2011 and April 2012, the percentage of applicants completing the assessment on a mobile device was small, 1.93%, but nevertheless represented more than 69,000 people. Overall, there were small test‐taker demographic differences in the use of mobile devices versus nonmobile devices in that mobile devices were slightly more likely to be used by women, African‐Americans and Hispanics, and younger applicants. Scores on a personality measure were similar for mobile and nonmobile devices but scores on a general mental ability test were substantially lower for mobile devices. Tests of measurement invariance also indicated equivalence across the mobile and nonmobile samples. Test taker and organizational implications for completing remotely delivered high‐stakes noncognitive and cognitive assessments on mobile versus nonmobile devices are discussed.     

Internet versus paper‐and‐pencil survey methods in psychological experiments: Equivalence testing of participant responses to health‐related messages

Despite experiments being increasingly conducted over the Internet, few studies have tested whether such experiments yield data equivalent to traditional methods' data. In the current study, data obtained via a traditional sampling method of undergraduate psychology students completing a paper‐and‐pencil survey (N = 107) were compared with data obtained from an Internet‐administered survey to a sample of self‐selected Internet‐users (N = 94). The data examined were from a previous study that had examined the persuasiveness of health‐related messages. To the extent that Internet data would be based on a sample at least as representative as data derived from a traditional student sample, it was expected that the two methodologies would yield equivalent data. Using formal tests of equivalence on persuasion outcomes, hypotheses of equivalence were generally supported. Additionally, the Internet sample was more diverse demographically than the student sample, identifying Internet samples as a valid alternative for future experimental research.     

How Secure are Unproctored Pre-Employment Tests? Analysis of Inconsistent Test Scores

Employment tests have long been scrutinized for psychometric considerations such as validity and reliability, but the extent to which cheating may occur on pre-employment tests has generally been overlooked. With the rise of unproctored, on-demand, online testing, the need has never been greater to focus on design and process considerations that can help mitigate the potential for cheating on employment tests. This paper builds on the limited existing research on the detection of inconsistent test scores on unproctored Internet testing. Job candidates (n = 4,026) completed a computer adaptive cognitive ability test under two conditions: an unproctored screening test followed by a proctored confirmation test. Analyses focused on detecting instances of inconsistent test scores based on comparison of standard errors of measure for the unproctored and proctored test scores (Guo and Drasgow, Int J Sel Assess 18:351–364, 2010). Results revealed a relatively low number of inconsistent scores and are discussed in the context of future research, application, and theory building on the nature of cheating on pre-employment tests.     

Two‐Step Testing in Employee Selection: Is score inflation a problem?

Unproctored Internet testing in employee selection has become increasingly popular over the past few years. However, there is a concern that cheating during unproctored administrations may influence the test results in terms of score inflation. The current research attempts to determine the extent of cheating on an unproctored Internet test of perceptual speed by analyzing data from a parallel version of the test administered in a proctored setting. Among a sample of 856 job applicants from nine European countries, there was no evidence that cheating occurred. In fact, evidence of an effect operating in the opposite direction was obtained. These results indicate that the use of unproctored Internet testing is appropriate for this type of test.     

Fakability of Implicit and Explicit Measures of the Big Five: Research findings from organizational settings

This study investigates the extent to which response distortion occur when implicit measures of personality traits are used in applied settings. Two groups composed of 58 security guards and 45 semiskilled workers, respectively, completed five Implicit Association Tests (IATs) for assessing the Big Five personality traits as a part of a personnel assessment program. They additionally completed a self‐report measure of the same personality dimensions. Scores on the Big Five IATs and self‐ratings of personality were obtained also by 52 volunteers who responded anonymously for research purposes. Results showed that participants under evaluative testing conditions scored significantly higher than volunteers on explicit measures of agreeableness, conscientiousness, and emotional stability. On the contrary, no significant differences were found in the Big Five IATs. Practical implications of findings and future research directions are discussed.     

The importance of traits and abilities in supervisors' hirability decisions as a function of method of assessment

Past research on the importance of traits and abilities in supervisors' hirability decisions has ignored the influence of the selection method used to derive information about these traits and abilities. In this study, experienced retail store supervisors (N=163) rated job applicant profiles that were described on the Big Five and General Mental Ability (GMA) personality dimensions. Contrary to past studies, the supervisors were also informed about the method of assessment used (paper‐and‐pencil test vs. unstructured interview). Hierarchical linear modelling analyses showed that the importance attached to extraversion and GMA was significantly moderated by the selection method, with extraversion and GMA decreasing in importance when store supervisors knew that scores on extraversion and GMA were derived from a paper‐and‐pencil test as opposed to from an unstructured interview. Store supervisors with more selection‐related experience also attached more importance to GMA. Results are discussed in relation to the practice–science gap and the extant literature on perceptions of selection procedures.     

Personality‐relevant attribute‐nouns: a taxonomic study in the Italian language

The present study was based on psycholexical approach premises and explored the structure of a large set of personality attribute‐nouns in the Italian language. Content inspection (based on Big Five categories) and quantitative indices were used to interpret the attribute‐noun dimensions. Results showed (a) a stable three‐component solution which replicated the Big Three; (b) an unstable five‐factor solution which did not reproduce the Big Five; (c) an unstable six‐factor solution which represented deviations from the Big Five system, which have been found in most psycholexical studies conducted in the Italian language. The six lexical dimensions were interpreted as follows: Conscientiousness (replicating the III of the Big Five); Self‐Assurance (combining the Big Five I assertiveness and IV fearfulness subcategories); Sociability (defined by the Big Five I sociableness and I impulse expression subcategories); Placidity (combining the Big Five II peacefulness, II unassertiveness, and IV irritableness subcategories); Honesty and Humility (comprising the Big Five II modesty and II helpfulness subcategories plus integrity values); Cleverness and Sophistication (defined by the Big Five V subcategories). The conclusion was that personality word organisation in the Italian language reflects the psycholexical Big Three and Big Six, but not the Big Five. Copyright © 2005 John Wiley & Sons, Ltd.     

IRT-related factor analytic procedures for testing the equivalence of paper-and-pencil and Internet-administered questionnaires

This article describes a general item response theory-based factor analytic procedure that allows assessment of the equivalence between 2 administrative modes of a questionnaire: paper and pencil, and Internet based. The theoretical relations between the present procedure and other methods used in previous empirical research are shown, and the advantages of the procedure are discussed. An empirical application based on 2 personality questionnaires is given, and the results are compared with the results of using traditional procedures for assessing equivalence. The substantive implications of the results, as well as suggestions for further research and methodology, are discussed.     

The relation between personality and prejudice: a variable‐ and a person‐centred approach

The relationship between Big Five personality (measured by the NEO‐PI) and prejudice was examined using a variable‐ and a person‐centred approach. Big Five scores were related to a generalized prejudice factor based on seven different prejudice scales (racial prejudice, sexism, etc). A correlation analysis disclosed that Openness to Experience and Agreeableness were significantly related to prejudice, and a multiple regression analysis showed that a variable‐centred approach displayed a substantial cross‐validated relationship between the five personality factors and prejudice. A cluster analysis of the Big Five profiles yielded, in line with previous research, three personality types, but this person‐centred approach showed a low cross‐validated relationship between personality and prejudice, where the overcontrolled type showed the highest prejudice and the undercontrolled the lowest, with the resilient falling in between. A head‐to‐head comparison sustained the conclusion that, based on people's Big Five personalities, their generalized prejudice could be predicted more accurately by the variable‐ than the person‐centred approach. Copyright © 2003 John Wiley & Sons, Ltd.     

The Stability of the Five‐Factor Model of Personality in Personnel Selection and Assessment in Greece

The stability and replicability of the Five‐Factor model of personality across samples and testing purposes remain a significant issue in personnel selection and assessment. The present study explores the stability of a new Greek Big Five personality measure (TPQue) across different samples in order to explore the suitability of the measure in personnel selection and assessment. The factor structure of the measure across three samples (students, employees, and job applicants) is examined. The results of exploratory and confirmatory factor analyses show that the five‐factor structure remains intact for the students’, the applicants’ and the employees’ samples – contrary to previous studies – with all the sub‐scales of the personality measure (TPQue) loading on the intended factors. Furthermore, congruence coefficients between the samples justify the stability of the model in the working settings.     

The Five Factor Model in personnel selection: Measurement equivalence between applicant and non-applicant groups

This study investigates the measurement equivalence of a Five Factor measure of personality between applicant and non-applicant samples. The Big Five Questionnaire-2 was administered in two samples: A group of volunteers (n = 903), who completed the test for research purposes, and a group applying for jobs, who completed the test during hiring procedures (n = 401). Multiple-group confirmatory factor analysis on item composites was conducted to test for the equivalence of factor covariance and mean structures of twenty facets of the Five Factors. Item-level analyses were carried out through analysis of variance to further examine the issue of measurement invariance. Findings suggested that personality facets have the same measurement unit across applicants and non-applicants, while a lack of equivalence was found in the origin of the scales. Similar results were found at the item-level. Implications for personality assessment are advanced and discussed.     

Personality traits and health‐risk behaviours in university students

Relations between personality and health‐risk behaviours in university undergraduates were examined using multiple measures of personality across multiple samples (N = 1151). Big Five personality variables, at both factor and facet levels, were used to predict three specific health‐risk behaviours: (a) tobacco consumption, (b) alcohol consumption and (c) speeding in an automobile. Our findings showed that low Conscientiousness and low Agreeableness were uniformly associated with this cluster of potentially health damaging behaviours. Extraversion was additionally associated with alcohol use. Interaction effects were found between Conscientiousness and Agreeableness on smoking and (for men only) on drinking. Other personality variables not centrally related to the Big Five, such as Risk‐Taking (high) and Integrity (low), were also implicated in the present health‐risk behaviours. Copyright © 2009 John Wiley & Sons, Ltd.     

Dimensions of Normal and Abnormal Personality: Elucidating DSM‐IV Personality Disorder Symptoms in Adolescents

ABSTRACT The present study aimed to elucidate dimensions of normal and abnormal personality underlying DSM‐IV personality disorder (PD) symptoms in 168 adolescents referred to mental health services. Dimensions derived from the Big Five of normal personality and from Livesley's (2006) conceptualization of personality pathology were regressed on interview‐based DSM‐IV PD symptom counts. When examined independently, both models demonstrated significant levels of predictive power at the higher order level. However, when added to the higher order Big Five dimensions, Livesley's higher and lower order dimensions afforded a supplementary contribution to the understanding of dysfunctional characteristics of adolescent PDs. In addition, they contributed to a better differentiation between adolescent PDs. The present findings suggest that adolescent PDs are more than extreme, maladaptive variants of higher order normal personality traits. Adolescent PDs seem to encompass characteristics that may be more completely covered by dimensions of abnormal personality. Developmental issues and implications of the findings are discussed.