Comparing Personality Test Formats and Warnings: Effects on criterion‐related validity and test‐taker reactions

The potential for applicant response distortion on personality measures remains a major concern in high‐stakes testing situations. Many approaches to understanding response distortion are too transparent (e.g., instructed faking studies) – or are too subtle (e.g., correlations with social desirability measures as indices of faking). Recent research reveals more promising approaches in two methods: using forced‐choice (FC) personality test items and warning against faking. The present study examined effects of these two methods on criterion‐related validity and test‐taker reactions. Results supported incremental validity for an FC and Likert‐scale measure in warning and no‐warning conditions, above and beyond cognitive ability. No clear differences emerged between the FC vs Likert measures or warning vs no‐warning conditions in terms of validity. However, some evidence suggested that FC measures and warnings may produce negative test‐taker reactions. We conclude with implications for implementation in selection settings.     

Predictive validity of an Implicit Association Test for assessing anxiety

The Implicit Association Test (IAT) was adapted to measure anxiety by assessing associations of self (vs. other) with anxiety-related (vs. calmness-related) words. Study 1 showed that the IAT-Anxiety exhibited good internal consistency and adequate stability. Study 2 revealed that the IAT-Anxiety was unaffected by a faking instruction. Study 3 examined the predictive validity of implicit and explicit measures and showed that the IAT-Anxiety was related to changes in experimenter-rated anxiety and performance decrements after failure. Study 4 found that several behavioral indicators of anxiety during a stressful speech were predicted by the IAT. Taken together, these studies show that the IAT-Anxiety is a reliable measure that is able to predict criterion variables above questionnaire measures of anxiety and social desirability.     

How do you fake a personality test? An investigation of cognitive models of impression-managed responding

Because faking poses a threat to the validity of personality measures, research has focused on ways of detecting faking, including the use of response times. However, the applicability and validity of these approaches are dependent upon the actual cognitive process underlying faking. This study tested three competing cognitive models in order to identify the process underlying faking and to determine whether response time patterns are a viable method of detecting faking. Specifically, we used a within-subjects manipulation of instructions (respond honestly, make a good impression, make a specific impression) to examine whether the distribution of response times across response scale options (e.g., disagree, agree) could be used to identify faking on the NEO PI-R. Our results suggest that individuals reference a schema of an ideal respondent when faking. As a result, response time patterns such as the well-known inverted-U cannot be used to identify faking.     

Do response time limitations counteract the effect of faking on personality inventory validity?

Different models of lying on personality scales make discrepant predictions on the association between faking and item response time. The current research investigated response time restriction as a method for reducing the influence of faking on personality scale validity. In 3 assessment simulations involving 540 university undergraduates responding to 2 common, psychometrically strong personality inventories, no evidence emerged to indicate that limiting respondents' answering time can attenuate the effects of faking on validity. Results were interpreted as failing to support a simple model of personality test item response dissimulation that predicts that lying takes time. Findings were consistent with models implying that lying involves primitive cognitive processing or that lying may be associated with complex processing that includes both primitive responding and cognitive overrides.     

The Psychometric Costs of Applicants' Faking: Examining Measurement Invariance and Retest Correlations Across Response Conditions

This study examines the stability of the response process and the rank-order of respondents responding to 3 personality scales in 4 different response conditions. Applicants to the University College of Teacher Education Styria (N = 243) completed personality scales as part of their college admission process. Half a year later, they retook the same personality scales in 1 of 3 randomly assigned experimental response conditions: honest, faking-good, or reproduce. Longitudinal means and covariance structure analyses showed that applicants' response processes could be partially reproduced after half a year, and respondents seemed to rely on an honest response behavior as a frame of reference. Additionally, applicants' faking behavior and instructed faking (faking-good) caused differences in the latent retest correlations and consistently affected measurement properties. The varying latent retest correlations indicated that faking can distort respondents' rank-order and thus the fairness of subsequent selection decisions, depending on the kind of faking behavior. Instructed faking (faking-good) even affected weak measurement invariance, whereas applicants' faking behavior did not. Consequently, correlations with personality scales—which can be utilized for predictive validity—may be readily interpreted for applicants. Faking behavior also introduced a uniform bias, implying that the classically observed mean raw score differences may not be readily interpreted.     

The effect of cognitive load on faking interrogative suggestibility on the Gudjonsson Suggestibility Scale

In the light of recent studies into the impact of cognitive load on detecting deception, the impact of cognitive load on faking on the Gudjonsson Suggestibility Scale (GSS) was investigated. Eighty undergraduate students participated in the study, and were randomly assigned to one of four conditions resulting from a combination of the factors: instruction type (genuine or instructed faking, see Hansen, Smeets, & Jelicic, 2009) and concurrent task (yes or no). Findings show that instructed fakers, not performing a concurrent task, score significantly higher on yield 1 in comparison to genuine interviewees. This is in-line with previous studies into faking on the GSS. However, instructed fakers, performing a concurrent task, achieved significantly lower yield 1 scores than instructed fakers not performing a concurrent task. Genuine (non fakers) showed a different response to increased cognitive load during the dual-task paradigm. This study suggests that increasing cognitive load may potentially indicate (and preclude) faking attempts on the yield dimension of the Gudjonsson Suggestibility Scale.     

Detection of feigned memory impairments using a Chinese word task

In this study the detection of feigned memory impairments was investigated using a forced-choice task with Chinese words as stimuli. Task difficulty on accuracy and response time and the faking strategies employed were examined for 85 participants, 42 men and 43 women, who were administered the experimental task. Following a simulation design, all participants performed under control and faking conditions. Their accuracy of performance and response times were measured. Analysis indicated significant differences on both outcome measures for the two conditions. Further, a significant difference in accuracy, but not in response time, among levels of task difficulty was observed. For classification accuracy, using the cut-off score criterion seemed more effective than using the below-chance criterion or response time. Implications for relying on chance criterion to identify deception are discussed.     

Criterion‐Related Validity of Dutch Police‐Selection Measures and Differences Between Ethnic Groups

This study investigated the criterion‐related validity of cognitive ability as well as non‐cognitive ability measures and differences between ethnic majority (N=2365) and minority applicants (N=682) in Dutch police officer selection. Findings confirmed the relatively low predictive validity of cognitive ability generally found for police jobs. Previous research reported no differential prediction. The present study, however, found small but systematic evidence for differences in validity for the ethnic majority and minority group of both cognitive and non‐cognitive measures. For the minority group, training performance appeared to be mainly predicted by the cognitive ability test. For the majority group, cognitive ability showed very little predictive power. Non‐cognitive ability variables appeared to be somewhat more predictive in this group.     

What if we fake emotional intelligence? A test of criterion validity attenuation

We evaluate the extent to which established self-report measures of emotional intelligence (EI) are susceptible to socially desirable responding. A study was conducted to assess the relationship between EI and multiple outcomes across 3 experimental faking conditions. Using structural equation modeling, we found that the criterion validities of self-report EI measures in the prediction of life satisfaction, psychological distress, rational coping, and detachment coping are not attenuated in moderate social desirability settings, but are somewhat attenuated when faking is maximized. Moreover, partialing out social desirability does not yield any improvement of the predictive validity of self-report EI.     

Biodata Validity Decay and Score Inflation with Faking: Do Item Attributes Explain Variance Across Items?

Rating scales were developed to assess the biodata dimensions offered by Mael (1991). Biodata items assessing conscientiousness were administered under honest-responding and faking-good conditions. Item attributes were examined to determine their value in predicting item validity for honest respondents and item validity for faking respondents. Analyses were also conducted to determine whether the degree of item faking was related to item attributes. Item attributes associated with item validity for honest respondents are not the same as the item attributes indicative of item validity for the faking respondents. We suggest that this makes it very difficult to develop a biodata questionnaire which will be equally valid for both honest and faking respondents.     

成年人个性特征与某些认知作业的关系

采用NEO-FFI个性问卷研究了个性特征与记忆、问题解决及视觉搜索反应时的关系。作业难度均属中下。被试为20－85岁四组成年人120名。结果：（1）相关和多元逐步回归分析结果完全一致：外向性（E）对人姓回忆及图形再认有负性预测关系,开放性（O）对人姓回忆及猜图用策略百分比有正性预测关系。（2）控制年龄因素的偏相关结果表明,神经质分（N）和清图用策略百分比,E分和图形再认、人姓回忆,认真负责分（C）和数字、字母、汉字搜索反应时均呈负相关。（3）认知指标加入个人情况诸因素作为自变量与个性特征求多元逐步回归方程显示,解决问题方案数量与E分呈负性预测关系,心理状态等因素与个性预测影响更大。（4）认知作业与个性特征关系各年龄组结果不同,两者关系青年组和老老年组较大,这可能与不同年龄成年人神经系统功能状态差异有关。     

Examining the item response process to personality measures in high-stakes situations: Issues of measurement validity and predictive validity

We conducted two experimental studies with between-subjects and within-subjects designs to investigate the item response process for personality measures administered in high- versus low-stakes situations. Apart from assessing measurement validity of the item response process, we examined predictive validity; that is, whether or not different response models entail differential selection outcomes. We found that ideal point response models fit slightly better than dominance response models across high- versus low-stakes situations in both studies. Additionally, fitting ideal point models to the data led to fewer items displaying differential item functioning compared to fitting dominance models. We also identified several items that functioned as intermediate items in both the faking and honest conditions when ideal point models were fitted, suggesting that ideal point model is “theoretically” more suitable across these contexts for personality inventories. However, the use of different response models (dominance vs. ideal point) did not have any substantial impact on the validity of personality measures in high-stakes situations, or the effectiveness of selection decisions such as mean performance or percent of fakers selected. These findings are significant in that although prior research supports the importance and use of ideal point models for measuring personality, we find that in the case of personality faking, though ideal point models seem to have slightly better measurement validity, the use of dominance models may be adequate with no loss to predictive validity.     

The impact of corrections for faking on the validity of noncognitive measures in selection settings

In selection research and practice, there have been many attempts to correct scores on noncognitive measures for applicants who may have faked their responses somehow. A related approach with more impact would be identifying and removing faking applicants from consideration for employment entirely, replacing them with high-scoring alternatives. The current study demonstrates that under typical conditions found in selection, even this latter approach has minimal impact on mean performance levels. Results indicate about .1 SD change in mean performance across a range of typical correlations between a faking measure and the criterion. Where trait scores were corrected only for suspected faking, and applicants not removed or replaced, the minimal impact the authors found on mean performance was reduced even further. By comparison, the impact of selection ratio and test validity is much larger across a range of realistic levels of selection ratios and validities. If selection researchers are interested only in maximizing predicted performance or validity, the use of faking measures to correct scores or remove applicants from further employment consideration will produce minimal effects.     

Ask and ye shall receive: behavioural specificity in the accuracy of subjective memory complaints

A cross‐sectional sample of adults completed an extensive set of cognitive tasks and a set of questionnaires measuring depressive affect, memory complaint, and other variables. During an interview about their prescribed medications, the participants also reported whether they were having problems remembering to take the medication as prescribed (an everyday prospective memory problem). Their medication adherence at home was then monitored for one month using pill bottles which microelectronic caps. Cognitive tasks correlated with memory complaints, as measured by the Memory Functioning Questionnaire, but not with problems in remembering to take medications. The highest correlations were with a free recall task. Conversely, reported problems with medication adherence during the interview had good predictive validity for subsequent adherence problems, but not for cognitive tasks, including a measure of prospective memory. Depressive affect was related to both the questionnaire and the interview complaints about medication adherence, but a structural equation model showed that the relationships of cognition and medication adherence to the different memory complaints were independent of depressive affect. The results are interpreted in terms of a behavioural specificity hypothesis, which states that adults' self‐reports of memory problems are valid when they focus directly on specific memory‐related behaviours in everyday contexts. Copyright © 2000 John Wiley & Sons, Ltd.     

Applicant Faking,Social Desirability,and the Prediction of Counterproductive Work Behaviors

Recent studies have pointed to within-subjects designs as an especially effective tool for gauging the occurrence of faking behavior in applicant samples. The current study utilized a within-subjects design and data from a sample of job applicants to compare estimates of faking via within-subjects score change to estimates based on a social desirability scale. In addition, we examined the impact of faking on the relationship between Conscientiousness and counterproductive work behaviors (CWBs), as well as the direct linkage between faking and CWBs. Our results suggest that social desirability scales are poor indicators of within-subjects score change, and applicant faking is both related to CWBs and has a negative impact on the criterion-related validity of Conscientiousness as a predictor of CWBs.     

The Impact of Exposure to Internet-Based Information About the Rorschach and the MMPI–2 on Psychiatric Outpatients’ Ability to Simulate Mentally Healthy Test Performance

To examine the impact of Internet-based information about how to simulate being mentally healthy on the Rorschach (Exner, 2003) and the MMPI–2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989), 87 psychiatric outpatients completed the tests under 4 conditions: uncoached and Internet-coached outpatients under faking healthy instructions (faking patients and Internet-faking patients) and patients and nonpatients under standard instructions (standard patients and standard nonpatients). On the Rorschach, faking patients and Internet-faking patients did not manage to portray healthy test performance and, like standard patients, revealed a significantly greater number of perceptual and cognitive disturbances than standard nonpatients. Faking patients scored in the psychopathological direction on most variables. Internet-faking patients produced constricted protocols with significantly higher F% (57%) and lower use of provoking and aggressive contents than the other groups. On the MMPI–2, faking patients and Internet-faking patients were able to conceal symptoms and, like standard nonpatients, scored in the normal range on the clinical scales. The validity scale L successfully detected the faking patients and the Internet-faking patients, whereas the F scale only distinguished the Internet-faking patients and K only the faking patients. We conclude that Internet-based information could threaten test validity.     

Are observer ratings of applicants’ personality also faked? Yes,but less than self‐reports

Although self‐report personality tests are a comparatively cheap and easy‐to‐administer personnel selection tool, researchers have criticized them for not predicting enough criterion‐related variance. Researchers have suggested using observer‐ratings of personality (e.g., as part of a reference check from a supervisor) because observer‐ratings have been reported to be more predictive. However, it is theoretically and empirically unclear whether supervisors also engage in faking (the intentional distortion of responses). Study 1 explored faking among managers who were first asked to imagine that a subordinate had to leave his/her job for private reasons and then to rate the personality of the subordinate. A week later, managers rated their subordinates honestly. A repeated‐measures MANOVA indicated that managers did fake. Study 2 (among supervisors of working students) replicated the above findings but also showed that there is less faking in supervisor‐ratings than in self‐ratings. Furthermore, we found no evidence that the validity of personality scales for predicting academic performance depends on self‐ versus observer‐ratings or on an applicant versus an honest condition. These two studies thus show that practitioners should not equate personality ratings obtained from observers in a selection context with honest personality ratings.     

A review of applicant faking in selection interviews

Interviews are commonly used for selection but research on interview faking only gained momentum relatively recently. We review both theoretical and empirical work on prevalence, antecedents, processes, and effects of interview faking. Most applicants fake at least to some degree. Personality (e.g., Conscientiousness, Honesty‐humility, the Dark Triad) and attitudes toward faking substantially correlate with faking behaviors. Research concerning applicants' ability, interview structure components, or contextual factors is limited. Furthermore, the impact of faking on interview ratings is mixed and effects on criterion‐related validity are not consistently negative. Finally, the detection of faking seems hardly possible and there are limited options available to reduce interview faking. Throughout our review, we describe important gaps and derive suggestions and propositions for future research.     

On the controllability of evaluative-priming effects: Some limits that are none

Two experiments examined recent claims of uncontrollability of the evaluative-priming effect. According to these claims, imposing an adaptive 600 ms response deadline prevents successful faking (Degner, 2009). Furthermore, strategic control attempts have been argued not to reduce the priming measure's sensitivity to spontaneous evaluations so that correlations of evaluative-priming effects with external criteria are not affected by attempts to fake (Bar-Anan, 2010). Here, we show that faking is possible even with an adaptive 600 ms response deadline when faking instructions do not conflict with speed pressures imposed thereby (Experiments 1 and 2). In addition, suitable faking instructions substantially affect the predictive validity of priming effects in terms of their correlations with (non-faked) self-report measures and the Implicit Association Test (Experiment 2). The previous claims about the uncontrollability of the evaluative-priming effect may thus have been premature.     

Variation in working memory capacity,fluid intelligence,and episodic recall: A latent variable examination of differences in the dynamics of free recall

A latent variable analysis was conducted to examine the nature of individual differences in the dynamics of free recall and cognitive abilities. Participants performed multiple measures of free recall, working memory capacity (WMC), and fluid intelligence (gF). For each free recall task, recall accuracy, recall latency, and number of intrusion errors were determined, and latent factors were derived for each. It was found that recall accuracy was negatively related to both recall latency and number of intrusions, and recall latency and number of intrusions were positively related. Furthermore, latent WMC and gF factors were positively related to recall accuracy, but negatively related to recall latency and number of intrusions. Finally, a cluster analysis revealed that subgroups of participants with deficits in focusing the search had deficits in recovering degraded representations or deficits in monitoring the products of retrieval. The results are consistent with the idea that variation in the dynamics of free recall, WMC, and gF are primarily due to differences in search set size, but differences in recovery and monitoring are also important.