Socially desirable responding and its elusive effects on the validity of personality assessments

Past studies of socially desirable self-reports on the items of personality measures have found inconsistent effects of the response bias on the measures' predictive validities, with some studies reporting small effects and other studies reporting large effects. Using Monte Carlo methods, we evaluated various models of socially desirable responding by systematically adding predetermined amounts of the bias to the simulated personality trait scores of hypothetical test respondents before computing test-criterion validity correlations. Our study generally supported previous findings that have reported relatively minor decrements in criterion prediction, even with personality scores that were massively infused with desirability bias. Furthermore, the response bias failed to reveal itself as a statistical moderator of test validity or as a suppressor of validity. Large differences between some respondents' obtained test scores and their true trait scores, however, meant that the personality measure's construct validity would be severely compromised and, more specifically, that estimates of those individuals' criterion performance would be grossly in error. Our discussion focuses on reasons for the discrepant results reported in the literature pertaining to the effect of socially desirable responding on criterion validity. More important, we explain why the lack of effects of desirability bias on the usual indicators of validity, moderation, and suppression should not be surprising.     

Big secrets do not necessarily cause hills to appear steeper

Slepian, Masicampo, Toosi, and Ambady (Journal of Experimental Psychology: General, 141, 619–624, 2012, Study 1) found that individuals recalling and writing about a big, meaningful secret judged a pictured hill as steeper than did those who recalled and wrote about a small, inconsequential secret (with estimates unrelated to physical effort unaffected). From an embodied cognition perspective, this result was interpreted as suggesting that important secrets weigh people down. Answering to mounting calls for the crucial need of independent direct replications of published findings to ensure the self-correcting nature of our science, we sought to corroborate Slepian et al.’s finding in two extremely high-powered, preregistered studies that were very faithful to all procedural and methodological details of the original study (i.e., same cover story, study title, manipulation, measures, item order, scale anchors, task instructions, sampling frame, population, and statistical analyses). In both samples, we were unsuccessful in replicating the target finding. Although Slepian et al. reported three other studies supporting the secret burdensomeness phenomenon, we advise that these three other findings need to be independently corroborated before the general phenomenon informs theory or health interventions.     

Understanding patterns of attitude change: When implicit measures show change, but explicit measures do not

A common assumption in research on attitudes is that indirect measures assess relatively stable implicit attitudes, whereas traditional self-report measures assess more recently acquired explicit attitudes that coexist with old, presumably stable implicit attitudes. This assumption seems difficult to reconcile with research showing experimentally induced changes on implicit but not explicit measures. The present research tested a process-account of such asymmetrical patterns. Specifically, we argue that implicit measures show experimental effects that do not emerge on explicit measures when (a) the pairing of an attitude object with positive or negative valence creates new automatic associations in memory, and, at the same time, (b) the consideration of additional information about the attitude object eliminates the impact of automatic associations on self-reported evaluative judgments. Results from three studies support these predictions. Implications for research on attitude change are discussed.     

How to find what's in a name: Scrutinizing the optimality of five scoring algorithms for the name‐letter task

Although the name‐letter task (NLT) has become an increasingly popular technique to measure implicit self‐esteem (ISE), researchers have relied on different algorithms to compute NLT scores and the psychometric properties of these differently computed scores have never been thoroughly investigated. Based on 18 independent samples, including 2690 participants, the current research examined the optimality of five scoring algorithms based on the following criteria: reliability; variability in reliability estimates across samples; types of systematic error variance controlled for; systematic production of outliers and shape of the distribution of scores. Overall, an ipsatized version of the original algorithm exhibited the most optimal psychometric properties, which is recommended for future research using the NLT. Copyright © 2009 John Wiley & Sons, Ltd.     

What Makes Mental Associations Personal or Extra‐Personal? Conceptual Issues in the Methodological Debate about Implicit Attitude Measures

Over the last decade, a new class of indirect measurement procedures has become increasingly popular in many areas of psychology. However, these implicit measures have also sparked controversies about the nature of the constructs they assess. One controversy has been stimulated by the question of whether some implicit measures (or implicit measures in general) assess extra‐personal rather than personal associations. We argue that, despite empirical and methodological advances stimulated by this debate, researchers have not sufficiently addressed the conceptual question of how to define extra‐personal in contrast to personal associations. Based on a review of possible definitions, we argue that some definitions render the controversy obsolete, whereas others imply fundamentally different empirical and methodological questions. As an alternative to defining personal and extra‐personal associations in an objective sense, we suggest an empirical approach that investigates the meta‐cognitive inferences that make a given association subjectively personal or extra‐personal for the individual.     

Implicit partner affect, relationship satisfaction, and the prediction of romantic breakup

The current research investigated the role of spontaneous partner feelings (implicit partner affect) in the dynamics of relationship satisfaction, commitment, and romantic dissolution. Participants completed a variant of the name-letter task as a measure of implicit partner affect, and self-report measures of relationship satisfaction and commitment. Approximately 4 months later, participants were contacted to assess their current relationship status. Overall, participants showed a biased preference for their partner’s initials (after adjusting for proper baselines), indicating the presence of positive implicit partner affect. Participants with more positive implicit partner affect were more satisfied with, but not more committed to, their relationship. Additionally, implicit partner affect exerted a significant indirect effect on relationship stability. These effects were independent of relationship length, age, and gender. Implications for the role of automatic affective processes in relationship processes and the utility of indirect measures for shedding light on relationship dynamics are discussed.     

What Do Implicit Measures Tell Us? Scrutinizing the Validity of Three Common Assumptions

ABSTRACT— Experimental paradigms designed to assess "implicit" representations are currently very popular in many areas of psychology. The present article addresses the validity of three widespread assumptions in research using these paradigms: that (a) implicit measures reflect unconscious or introspectively inaccessible representations; (b) the major difference between implicit measures and self-reports is that implicit measures are resistant or less susceptible to social desirability; and (c) implicit measures reflect highly stable, older representations that have their roots in long-term socialization experiences. Drawing on a review of the available evidence, we conclude that the validity of all three assumptions is equivocal and that theoretical interpretations should be adjusted accordingly. We discuss an alternative conceptualization that distinguishes between activation and validation processes.