Influence of recorder affect on the content of behavioural diaries and the recall of behaviours

Two experiments are described. In Experiment 1, supervisors (N = 4) kept daily performance diaries for each of four subordinates over an eight‐week period. In Experiment 2, students (N = 48) kept behavioural diaries for their instructor over a three‐week period. Daily measures of positive and negative affect were used to predict the favourability, person‐typicality and behavioural specificity of diary entries. Diary‐keepers tended to record behaviours that were consistent with their affect levels. Analyses of the variability of the favourability and person‐typicality ratings further suggested that high negative affect induced diary‐keepers to make fine discriminations among events, while high positive affect induced diary‐keepers to perceive events as similar. Diary‐keeper affect and the three diary content variables were used to predict memory for the diary entries. Diary‐keepers in Experiment 1 reported higher recall for negative behaviours than for positive behaviours, particularly if the behaviours were typical of others. This same negativity effect in recall emerged in Experiment 2, but only when there was low consistency in the positive affect experienced by diary‐keepers at encoding and recall. Copyright © 2001 John Wiley & Sons, Ltd.     

Fairness Reactions to Personnel Selection Methods: A look at professionals in Mumbai,India

This study expands upon Steiner and Gilliland's selection fairness research. Professionals (N = 114) from Mumbai, India rated 12 employee selection methods on favorability and provided the bases for those ratings. In line with previous research, interviews and resumes were rated most favorably, while graphology and honesty tests were rated least favorably. Perceived face validity, opportunity to perform, and widespread use of selection methods were highly correlated with favorability ratings, while interpersonal warmth, scientific evidence, and respectful of privacy exhibited weak correlations with favorability ratings. Work sample tests, which have previously been rated favorably, were rated unfavorably. Exploratory analysis showed that participants viewed assessment centers favorably and online information unfavorably. Outcome favorability was highly correlated with favorability ratings.     

Consensual validity parameters of the Zuckerman-Kuhlman personality questionnaire: evidence from self-reports and spouse reports

One way of validating questionnaire responses is correlating them with ratings made by external assessors who know the ratee well: This is known as consensual validity. In this study, we assessed the consensual validity of the Zuckerman-Kuhlman Personality Questionnaire (ZKPQ; Zuckerman, 2002; Zuckerman, Kuhlman, Joireman, Teta, & Kraft, 1993). A multitrait-multimethod matrix of self-reported and spouse reported personality ratings was used to establish convergent and discriminant validity by means of Campbell and Fiske's (1959) evaluative criteria. Self-reports of 86 men and 85 women were correlated with their spouses' reports. Intraclass correlations ranged from .47 to .63 for the 5 dimensions, providing strong evidence of convergent and discriminant validity. The results obtained favor the use of the ZKPQ as a valid self-report measure of personality traits.     

Affective forecasting and self-rated symptoms of depression,anxiety, and hypomania: Evidence for a dysphoric forecasting bias

Emerging research has examined individual differences in affective forecasting; however, we are aware of no published study to date linking psychopathology symptoms to affective forecasting problems. Pitting cognitive theory against depressive realism theory, we examined whether dysphoria was associated with negatively biased affective forecasts or greater accuracy. Participants (n=325) supplied predicted and actual emotional reactions for three days surrounding an emotionally evocative relational event, Valentine's Day. Predictions were made a month prior to the holiday. Consistent with cognitive theory, we found evidence for a dysphoric forecasting bias—the tendency of individuals in dysphoric states to overpredict negative emotional reactions to future events. The dysphoric forecasting bias was robust across ratings of positive and negative affect, forecasts for pleasant and unpleasant scenarios, continuous and categorical operationalisations of dysphoria, and three time points of observation. Similar biases were not observed in analyses examining the independent effects of anxiety and hypomania. Findings provide empirical evidence for the long-assumed influence of depressive symptoms on future expectations. The present investigation has implications for affective forecasting studies examining information-processing constructs, decision making, and broader domains of psychopathology.     

Evidence for the external validity of a social simulation measure of social skills

Two studies were conducted in order to obtain evidence regarding the external validity of a social simulation measure of social skills. The first study employed a contrasted group design, with the groups consisting of National Guardsmen and psychiatric patients. As predicted, trained judges rated the performances of the National Guardsmen in the simulations as more skillful and appearing less anxious than the psychiatric patients. In the second study, trained judges' ratings of psychiatric patients' performances in the simulations were found to be significantly related to ratings made by interviewers who had conducted a structured social history interview with the patients, nurses' ratings based on their observations of patients after 3 days on the psychiatric unit, a research assistant's ratings based on contact with the patient during the experiment, and finally, the subject's own self-ratings. In discussing the findings, it is hypothesized that one reason why support was found for the external validity of the simulation strategy in this study, in contrast to some previously reported studies, was the use of molar ratings in this study as opposed to the more molecular rating procedures used in those previous studies.These studies were funded in part by a research grant from the Veterans Administration.     

Measuring Creativity with Planned Missing Data

This paper introduces a method for the assessment of creativity that relies on creativity tasks, a subjective evaluation procedure, and a planned missing data design that offers a drastic reduction in the overall implementation costs (administration time and scoring procedure). This method was tested on a sample of 149 people, using three creativity tasks as a basis. Participants were instructed to produce several ideas in each task and then to select what they considered to be their best two ideas (i.e., “Top 2” procedure; Silvia, Winterstein, Willse, Barona, et al., Psychology of Aesthetics, Creativity, and the Arts, 2 , 2008 and 68). These ideas were then evaluated by a panel of peers and experts. Creativity ratings were analyzed with structural equations; measurement models were estimated for each task and correlations between factor-scores across the three tasks were investigated. Further insights regarding validity are provided through systematic investigation of the relationship between fluency scores, creativity ratings, intelligence tasks, self-reported idea generation abilities, and creative activities and achievements. Overall, the results support the viability of this new approach, providing evidence of convergent and discriminant validity. They are discussed in relation to past research and avenues for further extension are proposed.     

Psychometric properties of the mobility inventory for agoraphobia: convergent, discriminant, and criterion-related validity

Aims of this study were (a) to summarize the psychometric literature on the Mobility Inventory for Agoraphobia (MIA), (b) to examine the convergent and discriminant validity of the MIA's Avoidance Alone and Avoidance Accompanied rating scales relative to clinical severity ratings of anxiety disorders from the Anxiety Disorders Interview Schedule (ADIS), and (c) to establish a cutoff score indicative of interviewers’ diagnosis of agoraphobia for the Avoidance Alone scale. A meta-analytic synthesis of 10 published studies yielded positive evidence for internal consistency and convergent and discriminant validity of the scales. Participants in the present study were 129 people with a diagnosis of panic disorder. Internal consistency was excellent for this sample, α = .95 for AAC and .96 for AAL. When the MIA scales were correlated with interviewer ratings, evidence for convergent and discriminant validity for AAL was strong (convergent r with agoraphobia severity ratings = .63 vs. discriminant rs of .10–.29 for other anxiety disorders) and more modest but still positive for AAC (.54 vs. .01–.37). Receiver operating curve analysis indicated that the optimal operating point for AAL as an indicator of ADIS agoraphobia diagnosis was 1.61, which yielded sensitivity of .87 and specificity of .73.     

Constructs and developmental assessment centers: Further troubling empirical findings

Research indicates that assessment center (AC) ratings typically demonstrate poor construct validity; that is, they do not measure the intended dimensions of managerial performance (e.g., Sackett & Harris, 1988). The purpose of this study was to investigate the construct validity of dimension ratings from a developmental assessment center (N=102), using multitrait-multimethod analysis and factor analysis. The relationships between AC ratings, job performance ratings, and personality measures also were investigated. Results indicate that the AC ratings failed to demonstrate construct validity. The ratings did not show the expected relationships with the job performance and personality measures. Additionally, the factors underlying these ratings were found to be the AC exercises, rather than the managerial dimensions as expected. Potentially, this lack of construct validity of the dimension ratings is a serious problem for a developmental assessment center. There is little evidence that the managerial weaknesses identified by the AC are the dimensions that actually need to be improved on the job. Methods are discussed for improving the construct validity of AC ratings, for example, by decreasing the cognitive demands on the assessors.This study is based on a dissertation submitted to North Carolina State University. Portions of this paper were presented at the meeting of the Society for Industrial and Organizational Psychology in Montreal, Quebec, May, 1992. I am grateful to Paul Thayer, Bert Westbrook, James W. Cunningham, and Patrick Hauenstein for their contributions to this research. I also thank several anonymous reviewers for their comments on this article.     

A comparison of the accuracy and reactivity of methods of monitoring male sexual behavior

Two self-report methods of monitoring male sexuality have been compared—an ongoing method in the form of a diary card and one eliciting retrospective reports in the form of a monthly sexuality interview. Male volunteers were assigned at random to three testing conditions: (a) diary card and monthly interview for 3 months (N=29); (b) three monthly interviews (N=18); and (c) pre- and postinterview only, separated by 2 months (N=21). The correspondence among frequency reports, quality ratings, and number of sexual problems as obtained in diary card and interview measures was examined for respondents in the first condition, revealing that coital frequency and quality ratings were generally quite similar in the two modes of assessment. Interview responses for all subjects were examined for the existence of differential reactive effects according to the frequency of the assessment regimen. No differences emerged among conditions, although a trend for all groups over time was identified. Implications of the results are considered for clinical trials monitoring the effects of new contraceptive methods on sexual functioning.This study was supported by the World Health Organization's Special Programme of Research Development and Research Training in Human Reproduction (Project Nos. 76297 and 79003) and was conducted at King's College Hospital, London.     

Development and initial validation of an indirect measure of transformational leadership integrity

An indirect measure of transformational leadership integrity was developed across three studies. In Study 1, the transformational leadership integrity implicit association test (TLI-IAT) was developed and tested with 65 leaders across heterogeneous organizational contexts. Study 2 involved 51 coaches from 18 sports. Results from Studies 1 and 2 supported the construct validity of the instrument, providing evidence of the instrument's convergent and discriminant validity. Study 3 involved 32 coaches and 133 players from six sports. Findings supported the criterion validity of the measure, providing evidence for the instrument's predictive validity. In sum, evidence is presented that supports the TLI-IATs construct and criterion validity. As such, the present research has made significant advancements to the transformational leadership integrity literature and provides researchers with an indirect measure of automatic transformational leadership integrity self-attitudes.     

To Be,or Not to Be,Linear: An expanded representation of personality and its relationship to leadership performance

In this study, we tested for a set of complex, nonlinear relationships between derailing/dark side personality composites and leadership performance using two independent samples of managers/leaders (N=1306 and N=290 for Study 1 and 2, respectively). Based on the structure and characteristics of the derailing/dark side trait composites, we expected the relationship between these composites and leadership performance would best be described with an inverted U function. In Study 1, we found evidence of a nonlinear relationship between the derailing composite, as measured by the Global Personality Inventory^©, and behaviorally based, structured ratings of leadership performance in an assessment center. Similarly, Study 2 found evidence for a nonlinear relationship between dark side composites, as measured by the Hogan Development Survey^©, and supervisory/other ratings of leadership performance. We discuss the implications of these complex, nonlinear findings with respect to the continued use of personality for the selection and promotion of future leaders. Limitations of the current study and directions for future research are noted.     

The predictive validity of individual psychological assessments in selecting UK public sector senior managers

Individual Psychological Assessment (IPA) is a very widely offered service in Organizational Psychology. It generally consists of a psychologist or HR practitioner using a combination of interview and psychometrics to arrive at a detailed assessment of an individual's capabilities in relation to a job they are being considered for. Although much used, this practice has limited supporting evidence of its validity—not least because of the methodological difficulties in conducting research on this subject—and has been criticized accordingly. The current study examines the use of IPAs with 115 middle and senior management level candidates in a civil service context. All candidates completed a set of psychometric measures and had an in‐depth interview with a psychologist as part of a standardized process. The ratings made by the assessors were correlated with a criterion measure of potential for promotion derived from multisource feedback ratings obtained on these candidates some months later. Analysis of the results indicated that three of the four attributes rated by assessors correlated significantly with the criterion measure. Further, assessors’ ratings were found to show incremental validity over that provided by psychometric test scores alone. These findings are discussed in terms of the use of IPAs in senior level assessment.     

Comparing the Psychometric Characteristics of Ratings of Face‐to‐Face and Videotaped Structured Interviews

Videotaped interviews are used for both research and for making selection decisions in organizations. However, little research has examined the extent to which the psychometric characteristics of ratings of videotaped interviews are comparable with those of ratings made on the basis of face‐to‐face (FTF) interviews. Within a simulated selection setting, we compared ratings of interviewers who conducted FTF structured interviews to ratings of interviewers who viewed videotapes of those interviews. Results revealed that FTF ratings were significantly higher than video ratings of the same interviewees. We also found that the two sets of interviewers rated the relative performance of interviewees differently. For example, the correlation between FTF and video ratings (r=.31) was significantly smaller than the correlation between ratings of interviewers who conducted FTF panel interviews with the same interviewees (r=.73). Overall results suggest that researchers and practitioners should be cautious about generalizing research findings and selection decisions made on the basis of videotaped interviews to FTF interviews.     

Research Artifacts and the Social Psychology of Psychological Experiments

In psychological research with human subjects, experimenters need to anticipate potential artifacts that may be attributable to the social context of such research. Called research artifacts in this review, they are essentially uncontrolled, systematic errors (or biases) that threaten the degree of validity of statements made about whether changes in one variable result in changes in another variable. This discussion focuses on a proposed Markov‐like model emphasizing three mediating variables that operate in a theoretical chain of events. One variable refers to the likelihood of the subject's receptivity to task‐orienting cues (called demand characteristics), or incidental hints about the experimenter's expectations. A second variable refers to the likelihood of the subject's motivation (or willingness) to comply with those cues or hints. A third variable refers to the likelihood of the subject's capability of responding in accordance with the cues or hints. These three variables are discussed along with strategies that researchers can use to break the chain of events.     

The Prospective and Retrospective Memory Questionnaire (PRMQ): factorial structure, relations to global subjective memory ratings, and Swedish norms

The factorial structure of the Prospective and Retrospective Memory Questionnaire (PRMQ; Smith et al., 2000) was examined in a Swedish population based sample (N= 540, age range; 35-90 years). Concurrent validity was assessed by relating PRMQ to global ratings of memory. Confirmatory factor analyses of the PRMQ items indicated a superior fit of a three-factor model, with prospective and retrospective memory as orthogonal factors and episodic memory as a common factor. Furthermore, the PRMQ scales correlated with the global ratings of memory, suggesting that each rating contributed with unique variance in predicting PRMQ scores. Given differences in levels of complaints as compared with prior research (Crawford et al., 2003) norms for the Swedish version are provided. In conclusion, the present findings extend earlier work by providing additional support for the construct and concurrent validity of the PRMQ scales.     

DEVELOPMENT AND PSYCHOMETRIC EVALUATION OF THE INTERPERSONAL SEXUAL OBJECTIFICATION SCALE

This study reports on the development and psychometric evaluation of the Interpersonal Sexual Objectification Scale (ISOS). Data from 576 college women were collected in three studies. Exploratory factor analysis uncovered two factors: Body Evaluation and Unwanted Explicit Sexual Advances; confirmatory factor analysis supported this factor structure. ISOS scores were internally consistent and stable over a 3-week period. Supporting its construct validity, ISOS scores were (a) strongly related to sexist degradation; (b) slightly to moderately related to other sexist events, self-objectification (i.e., body surveillance and internalization of the thin-ideal), and body shame; and (c) unrelated to socially desirable responding. The relationship between ISOS scores and body shame was fully mediated by self-objectification, providing additional evidence for its construct validity. Furthermore, the ISOS garnered incremental validity, as it predicted self-objectification above and beyond the variance accounted for by sexist events.     

Personality,Autonomy, and Contextual Performance of Managers

This study contributes to research on assessment centers' (AC) external construct-related validity by investigating a potential moderator of the relationship between personality and AC dimension ratings. On the basis of recent insights in person-situation contingencies we hypothesized that individual differences in people's perception of situational demands moderate the relationship between personality and conceptually related AC dimension ratings. The hypotheses were tested with 108 individuals in two leaderless group discussion exercises. Results confirmed the hypotheses for two of the three traits (i.e., Agreeableness and Conscientiousness). In particular, people high on these traits who identified the situational demands received higher AC dimension ratings. People high on these traits who did not identify the situational demands received lower AC dimension ratings.