A cautionary note on the effects of range restriction on predictor intercorrelations

The purpose of this research report is to highlight a unique set of issues that arise when considering the effects of range restriction in the context of estimating predictor intercorrelations. Three approaches are used to illustrate the issue: simulation, a concrete applied example, and a reanalysis of a meta-analysis of ability-interview correlations. The general conclusion is that a predictor intercorrelation can differ dramatically from the population value when both predictors are used in a composite that is used operationally for selection. The compensatory nature of a composite means that low scorers on one predictor can only obtain high scores on the composite if they obtain very high scores on the other predictor; this phenomenon distorts the correlation between the predictors.     

Situational judgment tests in high-stakes settings: issues and strategies with generating alternate forms

This study used principles underlying item generation theory to posit competing perspectives about which features of situational judgment tests might enhance or impede consistent measurement across repeat test administrations. This led to 3 alternate-form development approaches (random assignment, incident isomorphism, and item isomorphism). The effects of these approaches on alternate-form consistency, mean score changes, and criterion-related validity were examined in a high-stakes context (N = 3,361). Generally, results revealed that even small changes in the context of the situations presented resulted in significantly lower alternate-form consistency. Conversely, placing more constraints on the alternate-form development process proved beneficial. The contributions, implications, and limitations of these results for the development of situational judgment tests and high-stakes testing are discussed.     

Video-based versus written situational judgment tests: a comparison in terms of predictive validity

In this study, the authors examined whether video-based situational judgment tests (SJTs) have higher predictive validity than written SJTs (keeping verbal content constant). The samples consisted of 1,159 students who completed a video-based version of an SJT and 1,750 students who completed the same SJT in a written format. The study was conducted in a high stakes testing context. The video-based version of an interpersonally oriented SJT had a lower correlation with cognitive ability than did the written version. It also had higher predictive and incremental validity for predicting interpersonally oriented criteria than did the written version. In this high stakes context, applicants also reacted relatively favorably to the SJTs, although there was no significant difference in face validity between the formats. These findings suggest that SJT format changes be made with caution and that validation evidence is needed when changes are proposed.     

Predicting adverse impact and mean criterion performance in multistage selection

The authors present an analytical method to assess the average criterion performance of the selected candidates as well as the adverse impact and the cost of general multistage selection decisions. The method extends previous work on the analytical estimation of multistage selection outcomes to the case in which the applicant pool is a mixture of applicant populations that differ in their average performance on the selection predictors. Next, the method was used to conduct 3 studies of important issues practitioners and researchers have with multistage selection processes. Finally, the authors indicate how the method can be integrated into a broader analytical framework to design multistage selection decisions that achieve intended levels of selection cost, workforce quality, and workforce diversity.     

A construct-driven investigation of gender differences in a leadership-role assessment center

This study examined gender differences in a large-scale assessment center for officer entry in the British Army. Subgroup differences were investigated for a sample of 1,857 candidates: 1,594 men and 263 women. A construct-driven approach was chosen (a) by examining gender differences at the construct level, (b) by formulating a priori hypotheses about which constructs would be susceptible to gender effects, and (c) by using both effect size statistics and latent mean analyses to investigate gender differences in assessment center ratings. Results showed that female candidates were rated notably higher on constructs reflecting an interpersonally oriented leadership style (i.e., oral communication and interaction) and on drive and determination. These results are discussed in light of role congruity theory and of the advantages of using latent mean analyses.     

Five types of personality continuity in childhood and adolescence

This study examines 5 types of personality continuity--structural, mean-level, individual-level, differential, and ipsative--in a representative population (N=498) and a twin and sibling sample (N=548) of children and adolescents. Parents described their children on 2 successive occasions with a 36-month interval using the Hierarchical Personality Inventory for Children (I. Mervielde & F. De Fruyt, 1999). There was evidence for structural continuity in the 2 samples, and personality was shown to be largely differentially stable. A large percentage had a stable trait profile indicative of ipsative stability, and mean-level personality changes were generally small in magnitude. Continuity findings were explained mainly by genetic and nonshared environmental factors.     

Personality trait development from age 12 to age 18: longitudinal,cross-sectional,and cross-cultural analyses

Three studies were conducted to assess mean level changes in personality traits during adolescence. Versions of the Revised NEO Personality Inventory (P. T. Costa, Jr., & R. R. McCrae, 1992a) were used to assess the 5 major personality factors. A 4-year longitudinal study of intellectually gifted students (N = 230) was supplemented by cross-sectional studies of nonselected American (N = 1,959) and Flemish (N = 789) adolescents. Personality factors were reasonably invariant across ages, although rank-order stability of individual differences was low. Neuroticism appeared to increase in girls, and Openness to Experience increased in both boys and girls; mean levels of Extraversion, Agreeableness, and Conscientiousness were stable. Results extend knowledge of the developmental curve of personality traits backward from adulthood and help bridge the gap with child temperament studies.     

Transsaccadic perception of saccade target and flanker objects

To account for location-dependent and location-independent preview benefits in transsaccadic object perception, J. M. Henderson (1994) and J. M. Henderson and M. D. Anes (1994) proposed a dual-route model in which both episodic object representations and long-term memory representations store information across a saccade. Four experiments are reported in which the dual-route model was assessed. Preview benefits for saccade target objects were found to be location independent, whereas preview benefits for flanker objects were location dependent. These results support a single-route, 2-stage model of transsaccadic object perception. First, preattentive object files are set up to parse a set of attentional and/or saccade targets from peripheral vision, causing location-dependent preview benefits. Second, 1 object is attentionally selected for further processing, activating long-term memory representations and resulting in location-independent preview benefits.     

Interference resolution moderates the impact of rumination and reappraisal on affective experiences in daily life

Research has shown that cognitive control processes play a central role in emotion regulation. While most research has examined whether individual differences in such processes are related to the use of these strategies, a crucial next step involves examining whether such differences influence their impact on people's feelings, especially in normal daily life. The present study examined whether impairments in cognitive control (measured using an affective interference resolution task) moderate the impact of using rumination and reappraisal on affective experiences in everyday life (assessed using experience sampling methods). Multilevel analyses revealed that difficulties removing previously relevant negative information from working memory were associated with a larger increase in negative affect following rumination, and smaller increase and decrease in positive and negative affect, respectively, following reappraisal. These findings show that impaired interference resolution for negative information aggravates the deleterious effects of rumination and curbs the benefits of reappraisal in daily life.