Why All Banding Procedures in Personnel Selection Are Logically Flawed

In this article, I first demonstrate that statistical significance testing of differ- ences between predictor scores (whether based on the standard error of mea- surement or any other statistic) is irrelevant to, and inconsistent with, the traditional, optimizing selection model. Second, I demonstrate that all band- ing procedures used in (or advocated for) personnel selection, including the sliding-band procedures advocated by Cascio, Outtz, Zedeck, and Goldstein (1991-this issue), are fatally flawed logically. I show that, when the number of applicants is large, all banding procedures logically lead to the absurd con- clusion that the only justifiable form of selection is random selection. Third, I present evidence that the empirical data set used by Cascio et al. to evaluate different selection strategies is anomalous and leads to results very different from those to be expected in typical and representative data. Specifically, the effect is to produce misleadingly small differences between the strategies in mean test scores of selectees and, therefore, in selection utility. In particular, selection utility losses from all forms of banding, in comparison to top-down selection, are understated. Finally, I show that, apart from the lethal logical flaw in banding procedures, Cascio et al. misinterpreted the meaning and na- ture of statistical significance testing.     

Why All Banding Procedures in Personnel Selection Are Logically Flawed

In this article, I first demonstrate that statistical significance testing of differ- ences between predictor scores (whether based on the standard error of mea- surement or any other statistic) is irrelevant to, and inconsistent with, the traditional, optimizing selection model. Second, I demonstrate that all band- ing procedures used in (or advocated for) personnel selection, including the sliding-band procedures advocated by Cascio, Outtz, Zedeck, and Goldstein (1991-this issue), are fatally flawed logically. I show that, when the number of applicants is large, all banding procedures logically lead to the absurd con- clusion that the only justifiable form of selection is random selection. Third, I present evidence that the empirical data set used by Cascio et al. to evaluate differmt selection strategies is anomalous and leads to results very different from those to be expected in typical and representative data. Specifically, the effect is to produce misleadingly small differences between the strategies in mean test scores of selectees and, therefore, in selection utility. In particular, selection utility losses from all forms of banding, in comparison to topdown selection, are understated. Finally, I show that, apart from the lethal logical flaw in banding procedures, Cascio et al. misinterpreted the meaning and na- ture of statistical significance testing.     

Cutoff Scores that Optimize the Utility of a Fixed Quota of Successful Employees Selection Decision

The new utility model, proposed previously by the author, is used to establish test cutoff scores that optimize the utility of a ‘fixed quota of successful employees selection decision’. Within the model, the problem can be formulated as a constrained non-linear program, and, to obtain the solution, a sequential quadratic programming method is implemented. Next, the approach is illustrated by means of an example application which relates to the selection of a fixed quota of five successful employees for an entrance management position. Apart from detailing certain computational aspects of the procedure, the example allows also to clarify the issue of practical relevance. Finally, the limitations, as well as a generalization of the procedure to the simultaneous estimation of both optimal test and criterion cutoffs, are discussed.     

THE FUTILITY OF UTILITY ANALYSIS REVISITED: WHEN EVEN AN EXPERT FAILS

The effect of utility analysis on managerial decisions regarding the use of a valid selection test was investigated. Experienced managers ( n = 41) were randomly assigned to one of three conditions. The control condition contained written advice to adopt new selection procedures from a hypothetical psychologist that included an explanation of validation procedures. The second condition contained the same written information and advice as the control condition plus written support of that advice from a hypothetical trusted adviser. The third condition contained all the material that was presented in the control condition plus a written explanation of utility analysis, an actual utility analysis showing that large financial benefits would flow from using the proposed procedures, and a video-taped presentation from an expert on utility analysis where the logic underlying utility analysis and its benefits were explained. The expert was then presented live to the audience to address any questions that might have arisen from the utility analysis or the video. An ANOVA revealed that the presentation of a positive utility analysis reduced support for implementing a valid selection procedure, even though the logic and merits of utility analysis as a decision-making tool were described by an internationally recognized authority.     

Weak measurement invariance with respect to unmeasured variables: An implication of strict factorial invariance

Measurement bias refers to systematic differences across subpopulations in the relation between observed test scores and the latent variant underlying the test scores. Comparisons of subpopulations with the same score on the latent variable can be expected to have the same observed test score. Measurement invariance is therefore one of the key issues in psychological testing. It has been established that strict factorial invariance (SFI) with respect to a selection variable V almost certainly implies weak measurement invariance with respect to V: given SFI, means and variances of observed scores do not depend on V. It is shown that this result can be extended. SFI in groups derived by selection on V has implications not only for V but also for potentially biasing variables W, if W and the selection variable V and/or if W and the factor underlying the observed test scores are statistically dependent. Given SFI with respect to V and prior knowledge concerning these dependencies, it is not necessary to measure and model variables W in order to exclude them as potentially biasing variables if the investigation focuses on groups selected on V.     

Comparison of optimal cutoff points for single and multiple tests in personnel selection

This paper compares the determination of optimal cutoff points for single and multiple tests in the field of personnel selection. Decisional skills of predictor tests composing the multiple test are assumed to be endogenous variables that depend on the cutting points to be set. It is shown how the predictor cutoffs and the collective decision rule are determined dependently by maximizing the multiple test's common expected utility. Our main result specifies the condition that determines the relationship between the optimal cutoff points for single and multiple tests, given the number of predictor tests, the collective decision rule (aggregation procedure of predictor tests' recommendations) and the function relating the tests' decisional skills to the predictor cutoff points. The proposed dichotomous decision-making method is illustrated by an empirical example of selecting trainees by means of the Assessment Center method.     

A compensatory approach to optimal selection with mastery scores

A Bayesian approach for simultaneous optimization of test-based decisions is presented using the example of a selection decision for a treatment followed by a mastery decision. A distinction is made between weak and strong rules where, as opposed to strong rules, weak rules use prior test scores as collateral data. Conditions for monotonicity of optimal weak and strong rules are presented. It is shown that under mild conditions on the test score distributions and utility functions, weak rules are always compensatory by nature. The authors are indebted to Wilbert Kallenberg for his valuable comments and to Jan Gulmans for providing the data for the empirical example. The names of the authors are alphabetical; they are equally responsible for the contents of this paper.     

The mean and variance of the selection differential after top‐down selection from a mixture of normal samples

A procedure is presented for determining the mean and variance of the selection differential for top‐down selections in which the candidates come from populations that have a different average score on the selection measure. Although the procedure is based on the same stochastic model and requires identical data to the currently available method for estimating the mean selection differential, it has the advantage that the resulting expressions are valid for finite‐sample selection decisions and that the variance of the selection differential can also be assessed. The difference between the two procedures is illustrated by means of an example application, and it is shown how the present results are particularly helpful in determining the expected utility of personnel selection decisions.     

RECRUITMENT AND RETENTION DECISIONS THAT MAXIMIZE THE UTILITY OF A PROBATIONARY SELECTION TO OBTAIN A FIXED QUOTA OF SUCCESSFUL SELECTEES

This article addresses the recruitment and retention decision problems that the selection practitioner faces when the objective is to maximize the utility of a fixed quota of successful selectees. Because the objective can be achieved by either a one-cohort or a multiple cohort with replacement approach, two optimizing procedures are presented. Given information on the selection predictor and the available recruiting sources, both procedures identify the criterion cutoff value (i.e., the critical value of the observed job performance at the end of the probationary period) that makes the best differentiation between successful and unsuccessful selectees. In addition, the proposals indicate the combination of recruiting sources (one-cohort approach) or the sequence of recruiting source combinations (multiple cohort approach) that, in combination with the optimal retention decision, result in the maximum possible utility of the predictor selected workforce. So, the article extends the previous contribution of, among others, Martin and Raju (1992) and Law and Myors (1993) to the case where not all the selectees are considered as successful. Also, the proposal recognizes the fact that the average quality of the applicants may vary across the recruitment sources.     

Optimal selection of E‐learning websites using multiattribute decision‐making approaches

A computational quantitative model based on weighted Euclidean distance‐based approximation and complex proportional assessment has been developed for the evaluation, selection, and ranking of various E‐learning websites in ascending or descending order based on their Euclidean distance value from the optimal website. The E‐learning website with rank 1 is considered the optimal selection on the particular dataset under consideration. The problem of the E‐learning website Selection, Evaluation and Ranking is modeled as a multiattribute decision‐making problem in which various interrelated attributes collectively termed as ranking criteria are identified to make the evaluation of available alternatives. In this research, 5 most popular E‐learning websites related to the C programming language for the software development have been considered to show the utility of developed model. Further, the concept of methodology validation strengthens this research by comparing the obtained results with the existing multiattribute decision‐making approach as analytical hierarchy process method.     

Pairwise multiple comparisons: A model comparison approach versus stepwise procedures

Researchers in the behavioural sciences have been presented with a host of pairwise multiple comparison procedures that attempt to obtain an optimal combination of Type I error control, power, and ease of application. However, these procedures share one important limitation: intransitive decisions. Moreover, they can be characterized as a piecemeal approach to the problem rather than a holistic approach. Dayton has recently proposed a new approach to pairwise multiple comparisons testing that eliminates intransitivity through a model selection procedure. The present study compared the model selection approach (and a protected version) with three powerful and easy‐to‐use stepwise multiple comparison procedures in terms of the proportion of times that the procedure identified the true pattern of differences among a set of means across several one‐way layouts. The protected version of the model selection approach selected the true model a significantly greater proportion of times than the stepwise procedures and, in most cases, was not affected by variance heterogeneity and non‐normality.     

Cutting scores and errors of measurement

Arguments for using multiple cutting scores are theoretically inapplicable when the selection measures are fallible. The effect of errors of measurement in altering the shape of some optimum selection regions is here investigated mathematically, with numerical illustrations, for the case of two selection variables.This work was supported by contract Nonr-2752(00) between the Office of Naval Research and Educational Testing Service. Reproduction in whole or in part for any purpose of the United States Government is permitted.     

Candidates' Ability to Identify Criteria in Nontransparent Selection Procedures: Evidence from an assessment center and a structured interview

In selection procedures like assessment centers (ACs) and structured interviews, candidates are often not informed about the targeted criteria. Previous studies have shown that candidates' ability to identify these criteria (ATIC) is related to their performance in the respective selection procedure. However, past research has studied ATIC in only one selection procedure at a time, even though it has been assumed that ATIC is consistent across situations, which is a prerequisite for ATIC to contribute to selection procedures' criterion‐related validity. In this study, 95 candidates participated in an AC and a structured interview. ATIC scores showed cross‐situational consistency across the two procedures and accounted for part of the relationship between performance in the selection procedures. Furthermore, ATIC scores in one procedure predicted performance in the other procedure even after controlling for cognitive ability. Implications and directions for future research are discussed.     

Toward an Integrative Model of Affect Regulation: Applications to Social-Psychological Research

ABSTRACT This article describes a model of affect regulation that integrates research and theory from psychoanalytic, cognitive, behavioral, and evolutionary perspectives on personality. It proposes that feelings are mechanisms for the selection and retention of behavioral and mental responses. Individuals select behaviors, coping strategies, and defensive strategies that regulate aversive affective states and maximize pleasurable ones. These affect regulation procedures are encoded as procedural knowledge and are activated under specific circumstances. Some regulation strategies are affect-specific, whereas others can be used to regulate multiple affects of similar valence. These procedures are often activated to resolve discrepancies between perceived and desired states of the self, significant others, and external circumstances. The utility of the model is demonstrated through a reinterpretation and integration of a number of disparate traditions in social psychology, including many of the classic experiments on social influence, in which the processing of emotionally relevant social information plays a substantial part.     

EXAMINING SELECTION UTILITY WHERE COMPETING PREDICTORS DIFFER IN ADVERSE IMPACT

This paper discusses the roles of validity, cut score choice, and adverse impact on selection system utility using data from two concurrent validation studies. We contrast an assessment center and published aptitude test on several metrics, including validity, testing costs, adverse impact, and utility. The assessment center produced slightly lower validity than the aptitude test while costing roughly 10 times as much per candidate. In spite of these advantages for the aptitude test, the assessment center produced so much less adverse impact its operational utility would be higher given cut scores likely to be chosen in this organization. Potential concerns with applying net utility models to this type of situation are discussed in comparison to gross utility models.     

A note on invariance in three-mode factor analysis

Previous results of the application of Lawley's selection theorem to the common factor analysis model are extended to a revision of Tucker's three-mode principal components model. If the regression of the three-mode manifest variates on variates used to select subpopulations is both linear and homoscedastic, the two factor pattern matrices, the core matrix, and the residual variance-covariance matrix in the three-mode model can all be assumed to be invariant across subpopulations. The implication of this finding for simple structure is discussed.     

Psychometric properties and predictive validity of the Mt Scale of the MMPI-2

The College Maladjustment (Mt) Scale is a 41-item supplementary scale of the Minnesota Multiphasic Personality Inventory. It was designed to identify college students classified as maladjusted. Very little research has been conducted on the Mt Scale in the 40 years since its inception. The current study had the following four goals: (a) provide additional data on the internal consistency reliability, (b) examine the relationship between Mt Scale scores and various school-related domains, (c) examine the relationship between Mt scores and indices of distress (trauma exposure, post-traumatic stress disorder [PTSD] symptoms, and history of psychological treatment), and (d) examine the diagnostic accuracy of cutting scores across the full spectrum of possible scores using receiver operating characteristic (ROC) analyses. The Mt Scale is relatively internally consistent (alpha = .85). Elevations on the Mt Scale were significantly associated with lower current grade point average, arriving late to classes more often, the PTSD symptoms of avoidance and arousal, and a history of treatment. Last, the findings from the ROC analyses suggest that the previously obtained cutting scores of 15 and 22 are too low and that values of 29 or higher are more likely to yield the best diagnostic utility values.     

Classification in psychopathology: rationale, alternatives, and standards

The elements that constitute a scientific taxonomy are outlined and presented with reference to the structure of psychopathologic categorization and diagnosis. The terminology, logic, and conceptual issues associated with clinical classification are discussed. Alternatives for the selection of substantive clinical attributes, the overall structural format into which categories are organized, and construction procedures used in developing a psychopathologic taxonomy are elaborated, as are a number of criteria for evaluating the taxonomy's utility and efficacy.