RECRUITMENT AND RETENTION DECISIONS THAT MAXIMIZE THE UTILITY OF A PROBATIONARY SELECTION TO OBTAIN A FIXED QUOTA OF SUCCESSFUL SELECTEES

This article addresses the recruitment and retention decision problems that the selection practitioner faces when the objective is to maximize the utility of a fixed quota of successful selectees. Because the objective can be achieved by either a one-cohort or a multiple cohort with replacement approach, two optimizing procedures are presented. Given information on the selection predictor and the available recruiting sources, both procedures identify the criterion cutoff value (i.e., the critical value of the observed job performance at the end of the probationary period) that makes the best differentiation between successful and unsuccessful selectees. In addition, the proposals indicate the combination of recruiting sources (one-cohort approach) or the sequence of recruiting source combinations (multiple cohort approach) that, in combination with the optimal retention decision, result in the maximum possible utility of the predictor selected workforce. So, the article extends the previous contribution of, among others, Martin and Raju (1992) and Law and Myors (1993) to the case where not all the selectees are considered as successful. Also, the proposal recognizes the fact that the average quality of the applicants may vary across the recruitment sources.     

Methodological pitfalls in predicting counseling success

Prediction models employing multiple linear regression of raw scores, multiple linear regression of factor scores, the single best predictor, and a nine-point decision rule index were compared. The subjects were 296 clients undergoing vocational counseling and evaluation. Predictor variables included performance ratings, demographic variables, and WAIS subtest scores; the criterion was employment status upon program completion. The least statistically sophisticated model, employing the single best predictor, was the most successful approach. Considerable shrinkage in power of prediction was demonstrated upon cross-validation particularly for multiple linear regression of raw scores model, indicating the necessity of cross-validating prediction schemes. Additional suggestions are made to those designing prediction studies.     

Can Conflicting Perspectives on the Role of g in Personnel Selection Be Resolved?

Cognitive ability tests represent the best single predictor of job performance, but also represent the predictor most likely to have substantial adverse impact on employment opportunities for members of several racial and ethnic minority groups. Debates over the use of these tests in selection often involve trade-offs between two criteria that are valued by decision makers-that is, efficiency and equity. Findings and methods from decision research can help us frame these trade-offs, but in most cases they cannot be avoided.     

Using aptitude measurements for the optimal assignment of subjects to treatments with and without mastery scores

For assigning subjects to treatments the point of intersection of within-group regression lines is ordinarily used as the critical point. This decision rule is critized and, for several utility functions and any number of treatments, replaced by optimal monotone, nonrandomized (Bayes) rules. Both treatments with and without mastery scores are considered. Moreover, the effect of unreliable criterion scores on the optimal decision rule is examined, and it is illustrated how qualitative information can be combined with aptitude measurements to improve treatment assignment decisions. Although the models in this paper are presented with special reference to the aptitude-treatment interaction problem in education, it is indicated that they apply to a variety of situations in which subjects are assigned to treatments on the basis of some predictor score, as long as there are no allocation quota considerations.     

Computing Pareto‐optimal Predictor Composites for Complex Selection Decisions

A complex selection situation encompasses vacancies for several different positions and applicants that apply simultaneously for one or several of these positions. This article presents an analytic method for estimating the expected selection quality, as well as the adverse impact ratio of these complex selections, when the decisions are based on a single predictor composite score. In addition, the method is integrated within a broader decision‐making framework for designing complex selection decisions that show a Pareto‐optimal balance between the selection quality and diversity goals. Finally, the decision aid is used to demonstrate the importance of applying the appropriate selection format (either the simple or the complex format) when exploring the front of Pareto‐optimal outcomes of planned selections.     

More powerful tests of predictor subsets in regression analysis under nonnormality

It is well-known that for normally distributed errors parametric tests are optimal statistically, but perhaps less well-known is that when normality does not hold, nonparametric tests frequently possess greater statistical power than parametric tests, while controlling Type I error rate. However, the use of nonparametric procedures has been limited by the absence of easily performed tests for complex experimental designs and analyses and by limited information about their statistical behavior for realistic conditions. A Monte Carlo study of tests of predictor subsets in multiple regression analysis indicates that various nonparametric tests show greater power than the F test for skewed and heavy-tailed data. These nonparametric tests can be computed with available software.     

Measurement scales and scale independence in the analytic hierarchy process

One approach to evaluate the relative performance of decision alternatives with respect to multiple criteria is provided by the analytic hierarchy process. The method is based on pairwise comparisons between attributes, and several numerical measurement scales for the ratio statements have been proposed. The choice of measurement scale is re‐examined, and new arguments supporting the measurement scale of geometric progression are derived. Separately from the measurement scale considerations, the effects of the scale parameter in geometric measurement scale are also studied. By using a regression model for pairwise comparisons data, it is shown that the statistical inference does not depend on the value of the scale parameter in the case of a single pairwise comparison matrix. It is also shown when the scale independence of statistical inference can be achieved in a decision hierarchy. This requires the use of the geometric‐mean aggregation rule instead of the traditional arithmetic‐mean aggregation. The results of the case study demonstrate that the measurement scale and the aggregation rule have potentially large impacts on decision support. Copyright © 2000 John Wiley & Sons, Ltd.     

计算机化分类测验终止规则的类别、特点及应用

计算机化分类测验(Computerized Classification Testing, CCT)能够高效地对被试进行分类, 已广泛应用于合格性测验及临床心理学中。作为CCT的重要组成部分, 终止规则决定测验何时停止以及将被试最终划分到何种类别, 因此直接影响测验效率及分类准确率。已有的三大类终止规则(似然比规则、贝叶斯决策理论规则及置信区间规则)的核心思想分别为构造假设检验、设计损失函数和比较置信区间相对位置。同时, 在不同测验情境下, CCT的终止规则发展出不同的具体形式。未来研究可以继续开发贝叶斯规则、考虑多维多类别情境以及结合作答时间和机器学习算法。针对测验实际需求, 三类终止规则在合格性测验上均有应用潜力, 而临床问卷则倾向应用贝叶斯规则。     

Some Reasons for Distinguishing Between “Non-normative Response” and “Irrational Decision”

Three concerns that limit the implications of normative decision research were described in this study. First, because there is no unequivocal way to determine whether any particular base rate is appropriate in a particular instance, the significance of the apparent base rate underutilization reported by a number of investigators is unclear. Second, because there are discrepancies between natural language and formal logic, word problems are an uncertain domain within which to gauge human inferential skills. Third, because different people will have different strategies and preferences, no single solution may be correct or optimal for all persons. In light of these limitations, a more functional approach to decision interpretation is discussed.     

Why Do People Engage in Collective Action? Revisiting the Role of Perceived Effectiveness

Research has shown limited support for the notion that perceived effectiveness of collective action is a predictor of intentions to engage in collective action. One reason may be that effectiveness has been in terms of whether the action will influence key decision makers. We argue that the effectiveness of collective action might be judged by other criteria, such as whether it influences third parties, builds an oppositional movement, and expresses values. Two hundred and thirty one attendees at a rally rated the effectiveness of the rally and their intentions to engage in future collective action. For those participants who were not members of an organization, intentions were linked to the perceived effectiveness of the rally in expressing values and influencing the public. For those who were members of an organization, intentions were linked only to the effectiveness of the rally in building an oppositional movement.     

Validation of intelligence and mechanical aptitude tests in selecting employees for manufacturing jobs

One hundred ninety-three manufacturing employees who produce electro-mechanical components participated in a concurrent criterion-related validity study. The employees were administered three tests: The Bennett Mechanical Comprehension Test (Form S); The Flanagan Aptitude Classification Test-Mechanics; and the Thurstone Test of Mental Alertness (Form A). Job performance was measured by a supervisor rating of fifteen job dimensions, assessed at two points in time separated by 60 days. Correlational and multiple regression analyses were used to assess the relationship between test scores and job performance ratings. The results revealed that the Bennett Mechanical Comprehension test was the best single predictor of job performance (uncorrectedr =.38), and the incremental gain in predictability from additional tests was not significant. The results were discussed in the context of the changing nature of manufacturing jobs and the inadequacy of conventional mechanical aptitude tests to be sensitive to these changes.     

Optimal correction for guessing in multiple-choice tests

Subjects’ decisions in multiple-choice tests are an interesting domain for the analysis of decision making under uncertainty. When the test is graded using a rule that penalizes wrong answers, each item can be viewed as a lottery where a rational examinee would choose whether to omit (sure reward) or answer (take the lottery) depending on risk aversion and level of knowledge. We formalize students as heterogeneous decision makers with different risk attitudes and levels of knowledge. Building on IRT, we compute the optimal penalty given students’ optimal behavior and the trade-off between bias and measurement error. Although MCQ examinations are frequently used, there is no consensus as to whether a penalty for wrong answers should be used or not. For example, examinations for medical licensing in some countries include MCQ sections with penalty while in others there is no penalty for wrong answers. We contribute to this discussion with a formal analysis of the effects of penalties; our simulations indicate that the optimal penalty is relatively high for perfectly rational students but also when they are not fully rational: even though penalty discriminates against risk averse students, this effect is small compared with the measurement error that it prevents.     

A multi-objective mathematical programming problem with fuzzy relation constraints

In this study we discuss the problem of multi-objective mathematical programming with constraints defined by ‘max-min’ composite fuzzy relation equations. Since the feasible region is normally non-convex, the properties of the efficient points of a non-convex feasible region under multi-objectives are investigated and illustrated by examples. The necessary and sufficient conditions are proposed and proved. To facilitate decisions, a procedure that transforms these efficient points of an interval-valued decision space into a constant-valued decision space is proposed when the level of confidence is given by a decision maker. Then the transformed problem becomes a multi-attribute decision problem that can be evaluated by Yager's method to find the optimal alternative.     

Designing Pareto-optimal selection systems: formalizing the decisions required for selection system development

The article presents an analytic method for designing Pareto-optimal selection systems where the applicants belong to a mixture of candidate populations. The method is useful in both applied and research settings. In an applied context, the present method is the first to assist the selection practitioner when deciding on 6 major selection design issues: (1) the predictor subset, (2) the selection rule, (3) the selection staging, (4) the predictor sequencing, (5) the predictor weighting, and (6) the stage retention decision issue. From a research perspective, the method offers a unique opportunity for studying the impact and relative importance of different strategies for reducing adverse impact.     

Further evaluating the conditional decision rule for comparing two independent means

Many books on statistical methods advocate a ‘conditional decision rule’ when comparing two independent group means. This rule states that the decision as to whether to use a ‘pooled variance’ test that assumes equality of variance or a ‘separate variance’ Welch t test that does not should be based on the outcome of a variance equality test. In this paper, we empirically examine the Type I error rate of the conditional decision rule using four variance equality tests and compare this error rate to the unconditional use of either of the t tests (i.e. irrespective of the outcome of a variance homogeneity test) as well as several resampling‐based alternatives when sampling from 49 distributions varying in skewness and kurtosis. Several unconditional tests including the separate variance test performed as well as or better than the conditional decision rule across situations. These results extend and generalize the findings of previous researchers who have argued that the conditional decision rule should be abandoned.     

An examination of learning processes during critical incident training: implications for the development of adaptable trainees

Three experiments are reported that examined the process by which trainees learn decision-making skills during a critical incident training program. Formal theories of category learning were used to identify two processes that may be responsible for the acquisition of decision-making skills: rule learning and exemplar learning. Experiments 1 and 2 used the process dissociation procedure (L. L. Jacoby, 1998) to evaluate the contribution of these processes to performance. The results suggest that trainees used a mixture of rule and exemplar learning. Furthermore, these learning processes were influenced by different aspects of training structure and design. The goal of Experiment 3 was to develop training techniques that enable trainees to use a rule adaptively. Trainees were tested on cases that represented exceptions to the rule. Unexpectedly, the results suggest that providing general instruction regarding the kinds of conditions in which a decision rule does not apply caused them to fixate on the specific conditions mentioned and impaired their ability to identify other conditions in which the rule might not apply. The theoretical, methodological, and practical implications of the results are discussed.     

Binary recursive partitioning: Background,methods, and application to psychology

Binary recursive partitioning (BRP) is a computationally intensive statistical method that can be used in situations where linear models are often used. Instead of imposing many assumptions to arrive at a tractable statistical model, BRP simply seeks to accurately predict a response variable based on values of predictor variables. The method outputs a decision tree depicting the predictor variables that were related to the response variable, along with the nature of the variables' relationships. No significance tests are involved, and the tree's ‘goodness’ is judged based on its predictive accuracy. In this paper, we describe BRP methods in a detailed manner and illustrate their use in psychological research. We also provide R code for carrying out the methods.