Methods of correcting correlations for indirect restriction of range with non-interval data

A derivation is presented for correcting correlations for indirect restriction of range when correlations between one selection variable and non-interval measures are involved. Following a systematic investigation of special cases, formulas are given for practical application in validation work. Procedures for the case with several selection variables are briefly discussed.This study was initiated when the author was with the Institute of Military Psychology, Sweden.     

Correcting for indirect range restriction in meta-analysis: testing a new meta-analytic procedure

Using computer simulation, the authors assessed the accuracy of J. E. Hunter, F. L. Schmidt, and H. Le's (2006) procedure for correcting for indirect range restriction, the most common type of range restriction, in comparison with the conventional practice of applying the Thorndike Case II correction for direct range restriction. Hunter et al.'s procedure produced more accurate estimates of both the mean and standard deviation in meta-analysis than the conventional procedure. Even when its key assumption that the effect of selection on a 3rd variable is fully mediated by the independent variable was violated, Hunter et al.'s procedure was still relatively more accurate than the conventional procedure. When applied to data from a previously published meta-analysis, the new procedure yielded results that led to different substantive conclusions.     

Implications of direct and indirect range restriction for meta-analysis methods and findings

Range restriction in most data sets is indirect, but the meta-analysis methods used to date have applied the correction for direct range restriction to data in which range restriction is indirect. The authors show that this results in substantial undercorrections for the effects of range restriction, and they present meta-analysis methods for making accurate corrections when range restriction is indirect. Applying these methods to a well-known large-sample empirical database, the authors estimate that previous meta-analyses have underestimated the correlation between general mental ability and job performance by about 25%, indicating that this is potentially an important methodological issue in meta-analysis in general.     

Bootstrap standard error and confidence intervals for the correlations corrected for indirect range restriction

The standard Pearson correlation coefficient, r, is a biased estimator of the population correlation coefficient, ρ(XY) , when predictor X and criterion Y are indirectly range-restricted by a third variable Z (or S). Two correction algorithms, Thorndike's (1949) Case III, and Schmidt, Oh, and Le's (2006) Case IV, have been proposed to correct for the bias. However, to our knowledge, the two algorithms did not provide a procedure to estimate the associated standard error and confidence intervals. This paper suggests using the bootstrap procedure as an alternative. Two Monte Carlo simulations were conducted to systematically evaluate the empirical performance of the proposed bootstrap procedure. The results indicated that the bootstrap standard error and confidence intervals were generally accurate across simulation conditions (e.g., selection ratio, sample size). The proposed bootstrap procedure can provide a useful alternative for the estimation of the standard error and confidence intervals for the correlation corrected for indirect range restriction.     

Communicating the validity of structured job interviews with graphical visual aids

The structured job interview is often resisted by human resource (HR) professionals despite its superior predictive validity compared to the traditional unstructured interview. However, HR professionals may underestimate the utility of structured interviews because of how validity information is presented. Three online experiments with MTurk samples were conducted to examine the effect of statistical visual aids (Binomial effect size display; expectancy chart; icon array) on people's perceived usefulness of validity information and intentions toward using the structured interview. Results showed that graphical visual aids (expectancy chart and icon array) were perceived as more useful for communicating validity evidence than a nongraphical effect size display. People also judged the structured interview as more useful and were more willing to use it when validity evidence was presented graphically. Individual differences in graph literacy and nationality were also examined.     

Using individual differences to predict job performance: correcting for direct and indirect restriction of range

The present study investigates the relationship between individual differences, indicated by personality (FFM) and general mental ability (GMA), and job performance applying two different methods of correction for range restriction. The results, derived by analyzing meta-analytic correlations, show that the more accurate method of correcting for indirect range restriction increased the operational validity of individual differences in predicting job performance and that this increase primarily was due to general mental ability being a stronger predictor than any of the personality traits. The estimates for single traits can be applied in practice to maximize prediction of job performance. Further, differences in the relative importance of general mental ability in relation to overall personality assessment methods was substantive and the estimates provided enables practitioners to perform a correct utility analysis of their overall selection procedure.     

Moving Forward with Systems of Care: Needs and New Directions

The articles in this Special Issue on system change within systems of care (SOCs) provide guidance regarding strategies for modifying SOCs to address the needs of different populations, and ways for changing systems to support more positive child and family outcomes. This paper frames central needs, unanswered questions, and issues that remain for those working to implement SOCs. Specific needs and new directions considered include: (1) rigorous implementation-focused research to identify the necessary and sufficient elements of SOCs and the primary practice approach currently used in SOCs, wraparound; (2) applied research to assess SOCs and document their effectiveness in non-standard or non-traditional settings (i.e., non-mental health settings, including child welfare, juvenile justice, local housing authorities); (3) controlled outcome studies for school-based wraparound initiatives; (4) research to document the effectiveness of the family support efforts that are part of most SOCs; and (5) attention to context, for families, service providers, and collaborative implementation efforts, by researchers and providers alike. Progress in these areas can inform well-targeted system change efforts in the context of SOCs, a critical need given changes in federal funding for these initiatives.     

Standard errors and confidence intervals for correlations corrected for indirect range restriction: A simulation study comparing analytic and bootstrap methods

A frequent topic of psychological research is the estimation of the correlation between two variables from a sample that underwent a selection process based on a third variable. Due to indirect range restriction, the sample correlation is a biased estimator of the population correlation, and a correction formula is used. In the past, bootstrap standard error and confidence intervals for the corrected correlations were examined with normal data. The present study proposes a large-sample estimate (an analytic method) for the standard error, and a corresponding confidence interval for the corrected correlation. Monte Carlo simulation studies involving both normal and non-normal data were conducted to examine the empirical performance of the bootstrap and analytic methods. Results indicated that with both normal and non-normal data, the bootstrap standard error and confidence interval were generally accurate across simulation conditions (restricted sample size, selection ratio, and population correlations) and outperformed estimates of the analytic method. However, with certain combinations of distribution type and model conditions, the analytic method has an advantage, offering reasonable estimates of the standard error and confidence interval without resorting to the bootstrap procedure's computer-intensive approach. We provide SAS code for the simulation studies.     

Moving Forward: The Existential Motion of the Self in Kierkegaard's Pseudonymous Works

Each of Kierkegaard's pseudonymous authors address the question of the existing individual and what it truly means to be a self. This article discusses the existing individual as depicted by the pseudonymous authors, focusing on motion, activity, and repetition. In particular, this article seeks to pinpoint the motion and activity of the self through Kierkegaard's notion of repetition, contending that repetition is a decisive action brought forward discontinuously through a breach in time's succession. I address the self's movement forward in four sections. In the first I introduce the overarching distinction between the motion of existing individuals and empirical objects, distinguishing the character of my investigation. After clarifying this distinction, the second section focuses on Anti-Climacus’ conception of the self as activity (a positing of the synthesis), revealing that for an existing individual the self is not given but a task to be actualized. The final section first details the positing of the synthesis as repetition and second contends that, through repetition, the inner workings of the self reveal its activity as a discontinuous breach in time that occurs by virtue of the eternal.     

Assessing personality with a structured employment interview: construct-related validity and susceptibility to response inflation

The authors evaluated the extent to which a personality-based structured interview was susceptible to response inflation. Interview questions were developed to measure facets of agreeableness, conscientiousness, and emotional stability. Interviewers administered mock interviews to participants instructed to respond honestly or like a job applicant. Interviewees completed scales of the same 3 facets from the NEO Personality Inventory, under the same honest and applicant-like instructions. Interviewers also evaluated interviewee personality with the NEO. Multitrait-multimethod analysis and confirmatory factor analysis provided some evidence for the construct-related validity of the personality interviews. As for response inflation, analyses revealed that the scores from the applicant-like condition were significantly more elevated (relative to honest condition scores) for self-report personality ratings than for interviewer personality ratings. In addition, instructions to respond like an applicant appeared to have a detrimental effect on the structure of the self-report and interview ratings, but not interviewer NEO ratings.     

High stakes testing in higher education and employment: appraising the evidence for validity and fairness

The authors review criticisms commonly leveled against cognitively loaded tests used for employment and higher education admissions decisions, with a focus on large-scale databases and meta-analytic evidence. They conclude that (a) tests of developed abilities are generally valid for their intended uses in predicting a wide variety of aspects of short-term and long-term academic and job performance, (b) validity is not an artifact of socioeconomic status, (c) coaching is not a major determinant of test performance, (d) tests do not generally exhibit bias by underpredicting the performance of minority group members, and (e) test-taking motivational mechanisms are not major determinants of test performance in these high-stakes settings.     

A Clinical Exploration of Moving Anger Forward: Intimacy,Anger, and Creative Freedom

This paper is an exploration of anger and its potential to impede or facilitate creative or analytic process. Conceptually I consider anger in terms of cognitive-affective linking and unlinking, and transference-countertransference enactments, and the repetition compulsion as an encoded “second language,” with reference to neuroscience research findings. At its experiential center, this paper tells the story of a story that could not be told about anger in a distressed clinical process.     

A cautionary note on the effects of range restriction on predictor intercorrelations

The purpose of this research report is to highlight a unique set of issues that arise when considering the effects of range restriction in the context of estimating predictor intercorrelations. Three approaches are used to illustrate the issue: simulation, a concrete applied example, and a reanalysis of a meta-analysis of ability-interview correlations. The general conclusion is that a predictor intercorrelation can differ dramatically from the population value when both predictors are used in a composite that is used operationally for selection. The compensatory nature of a composite means that low scorers on one predictor can only obtain high scores on the composite if they obtain very high scores on the other predictor; this phenomenon distorts the correlation between the predictors.     

The use of person-organization fit in employment decision making: an assessment of its criterion-related validity

Because measures of person-organization (P-O) fit are accountable to the same psychometric and legal standards used for other employment tests when they are used for personnel decision making, the authors assessed the criterion-related validity of P-O fit as a predictor of job performance and turnover. Meta-analyses resulted in estimated true criterion-related validities of .15 (k = 36, N = 5,377) for P-O fit as a predictor of job performance and .24 (k = 8, N = 2,476) as a predictor of turnover, compared with a stronger effect of .31 (k = 109, N = 108,328) for the more commonly studied relation between P-O fit and work attitudes. In contrast to the relations between P-O fit and work attitudes, the lower 95% credibility values for the job performance and turnover relations included zero. In addition, P-O fit's relations with job performance and turnover were partially mediated by work attitudes. Potential concerns pertaining to the use of P-O fit in employment decision making are discussed in light of these results.     

Linking Crimes Using Behavioural Clues: Current Levels of Linking Accuracy and Strategies for Moving Forward

The number of published studies examining crime linkage analysis has grown rapidly over the last decade, to the point where a special issue of this journal has recently been dedicated to the topic. Many of these studies have used a particular measure (the area under the receiver operating characteristic curve, or the AUC) to quantify the degree to which it is possible to link crimes. This article reviews studies that have utilised the AUC and examines how good we are currently at linking crimes (within the context of these research studies) and what factors impact linking accuracy. The results of the review suggest that, in the majority of cases, moderate levels of linking accuracy are achieved. Of the various factors that have been examined that might impact linking accuracy, the three factors that appear to have the most significant impact are crime type, behavioural domain, and jurisdiction. We discuss how generalisable these results are to naturalistic investigative settings. We also highlight some of the important limitations of the linking studies that we reviewed and offer up some strategies for moving this area of research forward. Copyright © 2013 John Wiley & Sons, Ltd.