Mindfulness and other Buddhist-derived interventions in correctional settings: A systematic review

Interest into the rehabilitative utility of Buddhist-derived interventions (BDIs) for incarcerated populations has been growing. The present paper systematically reviews the evidence for BDIs in correctional settings. Five databases were systematically searched. Controlled intervention studies of BDIs that utilized incarcerated samples were included. Jadad scoring was used to evaluate methodological quality. PRISMA (preferred reporting items for systematic reviews and meta-analysis) guidelines were followed. The initial search yielded 85 papers, but only eight studies met the inclusion criteria. The eight eligible studies comprised two mindfulness studies, four vipassana meditation studies, and two studies utilizing other BDIs. Intervention participants demonstrated significant improvements across five key criminogenic variables: (i) negative affect, (ii) substance use (and related attitudes), (iii) anger and hostility, (iv) relaxation capacity, and (v) self-esteem and optimism. There were a number of major quality issues. It is concluded that BDIs may be feasible and effective rehabilitative interventions for incarcerated populations. However, if the potential suitability and efficacy of BDIs for prisoner populations is to be evaluated in earnest, it is essential that methodological rigor is substantially improved. Studies that can overcome the ethical issues relating to randomization in correctional settings and employ robust randomized controlled trial designs are favored.     

Validity of General Mental Ability for the Prediction of Job Performance and Training Success in Germany: A meta‐analysis1

A meta‐analysis into the operational validity of general mental ability (GMA) measures in Germany is presented. The meta‐analysis addresses the question whether findings of US and European meta‐analyses are generalizable to Germany given the differences in the education systems of these countries. The high level of differentiation in the German educational system is expected to enhance the homogeneity of applicant pools resulting in a low level of variability in predictor scores which reduces the observed GMA–performance relationships. Our analysis is based on 54 independent German articles and unpublished reports. Results indicated an operational validity of ρ=.467 for training success (k=90; N=11,969) and ρ=.534 for job performance (k=9; N=746). Moderator analyses showed that job complexity and the year of publication are relevant moderator variables, with lower job complexity levels and older studies being associated with higher operational validities. Findings suggest that overall German operational validities are comparable with findings in the United States or other European countries. However, for training success operational GMA validities are slightly lower in Germany compared with US or European meta‐analyses.     

Psychometric Comparison of Dissociative Experiences Scales II and C: A Weak Trauma‐Dissociation Link

The debate regarding the relationship between dissociation and trauma has raised questions regarding the validity of measures of dissociation. Dalenberg et al.'s ( 2012 ) meta‐analysis included studies using the Dissociative Experiences Scale (DES II), but excluded the DES‐Comparison (DES‐C) scale, claiming that it lacked validity as a measure of dissociation. Lynn et al. ( 2014 ) contended that omitting those studies might have skewed the results. In the current study, we compared the psychometric properties of both measures in two nonclinical US adult (student, general population) samples to evaluate the convergent and discriminant validity of the scales. We found support for the DES‐II as a measure of dissociation as well as the validity and reliability of the DES‐C, which compares well to the DES II. Compared with studies in Dalenberg et al., we found lower correlations between trauma and dissociation. No empirical basis exists to exclude studies using the DES‐C in literature reviews. Copyright © 2017 John Wiley & Sons, Ltd.     

Comparison based satisfaction: contrast and empathy

The paper (1) presents a new contrast–empathy model to capture affective reactions following a social comparison, and (2) supports the applicability of the value function to social comparisons. Relationship quality and relevance of the comparison dimension are assumed to influence the interplay between contrast and empathy effects. The application of the value function to the contrast–empathy model leads to the predictions that (a) dissatisfaction looms larger than satisfaction and that (b) pity looms larger than empathetic joy. Three studies test the validity of the contrast empathy model. In all studies participants rated their satisfaction with income comparisons. Study 1 demonstrates the applicability of the value function to social comparisons. Study 2 extends the self‐evaluation model and supports the validity of the contrast–empathy model. Study 3 replicates parts of Study 2 in a between‐subjects design. Copyright © 2000 John Wiley & Sons, Ltd.     

In‐basket Validity: A systematic review

In‐baskets are high‐fidelity simulations often used to predict performance in a variety of jobs including law enforcement, clerical, and managerial occupations. They measure constructs not typically assessed by other simulations (e.g., administrative and managerial skills, and procedural and declarative job knowledge). We compiled the largest known database (k = 31; N = 3,958) to address the criterion‐related validity of in‐baskets and possible moderators. Moderators included features of the in‐basket: content (generic vs. job specific) and scoring approach (objective vs. subjective) and features of the validity studies: design (concurrent vs. predictive) and source (published vs. unpublished). Sensitivity analyses assessed how robust the results were to the influence of various biases. Results showed that the operational criterion‐related validity of in‐baskets was sufficiently high to justify their use in high‐stakes settings. Moderator analyses provided useful guidance for developers and users regarding content and scoring.     

Discriminating between strong and weak structures in three‐mode principal component analysis

Recently, a number of model selection heuristics (i.e. DIFFIT, CORCONDIA, the numerical convex hull based heuristic) have been proposed for choosing among Parafac and/or Tucker3 solutions of different complexity for a given three‐way three‐mode data set. Such heuristics are often validated by means of extensive simulation studies. However, these simulation studies are unrealistic in that it is assumed that the variance in real three‐way data can be split into two parts: structural variance, due to a true underlying Parafac or Tucker3 model of low complexity, and random noise. In this paper, we start from the much more reasonable assumption that the variance in any real three‐way data set is due to three different sources: (1) a strong Parafac or Tucker3 structure of low complexity, accounting for a considerable amount of variance, (2) a weak Tucker3 structure, capturing less prominent data aspects, and (3) random noise. As such, Parafac and Tucker3 simulation studies are run in which the data are generated by adding a weak Tucker3 structure to a strong Parafac or Tucker3 one and perturbing the resulting data with random noise. The design of these studies is based on the reanalysis of real data sets. In these studies, the performance of the numerical convex hull based model selection method is evaluated with respect to its capability of discriminating strong from weak underlying structures. The results show that in about two‐thirds of the simulated cases, the hull heuristic yields a model of the same complexity as the strong underlying structure and thus succeeds in disentangling strong and weak underlying structures. In the vast majority of the remaining third, this heuristic selects a solution that combines the strong structure and (part of) the weak structure.     

USING A META-ANALYTIC PERSPECTIVE TO ENHANCE JOB COMPONENT VALIDATION

This paper develops synthetic validity estimates based on a meta-analytic-weighted least squares (WLS) approach to job component validity (JCV), using position analysis questionnaire (PAQ) estimates of job characteristics, and the Data, People, & Things ratings from the  Dictionary of Occupational Titles  as indices of job complexity. For the general aptitude test battery database of 40,487 employees, nine validity coefficients were estimated for 192 positions. The predicted validities from the WLS approach had lower estimated variability than would be obtained from either the classic JCV approach or local criterion-related validity studies. Data, People, & Things summary ratings did not consistently moderate validity coefficients, whereas the PAQ data did moderate validity coefficients. In sum, these results suggest that synthetic validity procedures should incorporate a WLS regression approach. Moreover, researchers should consider a comprehensive set of job characteristics when considering job complexity rather than a single aggregated index.     

Comparing Personality Test Formats and Warnings: Effects on criterion‐related validity and test‐taker reactions

The potential for applicant response distortion on personality measures remains a major concern in high‐stakes testing situations. Many approaches to understanding response distortion are too transparent (e.g., instructed faking studies) – or are too subtle (e.g., correlations with social desirability measures as indices of faking). Recent research reveals more promising approaches in two methods: using forced‐choice (FC) personality test items and warning against faking. The present study examined effects of these two methods on criterion‐related validity and test‐taker reactions. Results supported incremental validity for an FC and Likert‐scale measure in warning and no‐warning conditions, above and beyond cognitive ability. No clear differences emerged between the FC vs Likert measures or warning vs no‐warning conditions in terms of validity. However, some evidence suggested that FC measures and warnings may produce negative test‐taker reactions. We conclude with implications for implementation in selection settings.     

Consumer susceptibility to interpersonal influence in Mainland China

To validate the consumer susceptibility to interpersonal influence (SUSCEP) scale in China, questionnaires were administered to 1114 adult employees and 350 university students. Confirmatory factor analyses showed that the consumer susceptibility to interpersonal influence scale had a stable three‐factor structure (i.e. the utilitarian, value‐expressive, and informational dimensions) in both samples, while the original two‐factor structure could not be retrieved. Using Chinese samples, the consumer susceptibility to interpersonal influence scale and its three sub‐scales demonstrated acceptable internal reliability, as well as convergent and discriminant validity. Positive correlations between the consumer susceptibility to interpersonal influence scale (as well as its sub‐scales) and attention to social comparison information also indicated good convergent validity. The distinction between utilitarian dimension and value‐expression dimension in Chinese samples as well as the non‐significant correlations of the utilitarian dimension with self‐esteem and the informational dimension with self‐esteem revealed cultural differences between the East and the West.     

Predicting job performance using FFM and non‐FFM personality measures

This study compares the criterion validity of the Big Five personality dimensions when assessed using Five‐Factor Model (FFM)‐based inventories and non‐FFM‐based inventories. A large database consisting of American as well as European validity studies was meta‐analysed. The results showed that for conscientiousness and emotional stability, the FFM‐based inventories had greater criterion validity than the non FFM‐based inventories. Conscientiousness showed an operational validity of .28 (N = 19,460, 90% CV = .07) for FFM‐based inventories and .18 (N =5,874, 90% CV = ‐.04) for non‐FFM inventories. Emotional stability showed an operational validity of .16 (N = 10,786, 90% CV = .04) versus .05 (N = 4,541, 90% CV = ‐.05) for FFM and non‐FFM‐based inventories, respectively. No relevant differences emerged for extraversion, openness, and agreeableness. From a practical point of view, these findings suggest that practitioners should use inventories based on the FFM in order to make personnel selection decisions.     

Designing Selection Systems for Medicine: The importance of balancing predictive and political validity in high‐stakes selection contexts

Selection into medical education and training is a high‐stakes process. A key unanswered issue is the effectiveness of measuring noncognitive predictors via both low‐fidelity and high‐fidelity selection approaches in this high‐stakes context. We review studies investigating the effectiveness of multiple selection instruments in terms of predictive validity, incremental validity, and applicant reactions in both entry‐level and advanced‐level medical selection. Our results show that the situational judgment test (SJT) is the best single predictor of performance, operationalized in multiple ways. In addition, the low‐fidelity SJT has incremental predictive power over cognitively oriented tests, and high‐fidelity assessment center (AC) exercises add incremental validity over the low‐fidelity (and less costly) selection methods. Concerning applicant reactions, results show that overall, the selection system is positively received. However, the method with the highest predictive validity – the SJT – received comparatively lower face validity ratings which may present a ‘justice dilemma’ for employers. Furthermore, various other stakeholders have a political interest in the selection methods used (e.g., government, the regulators and trade unions).     

The effect of the validity of co‐occurrence on automatic and deliberate evaluations

Co‐occurrence of an object and affective stimuli does not always mean that the object and the stimuli are the same valence (e.g., false accusations that Richard is a crook). Contemporary theory posits that information about the (in)validity of co‐occurrence has stronger influence on deliberate than automatic evaluation. However, available evidence supports that hypothesis only when the (in)validity information is delayed. Further, the existing evidence is open to alternative methodological accounts. In six high‐powered experiments (total N = 1750), we modified previous procedures to minimize alternative explanations and examine whether delayed (in)validity information has a discrepant effect on automatic versus deliberate evaluation. Casting doubt on the generality of the hypothesis, we found more sensitivity of deliberate than automatic evaluation to delayed validity information only when automatic evaluation was measured with the Implicit Association Test and not with the evaluative priming task or the affective misattribution procedure.     

Further examination of the convergent and discriminant validity of the student–teacher relationship scale

Two studies extended psychometric research on the Student–Teacher Relationship Scale (STRS) with kindergarten and preschool children (N₁ = 60–71; N₂ = 35) and their teachers. These studies used a multi‐method approach to replicate and extend previous findings concerning the convergent validity of the STRS Closeness, Conflict, and Dependency scale and to further examine the discriminant validity of the STRS. Study 1 investigated convergence between the STRS scales and child‐ and peer‐reports of the same constructs based on a multi‐trait multi‐method approach. Study 2 examined the pattern of associations between the STRS and indicators of teacher–child relationship quality rated by external observers. Support was found for the convergent validity and to a lesser extent the discriminant validity of the STRS Closeness and Conflict scale. For the STRS Dependency scale, additional research remains necessary. Copyright © 2009 John Wiley & Sons, Ltd.     

Racial/ethnic differences in the criterion-related validity of cognitive ability tests: a qualitative and quantitative review

The correlation between cognitive ability test scores and performance was separately meta-analyzed for Asian, Black, Hispanic, and White racial/ethnic subgroups. Compared to the average White observed correlation ( = .33, N = 903,779), average correlations were lower for Black samples ( = .24, N = 112,194) and Hispanic samples ( = .30, N = 51,205) and approximately equal for Asian samples ( = .33, N = 80,705). Despite some moderating effects (e.g., type of performance criterion, decade of data collection, job complexity), validity favored White over Black and Hispanic test takers in almost all conditions that included a sizable number of studies. Black-White validity comparisons were possible both across and within the 3 broad domains that use cognitive ability tests for high-stakes selection and placement: civilian employment, educational admissions, and the military. The trend of lower Black validity was repeated in each domain; however, average Black-White validity differences were largest in military studies and smallest in educational and employment studies. Further investigation of the reasons for these validity differences is warranted.     

A Matter of Context: A Meta‐Analytic Investigation of the Relative Validity of Contextualized and Noncontextualized Personality Measures

The empirical evidence that has accumulated in support of the notion that personality is a valid predictor of employee performance is vast, yet debate on the matter continues. This study investigates frame‐of‐reference effects as they relate to the validity of self‐report measures of personality. Specifically, we compare the validities of general, noncontextualized personality measures and work‐specific, contextualized measures. The findings suggest that personality measures are a more valid predictor of performance when the scale items or instructions are framed specifically so as to reference work‐specific behaviors. We found that the validities for noncontextualized measures of personality ranged from .02 to .22, with a mean validity of .11. The validities for contextualized measures ranged from .14 to .30, with a mean of .24 . Additional moderator analyses were conducted in an effort to examine several alternate explanations for these validity differences. Specifically, we examined differences between the developmental purpose (general use vs. workplace use) and reliabilities of each type of personality measure. We also compared the validities from published studies to those from unpublished studies. Results suggest that these moderators did not have an impact on the validity differences between noncontextualized and contextualized measures.     

Functions of Identity: Scale Construction and Validation

The purpose of this study was to construct a reliable indicator of the 5 functions of identity proposed by Adams and Marshall (1996). Following Loevinger's (1957) method of test construction, 2 related studies were completed to test the substantive, structural, and external validity of the Functions of Identity Scale. Study 1 provides support for the substantive validity (factor structure, internal consistency, and construct validity). Results of Study 2 reveal evidence for 4 functions of identity and support for the external validity of the scale. As predicted, the complexity of identity status (diffusion, foreclosure, moratorium, and achievement) was positively correlated with the functions of identity, except for the measure of harmonious goals. These findings also support the conceptualization of the process of identity formation based on the distinctions of active and passive identity. Passive identity (diffusion plus foreclosure) was associated with lower scores on the Functions of Identity Scale than was active identity (moratorium plus achievement).     

The Validity of Verifiable and Non‐verifiable Biodata Items: An Examination Across Applicants and Incumbents

This study examines the influence of item verifiability (non‐verifiable vs. verifiable), context (applicant vs. incumbent), and keying procedure on biodata mean test scores and validity. Concurrent and predictive validation studies were conducted using a sample of 425 call center incumbents and a sample of 410 call center applicants. Although applicants did not obtain significantly higher mean biodata scores, results provide support for the hypothesis that the non‐verifiable biodata composite would be less valid in the applicant context, while the verifiable biodata composite would be equally valid across both the applicant and incumbent contexts. The same pattern of results was obtained using both item‐ and option‐keying procedures. Implications for research and practice are discussed.     

Measurement and Application of Egoistic and Moralistic Self‐enhancement

Five studies were conducted to develop and validate the Egoistic and Moralistic Self‐enhancement (EMS) scale, a new self‐report measure of egoistic and moralistic self‐enhancement. Exploratory and confirmatory factor analysis provided evidence for a two‐factor structure comprising the tendencies to promote agentic (egoistic) and communal (moralistic) qualities (Study 1). Construct validity was supported by significant correlations with established measures of socially desirable responding, such as the Balanced Inventory of Desirable Responding (Study 2) and the Impression Management scale of the 16 Personality Factors (Study 3). When the EMS scale has been applied in organizational settings, we found that job applicants were more inclined than nonapplicants to promote their own agentic and communal qualities, although differences were more pronounced for egoistic self‐enhancement (Study 4). Findings also suggested that exaggeration of agentic qualities decrease the criterion‐related validity of conscientiousness in predicting job performance (Study 5). Taken together, findings suggest that the EMS scale provides a valid and reliable measure that can be fruitfully used for both theoretical and applied research.     

Knowing What NOT To Do Is a Critical Job Skill: Evidence from 10 different scoring methods

Situational judgment tests (SJTs) have become an increasingly important tool for predicting employee performance; however, at least two key areas warrant further investigation. First, prior studies of SJTs have generally relied on samples from the western world, leaving open the question of the validity of using SJTs in the developing world where the majority of the world's workforce resides. Second, there is currently no standardized, theoretically‐based method for the development and scoring of SJTs. Therefore, SJTs are highly domain‐specific and must be developed anew for each new context. We report the results of three studies, conducted in India, that aim to: (1) test the cross‐cultural validity of SJTs in a non‐western context, and (2) examine the differential validity of 10 different approaches to scoring SJTs, some of which have the potential to resolve the problem of developing a theoretically‐infused, standardized approach to scoring and future development.     

Forming Composites of Cognitive Ability and Alternative Measures to Predict Job Performance and Reduce Adverse Impact: Corrected Estimates and Realistic Expectations

Although there has been empirical attention paid to the criterion‐related validity of predictor composites, there has been much less attention paid to the standardized ethnic group differences associated with these composites. One important area of inquiry in predictor composite research is the influence of adding predictors to a test of general mental ability. The limited empirical literature on this practice is mixed, but the prevailing expectation is that there is likely to be higher validity and less adverse impact. Unfortunately, much of the previous work is limited by the presence of inaccurate validity and standardized ethnic group difference values. In this analysis we formed meta‐analytic matrices to more accurately estimate the validity and standardized ethnic group differences of several composites that combine a measure of cognitive ability with measures of conscientiousness, a structured interview, or biodata. While results were somewhat complex, we found that adding alternative predictors does not result in a situation in which validity automatically goes up and adverse impact potential automatically goes down. In fact, the reductions in adverse impact (if any) from adding “non‐cognitive” predictors were more modest than much of the literature suggests.