A meta-analytic investigation of conscientiousness in the prediction of job performance: examining the intercorrelations and the incremental validity of narrow traits

Researchers of broad and narrow traits have debated whether narrow traits are important to consider in the prediction of job performance. Because personality-performance relationship meta-analyses have focused almost exclusively on the Big Five, the predictive power of narrow traits has not been adequately examined. In this study, the authors address this question by meta-analytically examining the degree to which the narrow traits of conscientiousness predict above and beyond global conscientiousness. Results suggest that narrow traits do incrementally predict performance above and beyond global conscientiousness, yet the degree to which they contribute depends on the particular performance criterion and occupation in question. Overall, the results of this study suggest that there are benefits to considering the narrow traits of conscientiousness in the prediction of performance.     

Comment on a Monte Carlo investigation of scaling as an alternative to regression

In a recent study, Emery found that MONANOVA is “superior” to regression as a method of approximating data, whether the measurement scale of the criterion variable is interval or ordinal. However, there are three major reasons for these results. With comparison methods that are fairer to both analytical procedures, the performances of regression and of MONANOVA were found to be close even when the measurement scale of the criterion variable is ordinal.     

A Monte Carlo investigation of conjoint analysis index-of-fit: Goodness of fit,significance and power

Researchers in the field of conjoint analysis know the index-of-fit values worsen as the judgmental error of evaluation increases. This simulation study provides guidelines on the goodness of fit based on distribution of index-of-fit for different conjoint analysis designs. The study design included the following factors: number of profiles, number of attributes, algorithm used and judgmental model used. Critical values are provided for deciding the statistical significance of conjoint analysis results. Using these cumulative distributions, the power of the test used to reject the null hypothesis of random ranking is calculated. The test is found to be quite powerful except for the case of very small residual degrees of freedom.The authors thank the editor, the three reviewers and Ellen Foxman for helpful comments on the paper. Sanjay Mishra was a doctoral student at Washington State University at the time this research was completed. He is currently in the Department of Marketing at the University of Kansas.     

Sampling behaviour in estimating predictive validity in the context of selection and latent variable modelling: A Monte Carlo study

While the effect of selection in predictive validity studies has long been recognized and discussed in psychometric studies, little consideration has been given to this problem in the context of latent variable models. In a recent paper, Muthén & Hsu (1993) proposed and compared estimators of predictive validity of a multifactorial test. Both selectivity and measurement error were considered in the estimation of predictive validity. The purpose of the present paper is to expand on Muthén & Hsu (1993) by examining and comparing the sampling behaviour of three estimators for predictive validity, LQL (listwise, quasi-likelihood estimator), FQL (full, quasi-likelihood estimator) and FS (factor score estimator), using a Monte Carlo approach. Effects of selection procedures, selection ratios and sample sizes on the sampling behaviours of the estimators are also investigated. The results show that FQL and FS are the two preferred estimators and each has different strengths and weaknesses. A real data application is presented to illustrate the practical implementation of the estimators.     

The criterion-related validity of conscientiousness in personnel selection: A meta-analytic reality check

A key finding in personnel selection is the positive correlation between conscientiousness and job performance. Evidence predominantly stems from concurrent validation studies with incumbent samples but is readily generalized to predictive settings with job applicants. This is problematic because the extent to which faking and changes in personality affect the measurement likely vary across samples and study designs. Therefore, we meta-analytically investigated the relation between conscientiousness and job performance, examining the moderating effects of sample type (incumbent vs. applicant) and validation design (concurrent vs. predictive). The overall correlation of conscientiousness and job performance was in line with previous meta-analyses ($\overline{r}=.17,k=102,n=23,305$). In our analyses, the correlation did not differ across validation designs (concurrent: $\overline{r}=.18,k=78,n=19,132$; predictive: $\overline{r}=.15,k=24,n=4173$), sample types (incumbents: $\overline{r}=.18,k=92,n=20,808$; applicants: $\overline{r}=.14,k=10,n=2497$), or their interaction. Critically, however, our review revealed that only a small minority of studies (~12%) were conducted with real applicants in predictive designs. Thus, barely a fraction of research is conducted under realistic conditions. Therefore, it remains an open question if self-report measures of conscientiousness retain their predictive validity in applied settings that entail faked responses. We conclude with a call for more multivariate research on the validity of selection procedures in predictive settings with actual applicants.     

Taxometric analysis of fuzzy categories: a Monte Carlo study

A small Monte Carlo study examined the performance of a form of taxometric analysis (the MAXCOV procedure) with fuzzy data sets. These combine taxonic (categorical) and nontaxonic (continuous) features, containing a subset of casts with intermediate degrees of category membership. Fuzzy data sets tended to yield taxonic findings on plot inspection and two popular consistency tests, even when the degree of fuzziness, i.e., the proportion of intermediate cases, was large. These results suggest that fuzzy categories represent a source of pseudotaxonic inferences, if on is understood in the usual binary, "either-or" fashion. This in turn implies that dichotomous causes cannot be confidently inferred when taxometric analyses yield apparently taxonic findings.     

Magnetic properties of Mn-doped armchair ZnO nanotubes: a Monte Carlo study

The magnetic properties of a Mn-doped armchair ZnO nanotube have been studied using Monte Carlo simulation. The variation of zero-field-cooled and field-cooled magnetisation with reduced temperatures for different values of the dilution x (where x is the Mn concentration: Zn_1?xMn_xO) are given. The freezing temperatures and magnetisation vs. crystal field are calculated for different dilutions x. Finally, the hysteresis loops for different dilutions and temperatures are given for a fixed reduced temperature and crystal field. Superparamagnetic behaviour is observed for small values of x and low temperatures.     

Monte Carlo simulations and the chi-square test of independence

A Monte Carlo program for sampling 2 by 2 contingency tables from a user-specified population is discussed. Applications include computer-assisted instruction (CAI) of statistics, evaluation of actual vs nominal Type I error rates of the chi-square test of independence when expected frequencies are less than 10, and estimation of the power of the chi-square test.     

Synthesizing single-case studies: A Monte Carlo examination of a three-level meta-analytic model

Numerous ways to meta-analyze single-case data have been proposed in the literature; however, consensus has not been reached on the most appropriate method. One method that has been proposed involves multilevel modeling. For this study, we used Monte Carlo methods to examine the appropriateness of Van den Noortgate and Onghena's (2008) raw-data multilevel modeling approach for the meta-analysis of single-case data. Specifically, we examined the fixed effects (e.g., the overall average treatment effect) and the variance components (e.g., the between-person within-study variance in the treatment effect) in a three-level multilevel model (repeated observations nested within individuals, nested within studies). More specifically, bias of the point estimates, confidence interval coverage rates, and interval widths were examined as a function of the number of primary studies per meta-analysis, the modal number of participants per primary study, the modal series length per primary study, the level of autocorrelation, and the variances of the error terms. The degree to which the findings of this study are supportive of using Van den Noortgate and Onghena's (2008) raw-data multilevel modeling approach to meta-analyzing single-case data depends on the particular parameter of interest. Estimates of the average treatment effect tended to be unbiased and produced confidence intervals that tended to overcover, but did come close to the nominal level as Level-3 sample size increased. Conversely, estimates of the variance in the treatment effect tended to be biased, and the confidence intervals for those estimates were inaccurate.     

Self-monitoring personality at work: a meta-analytic investigation of construct validity

The validity of self-monitoring personality in organizational settings was examined. Meta-analyses were conducted (136 samples; total N = 23,191) investigating the relationship between self-monitoring personality and work-related variables, as well as the reliability of various self-monitoring measures. Results suggest that self-monitoring has relevance for understanding many organizational concerns, including job performance and leadership emergence. Sample-weighted mean differences favoring male respondents were also noted, suggesting that the sex-related effects for self-monitoring may partially explain noted disparities between men and women at higher organizational levels (i.e., the glass ceiling). Theory building and additional research are needed to better understand the construct-related inferences about self-monitoring personality, especially in terms of the performance, leadership, and attitudes of those at top organizational levels.     

Latent variables should remain as such: Evidence from a Monte Carlo study

Use of subject scores as manifest variables to assess the relationship between latent variables produces attenuated estimates. This has been demonstrated for raw scores from classical test theory (CTT) and factor scores derived from factor analysis. Conclusions on scores have not been sufficiently extended to item response theory (IRT) theta estimates, which are still recommended for estimation of relationships between latent variables. This is because IRT estimates appear to have preferable properties compared to CTT, while structural equation modeling (SEM) is often advised as an alternative to scores for estimation of the relationship between latent variables. The present research evaluates the consequences of using subject scores as manifest variables in regression models to test the relationship between latent variables. Raw scores and three methods for obtaining theta estimates were used and compared to latent variable SEM modeling. A Monte Carlo study was designed by manipulating sample size, number of items, type of test, and magnitude of the correlation between latent variables. Results show that, despite the advantage of IRT models in other areas, estimates of the relationship between latent variables are always more accurate when SEM models are used. Recommendations are offered for applied researchers.     

An investigation of the Mini-Mult validity scales

Examined the ability of the Mini-Mult validity scales to detect invalid MMPI profiles. When 34 invalid MMPI profiles were rescored with the Mini-Muir only 17 of the 34 profiles invalidated by the full MMPI were detected with the Mini-Mult. This included 14 of 27 profiles invalidated by an elevated F scale; 2 of 4 profiles invalidated by an elevated L scale and 1 of 3 profiles invalidated by an elevated K scale. Only 14 of 27 profiles invalidated by an F-K ratio of K11 were detected. When new conversion values for the Mini-Mult were utilized, the detection rate improved considerably for the F scale and the F-K ratio.     

缺失数据处理的期望-极大化算法与马尔可夫蒙特卡洛方法

缺失数据普遍存在于心理学研究中, 影响着统计推断。极大似然估计(MLE)与基于贝叶斯的多重借补(MI)是处理缺失数据的两类重要方法。期望-极大化算法(EM)是寻求MLE的一种强有力的方法。马尔可夫蒙特卡洛方法(MCMC)可以相对简易地实现MI, 而且可以适用于复杂情况下的缺失数据处理。结合研究的需要讨论了实现这两类方法的适用软件。     

Faking Fast and Slow: Within-Person Response Time Latencies for Measuring Faking in Personnel Testing

Purpose

Item response time (RT) latencies offer a potentially promising approach for measuring faking in personnel testing, but have been studied almost exclusively as either long or short RTs relative to group norms. As such, the ability to reliably assess faking RTs at the individual level remains a challenge. To address this issue, the present study set out to examine the usefulness of a within-person difference score index (DSI) method for measuring faking, in which “control question” (baseline) RTs were compared to “target question” RTs, within single test administrations.

Design/Methodology/Approach

Two hundred six participants were randomly selected to simulated faking or honest testing conditions, and were administered two types of integrity test items (overt and personality), whereby group classification (faking/honest) served as the main dependent variable.

Findings

Faking condition RTs were longer than honest condition RTs for both item types (overt: d = .43; personality: d = .47), and overt item RTs were slightly shorter than personality item RTs in both testing conditions (honest: d = .34; faking: d = .41). Finally, using a sample cut score, the DSI correctly classified an average of 26 % more cases of faking, and 53 % less false positives, compared to the traditional normative method.

Implications

The results suggest that the DSI can be an advantageous method for identifying faking in personnel testing scenarios.

Originality/Value

This is the one of the first studies to propose a practical method for identifying individual-level faking RTs within single test administrations.

     

Assessing categorization performance at the individual level: a comparison of Monte Carlo simulation and probability estimate model procedures

Two analytical procedures for identifying young children as categorizers, the Monte Carlo Simulation and the Probability Estimate Model, were compared. Using a sequential touching method, children aged 12, 18, 24, and 30 months were given seven object sets representing different levels of categorical classification. From their touching performance, the probability that children were categorizing was then determined independently using Monte Carlo Simulation and the Probability Estimate Model. The two analytical procedures resulted in different percentages of children being classified as categorizers. Results using the Monte Carlo Simulation were more consistent with group-level analyses than results using the Probability Estimate Model. These findings recommend using the Monte Carlo Simulation for determining individual categorizer classification.     

Comments on Monte Carlo simulations of void-lattice formation

We study the coalescence of neighbouring voids along close-packed directions in recent computer simulation studies of void-lattice formation. The stability against coalescence of a developing void lattice is found to depend very much on the detailed geometry of the local void distribution. The possibility of void coalescence as an artifact, caused by the incorrect assumption of homogeneous void nucleation in these simulations, is suggested and discussed.     

Why Monte Carlo Simulations Are Inferences and Not Experiments

Monte Carlo simulations arrive at their results by introducing randomness, sometimes derived from a physical randomizing device. Nonetheless, we argue, they open no new epistemic channels beyond that already employed by traditional simulations: the inference by ordinary argumentation of conclusions from assumptions built into the simulations. We show that Monte Carlo simulations cannot produce knowledge other than by inference, and that they resemble other computer simulations in the manner in which they derive their conclusions. Simple examples of Monte Carlo simulations are analysed to identify the underlying inferences.