Reduction of faking with the use of a forced-choice personality test: Cross-cultural comparisons between South Korea and the United States

Forced-choice format tests have been suggested as an alternative to Likert-scale measures for personnel selection due to robustness to faking and response styles. This study compared degrees of faking occurring in Likert-scale and forced-choice five-factor personality tests between South Korea and the United States. Also, it was examined whether the forced-choice format was effective at reducing faking in both countries. Data were collected from 396 incumbents participating in both honest and applicant conditions (N_SK = 179, N_US = 217). Cohen's d values for within-subjects designs (ds_within) for between the two conditions were utilized to measure magnitudes of faking occurring in each format and country. In both countries, the degrees of faking occurring in the Likert-scale were larger than those from the forced-choice format, and the magnitudes of faking across five personality traits were larger in South Korea by from 0.07 to 0.12 in ds_within. The forced-choice format appeared to successfully reduce faking for both countries as the average ds_within decreased by 0.06 in both countries. However, the patterns of faking occurring in the forced-choice format varied between the two countries. In South Korea, degrees of faking in Openness and Conscientiousness increased, whereas those in Extraversion and Agreeableness were substantially decreased. Potential factors leading to trait-specific faking under the forced-choice format were discussed in relation to cultural influence on the perception of personality traits and score estimation in Thurstonian item response theory (IRT) models. Finally, the adverse impact of using forced-choice formats on multicultural selection settings was elaborated.     

人才甄选中的人格测验作假研究的深入与拓展

企业人才甄选情境下求职者很容易在人格测验中作假。至今有关作假的研究已包含作假的内涵、来源和识别等多个方面,也诞生了多种心理模型尝试解释作假产生的心理机制,如作假动机与作假能力交互作用理论、作假计划行为理论、作假整合模型、一般作假行为模型以及作假的VIE模型,为后续理论研究点明方向。此外,作假应用领域中新兴的网络人格测验作假受到关注,在此介绍网络与纸笔测验两种形式下,人格测验作假行为、作假意向的不同。     

Significance tests to determine the direction of effects in linear regression models

Previous studies have discussed asymmetric interpretations of the Pearson correlation coefficient and have shown that higher moments can be used to decide on the direction of dependence in the bivariate linear regression setting. The current study extends this approach by illustrating that the third moment of regression residuals may also be used to derive conclusions concerning the direction of effects. Assuming non‐normally distributed variables, it is shown that the distribution of residuals of the correctly specified regression model (e.g., Y is regressed on X) is more symmetric than the distribution of residuals of the competing model (i.e., X is regressed on Y). Based on this result, 4 one‐sample tests are discussed which can be used to decide which variable is more likely to be the response and which one is more likely to be the explanatory variable. A fifth significance test is proposed based on the differences of skewness estimates, which leads to a more direct test of a hypothesis that is compatible with direction of dependence. A Monte Carlo simulation study was performed to examine the behaviour of the procedures under various degrees of associations, sample sizes, and distributional properties of the underlying population. An empirical example is given which illustrates the application of the tests in practice.     

A Statistical Hypothesis Testing Method for the Rank Ordering of the Priorities of the Alternatives in the Analytic Hierarchy Process

In analytic hierarchy process (AHP), a ratio scale (π₁, π₂, ⋯, π_t) for the priorities of the alternatives {T₁, T₂, ⋯, T_t} is used for a decision problem in which π_i/π_j is used to quantify the ratio of the priority of T_i to that of T_j. In a group decision‐making setup, the subjective estimates of π_i/π_j are obtained as entries of a pairwise comparison matrix for each member of the group. On the basis of these pairwise comparison matrices, one of the topics of interest in some situation is the total rank ordering of the priorities of the alternatives. In this article, a statistical method is proposed for testing a specific total rank ordering of the priorities of the alternatives. The method developed is then illustrated using numerical examples. Copyright © 2014 John Wiley & Sons, Ltd.     

Investigating Test-Taking Behaviors Using Timing and Process Data

The use of computer-based assessments makes the collection of detailed data that capture examinees’ progress in the tests and time spent on individual actions possible. This article presents a study using process and timing data to aid understanding of an international language assessment and the examinees. Issues regarding test-taking strategies, test speededness, test design, and their relationship to examinees’ demographic backgrounds and performance are also discussed.     

First Among Equals: Hybridization of Cognitive Diagnostic Assessment and Evidence-Centered Game Design

The objective of the present article is to explore differences and similarities between cognitive diagnostic assessment (CDA) and evidence-centered game design (ECgD) in the service of intentional hybridization. Although some testing specialists might argue that both are essentially the same given their origins in principled assessment design and equivalency of measurement models, this view misses differences in their focus and operationalization. Given the strengths of both CDA and ECgD, there is motivation to consider ways in which each can deliberately inform the other. The intentional hybridization of CDA and ECgD has, at least in principle, significant advantages to produce a stronger offspring than either parent alone. The article includes four sections: (1) conceptual differences between CDA and ECgD, (2) conceptual similarities between CDA and ECgD, (3) challenges with CDA and ECgD, including narrowness of cognitive models, fidelity with learning, ocean of data, sensitivity to diverse learners, reliance on multidimensional psychometric models, and how hybridization may help, and (4) implications for educational assessment in the twenty-first century around the globe.     

人格特质对考试焦虑的影响:多重中介效应分析

目的:探讨自尊、惧怕负面评价及无法忍受不确定性在人格特质和考试焦虑间的中介作用。方法:采用人格问卷简式量表EPQ-RSC、Rosenberg自尊问卷、惧怕负面评价量表简表、无法忍受不确定性量表和考试焦虑量表测量410名在校高中生和大学生。结果:(1)相关分析表明,神经质、惧怕负面评价、无法忍受不确定性及考试焦虑之间彼此正相关显著;外向性与自尊正相关显著,与惧怕负面评价、无法忍受不确定性及考试焦虑等负相关显著;精神质与自尊、惧怕负面评价及考试焦虑等负相关显著;自尊与神经质、惧怕负面评价、无法忍受不确定性及考试焦虑负相关显著。(2)回归分析表明,人格特质中的神经质和精神质能显著预测考试焦虑,无法忍受不确定性在人格特质和考试焦虑之间起部分中介作用;结构方程模型分析结果显示,人格特质中神经质和精神质对考试焦虑的直接效应显著,经由自尊、惧怕负面评价和无法忍受不确定性对考试焦虑的间接效应也极为显著。结论:自尊、惧怕负面评价和无法忍受不确定性在人格特质与考试焦虑间起着中介作用。     

A closer look at the discrimination outcomes in the IAT literature

To what extent the IAT (Implicit Association Test, Greenwald et al., 1998) predicts racial and ethnic discrimination is a heavily debated issue. The latest meta‐analysis by Oswald et al. (2013) suggests a very weak association. In the present meta‐analysis, we switched the focus from the predictor to the criterion, by taking a closer look at the discrimination outcomes. We discovered that many of these outcomes were not actually operationalizations of discrimination, but rather of other related, but distinct, concepts, such as brain activity and voting preferences. When we meta‐analyzed the main effects of discrimination among the remaining discrimination outcomes, the overall effect was close to zero and highly inconsistent across studies. Taken together, it is doubtful whether the amalgamation of these outcomes is relevant criteria for assessing the IAT's predictive validity of discrimination. Accordingly, there is also little evidence that the IAT can meaningfully predict discrimination, and we thus strongly caution against any practical applications of the IAT that rest on this assumption. However, provided that the application is thoroughly informed by the current state of the literature, we believe the IAT can still be a useful tool for researchers, educators, managers, and students who are interested in attitudes, prejudices, stereotypes, and discrimination.     

Improving Psychological Measurement: Does It Make a Difference? A Comment on Nesselroade and Molenaar (2016)

Nesselroade and Molenaar advocate the use of an idiographic filter approach. This is a fixed-effects approach, which may limit the number of individuals that can be simultaneously modeled, and it is not clear how to model the presence of subpopulations. Most important, Nesselroade and Molenaar's proposal appears to be best suited for modeling long time series on a few variables for a few individuals. Long time series are not common in psychological applications. Can it be applied to the usual longitudinal data we face? These are characterized by short time series (four to five points in time), hundreds of individuals, and dozens of variables. If so, what do we gain? Applied settings most often involve between-individual decisions. I conjecture that their approach will not outperform common, simpler, methods. However, when intraindividual decisions are involved, their approach may have an edge.