The Wished‐For Always Wins Until the Winner Was Inevitable All Along: Motivated Reasoning and Belief Bias Regulate Emotion During Elections

How do biases affect political information processing? A variant of the Wason selection task, which tests for confirmation bias, was used to characterize how the dynamics of the recent U.S. presidential election affected how people reasoned about political information. Participants were asked to evaluate pundit‐style conditional claims like “The incumbent always wins in a year when unemployment drops” either immediately before or immediately after the 2012 presidential election. A three‐way interaction between ideology, predicted winner (whether the proposition predicted that Obama or Romney would win), and the time of test indicated complex effects of bias on reasoning. Before the election, there was partial evidence of motivated reasoning—liberals performed especially well at looking for falsifying information when the pundit's claim predicted Romney would win. After the election, once the outcome was known, there was evidence of a belief bias—people sought to falsify claims that were inconsistent with the real‐world outcome rather than their ideology. These results suggest that people seek to implicitly regulate emotion when reasoning about political predictions. Before elections, people like to think their preferred candidate will win. After elections, people like to think the winner was inevitable all along.     

Panning for the gold in health research: Incorporating studies’ methodological quality in meta-analysis

Background: Systematic reviews now routinely assess methodological quality to gauge the validity of the included studies and of the synthesis as a whole. Although trends from higher quality studies should be clearer, it is uncertain how often meta-analyses incorporate methodological quality in models of study results either as predictors, or, more interestingly, in interactions with theoretical moderators.

Objective and methods: We survey 200 meta-analyses in three health promotion domains to examine when and how meta-analyses incorporate methodological quality.

Results: Although methodological quality assessments commonly appear in contemporary meta-analyses (usually as scales), they are rarely incorporated in analyses, and still more rarely analysed in interaction with theoretical determinants of the success of health promotions. The few meta-analyses (2.5%) that did include such an interaction analysis showed that moderator results remained significant in higher quality studies or were present only among higher quality studies. We describe how to model quality interactively with theoretically derived moderators and discuss strengths and weaknesses of this approach and in relation to current meta-analytic practice.

Conclusions: In large literatures exhibiting heterogeneous effects, meta-analyses can incorporate methodological quality and generate conclusions that enable greater confidence not only about the substantive phenomenon but also about the role that methodological quality itself plays.     

Developing mixed methods research in sport and exercise psychology: Critical reflections on five points of controversy

ObjectivesTo stimulate debate in sport and exercise psychology about the nature of mixed methods research as currently practiced and how this approach might develop in the future.DesignAn exploration of five points of controversy relating to mixed methods research.MethodA presentation of critical reflections on the following. (1) Mixing methods as a non-debate, (2) Purists, pragmatists and mixing paradigms, (3) Integrating findings and representational forms, (4) Judgment criteria and mixed methods research, and (5) Power, politics and what counts in mixed methods research.ResultsThe examples provided of mixed methods research in action indicate that a number of problematic issues regarding both process and product have been neglected.ConclusionsMixed methods research offers a number of conceptual, practical and pedagogical challenges that need to be addressed if this form of inquiry is to develop its full potential in sport and exercise psychology.     

Polytomous multilevel testlet models for testlet‐based assessments with complex sampling designs

A pplications of standard item response theory models assume local independence of items and persons. This paper presents polytomous multilevel testlet models for dual dependence due to item and person clustering in testlet‐based assessments with clustered samples. Simulation and survey data were analysed with a multilevel partial credit testlet model. This model was compared with three alternative models – a testlet partial credit model (PCM), multilevel PCM, and PCM – in terms of model parameter estimation. The results indicated that the deviance information criterion was the fit index that always correctly identified the true multilevel testlet model based on the quantified evidence in model selection, while the Akaike and Bayesian information criteria could not identify the true model. In general, the estimation model and the magnitude of item and person clustering impacted the estimation accuracy of ability parameters, while only the estimation model and the magnitude of item clustering affected the item parameter estimation accuracy. Furthermore, ignoring item clustering effects produced higher total errors in item parameter estimates but did not have much impact on the accuracy of ability parameter estimates, while ignoring person clustering effects yielded higher total errors in ability parameter estimates but did not have much effect on the accuracy of item parameter estimates. When both clustering effects were ignored in the PCM, item and ability parameter estimation accuracy was reduced.     

APPLICANT REACTIONS TO PRE-EMPLOYMENT APPLICATION BLANKS: A LEGAL AND PROCEDURAL JUSTICE PERSPECTIVE

The current research investigated applicant reactions derived from common application blanks. Previous work has found applications to be riddled with legally inadvisable items, but limited research has attempted to understand how such items might affect applicants. By using organizational justice theory as a framework, two studies examined applicant reactions from two application blanks: a legally advisable application and a legally problematic application. It was found that applicants completing the legally problematic application had lower perceptions of justice and expressed higher litigation intentions compared to those completing the legally advisable application, especially for those that were rejected without an explanation.

Don't hate me because I'm beautiful: Anti-attractiveness bias in organizational evaluation and decision making

Within organizational judgment and decision making contexts, biases based on an evaluated person's attractiveness are among the most salient and frequently investigated. An enormous amount of research indicates favoritism for attractive people compared to unattractive ones. The current research demonstrates that the nature of this bias depends on whether one is evaluating a member of the same sex or the opposite sex. Experiment 1 (n = 2639) investigated selection of scholarship applicants and demonstrated that a pro-attractiveness bias held only for selection of opposite-sex scholarship applicants; no such bias was observed for highly attractive same-sex applicants. Experiment 2 (n = 622) investigated evaluations of prospective job candidates and demonstrated again that pro-attractiveness bias was observed only for opposite-sex candidates; participants discriminated against highly attractive same-sex candidates. Moreover, this bias was not observed among highly attractive participants; it held only for moderately attractive participants, those for whom highly attractive same-sex individuals can pose especially potent social threats. Findings suggest that attractiveness biases in organizational decision making are rooted partly in the social threats and opportunities afforded by attractive people.     

Distribution-free tests of stochastic dominance for small samples

One variable is said to “stochastically dominate” another if the probability of observations smaller than x is greater for one variable than the other, for all x. Inferring stochastic dominance from data samples is important for many applications of econometrics and experimental psychology, but little is known about the performance of existing inferential methods. Through simulation, we show that three of the most widely used inferential methods are inadequate for use in small samples of the size commonly encountered in many applications (up to 400 observations from each distribution). We develop two new inferential methods that perform very well in a limited, but practically important, case where the two variables are guaranteed not to be equal in distribution. We also show that extensions of these new methods, and an improved version of an existing method, perform quite well in the original, unlimited case.     

On the Speed Sensitivity Parameter in the Lognormal Model for Response Times and Implications for High-Stakes Measurement Practice

In high-stakes testing, often multiple test forms are used and a common time limit is enforced. Test fairness requires that ability estimates must not depend on the administration of a specific test form. Such a requirement may be violated if speededness differs between test forms. The impact of not taking speed sensitivity into account on the comparability of test forms regarding speededness and ability estimation was investigated. The lognormal measurement model for response times by van der Linden was compared with its extension by Klein Entink, van der Linden, and Fox, which includes a speed sensitivity parameter. An empirical data example was used to show that the extended model can fit the data better than the model without speed sensitivity parameters. A simulation was conducted, which showed that test forms with different average speed sensitivity yielded substantial different ability estimates for slow test takers, especially for test takers with high ability. Therefore, the use of the extended lognormal model for response times is recommended for the calibration of item pools in high-stakes testing situations. Limitations to the proposed approach and further research questions are discussed.