A method of bias correction for maximal reliability with dichotomous measures

This paper is concerned with the reliability of weighted combinations of a given set of dichotomous measures. Maximal reliability for such measures has been discussed in the past, but the pertinent estimator exhibits a considerable bias and mean squared error for moderate sample sizes. We examine this bias, propose a procedure for bias correction, and develop a more accurate asymptotic confidence interval for the resulting estimator. In most empirically relevant cases, the bias correction and mean squared error correction can be performed simultaneously. We propose an approximate (asymptotic) confidence interval for the maximal reliability coefficient, discuss the implementation of this estimator, and investigate the mean squared error of the associated asymptotic approximation. We illustrate the proposed methods using a numerical example.     

Slope bias of psychometric functions derived from adaptive data

Several investigators have fit psychometric functions to data from adaptive procedures for threshold estimation. Although the threshold estimates are in general quite correct, one encounters a slope bias that has not been explained up to now. The present paper demonstrates slope bias for parametric and nonparametric maximum-likelihood fits and for Spearman-Kärber analysis of adaptive data. The examples include staircase and stochastic approximation procedures. The paper then presents an explanation of slope bias based on serial data dependency in adaptive procedures. Data dependency is first illustrated with simple two-trial examples and then extended to realistic adaptive procedures. Finally, the paper presents an adaptive staircase procedure designed to measure threshold and slope directly. In contrast to classical adaptive threshold-only procedures, this procedure varies both a threshold and a spread parameter in response to double trials.     

A meta-analysis of empirical studies of weight-based bias in the workplace

For nearly 30 years researchers have investigated how bodyweight affects evaluative workplace outcomes, such as hiring decisions and performance appraisals. Despite this, no meta-analytic review has been undertaken to quantify the negative impact that bodyweight has on such outcomes. The results of this meta-analytic study suggest that in relation to non-overweight individuals in the workplace, overweight individuals may be disadvantaged across evaluative workplace outcomes (d = −.52). Further, differences in magnitude of the effects of weight-based bias were found for hiring (d = −.70) and performance (d = −.23) outcomes.     

Six-month prognostic norms derived from studies of the Rorschach Prognostic Rating Scale

The Rorschach Prognostic Rating Scale (RPRS) was introduced in 1951 by Klopfer. Kirkner, Wisham, and Baker. The predictions of Klopfer et al. are compared to the outcomes in four studies of the RPRS. The originalinterpretation is shown to predict higher percentages of success than revealed by the empirical studies. A second interpretation of the scale is proposed on the basis of the experimental data. This interpretation relates RPRS scores to the chance for substantial improvement within 30 weeks of once weekly therapy by client-centered, rational-emotive, desensitization, aversion, or traditional methods. For any given RPRS score, the chance for substantial improvement is approximately the same for every type of therapy, and increases as the RPRS score increases. The second interpretation is proposed in both tubular and algebraic forms as a stimulus to further research and clinical applications.     

Slope bias of psychometric functions derived from adaptive data.

Several investigators have fit psychometric functions to data from adaptive procedures for threshold estimation. Although the threshold estimates are in general quite correct, one encounters a slope bias that has not been explained up to now. The present paper demonstrates slope bias for parametric and nonparametric maximum-likelihood fits and for Spearman-K?rber analysis of adaptive data. The examples include staircase and stochastic approximation procedures. The paper then presents an explanation of slope bias based on serial data dependency in adaptive procedures. Data dependency is first illustrated with simple two-trial examples and then extended to realistic adaptive procedures. Finally, the paper presents an adaptive staircase procedure designed to measure threshold and slope directly. In contrast to classical adaptive threshold-only procedures, this procedure varies both a threshold and a spread parameter in response to double trials.     

Mood acknowledgment and correction for the mood-congruency bias in social judgment

Past research has revealed a mood-congruency bias wherein people evaluate other individuals more positively when they are experiencing good moods than when they are experiencing bad moods. At times, however, people may attempt to prevent their transient mood states from biasing their evaluations of other people. It was proposed that the capacity to attend openly to one’s moods is an important precursor to such mood correction efforts. Two studies supported this hypothesis. People who were encouraged to attend to their feelings (Study 1), as well as people who are naturally inclined to acknowledge their feelings (Study 2), were more likely than their counterparts to prevent their positive and negative moods from biasing their judgments of a target person.     

Evaluation of publication bias in response interruption and redirection: A meta-analysis

Publication bias is the disproportionate representation of studies with large effects and statistically significant findings in the published research literature. If publication bias occurs in single-case research design studies on applied behavior-analytic (ABA) interventions, it can result in inflated estimates of ABA intervention effects. We conducted an empirical evaluation of publication bias on an evidence-based ABA intervention for children diagnosed with autism spectrum disorder, response interruption and redirection (RIRD). We determined effect size estimates for published and unpublished studies using 3 metrics, percentage of nonoverlapping data (PND), Hedges' g, and log response ratios (LRR). Omnibus effect size estimates across all 3 metrics were positive, supporting that RIRD is an effective treatment for reducing problem behavior maintained by nonsocial consequences. We observed larger PND for published compared to unpublished studies, small and nonsignificant differences in LRR for published compared to unpublished studies, and significant differences in Hedges' g for published compared to unpublished studies, with published studies showing slightly larger effect. We found little, if any, difference in methodological quality between published and unpublished studies. While RIRD appears to be an effective intervention for challenging behavior maintained by nonsocial consequences, our results reflect some degree of publication bias present in the RIRD research literature.     

Measurement bias and error correction in a two-stage estimation for multilevel IRT models

Among current state-of-the-art estimation methods for multilevel IRT models, the two-stage divide-and-conquer strategy has practical advantages, such as clearer definition of factors, convenience for secondary data analysis, convenience for model calibration and fit evaluation, and avoidance of improper solutions. However, various studies have shown that, under the two-stage framework, ignoring measurement error in the dependent variable in stage II leads to incorrect statistical inferences. To this end, we proposed a novel method to correct both measurement bias and measurement error of latent trait estimates from stage I in the stage II estimation. In this paper, the HO-IRT model is considered as the measurement model, and a linear mixed effects model on overall (i.e., higher-order) abilities is considered as the structural model. The performance of the proposed correction method is illustrated and compared via a simulation study and a real data example using the National Educational Longitudinal Survey data (NELS 88). Results indicate that structural parameters can be recovered better after correcting measurement biases and errors.     

Characterizing belief bias in syllogistic reasoning: A hierarchical Bayesian meta-analysis of ROC data

The belief-bias effect is one of the most-studied biases in reasoning. A recent study of the phenomenon using the signal detection theory (SDT) model called into question all theoretical accounts of belief bias by demonstrating that belief-based differences in the ability to discriminate between valid and invalid syllogisms may be an artifact stemming from the use of inappropriate linear measurement models such as analysis of variance (Dube et al., Psychological Review, 117(3), 831–863, 2010). The discrepancy between Dube et al.’s, Psychological Review, 117(3), 831–863 (2010) results and the previous three decades of work, together with former’s methodological criticisms suggests the need to revisit earlier results, this time collecting confidence-rating responses. Using a hierarchical Bayesian meta-analysis, we reanalyzed a corpus of 22 confidence-rating studies (N =?993). The results indicated that extensive replications using confidence-rating data are unnecessary as the observed receiver operating characteristic functions are not systematically asymmetric. These results were subsequently corroborated by a novel experimental design based on SDT’s generalized area theorem. Although the meta-analysis confirms that believability does not influence discriminability unconditionally, it also confirmed previous results that factors such as individual differences mediate the effect. The main point is that data from previous and future studies can be safely analyzed using appropriate hierarchical methods that do not require confidence ratings. More generally, our results set a new standard for analyzing data and evaluating theories in reasoning. Important methodological and theoretical considerations for future work on belief bias and related domains are discussed.     

A meta-analysis of the effect of cognitive bias modification on anxiety and depression

Cognitive biases have been theorized to play a critical role in the onset and maintenance of anxiety and depression. Cognitive bias modification (CBM), an experimental paradigm that uses training to induce maladaptive or adaptive cognitive biases, was developed to test these causal models. Although CBM has generated considerable interest in the past decade, both as an experimental paradigm and as a form of treatment, there have been no quantitative reviews of the effect of CBM on anxiety and depression. This meta-analysis of 45 studies (2,591 participants) assessed the effect of CBM on cognitive biases and on anxiety and depression. CBM had a medium effect on biases (g = 0.49) that was stronger for interpretation (g = 0.81) than for attention (g = 0.29) biases. CBM further had a small effect on anxiety and depression (g = 0.13), although this effect was reliable only when symptoms were assessed after participants experienced a stressor (g = 0.23). When anxiety and depression were examined separately, CBM significantly modified anxiety but not depression. There was a nonsignificant trend toward a larger effect for studies including multiple training sessions. These findings are broadly consistent with cognitive theories of anxiety and depression that propose an interactive effect of cognitive biases and stressors on these symptoms. However, the small effect sizes observed here suggest that this effect may be more modest than previously believed.     

Malleability of weight-biased attitudes and beliefs: A meta-analysis of weight bias reduction interventions

Weight bias exists across many important life domains, necessitating interventions designed to reduce weight-biased attitudes and beliefs. Though the effectiveness of weight bias interventions has been questioned, to our knowledge no meta-analysis of these interventions has been conducted. This meta-analysis evaluated the impact of weight bias interventions on weight-biased attitudes and beliefs and explored potential moderators. Interventions were eligible if they used an adult sample and a validated measure of weight-biased attitudes, which resulted in the inclusion of 30 studies represented in 29 articles. A random effects approach using inverse weights resulted in a mean effect size estimate of g = −0.33 (lower scores indicate less weight bias) for both attitudes and beliefs. Intervention type, publication type, and population type were not significant moderators but demonstrated noteworthy trends. Results reveal a small, positive effect of weight bias interventions on weight-biased attitudes and beliefs and provide useful information for future interventions.     

Parasite-stress, cultures of honor, and the emergence of gender bias in purity norms

Of the many far-reaching implications of Fincher & Thornhill's (F&T's) theory, we focus on the consequences of parasite stress for mating strategies, marriage, and the differing roles and restrictions for men and women. In particular, we explain how examination of cultures of honor can provide a theoretical bridge between effects of parasite stress and disproportionate emphasis on female purity.     

A basis for bias in geographical judgments

To determine why North Americans tend to locate European cities south of North American cities at similar latitudes (Tversky, 1981), we had observers provide bearing estimates between cities in the U.S. and Europe. Earlier research using latitude estimates of these cities has indicated that each continent has several subjective regions (Friedman & Brown, 2000a). Participants judged cities from two subjectively northern regions (Milwaukee-Munich), two subjectively southern regions (Memphis-Lisbon), and the two "crossed" regions (Albuquerque-Geneva; Minneapolis-Rome). Estimates were biased only when cities from the subjectively northern regions of North America were paired with cities from the subjectively southern region of Europe. In contrast to the view that biases are derived from distorted or aligned map-like representations, the data provide evidence that the subjective representation of global geography is principally categorical. Biases in numerical location estimates of individual cities and in bearing estimates between city pairs are derived from plausible reasoning processes operating on the same categorical representations.     

Theory-based bias correction in dispositional inference: The fundamental attribution error is dead,long live the correspondence bias

Social psychological research has repeatedly shown that perceivers draw correspondent dispositional inferences from observed behaviour even when this behaviour was highly constrained by situational factors (i.e., the correspondence bias). Even though this phenomenon has been proposed to be multiply determined, the most common explanation is still that perceivers ubiquitously consider situational factors to have little impact on human behaviour (i.e., the fundamental attribution error). The present chapter offers a critical analysis of the available empirical evidence on the correspondence bias from the perspective of theory-based bias correction. It is concluded that the correspondence bias results from a number of different processes associated with the application of perceivers' causal theories about situational influences on human behaviour. However, there is no evidence for the assumption that the correspondence bias is due to causal theories implying that situational factors have little impact on human behaviour. Theoretical and empirical implications are discussed.     

The development of intergroup bias in childhood: how social norms can shape children's racial behaviours

The present research examined the developmental course of racial behaviours in childhood. It tested the hypothesis that White children's expressions of racial prejudice do not necessarily decline in middle childhood due to the development of particular cognitive skills but that instead, as argued by the socio-normative approach, children older than seven will go on expressing prejudiced attitudes under appropriate conditions. This would be explained by the presence of an anti-racism norm, along with the existence of values promoting equal rights, which impede blatant expressions of racism. In the first study 283 White children aged 6-7 and 9-10 years performed a task of resource allocation to White and Black target children in conditions of high (White interviewer was present) or low (White interviewer was absent) salience of the anti-racist norm. The 6- to 7-year-old children discriminated against the Black target in both conditions whereas older children discriminated against the Black child only when the anti-racist norm was not salient. In Study 2, 187 White children aged 6-7 and 9-10 years performed the same resource allocation task in conditions of explicit activation of similarity vs dissimilarity or egalitarian vs merit-based norms regarding race relations. Supporting the hypothesis of the role of racist or anti-racist norms on the expression of intergroup discrimination, results have again shown that 6- to 7-year-old children discriminated against the Black target in both conditions while older children presented significantly different prejudiced/nonprejudiced behaviours consistent with the activated norms. These results were discussed in terms of the need for a reanalysis of the assumptions and research results of the cognitive-developmental theory and of further developments in the socio-normative approach regarding the development of prejudice in childhood.     

Comparability of two-point high-point code patterns from original MMPI norms to MMPI-2 norms for the restandardization sample

Using records from the 1,138 males and 1,462 females in the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) restandardization sample (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989), two-point high-point code patterns generated from the original norms were compared to the patterns that these subjects obtained from the new norms. Although some code patterns proved to be quite stable across both norms, code comparability was generally lower in this community-based sample than was true for the records from samples of psychiatric patients also reported in Butcher et al. (1989). The sources of differences between the original and the new norms were reviewed, and the implications for profile interpretation based on code patterns were pointed out. The differences arising from the use of the MMPI-2 norms are appreciable; they highlight the need for new empirical data on the correlates of coding patterns based on these norms.     

A set of semantic norms for German and Italian

The psychological community frequently investigates semantic norms of properties produced by native speakers after being presented concept words, and these norms are of great value for a wide variety of psychological experiments. This paper presents a new set of norms that includes a collection of properties from a production experiment for the German and the Italian languages. Stimuli consisted of 50 concrete objects taken from 10 different concept classes. The data comprise annotations of semantic relation types and several statistical measures, which facilitate the comparison of the two target languages.