INTERRATER CORRELATIONS DO NOT ESTIMATE THE RELIABILITY OF JOB PERFORMANCE RATINGS

Interrater correlations are widely interpreted as estimates of the reliability of supervisory performance ratings, and are frequently used to correct the correlations between ratings and other measures (e.g., test scores) for attenuation. These interrater correlations do provide some useful information, but they are not reliability coefficients. There is clear evidence of systematic rater effects in performance appraisal, and variance associated with raters is not a source of random measurement error. We use generalizability theory to show why rater variance is not properly interpreted as measurement error, and show how such systematic rater effects can influence both reliability estimates and validity coefficients. We show conditions under which interrater correlations can either overestimate or underestimate reliability coefficients, and discuss reasons other than random measurement error for low interrater correlations.     

Recovering the Lost: Remeasuring U.S. Religious Affiliation

Over the past several decades, survey research has found a growing percentage of Americans claiming no religious affiliation. In this article, we introduce a modified religious traditions (RELTRAD) typology to measure religious affiliation. The approach benefits from a more detailed data collection and coding scheme of religious tradition based upon religious family, denomination, and congregation. Using new national survey data from the Baylor Religion Survey, we find: (1) improvement to survey design and measurement makes it possible to accurately locate more Americans within established religious traditions; (2) Americans remain connected to congregations, but less so to denominations or more generic religious identity labels; and (3) religious adherents are considerably more evangelical than prior studies have found. Finally, we consider how affiliation as a form of religious belonging relates to religious beliefs and behaviors.     

LEARNING WITHIN A LEARNER CONTROL TRAINING ENVIRONMENT: THE INTERACTIVE EFFECTS OF GOAL ORIENTATION AND METACOGNITIVE INSTRUCTION ON LEARNING OUTCOMES

This study proposes that effective engagement in metacognitive activities is critical to success in high learner-control training environments. A model was developed that focuses on the interactive effects of goal orientation and a metacognitive training intervention on metacognitive activity and learning outcomes. Seventy-nine participants completed a Web-based training course on how to create Web pages. Consistent with expectations, learners reporting greater levels of metacognitive activity during training gained more declarative knowledge, performed better on a skill-based measure, and had higher levels of self-efficacy. Metacognitive activity was also found to partially mediate the relationship between goal orientation and learning outcomes. Finally, the expected goal orientation by intervention interaction was found. The metacognitive intervention was beneficial in enhancing metacognitive activity for trainees with low performance-avoidance orientations, but resulted in lower metacognitive activity among highly avoidant learners. Implications of these findings for research on self-regulation and training practice are discussed.     

MATCHING INDIVIDUALS TO OCCUPATIONS USING ABILITIES AND THE O*NET: ISSUES AND AN APPLICATION IN CAREER GUIDANCE

The Occupational Information Network (O*NET) is a modern computerized occupational database with the potential to be an important resource for numerous work-related applications. However, developing any O*NET-based application requires working through conceptual, methodological, and practical issues. This article discusses a set of major issues in the context of using the O*NET for person-occupation matching purposes, providing examples of how these issues were addressed in a systematic ability-based matching application developed for career guidance. Specifically, we (a) describe the O*NET and its potential for career guidance, (b) explain person-occupation fit and its positive consequences for individuals and organizations, (c) compare person-occupation fit with person-job fit, (d) discuss matching individuals to occupations empirically using abilities, (e) highlight issues faced in using the O*NJST in this process and give examples of how these issues were addressed in our matching application, and (f) evaluate the functioning of this example matching method.     

Ignoring God Triune? The Doctrine of the Trinity in Dutch Theology

This article presents an overview of Dutch trinitarian theology in the twentieth and twenty-first centuries. It will become clear how and why leading Dutch theologians either ignored the doctrine, or used it for their own purposes, or uttered critical reservations regarding its sense and significance. After having reviewed the ways in which this majority of Dutch theologians has dealt with the doctrine, some notable constructive exceptions are explored. In conclusion, the authors raise the question as to Low Dutch theology might, starting from its own tradition, catch up with the recent international and inter-denominational tendency towards a well-developed trinitarian theology.     

POTENTIAL EFFECTS OF BANDING AS A FUNCTION OF TEST RELIABILITY

Cascio, Outtz, Zedeck, and Goldstein (1991) described the application of a number of test score banding procedures in personnel selection. Equations are developed illustrating the relationship between the width of test score bands and test reliability. When reliability is moderate to low, bands are likely to be larger than the standard deviation of the test, and are likely to include a large proportion of the applicant pool. The relationships between band widths and the differences between higher scoring and lower scoring groups are also examined. When the band is smaller than the differences between groups (which may happen when highly reliable tests are used), banding may not by itself prove effective as a means of reducing the adverse impact of tests, even when banding systems that maximize opportunities for members of the lower scoring group are used.     

Perceived Family Criticism and Primary Care Utilization: Psychosocial and Biomedical Pathways

We explored the relationship of perceived family criticism to subsequent healthcare utilization in patients attending a family medicine center. We examined: a) the relationship of perceived criticism to subsequent utilization for biomedical and psychosocial/somatic problems; b) the mediating effects of self-rated mental health and physical function; and c) the mediating effects of social support. The analyses were adjusted for age, sex, race, education, health insurance, and marital status. Higher perceived criticism predicted more psychosocial/somatic and biomedical visits. The relationship of perceived criticism with psychosocial/somatic visits was entirely mediated through self-rated mental health. The relationship of perceived criticism with biomedical visits was partly mediated through self-rated physical function and, in part, independent. Social support played no role in explaining these relationships. Further research is needed to determine whether lowering perceived family criticism lowers primary care utilization.