Using the conditional grade-of-membership model to assess judgment accuracy

Consider the case whereJ instruments are used to classify each ofI objects relative toK nominal categories. The conditional grade-of-membership (GoM) model provides a method of estimating the classification probabilities of each instrument (or judge) when the objects being classified consist of both pure types that lie exclusively in one ofK nominal categories, and mixtures that lie in more than one category. Classification probabilities are identifiable whenever the sample of GoM vectors includes pure types from each category. When additional, relatively mild, assumptions are made about judgment accuracy, the identifiable correct classification probabilities are the greatest lower bounds among all solutions that might correspond to the observed multinomial process, even when the unobserved GoM vectors do not include pure types from each category. Estimation using the conditional GoM model is illustrated on a simulated data set. Further simulations show that the estimates of the classification probabilities are relatively accurate, even when the sample contains only a small percentage of approximately pure objects.The authors thank Max A. Woodbury, Kenneth G. Manton and H. Dennis Tolley for their help and four anonymous Psychometrika reviewers (including an associate editor) for their beneficial expository and technical suggestions. This work was supported by the Dean's Fund for Summer Research, Owen Graduate School of Management, Vanderbilt University.     

Effects of escalated exposure to information on accuracy of personality judgment

In an investigation of the effects of increasing exposure to information on accuracy of personality trait judgments, judges rated a single target on three successive occasions, each of which involved the introduction of a novel piece of personal information. Varying the order in which types of information were presented allowed us to jointly examine the effects of information quality and quantity on accuracy in personality judgment. We found that (a) in general, more information led to higher accuracy, (b) some types of information led to greater gains in distinctive or normative accuracy, and (c) introduction of lower quality information did not negatively affect accuracy.     

The effect of divided attention on global judgment of learning accuracy

This study examined the effect of divided attention (DA) on global judgment of learning (JOL) accuracy in a multitrial list learning paradigm. A word monitoring task was used to divide attention. Participants were assigned to an attention condition (DA at encoding, DA at judgment, DA at retrieval, or focused attention) and completed 4 learning trials, each comprising a study, judgment, and recall phase. Participants showed greater overconfidence in the DA at encoding (Trial 2) and DA at retrieval (Trials 1 and 2) conditions than in the focused attention condition. DA atjudgment did not affect JOL accuracy, and there was no effect of DA in Trials 3 and 4 on JOL accuracy across all attention conditions. Results indicate that participants consider conditions of encoding and retrieval but do not engage in recall when forming global JOLs. These findings suggest that people rely on extrinsic cues when making repeated, global metamemoryjudgments.     

Use of situational judgment tests to predict job performance: a clarification of the literature

Although situational judgment tests have a long history in the psychological assessment literature and continue to be frequently used in employment contexts, there has been virtually no summarization of this literature. The purpose of this article is to review the history of such tests and present the results of a meta-analysis on criterion-related and construct validity. On the basis of 102 coefficients and 10,640 people, situational judgment tests showed useful levels of validity (rho = .34) that were generalizable. A review of 79 correlations between situational judgment tests and general cognitive ability involving 16,984 people indicated that situational judgment tests typically evidence relationships with cognitive ability (rho = .46). On the basis of the literature review and meta-analytic findings, implications for the continued use of situational judgment tests are discussed, particularly in terms of recent investigations into tacit knowledge.     

At the doorstep to employment: Discrimination against immigrants as a function of applicant ethnicity,job type,and raters' prejudice

This study examined the impact of applicant ethnicity, job type, and prejudice on evaluation biases and intentions to interview in an experimental simulation. We suggest that bias and discrimination are more likely when foreign applicants who belong to disliked ethnic groups apply for jobs that require high interpersonal skills, and when raters are prejudiced against immigrants. Subjects were Swiss university students who evaluated Swiss, Spanish, and Kosovo Albanian fictitious applicants. Foreign applicants were second‐generation immigrants, i.e., Swiss‐born descendants of immigrants. Thus, all applicants had similar schooling and language proficiencies but differed with respect to ethnicity. As predicted, discrimination was only observed for members of the disliked ethnic group (Kosovo Albanian) and not for members of the well‐accepted group (Spanish). Moreover, this discrimination was only apparent when applying for a job requiring high interpersonal skills and not when applying for a job requiring high technical skills. Symbolic prejudice towards second‐generation immigrants interacted with applicant ethnicity and job type to affect evaluations of foreign applicants: Persons high in symbolic prejudice devalued foreign applicants belonging to the disliked group but only when applying for a job requiring high interpersonal skills. Overt prejudice was unrelated to evaluations and intentions to interview. These results suggest that discrimination against immigrants is highly specific, targeting only members of certain ethnic groups who apply for certain types of jobs. Moreover, evaluation biases may be more apparent in raters who are prejudiced. Thus, our results support the notion that discrimination for employment results from a complex interaction between characteristics of the applicant, the job, and the rater.     

When colors backfire: The impact of color cues on moral judgment

This article investigates if and how the valence of color cues affects moral acceptability of (un)desirable consumer behaviors. Study 1 uses colors with definite differences in terms of valence, namely, red and green. Study 2 applies an evaluative conditioning paradigm to endow initially neutral colors with negative versus positive valences. We find an ironic color effect: undesirable behaviors become more acceptable when presented with negatively valenced colors. In general, respondents find (un)desirable behaviors more acceptable when a background color is of the same valence rather than neutral or opposite in valence. Implications for promotion and prevention campaigns are discussed.     

The impact of state affect on job satisfaction

Affective events theory proposes affective experiences at work to cause job satisfaction. Using multiple measurements obtained in a diary study, affective experiences in terms of state positive and state negative affect (PA, NA) were related to state job satisfaction (N = 91). Trait measures were also collected. Results confirmed our hypothesis. First, aggregated state job satisfaction is strongly correlated with trait job satisfaction. Second, the relationship between state affect and state job satisfaction is not spurious: State affect impacts on state job satisfaction even if trait affect and trait job satisfaction are controlled. Third, the effect of state affect on job satisfaction measured by the faces scale is stronger than the effect of state affect on job satisfaction measured by facet scales. It is concluded that not all scales to measure job satisfaction are equally appropriate to investigate affect at work.     

The effects of domain knowledge on metacomprehension accuracy

In the present research, we examined the relationship between readers’ domain knowledge and their ability to judge their comprehension of novel domain-related material. Participants with varying degrees of baseball knowledge read five texts on baseball-related topics and five texts on non-baseball-related topics, predicted their performance, and completed tests for each text. Baseball knowledge was positively related to absolute accuracy within the baseball domain but was unrelated to relative accuracy within the baseball domain. Also, the readers showed a general underconfidence bias, but the bias was less extreme for higher knowledge readers. The results challenge common assumptions that experts’ metacognitive judgments are less accurate than novices’. Results involving topic familiarity ratings and a no-reading control group suggest that higher knowledge readers are not more likely to ignore text-specific cues in favor of a domain familiarity heuristic, but they do appear to make more effective use of domain familiarity in predicting absolute performance levels.     

The impact of life events on job satisfaction

Employing fixed effects regression techniques on longitudinal data, we investigate how life events affect employees' job satisfaction. Unlike previous work–life research, exploring mostly contemporaneous correlations, we look for evidence of adaptation in the years following major life events. We find evidence of adaptation following the first marriage event, but we find that the birth of the first child has a long-lasting impact on employees' job satisfaction. Our findings also suggest that there is a general boost in job satisfaction prior to first marriage and to a lesser extent prior to the birth of the first child, consistent with evidence of anticipation. Accordingly, our study provides some of the first evidence on the dynamic effect of non-work related factors on job satisfaction and it introduces a novel methodology and a new perspective for investigating the dynamic interaction between the work and life domains.     

Analyzing the accuracy of probability judgments for multiple events: An extension of the covariance decomposition

The probability score (PS) can be used to measure the overall accuracy of probability judgments for a single event, e.g., “Rain falls,” or “This patient has cancer.” It has been shown previously how a “covariance decomposition” of the mean of PS over many occasions indexes several distinct aspects of judgment performance (J. F. Yates, Organizational Behavior and Human Performance, 30, 132–156 (1982)). There are many situations in which probability judgments are reported for sample space partitions containing more than one event and its complement, e.g., medical situations in which a patient might suffer from Disease X, Disease Y, or Disease Z, or testing situations in which the correct answer to an item might be any one of alternatives (a) through (e). The probability score for multiple events (PSM) serves as a measure of the overall accuracy of probability judgments for the events in partitions of any size. The present article describes and interprets an extension of the covariance decomposition to the mean of PSM. The decomposition is illustrated with data from two contexts, medicine and education.     

The dimensional salience solution to the expectancy-value muddle: An extension

The theory of reasoned action (TRA) specifies a set of expectancy-value, belief-based frameworks that underpin attitude (behavioural beliefs × outcome evaluations) and subjective norm (normative beliefs × motivation to comply). Unfortunately, the most common method for analysing these frameworks generates statistically uninterpretable findings, resulting in what has been termed the ‘expectancy-value muddle’. Recently, however, a dimensional salience approach was found to resolve this muddle for the belief-based framework underpinning attitude. An online survey of 262 participants was therefore conducted to determine whether the dimensional salience approach could also be applied to the belief-based framework underpinning subjective norm. Results revealed that motivations to comply were greater for salient, as opposed to non-salient, social referents. The belief-based framework underpinning subjective norm was therefore represented by evaluating normative belief ratings for salient social referents. This modified framework was found to predict subjective norm, although predictions were greater when participants were forced to select five salient social referents rather than being free to select any number of social referents. These findings validate the use of the dimensional salience approach for examining the belief-based frameworks underpinning subjective norm. As such, this approach provides a complete solution to addressing the expectancy-value muddle in the TRA.     

Professional vs. amateur judgment accuracy: The case of foreign exchange rates

Highly knowledgeable people often fail to achieve highly accurate judgments, a phenomenon sometimes called the “process-performance paradox.” The present research tested for this paradox in foreign exchange (FX) rate forecasting. Forty professional and 57 sophisticated amateur forecasters made one-day and one-week-ahead FX predictions in deterministic and probabilistic formats. Among the conclusions indicated by the results are: (a) professional accuracy usually surpasses amateur accuracy, although many amateurs outperform many professionals; (b) professionals appear to achieve high proficiency via heavy reliance on predictive information (unlike what has been observed before, e.g., for stock prices); (c) forecast format strongly affects judgment accuracy and processes; and (d) apparent overconfidence can transform itself into underconfidence depending on when and how forecasters must articulate their confidence.     

Self‐confirmation strivings in depression: An extension to the affective domain using an experimental design

The purpose of the present study was to compare the emotional reactions of depressed and nondepressed individuals to experiences of romantic rejection versus acceptance. We tested our hypotheses in a sample of 28 depressed and 43 nondepressed undergraduate students. In support of self‐consistency theories, the results showed that depressed individuals reported significantly greater negative mood in the romantic acceptance versus rejection condition, while there was no significant difference across the two conditions in the self‐reported mood of nondepressed individuals. Further, symptoms of anxiety mediated the interactive effects of depression status and rejection status on mood. Our findings demonstrate how the responses of depressed individuals to interpersonal feedback contribute to their affective disturbance. Copyright © 2009 John Wiley & Sons, Ltd.     

Judging the accuracy of a likelihood judgment: the case of smoking risk

A standard method for assessing whether people have appropriate internal representations of an event's likelihood is to check whether their subjective probability or frequency estimates for the event correspond with the assumed objective value for that event. When a person's estimate for the event exceeds its assumed objective probability or frequency, the person's expectancy for the event is concluded to be greater than warranted. This paper describes three lines of reasoning as to why conclusions of this sort can be problematic. Recently published findings as well as data from two new experiments are described to support this main thesis. The case of smoking risk is used to illustrate the more general problem, and issues that must be considered to avoid or contend with the problem are discussed. Copyright © 2002 John Wiley & Sons, Ltd.     

Matrescence: lifetime impact of motherhood on cognition and the brain

Profound environmental, hormonal, and neurobiological changes mark the transition to motherhood as a major biosocial life event. Despite the ubiquity of motherhood, the enduring impact of caregiving on cognition and the brain across the lifespan is not well characterized and represents a unique window of opportunity to investigate human neural and cognitive development. By integrating insights from the human and animal maternal brain literatures with theories of cognitive ageing, we outline a framework for understanding maternal neural and cognitive changes across the lifespan. We suggest that the increased cognitive load of motherhood provides an initial challenge during the peripartum period, requiring continuous adaptation; yet when these demands are sustained across the lifespan, they result in increased late-life cognitive reserve.     

Assessing the impact of organizational communication on job satisfaction and job performance

The present paper analyses the impact of organizational communication on job satisfaction and job performance. Data were collected from 380 employees working at different managerial levels in various organizations in India by using Organizational Communication Scale (Roberts & O’Reilly, 1974), Job Satisfaction Survey scale (Spector, 1985), and Job Performance scale (Rodwell, Kienzle & Shadur, 1998). It was found that organizational communication had a significant effect on job satisfaction and job performance of the employees. The analysis further indicated that the employees at different levels perceived job satisfaction differently. Thus, it can be inferred that in Indian organizations, job satisfaction and performance are very much dependent on the communication behavior of the organization.     

The impact of information from similar or different advisors on judgment

People rely on others’ advice to make judgments on a daily basis. In three studies, we examine the differential impacts of similarity between the source of that advice and the person making the judgment in two settings: judging others’ behavior and judging one’s own actions. We find that similarity interacts with the target of the judgment. In particular, information received from a different advisor is more heavily weighed than from a similar advisor in judging others’ actions, but information from a similar advisor is more heavily weighed than from a different advisor in judging one’s own. We provide two potential explanations for this interaction, difficulty of the judgment and informativeness of the advice. Our analyses show a moderated mediating role of informativeness and difficulty in the relationship between the advisor’s similarity by judgment type interaction and advice use.     

An extension of item analysis procedures to the case of polychotomous response

Classical item analysis procedures were developed for dichotomously scored items and do not apply to items allowing multiple correct responses. Maximum likelihood procedures analogous to those employed in polychotomous bio-assay are presented which yield estimates of the sets of parameters for items having multiple nonordered responses. Expressions for the estimates of the asymptotic variances of the item parameters and on overall chi-square goodness of fit test are also provided.