What sources contribute to variance in observer ratings? Using generalizability theory to assess construct validity of psychological measures

Cronbach and Meehl (Psychol. Bull. 1955; 52:281–302) stated that the key question to be addressed when assessing construct validity is ‘What sources contribute to variance in test performance?’ We illustrate the utility of generalizability theory (GT) as a conceptual framework that encourages psychological researchers to address this question and as a flexible set of analytic tools that can provide answers to inform both substantive theory and measurement practice. To illustrate these capabilities, we analyze observer ratings of 27 caregiver–child dyads, focusing on the importance of situational (contextual) factors as sources of variance in observer ratings of caregiver–child behaviors. Cross‐situational consistency was relatively low for the categories of behavior analyzed, indicating that dyads vary greatly in their interactional patterns from one situation to the next, so that it is difficult to predict behavioral frequencies in one context from behaviors observed in a different context. Our findings suggest that single‐situation behavioral measures may have limited generalizability, either to behavior in other contexts or as measures of global interaction tendencies. We discuss the implications of these findings for research and measurement design in developmental psychology. Copyright © 2008 John Wiley & Sons, Ltd.     

Self-Other Perspective and Sexual Attitudes Affect Estimates of Sexual Risk

This study evaluated the effects of self-other perspective, sexual attitudes, and a sexual victimization history on women’s estimates of sexual risk for a set of vignettes describing situations that they may face when interacting socially with men. One hundred and twenty-four women from a medium sized southwestern university in the US were assigned to two different conditions (imagine self vs. an anonymous woman) and asked to rate how risky the situations described were in terms of having an unwanted sexual experience. Results revealed that participants asked to imagine an anonymous woman estimated sexual risk as higher than participants asked to imagine themselves, and that participants reporting liberal sexual attitudes estimated sexual risk as lower than participants reporting more conservative attitudes.     

The self-protective and undermining effects of attributional ambiguity

Two experiments assessed the self-protective and undermining effects of attributional ambiguity. Both studies utilized immersive virtual environment technology to achieve otherwise difficult manipulations of stigma. In Experiment 1, White and Latino participants were either stigmatized (represented as Latino) or not (represented as White) and given negative leadership performance feedback. Afterwards, stigmatized participants reported higher well-being and attributed negative feedback more to discrimination than nonstigmatized participants. In Experiment 2, Latinos represented veridically showed self-protective effects after receiving negative leadership feedback. Additionally, this experiment revealed undermining effects of attributional ambiguity such that those participants represented as Latino discounted positive feedback and reported lower well-being. Thus, attributional ambiguity of stigmatized individuals (real or induced) buffered well-being in the face of negative feedback but undermined the well-being effects of positive feedback.     

An additional antecedent of empathic concern: valuing the welfare of the person in need

Two experiments examined the role of valuing the welfare of a person in need as an antecedent of empathic concern. Specifically, these experiments explored the relation of such valuing to a well-known antecedent--perspective taking. In Experiment 1, both perspective taking and valuing were manipulated, and each independently increased empathic concern, which, in turn, increased helping behavior. In Experiment 2, only valuing was manipulated. Manipulated valuing increased measured perspective taking and, in part as a result, increased empathic concern, which, in turn, increased helping. Valuing appears to be an important, largely overlooked, situational antecedent of feeling empathy for a person in need.     

Weight beliefs and messages: Mindsets predict body‐shame and anti‐fat attitudes via attributions

In two samples (N = 247, N = 291), we examined the link between beliefs and messages about the changeable (incremental theory) vs. fixed (entity theory) nature of weight, attributions for weight, and body shame. We recruited participants using online sampling, employing a correlational design in Study 1 and an experimental design in Study 2. Across both studies, we found evidence for the stigma‐asymmetry effect—incremental, relative to entity beliefs/messages of weight predicted both (a) stronger onset responsibility attributions, indirectly increasing body shame and (b) stronger offset efficacy attributions, indirectly decreasing body shame. Study 2 replicated the stigma‐asymmetry effect with anti‐fat attitudes. We discuss implications for public health obesity messages with the goal of reducing stigma.     

Satisfaction with Body Image and Peer Relationships for Males and Females in a College Environment

The purpose of this study was to examine body image and relationship satisfaction in male and female college students. A total of 101 males and 187 females, primarily Caucasian and middle class, completed a questionnaire designed to measure satisfaction with specific body parts, overall body characteristics, and relationship status. These variables were examined in light of gender and body size differences. Results indicated that women, especially those under or above average weight, were more dissatisfied with their appearance than were men, whereas men were more dissatisfied with their relationships and sex lives than were women. In addition, those body parts with which each gender was most dissatisfied were consistent with the body parts emphasized through ideal images. These results are discussed in relation to gender role dynamics and societal pressures.     

Natural Disaster and Depression: A Prospective Investigation of Reactions to the 1993 Midwest Floods

A statewide sample of 1735 Iowa residents, approximately half of whom were victims of the 1993 Midwest Floods, participated in interviews 1 year prior to, and 30 to 90 days after, the disaster. Employing a rigorous methodology including both control-group comparisons and predisaster assessments, we performed a systematic evaluation of the disaster's impact. Overall, the disaster led to true but small rises in depressive symptoms and diagnoses 60–90 days postflood. The disaster–psychopathology effect was not moderated by predisaster depressive symptoms or diagnostically defined depression; rather, predisaster symptoms and diagnoses uniquely contributed to increases in postdisaster distress. However, increases in symptoms as a function of flood impact were slightly greater among respondents with the lowest incomes and among residents living in small rural communities, as opposed to on farms or in cities. Implications for individual- and community-level disaster response are discussed.     

Mindsets of health and healthy eating intentions

Across two studies, we examined the relation between mindsets of health, expectancy‐value, and eating intentions. We also explored if relations are stronger for African Americans compared to White Americans. In Study 1, we conducted a correlational study (N = 158) to examine initial relations among constructs. In Study 2, we employed an experimental design (N = 205), and randomly assigned participants to either a growth mindset or a fixed mindset of health condition. In both studies, we measured participants’ mindsets of health, expectancy‐value beliefs, healthy eating intentions, past eating habits, and demographics. In Study 1, stronger growth mindsets of health predicted healthier eating intentions. Expectancy‐value beliefs, namely, the extent to which individuals value healthy eating habits and expect to be able to manage their eating, mediated this relation. In Study 2, we successfully manipulated mindsets of health and individuals in the growth mindset condition reported healthier eating intentions, compared to those in the fixed mindset condition. Expectancy‐value beliefs again mediated this link. Race only moderated the relation in Study 1, such that effects of growth mindsets on outcomes (i.e., eating intentions and expectancy‐value beliefs) are stronger for African Americans compared to White Americans. Study 1 provided initial evidence of a relationship between stronger growth mindsets of health and healthier beliefs and intentions. Study 2 offered experimental evidence. We discuss theoretical and practical implications.