Data Collection in a Flat World: The Strengths and Weaknesses of Mechanical Turk Samples

Mechanical Turk (MTurk), an online labor system run by Amazon.com, provides quick, easy, and inexpensive access to online research participants. As use of MTurk has grown, so have questions from behavioral researchers about its participants, reliability, and low compensation. In this article, we review recent research about MTurk and compare MTurk participants with community and student samples on a set of personality dimensions and classic decision‐making biases. Across two studies, we find many similarities between MTurk participants and traditional samples, but we also find important differences. For instance, MTurk participants are less likely to pay attention to experimental materials, reducing statistical power. They are more likely to use the Internet to find answers, even with no incentive for correct responses. MTurk participants have attitudes about money that are different from a community sample's attitudes but similar to students' attitudes. Finally, MTurk participants are less extraverted and have lower self‐esteem than other participants, presenting challenges for some research domains. Despite these differences, MTurk participants produce reliable results consistent with standard decision‐making biases: they are present biased, risk‐averse for gains, risk‐seeking for losses, show delay/expedite asymmetries, and show the certainty effect—with almost no significant differences in effect sizes from other samples. We conclude that MTurk offers a highly valuable opportunity for data collection and recommend that researchers using MTurk (1) include screening questions that gauge attention and language comprehension; (2) avoid questions with factual answers; and (3) consider how individual differences in financial and social domains may influence results. Copyright © 2012 John Wiley & Sons, Ltd.     

Character Strengths: Stability and change from a military college education

A service academy military education seeks to develop character. Twenty-four character strengths measured by the Values in Action – Individual Strengths instrument are examined longitudinally to determine effects on character from a service academy education. Significant growth equations were obtained for Citizenship-Teamwork, Hope-Optimism, Fairness, Capacity to Love and be Loved, Appreciation of Beauty, Industry-Perseverance, Zest-Enthusiasm, Spirituality, Creativity, Leadership, Humor-Playfulness, Judgment, and Love of Learning from a mixed models analysis (overall Type 1 error set at .05). Based upon differences expected from a cross-sectional national sample it was inferred that the military college education differentially affected all character strengths except Gratitude, Citizenship-Teamwork, Creativity, Hope-Optimism, Capacity to Love and be Loved, Humor-Playfulness, Perspective, Modesty-Humility, Appreciation of Beauty, and Judgment.     

Toward Training Transformation

This article briefly discusses common themes concerning training transformation shared by articles in this special issue. Themes include: the continuing combat training center revolution, the interplay between theory and practice in military training programs, training as an engineering discipline, the role of training as a contributor to human performance in military operations, the use of training simulations to assess human performance, and data standards. The article emphasizes the need for researchers to better explain and articulate the value and return on investment of training in general and training transformation in particular. Meeting this need will better inform decisions and decision makers in allocating resources to training across the full range of defense requirements.     

Using a Values‐Based Taxonomy in Counselor Education

Values such as respect for the client's privacy and respect for diversity are integral to training new counselors. This requires learning in the affective domain, defined as the internalization of values ( Krathwohl, Bloom, & Masia, 1964 ). The affective taxonomy of Krathwohl et al. (1964) is presented as a means of teaching values and measuring the internalization of values. An example from a multicultural counseling course is offered as a template for implications in counselor education.     

The impact of career websites: what's the evidence?

ABSTRACT

Careers provision for young people in the UK is being re-formulated on the basis of a central role for career websites but this policy is based on unproven assumptions about their value. In this article we consider the use and impact of the two main career websites in Scotland on pupils' career management skills. We found that pupils at risk of not achieving positive post-school destinations were less likely to use the websites, as were minority ethnic pupils. Although similar in functions, the two websites differed in their effect: one had no impact while the other impacted on only one aspect of pupils' career management skills. Careers policy needs to be informed by more extensive research on career websites.     

Reducing demand for ineffective health remedies: overcoming the illusion of causality

Abstract

Objective: We tested a novel intervention for reducing demand for ineffective health remedies. The intervention aimed to empower participants to overcome the illusion of causality, which otherwise drives erroneous perceptions regarding remedy efficacy.

Design: A laboratory experiment adopted a between-participants design with six conditions that varied the amount of information available to participants (N?=?245). The control condition received a basic refutation of multivitamin efficacy, whereas the principal intervention condition received a full contingency table specifying the number of people reporting a benefit vs. no benefit from both the product and placebo, plus an alternate causal explanation for inefficacy over placebo.

Main outcome measures: We measured participants’ willingness to pay (WTP) for multivitamin products using two incentivized experimental auctions. General attitudes towards health supplements were assessed as a moderator of WTP. We tested generalisation using ratings of the importance of clinical-trial results for making future health purchases.

Results: Our principal intervention significantly reduced participants’ WTP for multivitamins (by 23%) and increased their recognition of the importance of clinical-trial results.

Conclusion: We found evidence that communicating a simplified full-contingency table and an alternate causal explanation may help reduce demand for ineffective health remedies by countering the illusion of causality.     

South African Educators' Voice on the Potential Role of Traditional Healers (THs) in Education Support Services

The aim of this qualitative research was to explore and describe perceptions of the potential role of traditional healers in education support services. Participants were 42 teachers (males = 13, females = 29) from a school district in the Eastern Cape province of South Africa. Interview and questionnaire data were collected and analysed, using a phenomenological method. Two main themes emerged from the analysis for the role of THs as education support providers: as 1) representatives of African culture and 2) contributors towards harmonising diversity in schools. The recommendations made were in line with the findings.     

Sex and the Right to HIV/AIDS Education in Early Childhood

This study explored how early childhood teachers (n=5) and young children (n=174) (age range 7 to 8 years; males = 81; females = 93) in two primary schools constructed and interpreted the right to HIV/AIDS education. Data were captured using individual interviews with teachers and group interviews with young children. Analysis of the data showed that teachers viewed young children's right to health information positively but did not consider the right to sexual information. Teachers operated within discourses which upheld the image of the child as innocent requiring protection from sexual knowledge. Children's perceptions of their rights to knowledge of sex in HIV/AIDS education showed ambiguity. Some accepted the right to know whilst others felt that knowledge about HIV/AIDS was inconsistent with childhood innocence.     

Change in physical self-perceptions across the transition to secondary school: Relationships with perceived teacher-emphasised achievement goals in physical education

ObjectivesThis study aimed to examine the effects of change in perceived teacher achievement goal emphasis in physical education (PE) on physical self-perceptions and self-esteem across the transition to secondary school.Design & methodsA longitudinal design was adopted with three time points, one at the end of primary school and two during the first year of secondary school. Participants (N = 491) were cross-classified by primary (N = 42) and secondary (N = 46) PE class in order to examine the association between perceived class-level teacher-emphasised goals and within-class student goals with self-beliefs.ResultsPersonal approach goals and class perceptions of teacher mastery approach goal promotion were all positively associated with ratings of co-ordination, sport competence, flexibility, and endurance in primary school. More favourable perceptions of coordination, sport competence, strength, flexibility, and endurance during the first year of secondary school were predicted by an increase in performance approach goal emphasis, whereas ratings of sport competence and flexibility were negatively associated with an increase in mastery approach goal emphasis.ConclusionsAlthough not entirely consonant with theoretical predictions, current findings suggest that teacher-emphasised performance approach goals in PE can promote development of several physical self-perceptions in the initial year of secondary school.