Amazon Mechanical Turk in Organizational Psychology: An Evaluation and Practical Recommendations

Purpose

Amazon Mechanical Turk is an increasingly popular data source in the organizational psychology research community. This paper presents an evaluation of MTurk and provides a set of practical recommendations for researchers using MTurk.

Design/Methodology/Approach

We present an evaluation of methodological concerns related to the use of MTurk and potential threats to validity inferences. Based on our evaluation, we also provide a set of recommendations to strengthen validity inferences using MTurk samples.

Findings

Although MTurk samples can overcome some important validity concerns, there are other limitations researchers must consider in light of their research objectives. Researchers should carefully evaluate the appropriateness and quality of MTurk samples based on the different issues we discuss in our evaluation.

Implications

There is not a one-size-fits-all answer to whether MTurk is appropriate for a research study. The answer depends on the research questions and the data collection and analytic procedures adopted. The quality of the data is not defined by the data source per se, but rather the decisions researchers make during the stages of study design, data collection, and data analysis.

Originality/Value

The current paper extends the literature by evaluating MTurk in a more comprehensive manner than in prior reviews. Past review papers focused primarily on internal and external validity, with less attention paid to statistical conclusion and construct validity—which are equally important in making accurate inferences about research findings. This paper also provides a set of practical recommendations in addressing validity concerns when using MTurk.

     

Holistic processing predicts face recognition

The concept of holistic processing is a cornerstone of face-recognition research. In the study reported here, we demonstrated that holistic processing predicts face-recognition abilities on the Cambridge Face Memory Test and on a perceptual face-identification task. Our findings validate a large body of work that relies on the assumption that holistic processing is related to face recognition. These findings also reconcile the study of face recognition with the perceptual-expertise work it inspired; such work links holistic processing of objects with people's ability to individuate them. Our results differ from those of a recent study showing no link between holistic processing and face recognition. This discrepancy can be attributed to the use in prior research of a popular but flawed measure of holistic processing. Our findings salvage the central role of holistic processing in face recognition and cast doubt on a subset of the face-perception literature that relies on a problematic measure of holistic processing.     

Tracking Multiple Statistics: Simultaneous Learning of Object Names and Categories in English and Mandarin Speakers

Two experiments were conducted to examine adult learners' ability to extract multiple statistics in simultaneously presented visual and auditory input. Experiment 1 used a cross‐situational learning paradigm to test whether English speakers were able to use co‐occurrences to learn word‐to‐object mappings and concurrently form object categories based on the commonalities across training stimuli. Experiment 2 replicated the first experiment and further examined whether speakers of Mandarin, a language in which final syllables of object names are more predictive of category membership than English, were able to learn words and form object categories when trained with the same type of structures. The results indicate that both groups of learners successfully extracted multiple levels of co‐occurrence and used them to learn words and object categories simultaneously. However, marked individual differences in performance were also found, suggesting possible interference and competition in processing the two concurrent streams of regularities.     

Intergenerational transmission of literacy skills among Filipino families

We examined the joint role of parental word reading skills and conventional home literacy environment measures among 320 Filipino low‐ to middle‐income families in Cebu City, Philippines with children aged 5–8 years old. A ranking of parent‐reported ratings of their frequency of engaging in home literacy activities and adult literacy practices revealed that book‐related behaviors were less frequently practiced relative to other behaviors, and mean ratings on the home literacy resources scale suggested a relatively print‐poor environment. Nevertheless, scale items about book reading and direct literacy instruction at home correlated with child's language and literacy skills. Structural equation modeling showed that parent's education and frequency of engaging in home literacy activities uniquely accounted for variance in child's oral language and print knowledge skills. In a second model, parent's word reading skills were significantly related to child's skills, but did not eliminate or attenuate influences from parent's education and home literacy activities. Results are important in relation to theories on the intergenerational transmission of literacy skills and the generalizability of findings from developed countries to developing country contexts.     

Clinical utility of the Cross-Cultural (Chinese) Personality Assessment Inventory (CPAI-2) in the assessment of substance use disorders among Chinese men

This study examined the clinical utility of the Cross-Cultural (Chinese) Personality Inventory (CPAI-2) in differentiating the personality characteristics of Chinese men with substance use disorders from other psychiatric patients and normal control participants. The CPAI-2 profile of 121 Chinese men with substance use disorders was contrasted with that of a matched psychiatric comparison group (n = 172) and a normal comparison group (n = 187). Multivariate analyses of variance and logistic regression results supported the utility of the CPAI-2 clinical scales, especially Pathological Dependence, Antisocial Behavior, and Depression, for assessing substance use disorders. The Pathological Dependence scale (cutoff T score of 64) achieved good sensitivity and specificity. Apart from the universal personality traits related to neuroticism, conscientiousness, and agreeableness found in Western studies, the indigenously derived CPAI-2 personality scales, including Family Orientation and Harmony, highlighted deficits in social adjustment and interpersonal relationship as important cultural features in the personality characteristics of these participants. The study provided a cross-cultural extension to research on the relationship between personality and substance use disorders and could assist clinicians in considering culturally relevant treatment approaches.     

Constitutive activation of the G-protein subunit Galphas within forebrain neurons causes PKA-dependent alterations in fear conditioning and cortical Arc mRNA expression

Memory formation requires cAMP signaling; thus, this cascade has been of great interest in the search for cognitive enhancers. Given that medications are administered long-term, we determined the effects of chronically increasing cAMP synthesis in the brain by expressing a constitutively active isoform of the G-protein subunit Galphas (Galphas*) in postnatal forebrain neurons of mice. Previously, we showed that Galphas* mice exhibit increased adenylyl cyclase activity but decreased cAMP levels in cortex and hippocampus due to a PKA-dependent increase in total cAMP phosphodiesterase (PDE) activity. Here, we extend previous findings by determining if Galphas* mice show increased activity of specific PDE families that are regulated by PKA, if Galphas* mice show PKA-dependent deficits in fear memory, and if these memory deficits are associated with PKA-dependent alterations in neuronal activity as mapped by Arc mRNA expression. Consistent with previous findings, we show here that Galphas* mice exhibit a significant compensatory increase in cAMP PDE1 activity and a trend toward increased cAMP PDE4 activity. Further, inhibiting the presumably elevated PKA activity in Galphas* mice fully rescues short- and long-term memory deficits in a fear-conditioning task, while extending the training session from one to four CS-US pairings partially rescues these deficits. Mapping of Arc mRNA levels suggests these PKA-dependent memory deficits may be related to decreased neuronal activity specifically within the cortex. Galphas* mice show decreased Arc mRNA expression in CA1, orbital cortex, and cortical regions surrounding the hippocampus; however, only the deficits in cortical regions surrounding the hippocampus are PKA dependent. Our results imply that chronically stimulating targets upstream of cAMP may detrimentally affect cognition.     

The effects of service participation, friendship networks, and family support on developmental outcomes: a study of young people from low-income families in Hong Kong

This research investigates the factors conducive to the success of young people growing up in low-income families. Many studies carried out locally and overseas focus on the risks and difficulties experienced by these young people; however, little attempt has been made to examine the factors that help them change their lives from failure, poverty, and social exclusion. Based on a quantitative survey of 405 young people recruited from schools and integrated youth service centers in Hong Kong, this research identifies a range of indicators of, and necessary conditions for, the positive development of young people with economic disadvantage. It also evaluates the way in which the functions of service participation, friendship networks, and various family factors support them to thrive in different areas. The study helps fill the gap of the existing literature and draws implications for policy and practice to address the needs of these youths. Future directions of research are also discussed.     

Commentary: Reintegrating psychology into an expanded biopsychosocial model of severe acute respiratory syndrome

The Special Issue highlights the importance of psychology in the research agenda on severe acute respiratory syndrome (SARS). The five articles examined relevant theories on social cognition and coping in the ecological context of the SARS outbreak. They provide preliminary tests to current psychological principles of coping and health psychology in a real crisis. The limitations of the studies are recognized. Other areas of psychological research on SARS are suggested. It is proposed that the biopsychosocial model of behavioral medicine could be expanded as a paradigm in public health to study and prepare for future emerging epidemics.     

Forming Life Satisfaction among Different Social Groups during the Modernization of China

Quality of life has become a notable concern in the modernization of China. However, there are no adequate data to chart the formation of quality of life, in terms of life satisfaction, and in relation to the societal conditions of modernization. Moreover, it is still a question concerning how life satisfaction forms among people of different social groups, differentiated by class, education, age, and gender. To answer the questions, the present study involved a random sample survey of 732 inhabitants in Beijing. Results indicate the contributions of income, age, perceived modern economic conditions in society, and satisfaction with societal economic conditions. In contrast, education and the ownership class appear to predict lower life satisfaction. Furthermore, there are significant variations in the effects of the factors in predicting the life satisfaction of people in different social groups. Notably, perceived economic conditions appear to be less important to people of higher class positions. This revised version was published online in August 2006 with corrections to the Cover Date.     

The Proofs Of The Grundgedanke In Wittgenstein's Tractatus

The Tractatus contains twodifferent proofs of the Grundgedanke, or thenonreferentiality of logical constants. In thispaper, I explicate the first proof in TLP 5.4s andreconstruct the less explicitly stated second proof. My explication of the first proof shows it to beelegant but based on an invalid inference. In myreconstruction of the second proof, the main argumentis that the sign of a logical constant does not denotebecause it possesses the punctuation-mark-nature. Andit possesses the punctuation-mark-nature because,given the analyticity thesis in TLP 5, one canestablish for everyday language an adequate symbolismwith N as the sole fundamental operation such that itssign is a bar indicating merely the order and scope ofits application.