Data Collection in a Flat World: The Strengths and Weaknesses of Mechanical Turk Samples

Mechanical Turk (MTurk), an online labor system run by Amazon.com, provides quick, easy, and inexpensive access to online research participants. As use of MTurk has grown, so have questions from behavioral researchers about its participants, reliability, and low compensation. In this article, we review recent research about MTurk and compare MTurk participants with community and student samples on a set of personality dimensions and classic decision‐making biases. Across two studies, we find many similarities between MTurk participants and traditional samples, but we also find important differences. For instance, MTurk participants are less likely to pay attention to experimental materials, reducing statistical power. They are more likely to use the Internet to find answers, even with no incentive for correct responses. MTurk participants have attitudes about money that are different from a community sample's attitudes but similar to students' attitudes. Finally, MTurk participants are less extraverted and have lower self‐esteem than other participants, presenting challenges for some research domains. Despite these differences, MTurk participants produce reliable results consistent with standard decision‐making biases: they are present biased, risk‐averse for gains, risk‐seeking for losses, show delay/expedite asymmetries, and show the certainty effect—with almost no significant differences in effect sizes from other samples. We conclude that MTurk offers a highly valuable opportunity for data collection and recommend that researchers using MTurk (1) include screening questions that gauge attention and language comprehension; (2) avoid questions with factual answers; and (3) consider how individual differences in financial and social domains may influence results. Copyright © 2012 John Wiley & Sons, Ltd.     

Faded Versus Increasing Feedback,Task Variability Trajectories,and Transfer of Training

We assessed the merit of the advice given in the literature to fade feedback over the course of training because it was not based on solid research evidence. In a transfer experiment, we compared the effects of faded versus increasing feedback-provided guidance over the course of training on performers' task variability patterns and the subsequent transfer of effective responses to different task conditions. We found that faded feedback led to a rapid change in the percentage of favorable (vs. unfavorable) task conditions experienced over the course of training, which facilitated the transfer of effective responses to favorable task conditions. Alternatively, increasing feedback-provided guidance resulted in a gradual increase in the percentage of favorable (vs. unfavorable) task conditions experienced over the course of training and more balanced exposure to different task conditions. This facilitated the transfer of effective responses to unfavorable task conditions and more equal transfer of effective responses to favorable and unfavorable task conditions. The results indicated that recommendations to fade feedback over training need to be qualified and have implications for the design of feedback interventions used in training and other developmental activities.     

Flexible Mediation Analysis in the Presence of Nonlinear Relations: Beyond the Mediation Formula

In the social sciences, mediation analysis has typically been formulated in the context of linear models using the Baron &; Kenny (1986) Baron, R. M. and Kenny, D. A. 1986. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic and statistical considerations. Journal of Personality and Social Psychology, 51: 1173–1182. [Crossref], [PubMed], [Web of Science ®] , [Google Scholar] approach. Extensions to nonlinear models have been considered but lack formal justification. By placing mediation analysis within the counterfactual framework of causal inference one can define causal mediation effects in a way that is not tied to a specific statistical model and identify them under certain no unmeasured confounding assumptions. Corresponding estimation procedures using parametric or nonparametric models, based on the so-called mediation formula, have recently been proposed in the psychological literature and made accessible through the R-package mediation. A number of limitations of the latter approach are discussed and a more flexible approach using natural effects models is proposed as an alternative. The latter builds on the same counterfactual framework but enables interpretable and parsimonious modeling of direct and mediated effects and facilitates tests of hypotheses that would otherwise be difficult or impossible to test. We illustrate the approach in a study of individuals who ended a romantic relationship and explore whether the effect of attachment anxiety during the relationship on unwanted pursuit behavior after the breakup is mediated by negative affect during the breakup.     

Psychotherapy as Moral Encounter: A Crisis of Modern Conscience

Psychotherapy can no longer be understood as a morally neutral or value-free practice. Many have argued that psychology is implicitly and inevitably laden with moral assumptions and has a significant impact on the moral and ethical fabric of its clients and surrounding society (Browning 1987; Doherty 1995; Dueck and Reimer 2003; Kelly and Strupp 1992; Rieff 1987, 2006). In this work, the authors highlight the profound identity crises within the field of psychology around the place of morality/ethics in its theories and practices. Chronicling some of the shifts between modern and postmodern theory will allow for a meaningful engagement with this issue. The postmodern shifts in the field have left clinicians ideologically hampered in their ability to attend to the moral dimensions of their clients’ lives. In conclusion, the authors posit a virtue ethics and consider avenues of holistically and meaningfully infusing clinical practice with moral sensibilities.     

The Value of Measuring Impact Alongside Symptoms in Children and Adolescents: A Longitudinal Assessment in a Community Sample

The impact that psychiatric symptoms have on the lives of young people is central to clinical practice and classification. However, there is relatively little research on impact and its association with symptoms. This paper examines how well impact can be measured and how it relates to psychiatric outcomes. On four separate occasions over 3 years, symptoms and impact were assessed in a UK epidemiological sample (n?=?4,479; 51.5 % boys) using the Strengths and Difficulties Questionnaire (SDQ) as reported by parents, youths and teachers. Disorders were ascertained using the Development and Well-Being Assessment. An impact scale made of items about distress and impairment demonstrated considerable internal consistency, cross-informant correlations, and longitudinal stability by all reporting sources. Impact at baseline was a strong predictor of psychiatric disorder 3 years later after accounting for psychiatric disorders and symptoms measured at baseline: odds ratio OR?=?2.10, 95 % Confidence Interval (CI) [1.50, 2.94] according to parent-rated impact and OR?=?1.71, CI [1.08, 2.72] according to teacher-rated impact. Changes in impact over time were predicted, but not fully accounted for, by symptoms measured at baseline. Impact can be reliably and easily measured across time, and it may be clinically useful as an independent predictor of future symptoms and psychiatric disorders. More studies are needed to understand inter-individual variation in the impact caused by equivalent symptoms.     

The experiential foundations of mathematical knowledge

A view of the sources of mathematical knowledge is sketched which emphasizes the close connections between mathematical and empirical knowledge. A platonistic interpretation of mathematical discourse is adopted throughout. Two skeptical views are discussed and rejected. One of these, due to Maturana, is supposed to be based on biological considerations. The other, due to Dummett, is derived from a Wittgensteinian position in the philosophy of language. The paper ends with an elaboration of Gödel's analogy between the mathematician and the physicist.     

Analytical Buddhism: The Two-Tiered Illusion of Self

In the contemporary debate on the nature of autonomy, it is generally taken for granted that autonomy is to be understood as rule by the self, with the debate centring on how that self is to be identified. Garnett [2013b Garnett, Michael 2013b. The Autonomous Life: A Pure Social View, Australasian Journal of Philosophy 91/4. [Google Scholar]] has recently proposed a theory that rejects the dominant understanding of autonomy as rule by the self and views it instead as resistance to rule by others. According to that theory, an autonomous agent is one whose various ‘autonomy traits’ render her sufficiently resistant to subjection to foreign wills. I argue that Garnett has not given us good reason to abandon the understanding of autonomy as rule by the self. His account is plausible only if it is understood to require for autonomy the possession of at least one trait with which the agent can be identified. And the best explanation of the necessity of some such trait(s) is offered, not by Garnett's, but by the dominant understanding of autonomy he rejects.     

Predictors of representational aggression in preschool children of low‐income urban African American adolescent mothers

Responses to five doll‐story stems thematically related to attachment experiences with the mother were videotaped in the home and used to evaluate child, maternal, and environmental predictors of representational aggression in 93 preschool children of African American women receiving public assistance who had become pregnant as teenagers. Significant correlations were found between representational aggression and child's gender (male), birth weight, maternal depressive affect, maternal educational attainment, recent employment, mother's historical residence with her own mother, and felt social support, accounting for 40% of the variance in representational aggression. A significant Felt Social Support × Gender interaction effect suggested that girls of mothers who perceive higher levels of felt social support are more likely to represent less aggression in their stories; felt social support was not associated with boys' representational aggression. A significant Felt Social Support × Employment interaction effect suggested that representational aggression is associated with lower levels of felt social support only among employed mothers. Findings suggest that different pathways exist for representational aggression in children of low‐income adolescent mothers, which nevertheless share predictors associated with poverty.     

On the space-time structure of human interlimb co-ordination

In three experiments we show, using behavioural measures of movement outcome, as well as movement trajectory information and resultant kinematic profiles, that there is a strong tendency for the limbs to be co-ordinated as a unitary structure even under conditions where the movements are of disparate difficulty. Environmental constraints (an obstacle placed in the path of one limb, but not in the other) are shown to modulate the space-time behaviour of both limbs (Experiment II). Our results obtain for symmetrical (Experiment I) as well as asymmetrical movements that involve non-homologous muscle groups (Experiment III). These findings suggest that in multi-joint limb movements, the many degrees of freedom are organised to function temporarily as a single coherent unit that is uniquely specific to the task demands placed on it. For movements in general, and two-handed movements in particular, such units are revealed in a partitioning of the relevant force demands for each component (a force scaling characteristic) and a preservation of the internal “topology” of the action, as indexed by the relative timing among components. These features, as well as systematic deviations from perfect synchrony between the limbs can be rationalised by a model that assumes the limbs behave qualitatively like non-linear oscillators.