A Systematic Scoping Review of the Choice Architecture Movement: Toward Understanding When and Why Nudges Work

In this paper, we provide a domain‐general scoping review of the nudge movement by reviewing 422 choice architecture interventions in 156 empirical studies. We report the distribution of the studies across countries, years, domains, subdomains of applicability, intervention types, and the moderators associated with each intervention category to review the current state of the nudge movement. Furthermore, we highlight certain characteristics of the studies and experimental and reporting practices that can hinder the accumulation of evidence in the field. Specifically, we found that 74% of the studies were mainly motivated to assess the effectiveness of the interventions in one specific setting, while only 24% of the studies focused on the exploration of moderators or underlying processes. We also observed that only 7% of the studies applied power analysis, 2% used guidelines aiming to improve the quality of reporting, no study in our database was preregistered, and the used intervention nomenclatures were non‐exhaustive and often have overlapping categories. Building on our current observations and proposed solutions from other fields, we provide directly applicable recommendations for future research to support the evidence accumulation on why and when nudges work. Copyright © 2017 John Wiley & Sons, Ltd.     

Exploratory factor analysis in transportation research: Current practices and recommendations

Exploratory factor analysis (EFA) is a widely used statistical method in traffic and transportation research, particularly for the development and validation of measurement instruments. This article critically examines current practices in conducting and reporting EFA in published transportation studies. One hundred and eighty papers published between 2016 and 2018 were examined, of which eighty-two were included in the present study after applying eligibility criteria. The review suggests that the quality of EFA reported in the field is routinely poor: (a) researchers fail to provide sufficient information to be able to adequately assess the appropriateness and quality of both the input data and the reported output; and (b) the decisions underlying the choices of EFA methods are not justified and rely mostly on procedures advised against, particularly the Little-Jiffy approach. In summary, a significant gap between current practice and experts' recommendations exists. We provide some guidelines that may help in conducting, reporting and reviewing EFA in transportation research.     

Say something nice: A meta-analytic review of peer reporting interventions

Peer reporting interventions (i.e., Positive Peer Reporting and tootling) are commonly used peer-mediated interventions in schools. These interventions involve training students to make reports about peers' prosocial behaviors, whether in oral or written form. Although peer reporting interventions have been included in meta-analyses of group contingencies, this study is the first meta-analytic review of single-case research focusing exclusively on peer reporting interventions. The literature search and application of inclusion criteria yielded 21 studies examining the impact of a peer reporting intervention on student behavior compared to baseline conditions. All studies used single-case experimental designs including at least three demonstrations of an effect and at least three data points per phase. Several aspects of studies, participants, and interventions were coded. Log response ratios and Tau were calculated as effect size estimates. Effect size estimates were synthesized in a multi-level meta-analysis with random effects for (a) studies and (b) cases within studies. Overall results indicated peer reporting interventions had a non-zero and positive impact on student outcomes. This was also true when data were subset by outcome (i.e., disruptive behavior, academically engaged behavior, and social behavior). Results were suggestive of more between- than within-study variability. Moderator analyses were conducted to identify aspects of studies, participants, or peer reporting interventions associated with differential effectiveness. Moderator analyses suggested published studies were associated with higher effect sizes than unpublished studies (i.e., theses/dissertations). This meta-analysis suggests peer reporting interventions are effective in improving student behavior compared to baseline conditions. Implications and directions for future investigation are discussed.     

Standards for the Reporting of Genetic Counseling Interventions in Research and Other Studies (GCIRS): an NSGC Task Force Report

As the demand for evidence to support the value of genetic counseling increases, it is critical that reporting of genetic counseling interventions in research and other types of studies (e.g. process improvement or service evaluation studies) adopt greater rigor. As in other areas of healthcare, the appraisal, synthesis, and translation of research findings into genetic counseling practice are likely to be improved if clear specifications of genetic counseling interventions are reported when studies involving genetic counseling are published. To help improve reporting practices, the National Society of Genetic Counselors (NSGC) convened a task force in 2015 to develop consensus standards for the reporting of genetic counseling interventions. Following review by the NSGC Board of Directors, the NSGC Practice Guidelines Committee and the editorial board of the Journal of Genetic Counseling, 23 items across 8 domains were proposed as standards for the reporting of genetic counseling interventions in the published literature (GCIRS: Genetic Counseling Intervention Reporting Standards). The authors recommend adoption of these standards by authors and journals when reporting studies involving genetic counseling interventions.     

Infidelity Treatment Patterns: A Practice-based Evidence Approach

Infidelity is a common presenting problem in couples therapy and can be challenging for therapists to treat. The current zeitgeist in mental health delivery systems includes a focus on developing treatment guidelines. The essence of this movement is that therapists' practice be informed by clinical research, or at the very least, be guided by the accumulated wisdom of other therapists. Practice-based evidence has been advocated as a method using a bottom-up approach of gathering data relying on the expertise of practicing clinicians to inform treatment guidelines. The goal of this paper was to distill and disseminate such aggregated wisdom related to treating infidelity into a practice-based evidence model. We reviewed the literature related to the treatment of infidelity and identified a core set of clinical recommendations that was similar across sources. The themes and patterns that emerged are divided into the following areas: goals of treatment, treatment engagement, treatment format, assessment, process and intervention, treatment adherence, and relapse prevention. We present this as a map or guide to treatment than can be individualized to a specific clinical situation, rather than as a specific, step-by-step manualized approach. We note cultural and ethical factors that may be relevant, as well discuss areas of debate and topics for future research.     

Content Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability

As a method specifically intended for the study of messages, content analysis is fundamental to mass communication research. Intercoder reliability, more specifically termed intercoder agreement, is a measure of the extent to which independent judges make the same coding decisions in evaluating the characteristics of messages, and is at the heart of this method. Yet there are few standard and accessible guidelines available regarding the appropriate procedures to use to assess and report intercoder reliability, or software tools to calculate it. As a result, it seems likely that there is little consistency in how this critical element of content analysis is assessed and reported in published mass communication studies. Following a review of relevant concepts, indices, and tools, a content analysis of 200 studies utilizing content analysis published in the communication literature between 1994 and 1998 is used to characterize practices in the field. The results demonstrate that mass communication researchers often fail to assess (or at least report) intercoder reliability and often rely on percent agreement, an overly liberal index. Based on the review and these results, concrete guidelines are offered regarding procedures for assessment and reporting of this important aspect of content analysis.     

Choosing a patient-reported outcome measure

There has been much philosophical interest regarding the ‘hierarchy of evidence’ used to determine which study designs are of most value for reporting on questions of effectiveness, prognosis, and so on. There has been much less philosophical interest in the choice of outcome measures with which the results of, say, an RCT or a cohort study are presented. In this paper, we examine the FDA’s recently published guidelines for assessing the psychometric adequacy of patient-reported outcome measures. We focus on their recommendations for demonstrating content validity and also for how researchers should weigh up the sum of psychometric evidence when choosing these measures. We argue that questions regarding judgment and understanding meaning of these measures should play a more central role in determining their adequacy.     

Beyond statistical significance: Five principles for the new era of data analysis and reporting

A crisis of confidence in research findings in consumer psychology and other academic disciplines has led to various proposals to abandon, replace, strengthen, or supplement the null hypothesis significance testing paradigm. The proliferation of such proposals, and their often-conflicting recommendations, can increase confusion among researchers. We aim to bring some clarity by proposing five simple principles for the new era of data analysis and reporting of research in consumer psychology. We avoid adding to researchers' confusion and proposing more onerous or rigid standards. Our goal is to offer straightforward practical principles that are easy for researchers to keep in mind while analyzing their data and reporting their findings. These principles involve (1) interpreting p-values as continuous measures of the strength of evidence, (2) being aware of assumptions that determine whether one can rely on p-values, (3) using theory to establish the applicability of findings to new settings, (4) employing multiple measures of evidence and various processes to obtain them, but assigning special privilege to none, and (5) reporting procedures and findings transparently and completely. We hope that these principles provide researchers with some guidance and help to strengthen the reliability of the conclusions derived from their data, analyses, and findings.     

A comparison of adults’ and children's metacognition for yes/no recognition decisions

Basic metacognitive development research suggests that metacognitive abilities develop before adolescence. However, this research has not used tasks that require the discrimination of seen from unseen stimuli, an important element of real-world recognition tasks such as eyewitness identification. We tested the idea that children would be less able to monitor and control the accuracy of their memories in such a task. We used a word-pair recognition task to compare children's (109 8–12 year olds) and adult's (102 first-year psychology students) ability to adaptively make, monitor, and control the reporting of yes/no recognition decisions about familiar stimuli in a task with no demand effects. We found that adults were substantially better at discriminating old from new stimuli, but no evidence of an age difference in metacognitive ability. Although these results do not explain children's poor metacognition in eyewitness identification, they suggest potential steps to improve children's identification performance.     

A methodological review of preference displacement research

The purpose of this review was to identify and compare methodological components of preference displacement research and outcomes. We coded, categorized, and defined patterns of preference displacement for a total of 133 participants across 10 studies. We found that patterns of displacement differed within and across studies, and over 46% of participants did not engage in patterns of displacement. Therefore, the commonly held notion that edible stimuli are more likely to displace tangible stimuli is not as ubiquitous as once thought. However, due to considerable variation in methodology and reporting across reviewed studies, it is difficult to determine what variables may be responsible for obtained study outcomes. We conclude that future researchers consider the importance of each methodological variable coded in our review, and make methodological decisions in the context of the research question they are looking to answer. We also provide additional suggestions for future research and clinical practice.     

The future of psychotherapy outcome research: science or political rhetoric?

Although the relationship between research and clinical psychology has at times been conflicted, it has also been productive. Psychologists from both specialties have benefited from each others' work. The area of psychotherapy outcome research represents an important interface between the fields of clinical and research psychology. In an era of scarce resources and demands for accountability, there is pressure for researchers to justify the value of clinical practices. Recently, numerous articles have appeared recommending changes to the way psychotherapy research is conducted. The authors of these articles emphasize with urgency the importance of conducting and reporting research in a manner that will influence the decisions of policymakers and sanction funding for psychotherapy services. This article is an exploration of the impact of these recommendations, whose objective appears to be the promotion of psychological techniques for inclusion in clinical practice guidelines. It is argued that such recommendations may be in conflict with the philosophy and methods of science and may adversely affect public perception, perhaps leading psychologists to be seen as political lobbyists rather than clinicians and researchers.     

The Relation Between Child Maltreatment and Adolescent Suicidal Behavior: A Systematic Review and Critical Examination of the Literature

A large body of research suggests that child maltreatment (CM) is associated with adolescent suicidal ideation and attempts. These studies, however, have not been critically examined and summarized in a manner that allows us to draw firm conclusions and make recommendations for future research and clinical work in this area. In this review, we evaluated all of the research literature to date examining the relationship between CM and adolescent suicidal ideation and attempts. Results generally suggest that childhood sexual abuse, physical abuse, emotional abuse, and neglect are associated with adolescent suicidal ideation and attempts across community, clinical, and high-risk samples, using cross-sectional and longitudinal research designs. In most studies, these associations remain significant when controlling for covariates such as youth demographics, mental health, family, and peer-related variables. When different forms of CM are examined in the same multivariate analysis, most research suggests that each form of CM maintains an independent association with adolescent suicidal ideation and suicide attempts. However, a subset of studies yielded evidence to suggest that sexual abuse and emotional abuse may be relatively more important in explaining suicidal behavior than physical abuse or neglect. Research also suggests an additive effect—each form of CM contributes unique variance to adolescent suicide attempts. We discuss the current limitations of this literature and offer recommendations for future research. We conclude with an overview of the clinical implications of this research, including careful, detailed screening of CM history, past suicidal behavior, and current suicidal ideation, as well as the need for integrated treatment approaches that effectively address both CM and adolescent suicidal ideation and suicide attempts.     

The effect of running on foot muscles and bones: A systematic review

Despite the widespread evidence of running as a health-preserving exercise, little is known concerning its effect on the foot musculature and bones. While running may influence anatomical foot adaptation, it remains unclear to what extent these adaptations occur. The aim of this paper is to provide a systematic review of the studies that investigated the effects of running and the adaptations that occur in foot muscles and bones. The search was performed following the PRISMA guidelines. Relevant keywords were used for the search through PubMed/MEDLINE, Scopus and SPORTDiscus. The methodological quality of intervention studies was assessed using the Downs and Black checklist. For cross-sectional studies, the Newcastle-Ottawa scale was used. Sixteen studies were found meeting the inclusion criteria. In general, the included studies were deemed to be of moderate methodological quality. Although results of relevant literature are limited and somewhat contradictory, the outcome suggests that running may increase foot muscle volume, muscle cross-sectional area and bone density, but this seems to depend on training volume and experience. Future studies conducted in this area should aim for a standard way of reporting foot muscle/bone characteristics. Also, herein, suggestions for future research are provided.     

Collecting Narrative Data on Amazon's Mechanical Turk

Memory researchers collect data using Amazon's Mechanical Turk (AMT). Despite numerous reports documenting the reliability of data collected on this web site, no study of its uses for narrative memory research has been published. Participants reported narratives of stressful events and were recruited via AMT and in various formats with college students, including completing Internet surveys, typing in the presence of an experimenter, and verbal reports to the experimenter. Data were compared on linguistic indicators, event type reported, and questionnaire responses about these event narratives. AMT participants reported shorter event narratives than all groups of college students but reported stressful events that they self‐reported as more difficult than college students reporting via the Internet and used proportionally more negative emotion terms than college students reporting verbally to an experimenter. Factors such as context and demographics are considered, and recommendations are made for researchers using this web site. Copyright © 2015 John Wiley & Sons, Ltd.     

The physical environment and child development: An international review

A growing body of research in the United States and Western Europe documents significant effects of the physical environment (toxins, pollutants, noise, crowding, chaos, and housing, school and neighborhood quality) on children and adolescents' cognitive and socioemotional development. Much less is known about these relations in other contexts, particularly the global South. We thus briefly review the evidence for relations between child development and the physical environment in Western contexts, and discuss some of the known mechanisms behind these relations. We then provide a more extensive review of the research to date outside of Western contexts, with a specific emphasis on research in the global South. Where the research is limited, we highlight relevant data documenting the physical environment conditions experienced by children, and make recommendations for future work. In these recommendations, we highlight the limitations of employing research methodologies developed in Western contexts (Ferguson & Lee, 2013). Finally, we propose a holistic, multidisciplinary, and multilevel approach based on Bronfenbrenner's (1979) bioecological model to better understand and reduce the aversive effects of multiple environmental risk factors on the cognitive and socioemotional development of children across the globe.     

Editorial: Evidence on Questionable Research Practices: The Good,the Bad,and the Ugly

Purpose

Questionable research or reporting practices (QRPs) contribute to a growing concern regarding the credibility of research in the organizational sciences and related fields. Such practices include design, analytic, or reporting practices that may introduce biased evidence, which can have harmful implications for evidence-based practice, theory development, and perceptions of the rigor of science.

Design/Methodology/Approach

To assess the extent to which QRPs are actually a concern, we conducted a systematic review to consider the evidence on QRPs. Using a triangulation approach (e.g., by reviewing data from observations, sensitivity analyses, and surveys), we identified the good, the bad, and the ugly.

Findings

Of the 64 studies that fit our criteria, 6 appeared to find little to no evidence of engagement in QRPs and the other 58 found more severe evidence (91 %).

Implications

Drawing upon the findings, we provide recommendations for future research related to publication practices and academic training.

Originality/value

We report findings from studies that suggest that QRPs are not a problem, that QRPs are used at a suboptimal rate, and that QRPs present a threat to the viability of organizational science research.