Editorial: Evidence on Questionable Research Practices: The Good,the Bad,and the Ugly

Purpose

Questionable research or reporting practices (QRPs) contribute to a growing concern regarding the credibility of research in the organizational sciences and related fields. Such practices include design, analytic, or reporting practices that may introduce biased evidence, which can have harmful implications for evidence-based practice, theory development, and perceptions of the rigor of science.

Design/Methodology/Approach

To assess the extent to which QRPs are actually a concern, we conducted a systematic review to consider the evidence on QRPs. Using a triangulation approach (e.g., by reviewing data from observations, sensitivity analyses, and surveys), we identified the good, the bad, and the ugly.

Findings

Of the 64 studies that fit our criteria, 6 appeared to find little to no evidence of engagement in QRPs and the other 58 found more severe evidence (91 %).

Implications

Drawing upon the findings, we provide recommendations for future research related to publication practices and academic training.

Originality/value

We report findings from studies that suggest that QRPs are not a problem, that QRPs are used at a suboptimal rate, and that QRPs present a threat to the viability of organizational science research.

     

Effects of a Peer-Delivered Corrective Reading Program

The effects of a peer-delivered Corrective Reading program (CRP) (Engelmann, Hanner, & Johnson, 1989) with repeated reading on the reading performance of at-risk high school students were examined. High school peer instructors and students in need of reading remediation were randomly assigned to dyads or triads. Peer instructors presented the CRP, conducted repeated reading trials, and took rate and accuracy measures. Pre to post test data were collected on vocabulary and comprehension subtests of the Gates–MacGinitie Reading Tests, oral reading fluency (words read per minute) and accuracy, number of repeated readings on initial and final lessons, and lessons completed. Results indicated that students' performance on the standardized reading assessment increased as did their oral reading fluency. In general, the number of repeated readings decreased from the initial lesson to the final lesson. Implications for further research are discussed.     

Hospital Restructuring and Nursing Staff Well-Being: The Role of Coping

A research model incorporating hospital restructuring stressors, job satisfaction, psychosomatic symptoms and active and escapist coping was tested in a large sample of nursing staff. Almost 1,400 staff nurses participated by completing anonymous questionnaires. Concepts in the model were assessed using multiple-item scales having satisfactory psychometric properties. A LISREL analysis was undertaken in which relationships among all variables in the model were considered simultaneously. Use of active coping was negatively related to the use of escapist coping; use of escapist coping was positively related to future threats to security and psychosomatic symptoms and negatively related to self-reported job satisfaction; use of active coping was related to perceptions of less extensive hospital restructuring, greater job satisfaction, and fewer psychosomatic symptoms.     

Resilient Response to Battering

Spouse abuse remains an important social problem. Recent research has begun to explore resiliency in a variety of areas: Why, despite harsh life circumstances, do some people survive and later thrive after experiencing trauma? What are the processes associated with a successful departure from abuse? The authors worked with a local shelter to identify three groups of women: (a) those currently at the shelter; (b) those who had been away from an abusive relationship for at least one year; and (c) those who were identified by shelter staff as having demonstrated a resilient response to battering, and compared experiences and recommendations between groups and discuss clinical and service implications.     

The relationship between hospital restructuring,anger, hostility and psychosomatics in nurses

This study examines the relationship between hospital restructuring, psychosomatic reactions, anger and hostility in nurses. Scales assessed extent of hospital restructuring. Anger was assessed using the State–Trait Anger Expression Inventory (STAXI) Scales. Results suggested that restructuring is a stressful and anger‐provoking experience which increases with more restructuring initiatives. Copyright © 2000 John Wiley & Sons, Ltd.     

Eliciting information and cues to deceit through sketching in interpreter‐based interviews

As interviewees typically say less when an interpreter is present, we examined whether this was caused by interpreters not interpreting everything interviewees says or by interviewees providing less information. We further examined (a) the effect of a model drawing on providing information and (b) the diagnostic value of total details and the proportion of complications as cues to deceit. Hispanic, Russian, and South Korean participants were interviewed by native interviewers or by a British interviewer through an interpreter. Truth tellers discussed a trip they had made; liars fabricated a story. Participants received no instruction (condition 1) or were instructed to sketch while narrating without (condition 2) or with (condition 3) being given examples of detailed sketches. Interviewees said less when an interpreter was present because they provided less information. Truth tellers gave more details and, particularly, obtained a higher proportion of complications than liars. The sketching manipulation had no effect.     

Take a break! Benefits of sleep and short breaks for daily work engagement

The current study investigates the benefits of a good night’s sleep and short work breaks for employees’ daily work engagement. It is hypothesized that sleep and self-initiated short breaks help restore energetic and self-regulatory resources which, in turn, enable employees to experience high work engagement. A daily diary study was conducted with 107 employees who provided data twice a day (before lunch and at the end of the working day) over 5 workdays (453 days in total). Multilevel regression analyses showed that sleep quality and short breaks were beneficial for employees’ daily work engagement. After nights employees slept better, they indicated higher work engagement during the day. Moreover, taking self-initiated short breaks from work in the afternoon boosted daily work engagement, whereas taking short breaks in the morning failed to predict daily work engagement. Taking short breaks did not compensate for impaired sleep with regard to daily work engagement. Overall, these findings suggest that recovery before and during work can foster employees’ daily work engagement.     

Potential Mediators in Parenting and Family Intervention: Quality of Mediation Analyses

Parenting and family interventions have repeatedly shown effectiveness in preventing and treating a range of youth outcomes. Accordingly, investigators in this area have conducted a number of studies using statistical mediation to examine some of the potential mechanisms of action by which these interventions work. This review examined from a methodological perspective in what ways and how well the family-based intervention studies tested statistical mediation. A systematic search identified 73 published outcome studies that tested mediation for family-based interventions across a wide range of child and adolescent outcomes (i.e., externalizing, internalizing, and substance-abuse problems; high-risk sexual activity; and academic achievement), for putative mediators pertaining to positive and negative parenting, family functioning, youth beliefs and coping skills, and peer relationships. Taken as a whole, the studies used designs that adequately addressed temporal precedence. The majority of studies used the product of coefficients approach to mediation, which is preferred, and less limiting than the causal steps approach. Statistical significance testing did not always make use of the most recently developed approaches, which would better accommodate small sample sizes and more complex functions. Specific recommendations are offered for future mediation studies in this area with respect to full longitudinal design, mediation approach, significance testing method, documentation and reporting of statistics, testing of multiple mediators, and control for Type I error.     

The nature of correlation perception in scatterplots

For scatterplots with gaussian distributions of dots, the perception of Pearson correlation r can be described by two simple laws: a linear one for discrimination, and a logarithmic one for perceived magnitude (Rensink & Baldridge, 2010). The underlying perceptual mechanisms, however, remain poorly understood. To cast light on these, four different distributions of datapoints were examined. The first had 100 points with equal variance in both dimensions. Consistent with earlier results, just noticeable difference (JND) was a linear function of the distance away from r = 1, and the magnitude of perceived correlation a logarithmic function of this quantity. In addition, these laws were linked, with the intercept of the JND line being the inverse of the bias in perceived magnitude. Three other conditions were also examined: a dot cloud with 25 points, a horizontal compression of the cloud, and a cloud with a uniform distribution of dots. Performance was found to be similar in all conditions. The generality and form of these laws suggest that what underlies correlation perception is not a geometric structure such as the shape of the dot cloud, but the shape of the probability distribution of the dots, likely inferred via a form of ensemble coding. It is suggested that this reflects the ability of observers to perceive the information entropy in an image, with this quantity used as a proxy for Pearson correlation.