Editorial: Evidence on Questionable Research Practices: The Good,the Bad,and the Ugly

Purpose

Questionable research or reporting practices (QRPs) contribute to a growing concern regarding the credibility of research in the organizational sciences and related fields. Such practices include design, analytic, or reporting practices that may introduce biased evidence, which can have harmful implications for evidence-based practice, theory development, and perceptions of the rigor of science.

Design/Methodology/Approach

To assess the extent to which QRPs are actually a concern, we conducted a systematic review to consider the evidence on QRPs. Using a triangulation approach (e.g., by reviewing data from observations, sensitivity analyses, and surveys), we identified the good, the bad, and the ugly.

Findings

Of the 64 studies that fit our criteria, 6 appeared to find little to no evidence of engagement in QRPs and the other 58 found more severe evidence (91 %).

Implications

Drawing upon the findings, we provide recommendations for future research related to publication practices and academic training.

Originality/value

We report findings from studies that suggest that QRPs are not a problem, that QRPs are used at a suboptimal rate, and that QRPs present a threat to the viability of organizational science research.

     

Evaluating Item-Sort Task Methods: The Presentation of a New Statistical Significance Formula and Methodological Best Practices

Purpose

An item-sort task is a common method to reduce over-representative item lists during the scale-creation process. The current article delineates the limitations and misapplications of the accepted statistical significance formula for item-sort tasks and proposes a new statistical significance formula with greater utility across a wider range of item-sort tasks.

Design

First, a simulation study compares the two formulas in an array of conditions that vary on sample size and number of assignment choices. Second, an empirical study compares the results of three separate item-sort tasks across the two formulas for statistical significance.

Findings

In the empirical study, the proposed formula produces more correct retention decisions than the existing formula across all three item-sort tasks. In the simulation study, the proposed formula is more appropriate than the existing formula under most conditions. The two formulas function identically in item-sort tasks with only two assignment choices.

Implications

Researchers could obtain erroneous results when misapplying the existing item-sort task statistical significance formula to cases with more than two assignment choices. The proposed formula corrects this limitation, ultimately providing accurate results more often than the existing formula. Applying the proposed formula could help future research and practice throughout the scale development process.

Originality

Despite widespread use, few attempts have been made to improve scale-creation pretest methods, particularly item-sort tasks. The current study demonstrates that even conventional statistical methods are susceptible to misuse and misapplication, and future research could benefit from the reexamination of other common methods.

     

Risk Factors Associated with Proactive and Reactive Aggression in a Child Psychiatric Inpatient Sample

Proactive and reactive aggression represent distinct functions of aggression that are associated with different risk factors, including individual and contextual characteristics. However, more research evaluating the interactive effects of risk factors is needed. The current study evaluated whether corporal punishment moderated the influence of neighborhood problems and anger coping on proactive and reactive aggression in a child psychiatric inpatient sample of 6 to 13 year olds (n?=?151). Findings were compared across child- and caregiver-reports of aggression. Consistent with expectations, anger coping was more strongly associated with reactive aggression than proactive aggression across informants. In contrast, perceived neighborhood problems were only associated with child-reports of proactive aggression, with corporal punishment moderating this association. Specifically, the association between neighborhood problems and proactive aggression was only evident at high levels of corporal punishment. Treatment implications and future directions are discussed.     

Psychopathic Traits Mediate the Relationship Between Exposure to Violence and Violent Juvenile Offending

Exposure to violence (ETV) has emerged as a key and stable predictor of violent offending. However, not all youth offenders who experience ETV go on to chronic violent offending. Consequently, it is possible that individual differences, such as psychopathic traits, may be an important factor in the link between ETV and violent offending. These traits are associated with exposure to violence and, separately, to violent offending. The present study used data from Pathways to Desistance, a multisite, longitudinal study of serious juvenile offenders (N = 1170, Mean_age = 16.05, SD = 1.16) to explore these relationships, simultaneously. First, autoregressive cross-lagged path models were used to examine the longitudinal bivariate relations among violent offending, ETV, and psychopathic traits. Second, latent class growth analysis was used to determine trajectories ETV. And third, the mediating influence of psychopathic traits was examined. Results indicated that ETV predicted later engagement in violence, but there was some degree of reciprocity between ETV and violence over time. Additionally, respondents with stable high or increasing trajectories of ETV reported more instances of violent offending. Finally, psychopathic traits mediated the relationship between ETV and violent offending. Together these findings support the notion that individuals with psychopathic traits perceive and internalize their environment differently than others and that this difference guides their own violent offending. Given the importance of psychopathic traits for understanding the influence of ETV on violent offending, prevention and intervention strategies must be developed that take into account both individual differences and environmental factors.     

Destination memory and cognitive theory of mind in normal ageing

Destination memory is the ability to remember the destination to which a piece of information has been addressed (e.g., “Did I tell you about the promotion?”). This ability is found to be impaired in normal ageing. Our work aimed to link this deterioration to the decline in theory of mind. Forty younger adults (M age = 23.13 years, SD = 4.00) and 36 older adults (M age = 69.53 years, SD = 8.93) performed a destination memory task. They also performed the False-belief test addressing cognitive theory of mind and the Reading the mind in the eyes test addressing affective theory of mind. Results showed significant deterioration in destination memory, cognitive theory of mind and affective theory of mind in the older adults. The older adults’ performance on destination memory was significantly correlated with and predicted by their performance on cognitive theory of mind. Difficulties in the ability to interpret and predict others’ mental states are related to destination memory decline in older adults.     

Finnish parents’ attitudes toward entrepreneurship education

This study set out to investigate parental attitudes toward entrepreneurship education as evaluative directing components of social representations. A nationwide sample of parents (N = 625) was asked to indicate their opinions on a set of statements about entrepreneurship education. The parents’ attitudinal orientation suggested that they would rather avoid entrepreneurship education than approach it. Additionally, parental evaluations contained indecision and polarised opinions, and most parents saw it as unlikely or were unsure whether their children would become entrepreneurs. Parental attitudes correlated with the established representations of education and subsequent policy stands, suggesting that entrepreneurship education has been made familiar by anchoring attitudes in the field of educational debates. It was suggested that current trends in educational policy and work life may stimulate the formation of polemical social representations pertaining to entrepreneurship education.     

Empowering and legitimizing the fundamental attribution error: Power and legitimization exacerbate the translation of role‐constrained behaviors into ability differences

Our daily interactions are influenced by the social roles we endorse. People however underestimate these role constraints in their everyday explanation relying on individual dispositions to make sense of behaviors. Two studies investigated whether this bias is exacerbated when role structure is legitimated and when power matches the advantages conferred by the social roles of a quiz game. Legitimacy as well as power increased the tendency for both advantaged (questioner) and disadvantaged (answerer) actors (Study 1) as well as naïve observers of the quiz game (Study 2) to attribute to ability the behaviors elicited by social roles. These results extend previous findings. People are more prone to explain constrained behaviors by differences in ability when role structure is legitimated and when power asymmetry matches role structure. Legitimacy and power may then play an important role in the translation of role constraints into inferences about ability.     

Cognitive costs of encoding novel natural activities: Can “learning by doing” be distracting and deceptive?

Findings from action memory research suggest that the enactment of simple actions and naturalistic activities results in similar memory performance to that from their observation. However, little is known about potential differences between the conditions during the encoding of the to-be-studied actions and activities. We analysed the cognitive costs of encoding two novel naturalistic activities studied via enactment or via observation in four experiments. In addition to memory performance, we measured objective cognitive costs with a secondary task and subjective cognitive costs with repeated ratings of mental effort and estimates of general activity difficulty. Memory performance was comparable across study conditions throughout all experiments. The enactment of activities repeatedly resulted in slower reaction times in the secondary task than did observation, suggesting higher objective costs. In contrast, subjective costs were rated lower after enactment than after observation. Findings from a pantomimic enactment condition suggested that the low ratings of subjective costs after enactment represent a misinterpretation of task demands. Our findings imply that the widespread belief about “learning by doing” as an easy way of learning does not stem from an actual advantage in memory performance, but rather from continuous feedback about one's performance resulting from enactment.