Editorial: Evidence on Questionable Research Practices: The Good,the Bad,and the Ugly

Purpose

Questionable research or reporting practices (QRPs) contribute to a growing concern regarding the credibility of research in the organizational sciences and related fields. Such practices include design, analytic, or reporting practices that may introduce biased evidence, which can have harmful implications for evidence-based practice, theory development, and perceptions of the rigor of science.

Design/Methodology/Approach

To assess the extent to which QRPs are actually a concern, we conducted a systematic review to consider the evidence on QRPs. Using a triangulation approach (e.g., by reviewing data from observations, sensitivity analyses, and surveys), we identified the good, the bad, and the ugly.

Findings

Of the 64 studies that fit our criteria, 6 appeared to find little to no evidence of engagement in QRPs and the other 58 found more severe evidence (91 %).

Implications

Drawing upon the findings, we provide recommendations for future research related to publication practices and academic training.

Originality/value

We report findings from studies that suggest that QRPs are not a problem, that QRPs are used at a suboptimal rate, and that QRPs present a threat to the viability of organizational science research.

     

Evaluating Item-Sort Task Methods: The Presentation of a New Statistical Significance Formula and Methodological Best Practices

Purpose

An item-sort task is a common method to reduce over-representative item lists during the scale-creation process. The current article delineates the limitations and misapplications of the accepted statistical significance formula for item-sort tasks and proposes a new statistical significance formula with greater utility across a wider range of item-sort tasks.

Design

First, a simulation study compares the two formulas in an array of conditions that vary on sample size and number of assignment choices. Second, an empirical study compares the results of three separate item-sort tasks across the two formulas for statistical significance.

Findings

In the empirical study, the proposed formula produces more correct retention decisions than the existing formula across all three item-sort tasks. In the simulation study, the proposed formula is more appropriate than the existing formula under most conditions. The two formulas function identically in item-sort tasks with only two assignment choices.

Implications

Researchers could obtain erroneous results when misapplying the existing item-sort task statistical significance formula to cases with more than two assignment choices. The proposed formula corrects this limitation, ultimately providing accurate results more often than the existing formula. Applying the proposed formula could help future research and practice throughout the scale development process.

Originality

Despite widespread use, few attempts have been made to improve scale-creation pretest methods, particularly item-sort tasks. The current study demonstrates that even conventional statistical methods are susceptible to misuse and misapplication, and future research could benefit from the reexamination of other common methods.

     

Risk Factors Associated with Proactive and Reactive Aggression in a Child Psychiatric Inpatient Sample

Proactive and reactive aggression represent distinct functions of aggression that are associated with different risk factors, including individual and contextual characteristics. However, more research evaluating the interactive effects of risk factors is needed. The current study evaluated whether corporal punishment moderated the influence of neighborhood problems and anger coping on proactive and reactive aggression in a child psychiatric inpatient sample of 6 to 13 year olds (n?=?151). Findings were compared across child- and caregiver-reports of aggression. Consistent with expectations, anger coping was more strongly associated with reactive aggression than proactive aggression across informants. In contrast, perceived neighborhood problems were only associated with child-reports of proactive aggression, with corporal punishment moderating this association. Specifically, the association between neighborhood problems and proactive aggression was only evident at high levels of corporal punishment. Treatment implications and future directions are discussed.     

Psychopathic Traits Mediate the Relationship Between Exposure to Violence and Violent Juvenile Offending

Exposure to violence (ETV) has emerged as a key and stable predictor of violent offending. However, not all youth offenders who experience ETV go on to chronic violent offending. Consequently, it is possible that individual differences, such as psychopathic traits, may be an important factor in the link between ETV and violent offending. These traits are associated with exposure to violence and, separately, to violent offending. The present study used data from Pathways to Desistance, a multisite, longitudinal study of serious juvenile offenders (N = 1170, Mean_age = 16.05, SD = 1.16) to explore these relationships, simultaneously. First, autoregressive cross-lagged path models were used to examine the longitudinal bivariate relations among violent offending, ETV, and psychopathic traits. Second, latent class growth analysis was used to determine trajectories ETV. And third, the mediating influence of psychopathic traits was examined. Results indicated that ETV predicted later engagement in violence, but there was some degree of reciprocity between ETV and violence over time. Additionally, respondents with stable high or increasing trajectories of ETV reported more instances of violent offending. Finally, psychopathic traits mediated the relationship between ETV and violent offending. Together these findings support the notion that individuals with psychopathic traits perceive and internalize their environment differently than others and that this difference guides their own violent offending. Given the importance of psychopathic traits for understanding the influence of ETV on violent offending, prevention and intervention strategies must be developed that take into account both individual differences and environmental factors.     

Finnish parents’ attitudes toward entrepreneurship education

This study set out to investigate parental attitudes toward entrepreneurship education as evaluative directing components of social representations. A nationwide sample of parents (N = 625) was asked to indicate their opinions on a set of statements about entrepreneurship education. The parents’ attitudinal orientation suggested that they would rather avoid entrepreneurship education than approach it. Additionally, parental evaluations contained indecision and polarised opinions, and most parents saw it as unlikely or were unsure whether their children would become entrepreneurs. Parental attitudes correlated with the established representations of education and subsequent policy stands, suggesting that entrepreneurship education has been made familiar by anchoring attitudes in the field of educational debates. It was suggested that current trends in educational policy and work life may stimulate the formation of polemical social representations pertaining to entrepreneurship education.     

Mapping and correcting the influence of gaze position on pupil size measurements

Pupil size is correlated with a wide variety of important cognitive variables and is increasingly being used by cognitive scientists. Pupil data can be recorded inexpensively and non-invasively by many commonly used video-based eye-tracking cameras. Despite the relative ease of data collection and increasing prevalence of pupil data in the cognitive literature, researchers often underestimate the methodological challenges associated with controlling for confounds that can result in misinterpretation of their data. One serious confound that is often not properly controlled is pupil foreshortening error (PFE)—the foreshortening of the pupil image as the eye rotates away from the camera. Here we systematically map PFE using an artificial eye model and then apply a geometric model correction. Three artificial eyes with different fixed pupil sizes were used to systematically measure changes in pupil size as a function of gaze position with a desktop EyeLink 1000 tracker. A grid-based map of pupil measurements was recorded with each artificial eye across three experimental layouts of the eye-tracking camera and display. Large, systematic deviations in pupil size were observed across all nine maps. The measured PFE was corrected by a geometric model that expressed the foreshortening of the pupil area as a function of the cosine of the angle between the eye-to-camera axis and the eye-to-stimulus axis. The model reduced the root mean squared error of pupil measurements by 82.5 % when the model parameters were pre-set to the physical layout dimensions, and by 97.5 % when they were optimized to fit the empirical error surface.     

Interfering with activity in the dorsomedial prefrontal cortex via TMS affects social impressions updating

In our everyday social interactions we often need to deal with others’ unpredictable behaviors. Integrating unexpected information in a consistent representation of another agent is a cognitively demanding process. Several neuroimaging studies point to the medial prefrontal cortex (mPFC) as a critical structure in mediating social evaluations. Our aim here was to shed light on the possible causal role of the mPFC in the dynamic process of forming and updating social impressions about others. We addressed this issue by suppressing activity in the mPFC by means of 1 Hz offline transcranial magnetic stimulation (TMS) prior to a task requiring participants to evaluate other agents’ trustworthiness after reading about their social behavior. In two different experiments, we found that inhibiting activity in the mPFC increased perceived trustworthiness when inconsistent information about one agent’s behavior was provided. In turn, when only negative or positive behaviors of a person were described, TMS over the mPFC did not affect judgments. Our results indicate that the mPFC is causally involved in mediating social impressions updating—at least in cases in which judgment is uncertain due to conflicting information to be processed.     

Pathologisches Glücksspiel und Delinquenz

Pathological gambling is characterized by excessive gambling behavior which influences the life and well-being of a person in a dysfunctional way. It often leads to severe consequences in social and family life, career and material matters. This can also include illegal acts. In the Diagnostic and Statistical Manual of Mental Disorders IV (DSM-IV) excessive gambling was classified under impulse-control disorders but was renamed as gambling disorder (GD) and reclassified under the category “substance-related and addictive disorders” in DSM-5, due to parallels to substance use disorders in the clinical characteristics as well as genetic and neurobiological factors. In Germany, approximately 0.5?% of the adult population suffer from GD. Especially gambling machines are associated with an increased risk of gambling-related problems. Common comorbidities are substance-related and affective disorders. The most important risk factors are young age, male gender, a history of migration, gambling-related problems in the family, unemployment, a low educational level and low household income. Delinquent behavior was not considered to be a sufficiently discriminative factor for GD, although there is a correlation between delinquency and gambling behavior. In addition, those GD subjects who show delinquent behavior also show a specific risk profile. Thus, mental health professionals should be aware of possible psychosocial consequences in GD, such as illegal acts, and this should be addressed in the effective treatment strategies, such as cognitive behavioral therapy. Moreover, it is necessary to improve the access routes for affected people into healthcare system in order to ensure early detection and treatment of the disease.     

Associations between parental gendered attitudes and behaviours and children’s gender development across middle childhood

The present study used a longitudinal, multi-informant approach to examine which specific elements of parents’ gendered attitudes and behaviours were predictive of multiple dimensions of children’s gender development. One hundred and six families with two children (older sibling M = 7.45 years and younger sibling M = 5.19 years at Time 1) were assessed at two time points four to five years apart. At Time 1, parents reported on division of household labour, their own gender-role attitudes (GRAs), and children’s gendered preferences. At Time 2, children reported on their gendered preferences, GRAs and gendered personality traits. Results from multilevel modelling showed that fathers’ egalitarian GRA and egalitarian division of household tasks were predictive of egalitarian child GRA and outcomes, but child sex did not moderate these relationships. As some gender measures were more strongly correlated than others, these findings highlight the importance of examining multiple dimensions of both children’s and parents’ gendered attitudes and behaviours.