Why researchers don't study women: The responses of 62 researchers

Sixty-two researchers accounting for 64 single-sex studies that appeared in 1970 1971 issues of the Journal of Personality and Social Psychology reported why they had limited their studies to a single sex and why they had preferred either males or females for their studies. Also, articles and abstracts were read as a way of checking researchers' reporting practices. The replies were analyzed for thematic content and formed three major types: scientific (56%), practical (28%), and extrascientific (15%). The responses helped to explain why there has been an imbalance in psychology which led to the study of men rather than women instead of the exploration of both sexes. Sggestions are offered to help counteract this trend in future psychological research and reporting.     

JEAB Research Over Time: Species Used,Experimental Designs,Statistical Analyses,and Sex of Subjects

We examined the species used as subjects in every article published in the Journal of the Experimental Analysis of Behavior (JEAB) from 1958 through 2013. We also determined the sex of subjects in every article with human subjects (N = 524) and in an equal number of randomly selected articles with nonhuman subjects, as well as the general type of experimental designs used. Finally, the percentage of articles reporting an inferential statistic was determined at 5-year intervals. In all, 35,317 subjects were studied in 3,084 articles; pigeons ranked first and humans second in number used. Within-subject experimental designs were more popular than between-subjects designs regardless of whether human or nonhuman subjects were studied but were used in a higher percentage of articles with nonhumans (75.4 %) than in articles with humans (68.2 %). The percentage of articles reporting an inferential statistic has increased over time, and more than half of the articles published in 2005 and 2010 reported one. Researchers who publish in JEAB frequently depart from Skinner’s preferred research strategy, but it is not clear whether such departures are harmful. Finally, the sex of subjects was not reported in a sizable percentage of articles with both human and nonhuman subjects. This is an unfortunate oversight.     

Sexual Double Standard: A Review of the Literature Between 2001 and 2010

The present article is a review of 26 studies on the sexual double standard. This phenomenon consists of different rules and standards of sexual behavior for men and women. Only scientific articles reporting empirical research published between 2001 and 2010 and available online in full were included. Our review was focused on the evidence of the existence of the sexual double standard and on the methods that have been used in the studies in this field. Most studies used quantitative methods based on scales or vignette tasks with a target to be evaluated by respondents. There was greater variability of results among the 6 studies using qualitative methods. These studies employed different interview techniques, focus groups and participant observation. There were also 2 studies reporting the use of mixed methods. The results of the studies showed that the sexual double standard still can be identified nowadays, although it seems that sexual behaviors and situations influenced by this phenomenon are changing. Today, premarital sex and sexual intercourse outside of committed relationships are more accepted for both genders. However, evaluations of other facets of sexuality continue to be based on different criteria for men and women.     

Questionable research practices among Brazilian psychological researchers: Results from a replication study and an international comparison

Research on scientific integrity is growing in psychology, and questionable research practices (QRPs) have received more attention due to its harmful effect on science. By replicating the procedures of previous research, the present study aimed at describing the use of QRPs among Brazilian psychological researchers and to make an international comparison with previous studies in other countries—the US and Italy. Two hundred and thirty-two Brazilian researchers in the field of psychology answered questions related to 10 different QRPs. Brazilian researchers indicated a lower tendency to engage in two QRPs (failing to report all of a study's dependent measures; deciding whether to collect more data after looking to see whether the results were significant) when compared to their Italian and North American counterparts, but indicated a higher tendency to engage in two other QRPs (selectively reporting studies that “worked”; not reporting all of a study's conditions). Most of the sample did not admit integrity conflicts in their own research but indicated that others have integrity problems, as observed in previous studies. Those discrepancies could be attributed to contextual and systemic factors regarding different publication demands among the different nations. Further studies should focus on identifying the antecedents of QRPs.     

Accounting for Diversity in Suicide Research: Sampling and Sample Reporting Practices in the United States

Research on suicidal thoughts and behaviors (STB ) has identified many risk factors, but whether these findings generalize to diverse populations remains unclear. We review longitudinal studies on STB risk factors over the past 50 years in the United States and evaluate the methodological practices of sampling and reporting sample characteristics. We found that articles frequently reported participant age and sex, less frequently reported participant race and ethnicity, and rarely reported participant veteran status or lesbian, gay, bisexual, and transgender status. Sample reporting practices modestly and inconsistently improved over time. Finally, articles predominantly featured White, non‐Hispanic, young adult samples.     

Don't ask,don't tell: a systematic review of the extent to which participant characteristics are reported in social anxiety treatment studies

Background and Objectives: This study examined the extent to which social anxiety treatment studies report the demographic characteristics of their participants. One hundred and 56 treatment studies published in English between 2001 and 2012 articles were collected. Methods: Each study was evaluated on whether or not it reported information on gender, age, race, relationship status, education, socioeconomic status, sexual orientation, and disability and also the extent to which the racial composition of the sample was described. Results: The majority of studies reported information on age (96.2%) and gender (94.2%), but the percentage of studies that reported anything else is much lower: race (50.0%), education (42.3%), relationship status (37.8%), socioeconomic status (5.1%), disability (2.6%), and sexual orientation (1.3%). One third (34.0%) of studies reported the race of all participants in their samples, while the remaining reported no information or information for only a subset of participants (e.g. “mostly white”). Conclusions: Participants of social anxiety disorder treatment studies generally are not described beyond their age and gender. Standards for reporting participant characteristics of treatment studies (similar to standards for reporting the methodology of treatment studies) could improve clinical researchers’ and clinicians’ ability to evaluate the external validity of this body of work.     

Treatment integrity of school-based interventions with children in the journal of applied behavior analysis 1991-2005

We reviewed all school-based experimental studies with individuals 0 to 18 years published in the Journal of Applied Behavior Analysis (JABA) between 1991 and 2005. A total of 142 articles (152 studies) that met review criteria were included. Nearly all (95%) of these experiments provided an operational definition of the independent variable, but only 30% of the studies provided treatment integrity data. Nearly half of studies (45%) were judged to be at high risk for treatment inaccuracies. Treatment integrity data were more likely to be included in studies that used teachers, multiple treatment agents, or both. Although there was a substantial increase in reporting operational definitions of independent variables, results suggest that there was only a modest improvement in reported integrity over the past 30 years of JABA studies. Recommendations for research and practice are discussed.     

Reporting observer agreement on stuttering event judgments: A survey and evaluation of current practice

A survey of articles appearing in three national journals over a five-year period was conducted in order to examine current practice in reporting observer agreement on stuttering event judgments. Of 55 articles in which observer agreement was deemed critical, 47 reported at least one observer agreement procedure. A total of eleven agreement procedures were reported and current practice is characterized by five procedures which account for 92% of the articles in which procedures are reported. Problems with using these five indices to represent observer agreement are detailed and several alternatives are discussed.     

The correlates and consequences of newspaper reports of research on sex differences

Correlates and consequences of newspaper accounts of research on sex differences were examined. In Study 1, articles from high-circulation newspapers were coded for the degree to which biological factors were used to explain sex differences. Results showed that political conservatism and traditional attitudes toward gender roles coded from other newspaper sections predicted greater use of biological explanations than did political liberalism and less traditional attitudes toward gender roles. In Studies 2 and 3, participants read a fictional newspaper article reporting research on a gender difference that cited either biological or sociocultural factors as explaining the difference. Results showed that exposure to biological explanations significantly increased participants' endorsement of gender stereotypes. Moreover, exposure to social explanations significantly increased participants' belief in the mutability of human behavior. Together, these studies show that political ideology influences how the popular press reports research findings and that such reporting in turn affects readers' beliefs and attitudes.     

Publication bias in psychological science: comment on Ferguson and Brannick (2012)

It is well documented that studies reporting statistically significant results are more likely to be published than are studies reporting nonsignificant results--a phenomenon called publication bias. Publication bias in meta-analytic reviews should be identified and reduced when possible. Ferguson and Brannick (2012) argued that the inclusion of unpublished articles is ineffective and possibly counterproductive as a means of reducing publication bias in meta-analyses. We show how idiosyncratic choices on the part of Ferguson and Brannick led to an erroneous conclusion. We demonstrate that their key finding--that publication bias was more likely when unpublished studies were included--may be an artifact of the way they assessed publication bias. We also point out how the lack of transparency about key choices and the absence of information about critical features of Ferguson and Brannick's sample and procedures might have obscured readers' ability to assess the validity of their claims. Furthermore, we demonstrate that many of the claims they made are without empirical support, even though they could have tested these claims empirically, and that these claims may be misleading. With their claim that addressing publication bias introduces subjectivity and bias into meta-analysis, they ignored a large body of evidence showing that including unpublished studies that meet the inclusion criteria of a meta-analysis decreases (rather than increases) publication bias. Rather than exclude unpublished studies, we recommend that meta-analysts code study characteristics related to methodological quality (e.g., experimental vs. nonexperimental design) and test whether these factors influence the meta-analytic results.     

Reporting ethical practices in journal articles

Little attention has focused on the reporting of ethical research practices in journal articles. In Study 1, published articles in 2 psychopathology journals were reviewed to ascertain the types of ethical research information that were reported. In Study 2, a survey was sent to authors in Study 1 to determine which ethical practices they engaged in, if they reported this information, and reasons for not including this information in their article. In general, there is a great variability regarding the types of ethical research practices reported in journal articles. Commonly cited reasons for not including ethical research practice information in the articles included the need for brevity, belief that it was common practice, and lack of relevance for the project. These results suggest that there is no standard practice for reporting research practices in journal articles and great variability in the implementation of procedures that are generally considered standard.     

Use of statistical techniques in studies of suicide seasonality, 1970 to 1997

The effect of seasons on suicides has been suggested repeatedly. In order to reveal a true seasonal pattern, an appropriate statistical technique, which is sensitive to a specific type of cyclic variation in the data, must be chosen. This study is a review of the use of statistical techniques for seasonality and of some important characteristics of study samples that were evaluated from 46 original suicide seasonality articles published in major psychiatric journals. The results showed that statistical techniques were applied in a majority of articles, but they were commonly lacking regarding analyses, which compared seasonal patterns among subgroups of a population. In recent studies more sophisticated statistical techniques were utilized for seasonality, like spectral analyses, as compared with earlier studies, in which the emphasis was on chi-square tests. Lack of reporting essential features of the data, such as the sample size and monthly values of suicides, were frequent. The calendar effect was adjusted only in 11 studies. Some recommendations concerning the methodological and reporting issues are summarized for future articles on the seasonal affect on suicides.     

Reporting and referring research participants: ethical challenges for investigators studying children and youth

Researchers studying at-risk and socially disenfranchised child and adolescent populations are facing ethical dilemmas not previously encountered in the laboratory or the clinic. One such set of ethical challenges involves whether to: (a) share with guardians research-derived information regarding participant risk, (b) provide participants with service referrals, or (c) report to local authorities problems uncovered during the course of investigation. The articles assembled for this special section address the complex issues of deciding if, when, and how to report or provide referrals for research participants who are minors (referred to hereafter as minor research participants). This paper focuses on two factors underlying these decisions: the validity of risk estimates and meta-ethical positions on scientific responsiblity. It is suggested that, before sharing information about minor research participants investigators should do the following: critically examine the diagnostic validity of developmental measures, include the scope and limitations of information sharing in informed consent procedures, and become familiar with state reporting laws. I discuss the impact of the traditionally accepted act utilitarian meta-ethical position on the investigator-participant relationship, and I recommend consideration of alternative positions as a step toward developing a research ethic of scientific responsibility and care.     

Suicide surveillance in the U.S. Military--reporting and classification biases in rate calculations

The military has a well-defined population with suicide prevention programs that have been recognized as possible models for civilian suicide prevention efforts. Monitoring prevention programs requires accurate reporting. In civilian settings, several studies have confirmed problems in the reporting and classification of suicides. This analysis evaluated whether suicides were underreported or misclassified under accident or undetermined manner of death in the military system. We reviewed all 1998 and 1999 military deaths using official death reports and compared these data with additional sources, most importantly the DoD Medical Mortality Registry. We assessed for evidence of expressed suicidal intent and past psychiatric history among deaths classified as undetermined and accidents due to gunshot, overdose, drowning, falls, or asphyxia. Using sources other than official records, we found 17% more suicides than were reported, and an additional 4% of deaths that were suspicious for suicide. This study suggests that reporting and classification errors may account for 21% additional suicides in the military. These findings are comparable to rates seen in civilian studies and add to the literature regarding the problems inherent in using administrative death classification data for medical surveillance purposes.     

The (mis)reporting of statistical results in psychology journals

In order to study the prevalence, nature (direction), and causes of reporting errors in psychology, we checked the consistency of reported test statistics, degrees of freedom, and p values in a random sample of high- and low-impact psychology journals. In a second study, we established the generality of reporting errors in a random sample of recent psychological articles. Our results, on the basis of 281 articles, indicate that around 18% of statistical results in the psychological literature are incorrectly reported. Inconsistencies were more common in low-impact journals than in high-impact journals. Moreover, around 15% of the articles contained at least one statistical conclusion that proved, upon recalculation, to be incorrect; that is, recalculation rendered the previously significant result insignificant, or vice versa. These errors were often in line with researchers’ expectations. We classified the most common errors and contacted authors to shed light on the origins of the errors.     

Missing data: prevalence and reporting practices

Results are described for a survey assessing prevalence of missing data and reporting practices in studies with missing data in a random sample of empirical research journal articles from the PsychINFO database for the year 1999, two years prior to the publication of a special section on missing data in Psychological Methods. Analysis indicates missing data problems were found in about one-third of the studies. Further, analytical methods and reporting practices varied widely for studies with missing data. One may consider these results as baseline data to assess progress as reporting standards evolve for studies with missing data. Some potential reporting standards are discussed.     

An evaluation of the quantity and quality of empirical research in three pastoral care and counseling journals, 1990-1999: has anything changed?

This article summarizes a review of all articles published in Pastoral Psychology, The Journal of Rleigion and Health, and The Journal of Pastoral Care between 1900 and 1999, identifying a total of 737 scholarly articles, of which 165 (22.4%) were research studies. The proportion of research studies, especially quantitative studies, increased significantly between the first and second half of the study period (p < .05). There was a significant positive correlation between compliance with three out of four criteria of internal validity. Three of five criteria of external validity were also positively related to one another. Compared to previous research using identical criteria to assess quantitative studies in the same journals in 1980-1989, the 1990-1999 sample showed improved compliance with respect to specifying the sampling method (p < .001), reporting the response rate (p < .05), and discussing the limitations of research studies (p < .001). However, the overall findings suggest that many researchers in the field do not have a sophisticated knowledge of statistical sampling, statistical analysis, or research design. Several recommendations for increasing the quality of quantitative research are offered.