The (mis)reporting of statistical results in psychology journals

In order to study the prevalence, nature (direction), and causes of reporting errors in psychology, we checked the consistency of reported test statistics, degrees of freedom, and p values in a random sample of high- and low-impact psychology journals. In a second study, we established the generality of reporting errors in a random sample of recent psychological articles. Our results, on the basis of 281 articles, indicate that around 18% of statistical results in the psychological literature are incorrectly reported. Inconsistencies were more common in low-impact journals than in high-impact journals. Moreover, around 15% of the articles contained at least one statistical conclusion that proved, upon recalculation, to be incorrect; that is, recalculation rendered the previously significant result insignificant, or vice versa. These errors were often in line with researchers’ expectations. We classified the most common errors and contacted authors to shed light on the origins of the errors.     

Disability research in counseling psychology journals: a 20-year content analysis

We conducted an exploratory content analysis of disability research in 5 major counseling psychology journals between 1990 and 2010. The goal was to review the counseling psychology literature to better understand the prevalence of disability research, identify research methods most often conducted, and elucidate the types of concerns most studied. We searched 5 journals (Journal of Counseling & Development, Journal of Counseling Psychology, The Counseling Psychologist, Journal of Multicultural Counseling and Development, and Cultural Diversity and Ethnic Minority Psychology) using keywords derived from disability terms defined by the Individuals With Disabilities Education Improvement Act of 2004 (IDEA). Articles were categorized by methodology and disability category examined. We found that disability research continues to comprise an extremely small amount (from less than 1% to 2.7%) of the counseling psychology literature, with the frequency of articles plateauing in recent years. The research design of articles published has changed, with an increased number of empirical articles and a decrease in literature reviews. We conclude by calling for increased empirical investigation of disability among journals specific to counseling psychology to recognize disability as an important aspect of diversity within the field.     

Survey of current clinical and research trends in clinical psychology

A survey of the directors of APA accredited clinical psychology programs was conducted to explore current trends in the training of clinical psychologists. Specifically, information was obtained regarding the research areas represented at each school, the types of clinical training sites available to graduate students, and the prevalence of different clinical orientations (e.g., cognitive-behavioral, systems, or psychoanalytic). Results of the survey provide a summary of the activities of faculty and students at the more than 100 fully accredited clinical psychology programs.     

The moderator effect that wasn't there: statistical problems in ambivalence research

Ambivalence researchers often collapse separate measures of positivity and negativity into a single numerical index of ambivalence and refer to it as objective, operative, or potential ambivalence. The authors argue that this univariate approach to ambivalence models undermines the validity of subsequent statistical analyses because it confounds the effects of the index and its components. To remedy this situation, they demonstrate how the assumptions underlying the indices derived from the conflicting reactions model and similarity-intensity model can be tested using a multivariate approach to ambivalence models. On the basis of computer simulations and reanalyses of published moderator effects, the authors show that the frequently reported moderating influence of ambivalence on attitude effects may be a statistical artifact resulting from unmodeled correlations of positivity and negativity with attitude and the dependent variable. On the basis of extensive power analyses, they conclude that it may be extremely difficult to detect moderator effects of ambivalence in observational data. Therefore, they encourage ambivalence researchers to take an experimental approach to study design and a multivariate approach to data analysis.     

The accuracy of abstracts in psychology journals

This article provides an empirically supported reminder of the importance of accuracy in scientific communication. The authors identify common types of inaccuracies in research abstracts and offer suggestions to improve abstract-article agreement. Abstracts accompanying 13% of a random sample of 400 research articles published in 8 American Psychological Association journals during 1997 and 1998 contained data or claims inconsistent with or missing from the body of the article. Error rates ranged from 8% to 18%, although between-journal differences were not significant. Many errors (63%) were unlikely to cause substantive misinterpretations. Unfortunately, 37% of errors found could be seriously misleading with respect to the data or claims presented in the associated article. Although deficient abstracts may be less common in psychology journals than in major medical journals (R. M. Pitkin, M. A. Branagan, & L. F. Burmeister, 1999), there is still cause for concern and need for improvement.     

Estimating sample-size requirements in research design: A study of intuitive statistical judgment

Subjects were given an experimental task in which they had to play the role of a quality-control researcher for a company. They had to consider a hypothetical experiment that involves testing a sample of batteries from a truck load, which may or may not be substandard. In the main experiment, subjects were given information about the prior probability of substandard truck loads (base rate), the degree of variability of battery life, and the mean difference between standard and substandard batteries, all of which are formally relevant to the decision, and they were also told the number of batteries in the truck (population size) that is formally irrelevant. The task was to decide (intuitively) how many batteries to test to achieve a specified error rate using a specified decision rule. In a second study, subjects were given a similar scenario, but asked to rate which pieces of information would be relevant to the decision. Subjects showed themselves to be sensitive to the effects of sample variability and base rate when making intuitive design decisions, though an odd effect of the mean difference factor was observed. There is also clear confirmation of a bias-to-weight sample size by population size as reported in earlier research using different kinds of judgmental tasks.     

Qualitative research in psychology: Attitudes of psychology students and academic staff

Qualitative research is experiencing a resurgence within the field of psychology. This study aimed to explore the range of attitudes towards qualitative research in psychology held by students and academics, using the model of attitudes by Eagly and Chaiken as a framework. Twenty‐one psychology students and academics were interviewed about their attitudes towards qualitative research. Interviews were transcribed and analysed using thematic analysis. While qualitative research was described as inherent to the psychology profession and useful for generating rich data, some participants felt that this approach was not well respected or considered as legitimate as quantitative methods. Reflecting common misperceptions about qualitative research, participants also expressed concerns that qualitative research was too subjective and had limited generalisability. Furthermore, some participants felt that they lacked the skills and confidence necessary to conduct qualitative research. Large investments in time and resources were identified as barriers to undertaking qualitative research. Identifying attitudes towards qualitative research provides a basis for future work in dispelling myths, promoting attitudinal change and increasing both the use and teaching of qualitative approaches in psychology.     

Boundaries,journals and counselling psychology: Stephen Palmer interviews Kasia Szymanska

You have just retired from editing Counselling Psychology Review, the in-house journal of the British Psychological Society, Division of Counselling Psychology. It seems an appropriate moment to interview you for Counselling Psychology Quarterly. In this interview I want to focus on a topic I know you are interested in, client-therapist boundary issues, and your experience editing the journal. But I'd like to start off at a slight tangent. What encouraged you to go into counselling psychology as a profession?     

Thoughts and recommendations concerning impact and productivity in school psychology journals

Having been personally involved in several examinations of productivity in various sub-disciplines of psychology, I read with great interest the articles in this special issue. I commend the authors for engaging in this activity, as I feel that a field benefits from occasional self-examinations. In this commentary, I caution against placing too much emphasis on impact factors, arguing that they are not measured without error and unethical behaviors could increase if they are weighted too heavily. I recommend examinations of methodological trends, whether implications for practice are based on observational data, whether female participation in the publication process matches their participation as members of the field, and finally including other characteristics in defining highly productive scholars that would attract potential graduate students.     

Paul Meehl and the evolution of statistical methods in psychology

In his landmark 1978 paper, Paul Meehl delineated, with remarkable clarity, some fundamental challenges facing soft psychology as it attempts to test theory with data. In the quarter century that followed, Meehl’s views stimulated much debate and progress, while continually evolving to keep pace with that progress. This paper pays homage to Meehl’s prescience, and traces the impact of his ideas on the recent shift of emphasis away from hypothesis testing and toward confidence interval estimates of effect size.