Missing data: prevalence and reporting practices

Results are described for a survey assessing prevalence of missing data and reporting practices in studies with missing data in a random sample of empirical research journal articles from the PsychINFO database for the year 1999, two years prior to the publication of a special section on missing data in Psychological Methods. Analysis indicates missing data problems were found in about one-third of the studies. Further, analytical methods and reporting practices varied widely for studies with missing data. One may consider these results as baseline data to assess progress as reporting standards evolve for studies with missing data. Some potential reporting standards are discussed.     

Science and ethics in conducting, analyzing, and reporting psychological research

The relationship between scientific quality and ethical quality is considered for three aspects of the research process: conduct of the research, data analysis, and reporting of results. In the area of conducting research, issues discussed involve design, recruitment, causism, scientific quality, and costs and utilities. The discussion of data analysis considers data dropping, data exploitation, and meta-analysis. Issues regarding reporting of results include misrepresentation of findings, misrepresentation of credit, and failure to report results as a result of self censoring or external censoring.     

Exploring a Predictive Model

The purposes of this study were (a) to explore the utility of a model for predicting reporting of sexual harassment and (b) to collect data on the incidence of sexual harassment using the Sexual Experiences Questionnaire (SEQ; Fitzgerald et al., 1988). The sample included 214 tenuretrack faculty women and 276 women graduate students. The full model postulated that age, marital status, feminist ideology, and frequency of behavior would be directly related to perceived offensiveness at the behavior. In turn, perceived offensiveness, normative expectations for reporting, and perceived outcomes of reporting would directly influence reporting. As predicted, perceived offensiveness showed a direct influence on reporting, and feminist ideology and frequency of behavior were significant predictors of perceived offensiveness. Incidence data showed that the most frequently experienced situations involved gender harassment and seductive behaviors. Results suggest that educating women about the offensiveness of sexual harassment might increase frequency of reporting. Also, further exploration of the model seems warranted.     

Selected Ethical Issues in the Analysis and Reporting of Research: Survey of Business School Faculty in Malaysia

This study reports the perceptions of business school faculty on ethical behaviors related to data analysis and research reporting as well as the prevalence of such behaviors in their academic environment. Survey data for the study were obtained from a sample of 102 business school faculty from five government-funded universities in Malaysia. Study results showed that a majority of the respondents considered practices such as fabrication, manipulation, and distortion of data to be ethically unacceptable, and these behaviors were reported to be least prevalent. In contrast, the practice of misapplying statistical techniques was considered ethically acceptable and reported to be quite prevalent. On research reporting, although a majority of the respondents agreed that plagiarism and taking undeserved authorship credit were ethically unacceptable, they also reported having observed the frequent occurrence of such behaviors. Finally, practices such as cutting up research data and simultaneous submissions to more than one publication outlet at the same time were less likely to be viewed as unethical and seen to be quite a common practice. In general, the findings of this study indicate that the perceptions of the ethicality and frequency of occurrence of behaviors related to data analysis and research reporting vary among business school faculty.     

Reporting subscores for institutions

Recently, there has been an increasing level of interest in reporting subscores for components of larger assessments. This paper examines the issue of reporting subscores at an aggregate level, especially at the level of institutions to which the examinees belong. A new statistical approach based on classical test theory is proposed to assess when subscores at the institutional level have any added value over the total scores. The methods are applied to two operational data sets. For the data under study, the observed results provide little support in favour of reporting subscores for either examinees or institutions.     

Women’s Reporting of Physical Assaults to Police in a National Sample: A Brief Report

Research examining reporting decisions of women physically assaulted by any type of perpetrator is scarce. This study used data from the National Violence Against Women Survey (NVAWS) to examine how victim characteristics, assault characteristics, emotions during the assault, and postassault disclosure to informal social support sources affect victim and third-party reporting decisions in physical assault incidents. Results showed that victim education, race and ethnicity, victim–offender relationship, crime severity, perceived life threat, and disclosure to informal social support sources were related to victim reporting, whereas victim employment and marital status were associated with third-party reporting.     

The influence of sexual harassment frequency and perceptions of organizational justice on victim responses to sexual harassment

Organizational justice theory was used to understand the conditions that influence how women respond when sexually harassed. Specifically, this study examined whether sexual harassment frequency interacts with perceptions of four types of organizational justice (procedural, distributive, interpersonal, and informational) to predict two types of victim responses (confrontation and reporting). With data collected from 257 female employees, it was found that the interaction between sexual harassment frequency and perceptions of distributive justice and the interaction between sexual harassment frequency and perceptions of procedural justice predicted reporting, whereas the interaction between sexual frequency and perceptions of distributive justice predicted confrontation. The interaction between sexual harassment frequency and perceptions of informational justice predicted both confrontation and reporting. Implications for organizations are discussed.     

Age differences in encoding and retrieving details of a pediatric examination

Baker-Ward, Gordon, Ornstein, Larus, and Clubb (1993) showed that recall improves over ages 3–7 for events experienced during a physical examination. We used a joint multinomial model to ask whether the improvement was due to encoding, to retrieval, or to likelihood to report. The model fit the Baker-Ward et al. data well and showed that (1) retrieval and reporting cannot be distinguished and (2) the observed effects were due primarily to age-related improvement in retrieval reporting rather than in encoding.     

Observer agreement, credibility, and judgment: some considerations in presenting observer agreement data

Graphical and statistical indices employed to represent observer agreement in interval recording are described as "judgmental aids", stimuli to which the researcher and scientific community must respond when viewing observer agreement data. The advantages and limitations of plotting calibrating observer agreement data and reporting conventional statistical aids are discussed in the context of their utility for researchers and research consumers of applied behavior analysis. It is argued that plotting calibrating observer data is a useful supplement to statistical aids for researchers but is of only limited utility for research consumers. Alternatives to conventional per cent agreement statistics for research consumers include reporting special agreement estimates (e.g., per cent occurrence agreement and nonoccurrence agreement) and correlational statistics (e.g., Kappa and Phi).     

Recent reliability reporting practices in Psychological Assessment: recognizing the people behind the data

Helms, Henze, Sass, and Mifsud (2006) defined good practices for internal consistency reporting, interpretation, and analysis consistent with an alpha-as-data perspective. Their viewpoint (a) expands on previous arguments that reliability coefficients are group-level summary statistics of samples' responses rather than stable properties of scales or measures and (b) encourages researchers to investigate characteristics of reliability data for their own samples and subgroups within their samples. In Study 1, we reviewed past and current reliability reporting practices in a sample of Psychological Assessment articles published across 3 decades (i.e., from the years 1989, 1996, and 2006). Results suggested that contemporary and past researchers' reliability reporting practices have not improved over time and generally were not consistent with good practices. In Study 2, we analyzed an archival data set to illustrate the real-life repercussions of researchers' ongoing misconstrual and misuse of reliability data. Our analyses suggested that researchers should conduct preliminary analyses of their data to determine whether their data fit the assumptions of their reliability analyses. Also, the results indicated that reliability coefficients varied across racial or ethnic and gender subgroups, and these variations had implications for whether the same depression measure should be used across groups. We concluded that the alpha-as-data perspective has implications for one's choice of psychological measures and interpretation of results, which subsequently affect conclusions and recommendations. We encourage researchers to recognize the people behind their data by adopting better practices in internal consistency reporting, analysis, and interpretation.     

Participant Protection With the Use of Records: Ethical issues and recommendations

Fraudulent analysis and reporting of psychological data have the potential to contaminate the scientific knowledge base and eventuate in the unjustified expenditure of public money and scientific effort (Koocher &; Keith-Spiegel, 1998). Traditionally, the field has relied on quantitative methodologists to educate researchers in proper analysis and reporting practices, and to examine these via peer review. The field has also relied on psychologists with training or board service in ethics to establish standards and implement strategies to discourage misconduct. However, this division of responsibility for examination, standard setting, and deterrence is shown to compromise the effectiveness of methodologists' and ethicists' respective gatekeeping efforts. Methodologically and ethically trained specialists instead need to coordinate efforts to safeguard analysis and reporting procedures. Researchers also need to increase self-monitoring. Potential obstacles to achieving these ends are considered, and three tactics are proposed to overcome them.     

Beyond statistical significance: Five principles for the new era of data analysis and reporting

A crisis of confidence in research findings in consumer psychology and other academic disciplines has led to various proposals to abandon, replace, strengthen, or supplement the null hypothesis significance testing paradigm. The proliferation of such proposals, and their often-conflicting recommendations, can increase confusion among researchers. We aim to bring some clarity by proposing five simple principles for the new era of data analysis and reporting of research in consumer psychology. We avoid adding to researchers' confusion and proposing more onerous or rigid standards. Our goal is to offer straightforward practical principles that are easy for researchers to keep in mind while analyzing their data and reporting their findings. These principles involve (1) interpreting p-values as continuous measures of the strength of evidence, (2) being aware of assumptions that determine whether one can rely on p-values, (3) using theory to establish the applicability of findings to new settings, (4) employing multiple measures of evidence and various processes to obtain them, but assigning special privilege to none, and (5) reporting procedures and findings transparently and completely. We hope that these principles provide researchers with some guidance and help to strengthen the reliability of the conclusions derived from their data, analyses, and findings.     

Suicide surveillance in the U.S. Military--reporting and classification biases in rate calculations

The military has a well-defined population with suicide prevention programs that have been recognized as possible models for civilian suicide prevention efforts. Monitoring prevention programs requires accurate reporting. In civilian settings, several studies have confirmed problems in the reporting and classification of suicides. This analysis evaluated whether suicides were underreported or misclassified under accident or undetermined manner of death in the military system. We reviewed all 1998 and 1999 military deaths using official death reports and compared these data with additional sources, most importantly the DoD Medical Mortality Registry. We assessed for evidence of expressed suicidal intent and past psychiatric history among deaths classified as undetermined and accidents due to gunshot, overdose, drowning, falls, or asphyxia. Using sources other than official records, we found 17% more suicides than were reported, and an additional 4% of deaths that were suspicious for suicide. This study suggests that reporting and classification errors may account for 21% additional suicides in the military. These findings are comparable to rates seen in civilian studies and add to the literature regarding the problems inherent in using administrative death classification data for medical surveillance purposes.     

Principles and practice in reporting structural equation analyses

Principles for reporting analyses using structural equation modeling are reviewed, with the goal of supplying readers with complete and accurate information. It is recommended that every report give a detailed justification of the model used, along with plausible alternatives and an account of identifiability. Nonnormality and missing data problems should also be addressed. A complete set of parameters and their standard errors is desirable, and it will often be convenient to supply the correlation matrix and discrepancies, as well as goodness-of-fit indices, so that readers can exercise independent critical judgment. A survey of fairly representative studies compares recent practice with the principles of reporting recommended here.     

Reporters and a Free Press

ABSTRACT A necessary condition for news to be produced under conditions of 'negative' freedom is for newspapers to become 'reporter codetermined newspapers', where reporters, along with editors and publishers, have a collective 'positive' freedom to vote on news-policy, select editors and hire reporters. 'Publisher-controlled' newspapers systematically prevent reporters from reporting some news-stories and coerce and manipulate reporters into reporting others. It is argued here that all newspapers should be legally required to become reporter codetermined newspapers. This change is also required by Rawls' and Mill's arguments for freedom of the press. On the basis of empirical data it is unlikely that the demands of freedom or justice can be met within existing institutional structures.     

Reporting of life events over time: methodological issues in a longitudinal sample of women

The number of life events reported by study participants is sensitive to the method of data collection and time intervals under consideration. Individual characteristics also influence reporting; respondents with poor mental health report more life events. Much current research on life events is cross-sectional. Data from a longitudinal study of women's health from 4 waves over a decade suggest that over time additional systematic biases in reporting life events occur. Inconsistency over time is due to both fall-off of reporting and telescoping. Intracategory variability and ambiguity of items, as well as respondent characteristics, also potentially contribute to response biases. Although some factors (e.g., item wording) are controllable, others (e.g., respondents' mental health) are not and must be factored into data analysis and interpretation.     

Say something nice: A meta-analytic review of peer reporting interventions

Peer reporting interventions (i.e., Positive Peer Reporting and tootling) are commonly used peer-mediated interventions in schools. These interventions involve training students to make reports about peers' prosocial behaviors, whether in oral or written form. Although peer reporting interventions have been included in meta-analyses of group contingencies, this study is the first meta-analytic review of single-case research focusing exclusively on peer reporting interventions. The literature search and application of inclusion criteria yielded 21 studies examining the impact of a peer reporting intervention on student behavior compared to baseline conditions. All studies used single-case experimental designs including at least three demonstrations of an effect and at least three data points per phase. Several aspects of studies, participants, and interventions were coded. Log response ratios and Tau were calculated as effect size estimates. Effect size estimates were synthesized in a multi-level meta-analysis with random effects for (a) studies and (b) cases within studies. Overall results indicated peer reporting interventions had a non-zero and positive impact on student outcomes. This was also true when data were subset by outcome (i.e., disruptive behavior, academically engaged behavior, and social behavior). Results were suggestive of more between- than within-study variability. Moderator analyses were conducted to identify aspects of studies, participants, or peer reporting interventions associated with differential effectiveness. Moderator analyses suggested published studies were associated with higher effect sizes than unpublished studies (i.e., theses/dissertations). This meta-analysis suggests peer reporting interventions are effective in improving student behavior compared to baseline conditions. Implications and directions for future investigation are discussed.     

SIGNIFICANCE AND SUBSTANCE: AN EXAMINATION OF EXPERIMENTAL EFFECTS

The dimension of obtained effect size (OES) was examined in this report. The 1977 volumes of Communication Monographs, Human Communication Research, and the Journal of Communication were reviewed, and a comparison of OES magnitude, reporting, and interpretation was conducted. Additional data gleaned from the 1975, 1976, and 1978 volumes of Human Communication Research were also included. The magnitude of experimental effects was relatively high in Communication Monographs and Human Communication Research. The number of authors reporting OES was most pronounced in Human Communication Research. The role of OES in relation to significance was discussed, and some recommendations for the reporting of data were advanced.     

Trust differences between supervisors and subordinates: Examining the role of race and gender

This study examined trust differences between supervisor and subordinate dyads that differ in race or gender as well as in dyads that were similar. The results of 301 (the sample consisted of 241 whites and 41 blacks, 216 were women and 86 were men) respondents indicated gender and race do affect trust. More specifically, the highest level of trust was found between female subordinates reporting to male supervisors. It was also found that whites reporting to whites perceived significantly less trust than blacks reporting to blacks, but significantly greater trust than whites reporting to blacks. The results also indicated that whites reporting to blacks and blacks reporting to whites perceive significantly less trust than blacks reporting to blacks. A discussion of these results is also included.