Observer agreement, credibility, and judgment: some considerations in presenting observer agreement data

Graphical and statistical indices employed to represent observer agreement in interval recording are described as "judgmental aids", stimuli to which the researcher and scientific community must respond when viewing observer agreement data. The advantages and limitations of plotting calibrating observer agreement data and reporting conventional statistical aids are discussed in the context of their utility for researchers and research consumers of applied behavior analysis. It is argued that plotting calibrating observer data is a useful supplement to statistical aids for researchers but is of only limited utility for research consumers. Alternatives to conventional per cent agreement statistics for research consumers include reporting special agreement estimates (e.g., per cent occurrence agreement and nonoccurrence agreement) and correlational statistics (e.g., Kappa and Phi).     

Barking up the wrong tree? Industry funding of academic research

This case raises ethical issues involving conflicts of interest arising from industrial funding of academic research; ethical responsibilities of laboratories to funding agencies; ethical responsibilities in the management of a research lab; ethical considerations in appropriate research design; communication in a research group; communication between advisor and graduate student; responsibilities of researchers for the environment; misrepresentation or withholding of scientific results.     

Barking up the wrong tree? Industry funding of academic research. A case study with commentaries

This case raises ethical issues involving conflicts of interest arising from industrial funding of academic research; ethical responsibilities of laboratories to funding agencies; ethical responsibilities in the management of a research lab; ethical considerations in appropriate research design; communication in a research group; communication between advisor and graduate student; responsibilities of researchers for the environment; misrepresentation or withholding of scientific results.     

Tolerance of ambiguity, information, and negotiation.

This study examines the relationship between the tolerance of ambiguity and the outcomes of negotiation and the misrepresentation of information. The data were obtained from 98 middle managers of a bank in Turkey. A positive correlation of .74 (p < .01) obtained between scores on tolerance of ambiguity with the outcomes of negotiation and a negative correlation of -.66 (p < .01) with misrepresentation of information.     

Some dubious premises in research and theory on racial differences. Scientific, social, and ethical issues

The scientific premises for looking for statistical differences between groups designated as races (on somewhat arbitrary grounds) are questionable. The explanation of such differences in strictly biological-evolutionary terms is even more dubious. Studies of temperament, basic personality traits, disorders (such as antisocial personality), and specific genetic markers show that there is much more variation within groups designated as races than between such groups. Investigators and theoreticians interpreting such differences on the basis of limited sampling within the three broad racial groups should be careful to avoid selectivity and misrepresentation of data that serve racist ideology, and should be cautious about presenting their theories to the public through inappropriate media forums.     

Family therapy outcome research: a trial for families, therapists, and researchers

This article describes the process of an evolving research project. Initially conceived as a study investigating outcome measures and their sensitivity to change after a course of family therapy, the project soon changed its focus. As unexpected results were recorded, the clinical research team became destabilized and the individual team members responded by making their own "sense" of the data, reflecting their respective clinical and scientific positions. As clinicians and researchers began to challenge each other's belief systems, the project entered a new stage. The interactions within the team became of increasing interest and themselves objects of research. The recursive nature of re-search was demonstrated, and the act of writing this report completed the circle, as the various authors tried to achieve a balance between reporting the content and the process of this project.     

Speak No Evil: Scientists,Responsibility, and the Public Understanding of Science

In this paper, I will discuss the responsibilities that scientists have for ensuring their work is interpreted correctly. I will argue that there are three good reasons for scientists to work to ensure the appropriate communication of their findings. First, I will argue that scientists have a general obligation to ensure scientific research is communicated properly based on the vulnerability of others to the misrepresentation of their work. Second, I will argue that scientists have a special obligation to do so because of the power we as a society invest in them as specialists and professionals. Finally, I will argue that scientists ought to ensure their work is interpreted correctly based on prudential, self-interested considerations. I will conclude by offering suggestions regarding policy considerations.     

Of whistleblowers, investigators, and judges

Conclusion: Deliberate misrepresentation continues. Needleman attacks the whistleblowers, the NIH, and his own University to deflect attention from the indications of misconduct in his research. Federal agencies base policies on his research and on his assertions regarding the purported effects of low-level lead exposure on children. The policies divert resources and attention from the real needs of children. Misconduct issues will ultimately be resolved by the ORI and the courts. Public policy issues may require reconsideration in light of further evidence.     

Conceptualizing Religious Practices in Psychological Research: Problems and Prospects

This article addresses the problems and prospects that follow from the conceptualization of religious phenomena and practices for scientific investigation in the psychology of religion. Two Western research traditions—instrumentalism and operationalism—are described and their potential contribution to a mismatch between what researchers intend to study and what they actually study is illustrated through two exemplar studies. The exemplar studies show how researchers’ concern with methodological rigor can compromise the rich and thick meanings of religious practices, resulting in the misrepresentation of the practices and misleading both the psychological and religious research consumer. Several suggestions for dealing with these problems are discussed.     

Mediation analysis with structural equation models: Combining theory,design, and statistics

Statistical tests of indirect effects can hardly distinguish between genuine and spurious mediation effects. The present research demonstrates, however, that mediation analysis can be improved by combining a significance test of the indirect effect with assessing the fit of causal models. Testing only the indirect effect can be misleading, because significant results may also be obtained when the underlying causal model is different from the mediation model. We use simulated data to demonstrate that additionally assessing the fit of causal models with structural equation models can be used to exclude subsets of models that are incompatible with the observed data. The results suggest that combining structural equation modeling with appropriate research design and theoretically stringent mediation analysis can improve scientific insights. Finally, we discuss limitations of the structural equation modeling approach, and we emphasize the importance of non‐statistical methods for scientific discovery.     

Editors’ Overview: Topics in the Responsible Management of Research Data

Responsible data management is a multifaceted topic involving standards within the research community regarding research design and the sharing of data as well as the collection, selection, analysis and interpretation of data. Transparency in the manipulation of images is increasingly important in order to avoid misrepresentation of research findings, and research oversight is also critical in helping to assure the integrity of the research process. Intellectual property issues both unite and divide academe and industry in their approaches to data management. Central to the realization and promulgation of responsible data management is clear and careful communication of standards and expectations within the research community to trainees as well as among colleagues. These topics are examined and explored in depth in a special issue of Science and Engineering Ethics on responsible data management.     

Reporting Ethics Committee Approval in Public Administration Research

While public administration research is thriving because of increased attention to social scientific rigor, lingering problems of methods and ethics remain. This article investigates the reporting of ethics approval within public administration publications. Beginning with an overview of ethics requirements regarding research with human participants, I turn to an examination of human participants protections for public administration research. Next, I present the findings of my analysis of articles published in the top five public administration journals over the period from 2000 to 2012, noting the incidences of ethics approval reporting as well as funding reporting. In explicating the importance of ethics reporting for public administration research, as it relates to replication, reputation, and vulnerable populations, I conclude with recommendations for increasing ethics approval reporting in public administration research.     

Participant Protection With the Use of Records: Ethical issues and recommendations

Fraudulent analysis and reporting of psychological data have the potential to contaminate the scientific knowledge base and eventuate in the unjustified expenditure of public money and scientific effort (Koocher &; Keith-Spiegel, 1998). Traditionally, the field has relied on quantitative methodologists to educate researchers in proper analysis and reporting practices, and to examine these via peer review. The field has also relied on psychologists with training or board service in ethics to establish standards and implement strategies to discourage misconduct. However, this division of responsibility for examination, standard setting, and deterrence is shown to compromise the effectiveness of methodologists' and ethicists' respective gatekeeping efforts. Methodologically and ethically trained specialists instead need to coordinate efforts to safeguard analysis and reporting procedures. Researchers also need to increase self-monitoring. Potential obstacles to achieving these ends are considered, and three tactics are proposed to overcome them.     

Exploratory factor analysis in transportation research: Current practices and recommendations

Exploratory factor analysis (EFA) is a widely used statistical method in traffic and transportation research, particularly for the development and validation of measurement instruments. This article critically examines current practices in conducting and reporting EFA in published transportation studies. One hundred and eighty papers published between 2016 and 2018 were examined, of which eighty-two were included in the present study after applying eligibility criteria. The review suggests that the quality of EFA reported in the field is routinely poor: (a) researchers fail to provide sufficient information to be able to adequately assess the appropriateness and quality of both the input data and the reported output; and (b) the decisions underlying the choices of EFA methods are not justified and rely mostly on procedures advised against, particularly the Little-Jiffy approach. In summary, a significant gap between current practice and experts' recommendations exists. We provide some guidelines that may help in conducting, reporting and reviewing EFA in transportation research.     

Sleep quality, sleep propensity and academic performance

We examined associations between measures of sleep propensity on the Epworth Sleepiness Scale, sleep quality on the Pittsburgh Sleep Quality Index and academic performance by GPA and grades in introductory psychology for 414 students. In the total sample, neither sleep propensity nor sleep quality correlated with GPA or introductory psychology grades. However, among students carrying a full course load, those reporting poor sleep quality performed less well on academic measures than those reporting a better quality of sleep. Further research is needed to assess the moderating influence of overall demands of daytime functioning on the association between sleep quality and academic performance.