Publication bias and the failure of replication in experimental psychology

Replication of empirical findings plays a fundamental role in science. Among experimental psychologists, successful replication enhances belief in a finding, while a failure to replicate is often interpreted to mean that one of the experiments is flawed. This view is wrong. Because experimental psychology uses statistics, empirical findings should appear with predictable probabilities. In a misguided effort to demonstrate successful replication of empirical findings and avoid failures to replicate, experimental psychologists sometimes report too many positive results. Rather than strengthen confidence in an effect, too much successful replication actually indicates publication bias, which invalidates entire sets of experimental findings. Researchers cannot judge the validity of a set of biased experiments because the experiment set may consist entirely of type I errors. This article shows how an investigation of the effect sizes from reported experiments can test for publication bias by looking for too much successful replication. Simulated experiments demonstrate that the publication bias test is able to discriminate biased experiment sets from unbiased experiment sets, but it is conservative about reporting bias. The test is then applied to several studies of prominent phenomena that highlight how publication bias contaminates some findings in experimental psychology. Additional simulated experiments demonstrate that using Bayesian methods of data analysis can reduce (and in some cases, eliminate) the occurrence of publication bias. Such methods should be part of a systematic process to remove publication bias from experimental psychology and reinstate the important role of replication as a final arbiter of scientific findings.     

Prior Publication and Redundancy in Contemporary Science: Are Authors and Editors at the Crossroads?

We discuss prior publication and redundancy in contemporary science in the context of changing perceptions of originality in the communication of research results. These perceptions have been changing in the publication realm, particularly in the last 15 years. Presenting a brief overview of the literature, we address some of the conflicts that are likely to arise between authors and editors. We illustrate our approach with conference presentations that are later published as journal articles and focus on a recent retraction of an article that had been previously published as a conference proceedings. Although we do not make definitive pronouncements on the matter—as many concepts are evolving—we do argue that conference papers that contain sufficient details for others to attempt a replication and are indexed in scientific databases such as PubMed, challenge some currently held assumptions of prior publication and originality in the sciences. Our view is that these important issues are in need of further clarification and harmonization within the science publishing community. This need is more evident when we consider current notions of research integrity when it comes to communication to peers. Revisiting long-standing views about what constitutes prior publication and developing a clearer set of guidelines for authors and editors to follow should reduce conflicts in the research environment, which already exerts considerable pressure, especially on newcomers in academia. However, while clearer guidelines are timely, developing them is only part of the challenge. The present times seem to call for deeper changes in the research and publication systems.     

Being there on time: Reinforcer effects on timing and locating

In research on timing, reinforcers often are assumed to influence discrimination of elapsed time. We asked whether changes in choice used to measure timing arise because of joint control by elapsed time and reinforcers, rather than from the direct modification of control by elapsed time by reinforcers. Pigeons worked on a concurrent-choice task in which 1 response was 9 times more likely to produce a reinforcer, reversing between locations when 19 s had elapsed since the marker event. Across conditions, we manipulated the percentage of reinforcers arranged before the probability reversal from 5 to 95%. These changes in reinforcer percentages altered control by location-based elements of the contingency, but not by time-based elements. Choice was well described by a model that assumes that control by the contingency is weakened by generalization across the time and location of reinforcers, and that these generalizations become more likely at later times since a marker. These findings add to a growing body of research that suggests that reinforcers share the same function as other environmental events in determining how the environment controls behavior.     

Comparing the reinforcing efficacy of tokens and primary reinforcers

Token economies are one of the most commonly used behavior-analytic interventions. Despite literature supporting the use of tokens as tools for behavior change, little is known about the efficacy of tokens compared to that of the items for which they are exchanged. Results of previous research comparing the reinforcing efficacy of tokens and primary reinforcers have shown that both produce similar effects on responding. However, published findings have been confounded given the inclusion of primary reinforcers in the token-reinforcer test conditions. In this study, we established novel tokens as reinforcers. We then conducted a conditioned-reinforcer assessment using a tandem control to ensure that the tokens functioned as reinforcers. We used progressive-ratio schedules to compare the reinforcing efficacy of the tokens to high- and low-preference edibles that were also used as backup reinforcers. For both participants, we found that high-preference primary reinforcers maintained higher response frequencies than did tokens.     

Science and Family Therapy

Family therapists are being exhorted to adopt a social constructionist epistemology and to abandon science as a valid form of knowing. Epistemological objections to science are usually directed at 'positivism', an outmoded view that is largely discredited among practising scientists. Contemporary science, is rather, constructionist, differing from other constuctionist approaches only in ways of arbitrating between world views. Criticisms aimed at science apply to all forms of discourse, including family therapy. I conclude that science has failed family therapy at the practical, not the philosophical level.     

The Philosophical Use and Misuse of Science

Science is our best way of finding out about the natural world, and philosophers who write about that world ought to be sensitive to the claims of our best science. There are obstacles, however, to outsiders using science well. We think philosophers are prone to misuse science: to give undue weight to results that are untested; to highlight favorable and ignore unfavorable data; to give illegitimate weight to the authority of science; to leap from scientific premises to philosophical conclusions without spelling out their relevance; to treat mere resonance between a scientific theory and a philosophical view as empirical evidence for the philosophical view. This article identifies and illustrates some of the ways in which philosophers misuse science, explains why these pitfalls are easy to fall into, and concludes with suggestions for avoiding them.     

Arguing for Wisdom in the University: An Intellectual Autobiography

For 40?years I have argued that we urgently need to bring about a revolution in academia so that the basic task becomes to seek and promote wisdom. How did I come to argue for such a vast, wildly ambitious intellectual revolution? It goes back to my childhood. From an early age, I desired passionately to understand the physical universe. Then, around adolescence, my passion became to understand the heart and soul of people via the novel. But I never discovered how to tell stories in order to tell the truth. So, having failed to become a physicist, and failed to become a novelist, I studied philosophy at Manchester University and then, in 6?weeks of inspiration, discovered that the riddle of the universe is the riddle of our desires. Philosophy should be about how to live, and should not just do conceptual analysis. I struggled to reconcile the two worlds of my childhood ambitions, the physical universe and the human world. I decided they could be reconciled with one another if one regarded the two accounts of them, physics and common sense, as myths, and not as literal truths. But then I discovered Karl Popper: truth is too important to be discarded. I revised my ideas: physics seeks to depict truly only an aspect of all that there is; in addition, there is the experiential aspect of things??the world as we experience it. I was immensely impressed with Popper??s view that science makes progress, not by verification, but by ferocious attempted falsification of theories. I was impressed, too, with his generalization of this view to form critical rationalism. Then it dawned on me: Popper??s view of science is untenable because it misrepresents the basic aim of science. This is not truth as such; rather it is explanatory truth??truth presupposed to be unified or physically comprehensible. We need, I realized, a new conception of science, called by me aim-oriented empiricism, which acknowledges the real, problematic aims of science, and seeks to improve them. Then, treading along a path parallel to Popper??s, I realized that aim-oriented empiricism can be generalized to form a new conception of rationality, aim-oriented rationality, with implications for all that we do. This led on to a new conception of academic inquiry. From the Enlightenment we have inherited the view that academia, in order to help promote human welfare, must first acquire knowledge. But this is profoundly and damagingly irrational. If academia really does seek to help promote human welfare, then its primary tasks must be to articulate problems of living, and propose and critically assess possible solutions??possible actions, policies, political programmes, philosophies of life. The pursuit of knowledge is secondary. Academia needs to promote cooperatively rational problem solving in the social world, and needs to help humanity improve individual and institutional aims by exploiting aim-oriented rationality, arrived at by generalizing the real progress-achieving methods of science. We might, as a result, get into life some of the progressive success that is such a marked feature of science. Thus began my campaign to promote awareness of the urgent need for a new kind of academic inquiry rationally devoted to helping humanity create a wiser world.     

Too good to be true: Publication bias in two prominent studies from experimental psychology

Empirical replication has long been considered the final arbiter of phenomena in science, but replication is undermined when there is evidence for publication bias. Evidence for publication bias in a set of experiments can be found when the observed number of rejections of the null hypothesis exceeds the expected number of rejections. Application of this test reveals evidence of publication bias in two prominent investigations from experimental psychology that have purported to reveal evidence of extrasensory perception and to indicate severe limitations of the scientific method. The presence of publication bias suggests that those investigations cannot be taken as proper scientific studies of such phenomena, because critical data are not available to the field. Publication bias could partly be avoided if experimental psychologists started using Bayesian data analysis techniques.     

Positive behavioral contrast when pigeons press treadles during multiple schedules

Pigeons were placed on multiple variable-interval 15-second variable-interval 15-second and on multiple variable-interval 15-second extinction schedules in which treadle presses produced food reinforcers. Positive behavioral contrast occurred. That is, rates of responding were higher during the variable-interval 15-second component when the other component was extinction than when it was another variable-interval 15-second schedule. These results contradict the findings of other studies, which failed to find positive contrast when pigeons pressed treadles for food reinforcers. They may also question the additive theories of behavioral contrast that predict that contrast should not occur in this situation.     

Justice,authority, and the world order

This paper defends the pertinence of global justice in the contemporary world. It accepts, for the sake of argument, Nagel's view that matters of justice arise only when political authority is asserted or exercised and, connectedly, his rejection of the cosmopolitan thesis. However, it challenges his conclusion that considerations of justice do not apply beyond the state. It argues that on any plausible account of the relationship between authority and justice international institutions, such as the World Trade Organisation, are now authoritative in the right way to justify their evaluation from the point of view of justice.     

Control by reinforcers across time and space: A review of recent choice research

Reinforcers affect behavior. A fundamental assumption has been that reinforcers strengthen the behavior they follow, and that this strengthening may be context‐specific (stimulus control). Less frequently discussed, but just as evident, is the observation that reinforcers have discriminative properties that also guide behavior. We review findings from recent research that approaches choice using nontraditional procedures, with a particular focus on how choice is affected by reinforcers, by time since reinforcers, and by recent sequences of reinforcers. We also discuss how conclusions about these results are impacted by the choice of measurement level and display. Clearly, reinforcers as traditionally considered are conditionally phylogenetically important to animals. However, their effects on behavior may be solely discriminative, and contingent reinforcers may not strengthen behavior. Rather, phylogenetically important stimuli constitute a part of a correlated compound stimulus context consisting of stimuli arising from the organism, from behavior, and from physiologically detected environmental stimuli. Thus, the three‐term contingency may be seen, along with organismic state, as a correlation of stimuli. We suggest that organisms may be seen as natural stimulus‐correlation detectors so that behavioral change affects the overall correlation and directs the organism toward currently appetitive goals and away from potential aversive goals. As a general conclusion, both historical and recent choice research supports the idea that stimulus control, not reinforcer control, may be fundamental.     

Capacities,explanation and the possibility of disunity

Nancy Cartwright argues that so-called capacities, not universal laws of nature, best explain the often complex way events actually unfold. On this view, science would represent a world that is fundamentally "dappled", or disunified, and not, as orthodoxy would perhaps have it, a world unified by universal laws of nature. I argue, first, that the problem Cartwright raises for laws of nature seems to arise for capacities too, so why reject laws of nature? Second, that in so far as there is a problem, it concerns the role of counterfactuals in explanation; I then briefly propose a simple model of counterfactual explanation. Finally, I investigate how a sophisticated version of the regularity theory of laws of nature (that of Ramsey-Lewis) can be neutral between the empirical hypotheses that the world is unified, and that the world is disunified.     

Behavioral contrast in rats with different reinforcers and different response topographies

Experiment I demonstrated positive behavioral contrast in rats when one of two qualitatively different reinforcers (milk and pellets) was removed from a component of a multiple schedule. The contrast effect was larger and more enduring when milk was removed. Experiment II showed that the rats spent more time on the side of a shuttle-box on which milk was freely available than on the side on which pellets were freely available. Experiment III, a partial replication of Experiment I, failed to demonstrate the contrast effect of Experiment I. Experiment IV demonstrated contrast when two topographically distinct responses, nose-key poking and lever pressing, were required in different components of a multiple schedule. These results extend the conditions that generate behavioral contrast in rats.     

Number tags applied by rats to reinforcers are general and exert powerful control over responding

Rats are capable of employing the number of successively presented food reinforcers in a runway as a discriminative cue. Trained, for example, under a series in which three food-reinforced trials precede a non-reinforced trial (RRRN), rats have run fast to R, R, and R, slowly to N. Such numerical discriminations have been shown to be general to some extent, being maintained in connection with either novel reinforcers suddenly introduced, or various alterations of the old series of reinforcers. In the two investigations reported here, the four-event series R'RRN was employed, R' and R being qualitatively different reinforcers. On being shifted to more severe alterations of this series than heretofore, the previously established numerical discrimination was maintained, indicating that the number tags rats apply to reinforcing events are to a considerable extent general. In Experiment 1 rats trained under the two series R'RRN and RRN maintained the numerical discrimination when shifted to the series R'RN and NR'RN, despite the fact that the specific events R'R that signalled R in original training (R'RRN series) now signalled N in the shift or transfer phase. In Experiment 2 rats trained under R'RRN and RRN maintained the numerical discrimination on being shifted to the series NRRRN and RRRN (count to three), despite the fact that they had previously received the RRN series (count to two). Various other findings obtained here are consistent with the view that the rat's number tags are to a considerable extent general.     

Pedagogy as a Framework for a Proper Dialogue between Science and Literature

An aim of science is to find truths about reality. These truths are collected together to form systematic knowledge structures called theories. Theories are intended to create a truthful picture of the reality behind the study. Together with all the other fields of science we get a (complete) scientific picture or a world view. This scientific world view is open in the sense that not all truths are known by scientists and not all present day theories are true. So, there is no reason to assume that any field of science has been or will be completed; science is essentially progressive, or an open ended approach. Science is not the only method of picturing, but the view that science is the best human method for knowledge acquisition is well justified. Knowledge is not all we humans are looking for; we need to make our lives meaningful. The meaningfulness may be achieved through science, art, or practical activity. It is interesting to compare literature and science. Both are expressed linguistically, but the aims of literature and science seem to be very different. We are looking for the theoretical interconnection between science and literature, but the proper dialogue can be found not through science, literature or philosophy, but through pedagogy.     

THE RELATIVE MOTIVATIONAL PROPERTIES OF SENSORY AND EDIBLE REINFORCERS IN TEACHING AUTISTIC CHILDREN

We compared the effects of sensory and edible reinforcers on resistance to satiation in three autistic children while learning visual discrimination tasks. Within-subject designs were used to compare a single sensory reinforcer with a single edible reinforcer and to compare multiple sensory reinforcers with multiple edibles. Results indicated that multiple sensory reinforcers maintained responding over more trials than did multiple edible reinforcers; however, the use of single sensory reinforcers and single edibles resulted in about equal numbers of trials to satiation. Both multiple and single sensory reinforcers produced higher percentages of correct responses than edible reinforcers. The findings are discussed in terms of the advantages of sensory reinforcers in teaching autistic children.     

Economic concepts for the analysis of behavior

A review of the relationship between schedule of reinforcement, response rate, and choice suggests that certain unifying concepts from economics can contribute to a more complete science of behavior. Four points are made: 1) a behavioral experiment is an economic system and its characteristics—open or closed—can strongly determine the results; 2) reinforcers can be distinguished by a functional property called elasticity; 3) reinforcers may interact as complements as well as substitutes; 4) no simple choice rule, such as strict matching, can account for all choice behavior.     

ISLAM AND ENVIRONMENTAL ETHICS: TRADITION RESPONDS TO CONTEMPORARY CHALLENGES

Abstract. Mounting globed environmental challenges beg for cross-cultural discussions that highlight underlying cultural values regarding nature. This paper explores the insights of Islamic scholars as they examine the interaction of Islamic culture and the West. The Western worldview that separates religion and science, value and fact, in particular differs from Islamic tradition, which sees all facets of life and affairs as interconnected by virtue of their common source—the Creator. As traditional Islamic values have been abandoned to adopt modern Western technologies, environmental problems have intensified in the Muslim world. Islamic scholars urge a return to Islamic ideals that reflect a sacramental view of the physical universe, and they champion the revival of an Islamic science that synthesizes empirical study and symbolic cognition.