Bayesian t tests for accepting and rejecting the null hypothesis

Progress in science often comes from discovering invariances in relationships among variables; these invariances often correspond to null hypotheses. As is commonly known, it is not possible to state evidence for the null hypothesis in conventional significance testing. Here we highlight a Bayes factor alternative to the conventional t test that will allow researchers to express preference for either the null hypothesis or the alternative. The Bayes factor has a natural and straightforward interpretation, is based on reasonable assumptions, and has better properties than other methods of inference that have been advocated in the psychological literature. To facilitate use of the Bayes factor, we provide an easy-to-use, Web-based program that performs the necessary calculations.     

Accepting the null hypothesis

This article concerns acceptance of the null hypothesis that one variable has no effect on another. Despite frequent opinions to the contrary, this null hypothesis can be correct in some situations. Appropriate criteria for accepting the null hypothesis are (1) that the null hypothesis is possible; (2) that the results are consistent with the null hypothesis; and (3) that the experiment was a good effort to find an effect. These criteria are consistent with the meta-rules for psychology. The good-effort criterion is subjective, which is somewhat undesirable, but the alternative—never accepting the null hypothesis—is neither desirable nor practical.     

Constant latent odds-ratios models and the mantel-haenszel null hypothesis

In the present paper, a new family of item response theory (IRT) models for dichotomous item scores is proposed. Two basic assumptions define the most general model of this family. The first assumption is local independence of the item scores given a unidimensional latent trait. The second assumption is that the odds-ratios for all item-pairs are constant functions of the latent trait. Since the latter assumption is characteristic of the whole family, the models are called constant latent odds-ratios (CLORs) models. One nonparametric special case and three parametric special cases of the general CLORs model are shown to be generalizations of the one-parameter logistic Rasch model. For all CLORs models, the total score (the unweighted sum of the item scores) is shown to be a sufficient statistic for the latent trait. In addition, conditions under the general CLORs model are studied for the investigation of differential item functioning (DIF) by means of the Mantel-Haenszel procedure. This research was supported by the Dutch Organization for Scientific Research (NWO), grant number 400-20-026.     

The fallacy of the null hypothesis in soft psychology

In his classic article on the fallacy of the null hypothesis in soft psychology [J. Consult. Clin. Psychol. 46 (1978)], Paul Meehl claimed that, in nonexperimental settings, the probability of rejecting the null hypothesis of nil group differences in favor of a directional alternative was 0.50—a value that is an order of magnitude higher than the customary Type I error rate. In a series of real data simulations, using Minnesota Multiphasic Personality Inventory-Revised (MMPI-2) data collected from more than 80,000 individuals, I found strong support for Meehl’s claim.     

Two tests of the stuck-in-time hypothesis

The authors report 2 experiments that test the stuck-in-time hypothesis, which argues that animals cannot time-date events and thus do not remember when events occurred and do not anticipate future events. In Experiment 1, rats in the experimental condition could earn a large reward by reentering the 1st arm that they visited on a radial maze. They did not learn to reenter this arm early and did no better than did a control group that was not given a large reward for reentering the first arm. In Experiment 2, rats in the experimental group could earn a large reward by delaying entry into a distinctive arm. These rats did not learn to delay entry into the distinctive arm, and they performed no better than did the control-group rats, which did not receive a large reward for delayed entry. These experiments provide further evidence in support of the stuck-in-time hypothesis.     

Statistical requirements for properly investigating a null hypothesis

Issues involved in the evaluation of null hypotheses are discussed. The use of equivalence testing is recommended as a possible alternative to the use of simple t or F tests for evaluating a null hypothesis. When statistical power is low and larger sample sizes are not available or practical, consideration should be given to using one-tailed tests or less conservative levels for determining criterion levels of statistical significance. Effect sizes should always be reported along with significance levels, as both are needed to understand results of research. Probabilities alone are not enough and are especially problematic for very large or very small samples. Pre-existing group differences should be tested and properly accounted for when comparing independent groups on dependent variables. If confirmation of a null hypothesis is expected, potential suppressor variables should be considered. If different methods are used to select the samples to be compared, controls for social desirability bias should be implemented. When researchers deviate from these standards or appear to assume that such standards are unimportant or irrelevant, their results should be deemed less credible than when such standards are maintained and followed. Several examples of recent violations of such standards in family social science, comparing gay, lesbian, bisexual, and transgender families with heterosexual families, are provided. Regardless of their political values or expectations, researchers should strive to test null hypotheses rigorously, in accordance with the best professional standards.     

Multiplicity,directional (type III) errors,and the null hypothesis

L. V. Jones and J. W. Tukey (2000) pointed out that the usual 2-sided, equal-tails null hypothesis test at level alpha can be reinterpreted as simultaneous tests of 2 directional inequality hypotheses, each at level alpha/2, and that the maximum probability of a Type I error is alpha/2 if the truth of the null hypothesis is considered impossible. This article points out that in multiple testing with familywise error rate controlled at alpha, the directional error rate (assuming all null hypotheses are false) is greater than alpha/2 and can be arbitrarily close to alpha. Single-step, step-down, and step-up procedures are analyzed, and other error rates, including the false discovery rate, are discussed. Implications for confidence interval estimation and hypothesis testing practices are considered.     

Sex comparisons on causal attributions: Another look at the null hypothesis

Eighty-two subjects (49 women, 33 men) solved a difficult or easy set of anagrams, after which they indicated their subjective level of performance, provided causal attributions, and stated expectations for future performance. Sex comparisons were conducted on each element in the attribution model. Results indicated no sex differences on subjective evaluation of success, on causal dimensions, on expectations for future performance, or on performance on a final set of anagrams.     

SNOOP: A program for demonstrating the consequences of premature and repeated null hypothesis testing

The ease with which data can be collected and analyzed via personal computer makes it potentially attractive to “peek” at the data before a target sample size is achieved. This tactic might seem appealing because data collection could be stopped early, which would save valuable resources, if a peek revealed a significant effect. Unfortunately, such data snooping comes with a cost. When the null hypothesis is true, the Type I error rate is inflated, sometimes quite substantially. If the null hypothesis is false, premature significance testing leads to inflated estimates of power and effect size. This program provides simulation results for a wide variety of premature and repeated null hypothesis testing scenarios. It gives researchers the ability to know in advance the consequences of data peeking so that appropriate corrective action can be taken.     

Mechanisms of eyewitness suggestibility: tests of the explanatory role hypothesis

In a recent paper, Chrobak and Zaragoza (Journal of Experimental Psychology: General, 142(3), 827–844, 2013) proposed the explanatory role hypothesis, which posits that the likelihood of developing false memories for post-event suggestions is a function of the explanatory function the suggestion serves. In support of this hypothesis, they provided evidence that participant-witnesses were especially likely to develop false memories for their forced fabrications when their fabrications helped to explain outcomes they had witnessed. In three experiments, we test the generality of the explanatory role hypothesis as a mechanism of eyewitness suggestibility by assessing whether this hypothesis can predict suggestibility errors in (a) situations where the post-event suggestions are provided by the experimenter (as opposed to fabricated by the participant), and (b) across a variety of memory measures and measures of recollective experience. In support of the explanatory role hypothesis, participants were more likely to subsequently freely report (E1) and recollect the suggestions as part of the witnessed event (E2, source test) when the post-event suggestion helped to provide a causal explanation for a witnessed outcome than when it did not serve this explanatory role. Participants were also less likely to recollect the suggestions as part of the witnessed event (on measures of subjective experience) when their explanatory strength had been reduced by the presence of an alternative explanation that could explain the same outcome (E3, source test + warning). Collectively, the results provide strong evidence that the search for explanatory coherence influences people’s tendency to misremember witnessing events that were only suggested to them.     

The generation effect: Further tests of the lexical activation hypothesis

Three experiments compared recognition memory for word versus nonword responses when they had been either read or generated using a rhyme rule and either a word or nonword stimulus. That is, either the wordshop or the nonwordthop was generated from either the wordchop or the nonwordphop. In Experiment 1, the lexicality of the stimulus and response terms was manipulated orthogonally between subjects; in Experiments 2 and 3, it was manipulated orthogonally within subjects. In Experiment 3, subjects also made a lexical (word-nonword) decision about each response term after it had been read or generated. In all three experiments, memory performance was better for generated than read responses. This generation effect occurred only if the response term was a word, regardless of whether the stimulus term was a word or a nonword. These results are discussed in terms of the roles that lexical memory and response unitization play in the generation effect.     

The semantic congruity effect: Some tests of the expectancy hypothesis

Subjects comparing items in memory along some dimension are usually quicker to specify the lesser (than the greater) of two low magnitude items and the greater (than the lesser) of two high magnitude ones. One account explains this congruity effect as due to subjects instructed to specify the higher as expecting high magnitude items to follow and the reverse being true for subjects specifying the lesser. Three experiments tested this expectancy hypothesis. In experiment 1, subjects were set to the actual size range of each pair before the pair was shown but the congruity effect still occurred. In experiments 2 and 3, subjects compared critical pairs from a narrow size range plus more from either the same or much broader ranges. Times to compare the critical pairs were the same regardless of the range of the other pairs that subjects were exposed to. These results are strong evidence against the expectancy hypothesis.     

Infants perceiving and acting on the eyes: tests of an evolutionary hypothesis

It has been hypothesized that an evolutionarily ancient mechanism underlies the ability of human infants to detect and act upon the direction of eye gaze of another human face. However, the evidence from behavioral studies with infants is also consistent with a more domain-general system responsive to the lateral motion of stimuli regardless of whether or not eyes are involved. To address this issue three experiments with 4-month-old infants are reported that utilize a standard face-cueing paradigm. In the first experiment an inverted face was used to investigate whether the motion of the pupils elicits the cueing effect regardless of the surrounding face context. In the second experiment pupil motion and eye gaze direction were opposed, allowing us to assess their relative importance. In a third experiment, a more complex gaze shift sequence allowed us to analyse the importance of beginning with a period of mutual gaze. Overall, the results were consistent with the importance of the perceived direction of motion of pupils. However, to be effective in cueing spatial locations this motion needs to be preceded by a period of direct mutual gaze (eye contact). We suggest that evolution results in information-processing biases that shape and constrain the outcome of individual development to eventually result in adult adaptive specializations.     

Effects of mood on memory: Experimental tests of a mood-state-dependent retrieval hypothesis and of a mood-congruity hypothesis

Summary In the present experiment a mood-state-dependent retrieval hypothesis and a mood-congruity hypothesis are tested. The first hypothesis states that recall is higher when mood at test matches mood at input than when it does not match. The second hypothesis states that people learn more about events that match their mood state. Both hypotheses are consistent with the network model of mood and memory proposed by Bower and his colleagues. Our study was conducted to test the cited hypotheses with normal subjects (instead of highly hypnotizable subjects as in the studies by Bower et al.) and with a comparatively mild mood-induction technique. Another aim was to test different predictions from the two hypotheses.We experimentally varied mood (happy vs. sad), both at learning and at recall by using the Velten technique. Additionally, the emotional content (pleasant vs. neutral vs. unpleasant) of the text to be learned was manipulated.The empirical evidence with some restrictions favored the mood-state-dependent hypothesis and disagreed with the mood-congruity hypothesis. Possible explanations for the failure to obtain a mood-congruity effect are offered, and the results are discussed with reference to a network model of mood and memory.