Theory Testing Using Quantitative Predictions of Effect Size

Traditional Null Hypothesis Testing procedures are poorly adapted to theory testing. The methodology can mislead researchers in several ways, including: (a) a lack of power can result in an erroneous rejection of the theory; (b) the focus on directionality (ordinal tests) rather than more precise quantitative predictions limits the information gained; and (c) the misuse of probability values to indicate effect size. An alternative approach is proposed which involves employing the theory to generate explicit effect size predictions that are compared to the effect size estimates and related confidence intervals to test the theoretical predictions. This procedure is illustrated employing the Transtheoretical Model. Data from a sample ( N = 3,967) of smokers from a large New England HMO system were used to test the model. There were a total of 15 predictions evaluated, each involving the relation between Stage of Change and one of the other 15 Transtheoretical Model variables. For each variable, omega-squared and the related confidence interval were calculated and compared to the predicted effect sizes. Eleven of the 15 predictions were confirmed, providing support for the theoretical model. Quantitative predictions represent a much more direct, informative, and strong test of a theory than the traditional test of significance.     

The effects of flicker on eye movement control

Groups of typists with extensive experience of screen-based editing and groups of students with no such experience carried out a reading task under three conditions of illumination (50-Hz flicker, 100-Hz flicker, and steady illumination). Subjects read a sentence, which was followed by the presentation of a single stimulus word on the same line to the right-hand side of the display. The task was to decide whether or not the stimulus was present in the sentence. Subjects were free to re-inspect the sentence when making the decision. Eye movements were measured as subjects completed the task. In comparison with students, typists adopted a more cautious reading style, making more right-to-left saccades, shorter saccades, and more corrective eye movements. Flicker affected the performance of both groups of subjects in the first pass, leading to shorter saccades. In the second pass, its effect for students was to shorten the extent of large saccades made to check the presence of the stimulus word. In the group of typists, flicker led to an increase in the variability of saccade extent and a doubling in the number of small corrective saccades. The results are consistent with the view that flicker has two distinct effects on reading, both of which are potentially disruptive. The first relates to an increase in the number of prematurely triggered saccades, which are, as a result, less accurate. The second is an increase in the number of saccades perturbed in flight, which land short of their intended target. These two mechanisms may have different consequences for readers, depending on their reading style.