RULES OF EVIDENCE:

Abstract— Statistical tests define rules of evidence in psychological science Agreed-upon rules of evidence are needed to discuss what is true Statistics are probability statements about the reliability of associations In public and journalistic debates about psychological topics, the need far statistical rules is evident Anecdotes, intuitions, and sensational and unlikely events often dominate popular discussions Psychological science demands that the probability of events and associations be evaluated by statistical means Understanding of statistical tests can be improved by abandoning the term significance, winch means reliability to psychologists, but in everyday English, means importance. The importance of a result depends on magnitude of association, not on whether the event has a nonzero chance of recurring. Given large samples and multiple measures, statistical tests are used too often to draw misleading conclusions about the importance of research results. Better uses of statistics would focus on the magnitude of effects and error estimates     

Lack of early child care effects on school-age children's social competence and academic achievement

Teacher ratings of social competence and academic achievement were obtained from a sample of 127 Bermudian children at ages 5, 6, 7 and 8 years. The children were studied first during their pre-school years, when they had been exposed to various amounts and qualities of day care. Quality of care was found to be important to the children's intellectual and social development while they were in the day care settings but not after 1–4 years of primary schooling. In hierarchical and simultaneous regressions, family background characteristics, not child care amounts or qualities, were found to be predictive of social competence and academic achievement in the primary grades. By school age, the effects of infant and preschool child care experiences were no longer influential in children's development, but family background continued to be important. These findings and their implications for child care are discussed.     

Children's memory of events: effects of stress,age, time delay and location of interview

To examine the ability of children aged between 4 and 12 years to recall a stressful event (venipuncture) compared with a non-stressful event (demonstration of venipuncture), recall was tested after 6–8 weeks. Half also had recall tested after 2–7 days. Testing took place where the stressful event occurred (n=122) or at a neutral location (n=127). Children who experienced the stressful event were less likely to give inaccurate responses in free recall or to acquiesce to suggestive misleading questions. Apart from incorrect responses in free recall, correct responses increased and incorrect responses decreased with increasing age. Recall after 2–7 days was superior to recall after 6–8 weeks. Those who had an early and a late interview had better recall at the late interview than those who had a late interview only. The location of interview showed no effects on recall. © 1998 John Wiley & Sons, Ltd.     

Latent Trait Modeling with the Common beliefs Survey III: Using Item Response Theory to Evaluate an Irrational Beliefs Inventory

Assessment of irrational beliefs by such measures as the Common Beliefs Survey III (CBS) has traditionally relied upon classical test theory assumptions, in which the properties of specific test items are less important than the total test score as the aggregate of all item responses. An alternative approach using item response theory (IRT) methodology allows one to specify the parameters of difficulty and discrimination for each test item. Difficulty levels of CBS items range along a continuum of irrationality, the implied latent trait measured by responses to the questionnaire as a whole. We evaluated the CBS responses of 605 individuals from clinical and college settings, drawing from current and archival data. The original Likert scale ratings were recoded into dichotomous scores. Fourteen of the 54 items were highly or very highly discriminating in distinguishing respondents with high and low irrationality levels. However, discriminating items exhibited a very narrow range of difficulty; most functioned at a point a little above the halfway mark on the continuum of irrationality. Item characteristic curves and test information curves were very similar for female (n = 424) and male (n = 179) respondents. We derived a 4-item screening test for irrationality from our IRT analyses of the 54 CBS items. Further test development, focused on the selection and scaling of items with a much broader range of difficulty, would facilitate evaluation of the hierarchical structure of irrational beliefs. Portions of this paper were presented at the 39th Annual Convention of the Association for Behavioral and Cognitive Therapies, Washington, DC, November, 2005.     

Effects of setting and number of observations on generalizability of parent-child interactions in childhood obesity treatment

The generalizability of behaviors across observational conditions is a critical issue in behavioral assessment. Generalizability theory was used to examine two aspects of audio recorded parent-child interactions recorded over 6 days of home measurement and 1 day of laboratory measurement in a behavioral treatment program for childhood obesity. Families audiotaped parent-child home meetings during which they reviewed self-monitored diet and exercise records that were coded for the following types of interactions: praise statements, negative statements, prompts for new behaviors, and statements promoting problem solving. A similar meeting was audiotaped in our laboratory. The first question explored was the number of measurements needed to generalize to the universe of the six home measures. Results showed an increase in generalizability over measurements for each behavioral category. Using generalizability coefficients of .60 or more, praise, negative comments and prompts, respectively, could be reliably observed based on 1, 4, or 4 days of measurement. Second, the effects of setting (laboratory versus home) were assessed for 1 day of measurement in each environment. Again using generalizability coefficients of .60, generalizability analysis showed that the lab setting could not be generalized to the home setting based on 1 day of measurement, with generalizability coefficients ranging from .27 for negative comments to .57 for praise. Results suggest that 4 days of behavioral assessment in the home can be used to establish generalizable data for all the dependent measures studied. However, generalizability coefficients suggested that 1 day of laboratory measurement was not adequate to generalize to typical home behavior.This research was supported in part by Grant NIH HD 23713 awarded to the third author.