ANCOVA Versus CHANGE From Baseline in Nonrandomized Studies: The Difference

The pretest-posttest control group design can be analyzed with the posttest as dependent variable and the pretest as covariate (ANCOVA) or with the difference between posttest and pretest as dependent variable (CHANGE). These 2 methods can give contradictory results if groups differ at pretest, a phenomenon that is known as Lord's paradox. Literature claims that ANCOVA is preferable if treatment assignment is based on randomization or on the pretest and questionable for preexisting groups. Some literature suggests that Lord's paradox has to do with measurement error in the pretest. This article shows two new things: First, the claims are confirmed by proving the mathematical equivalence of ANCOVA to a repeated measures model without group effect at pretest. Second, correction for measurement error in the pretest is shown to lead back to ANCOVA or to CHANGE, depending on the assumed absence or presence of a true group difference at pretest. These two new theoretical results are illustrated with multilevel (mixed) regression and structural equation modeling of data from two studies.     

Help for the Significant Others of Bulimics

A quasi-experimental design was used to assess the effects of a 5-week assistance program for the significant others of bulimics. A knowledge test, an attitude survey, and a behavior device were used to evaluate the effects of the program. One or more of the three instruments was administered to an experimental group of significant others before (pretest) and after (posttest) they attended the assistance program. One or more of the three instruments was also administered to a control group of significant others before (pretest) and after (posttest) a 5-week period. During this period they did not attend the assistance program. No significant difference was found between the pretest and posttest scores of the control group on the three instruments at the .05 probability level. Experimental group posttest scores were significantly higher than their pretest scores on the three instruments at the .05 probability level. An anonymous evaluation conducted following the program also indicated significant others who attended the program found it to be beneficial.     

Improving the measurement of spatial aptitude by dynamic testing

Can spatial aptitude scores be increased by test-related training? Is construct validity influenced? The traditional psychometric view of test-related training as coaching, which threatens test validity, contrasts sharply with the view that performance modifiability measures a theoretically important construct that increases test validity (e.g., Vygotsky, 1978). This article reports on three studies of the dynamic testing of spatial ability in which training on the physical analogue of the mental folding task intervenes between a pretest and a posttest. Experiment 1 found that spatial ability can be significantly and substantially increased. Experiment 2 examined the impact of the dynamic testing procedure on the relationship of ability scores to other measures. Although sex differences in spatial ability were unchanged, the dynamic testing procedure increased predictive validity for training in text editing. Experiment 3 examined the impact on construct representation. The mathematical models of task difficulty indicated that the posttest more clearly represented rotational processing while the psychometric models supported the measurement of two abilities (i.e., initial status and gain). The implications for measuring learning ability by dynamic testing are discussed.     

Reliability and validity of gain scores considered graphically

The ordinary gain score, g, is defined as g = x2-x1, where x1 is the pretest score and x2 is the posttest score. The present study extends and refines previous research on the reliability and validity of gain scores. Using particular values as stated in the tables and graphs, the pre- and posttest reliabilities, pre- and posttest validities, ratios of pretest to posttest standard deviations, and correlations between the pretest and posttest were varied systematically to examine the effects of these parameter configurations on gain scores' reliability and validity. Results plotted graphically provide insight via visual interpretation not easily inferred using only values from a table. One interesting finding was that the reliability of a gain score can be at a maximum when the validity is at a minimum. Another is that a high correlation between pre- and posttest was beneficial to the validity of the gain score but detrimental to its reliability. By identifying the situations in which gain scores can be reliable and valid, findings inform researchers when gain scores should or should not be used.     

The Effects of Weight Loss Attempts, Exercise Initiation, and Dietary Practices on Health Related Quality of Life

This study (N = 98) randomly placed participants in either a higher protein or American Heart Association diet group with each group initiating an aerobic exercise program. Body composition (BC) was measured using four different methodologies (DEXA, BIA, skinfold, BMI) along with height and weight, while Health-Related Quality of Life (HRQOL) changes were measured using the Medical Outcomes Short Form 36 (SF-36). There were no significant time/diet interactions in HRQOL scores, body fat measures, and weight with the exception of Role-Physical scores (p = .006) with posttest scores higher in participants in Diet 1. Role-Physical scores changed significantly in participants in Diet 1 (p = .0001) and Diet 2 (p = .026) with higher posttest scores in each group. Social functioning had a significant main effect for time in both groups with pretest scores in both diet groups lower than their respective posttest scores (p = .015). HRQOL pretest scores were not significantly different between groups at baseline (p = .56). ANOVA revealed no significant differences in average calories between groups (p = .88). Analysis revealed no differences in exercise in minutes (p = .66) nor statistically significant changes in BC or weight (p = .65). Conclusions: Improvements in HRQOL were demonstrated in two subscales related to physiological and psychological functioning. These novel findings add to literature suggesting that HRQOL, at least concerning social functioning, work and other daily activities (Role-Physical) may improve by initiating an exercise and dietary program even when weight loss or BC improvements have been unsuccessful.     

Considerations in the use of difference scores to identify learning-disabled children

Although difference scores are widely used in classifying children as learning-disabled, their psychometric properties are often not well understood. Such scores generally contain more error than single test scores. Reliability and standard error of measurement figures for several combinations of ability and achievement measures are presented. The rates and types of errors that occur when such scores are used to classify children as learning-disabled are discussed. Three recommendations for using difference scores are given: (a) combinations of ability and achievement tests that yield difference score reliabilities higher than .80 should be used when classifying children; (b) scores should be reported as a band of scores (± one standard error of measurement) to inform decision-makers regarding the amount of error estimated to be in the score, and (c) the criterion score for classifying the learning disabled should be set after consideration of the rate and types of errors likely to occur.     

Best univocal estimates of orthogonal common factors

A general formulation is presented for obtaining conditionally unbiased, univocal common-factor score estimates that have maximum validity for the true orthogonal factor scores. We note that although this expression is formally different from both Bartlett's formulation and Heermann's approximate expression, all three, while developed from very different rationales, yield identical results given that the common-factor model holds for the data. Although the true factor score validities can be raised by a different non-orthogonal transformation of orthogonalized regression estimates—as described by Mulaik—the resulting estimates lose their univocality.     

Instructor Feedback for College Writing Assignments in Introductory Classes

We compared three instructor feedback protocols to determine whether minimal instructor feedback could improve undergraduate students' writing skills in large introductory classes. The three feedback conditions were specific feedback, nonspecific feedback, and grade only. Sixty community college students participated in the study. Sections of an Introduction to Human Services course were randomly assigned to the three feedback conditions. Students wrote 9 timed essays during the semester. Pretest data were collected from the first essay at the beginning of the semester. Posttest data were taken from the final essay. Data for each student's essays consisted of the number of words written, the number of spelling errors, and the number of syntax grammar errors. Between the pretest and the posttest, students wrote 7 timed essays during class periods. Students in the specific feedback condition received a tally of spelling and grammar errors on grade criteria sheets and the location of the errors marked on their essays. In the nonspecific feedback condition, students received a tally of the errors on their grade criteria sheets but the locations of the errors were not marked on the essays. In the grade only condition, students received feedback on the number of points they had earned for each essay. Significant differences were found for pretest–posttest difference (gain) scores for the proportion of errors to the number of words written between the specific feedback condition and the grade only condition. Specific feedback decreased the proportion of errors to words written. The grade only condition maintained the same proportion of errors to words written from pretest to posttest. Significant differences were also found between difference (gain) scores for total errors between the specific feedback condition and the grade only condition and between the nonspecific feedback condition and the grade only condition.     

Effects of test-wiseness training on test anxiety,locus of control and reading achievement in elementary school children

Abstract

Fifth and sixth grade children were randomly assigned to either a control or an experimental condition in a pretest, posttest, delayed posttest design. The experimental group received instruction and practice in test-taking, and the control group received a placebo treatment. Dependent variables were four components of test anxiety, locus of control and achievement. Factor analysis of pretest data supported I. Sarason's four-factor conceptualization of test anxiety (worry, test-irrelevant thinking, tension, and bodily reactions to test taking). Posttest data generally supported the hypothesized treatment effects. As predicted, children in the experimental condition had higher reading scores, greater internality, and less test-irrelevant thinking. The delayed posttest scores indicated a diminished effect on the achievement and internality measures; however, the amount of test-irrelevant thinking remained lower in the experimental condition.     

Preschoolers' Acquisition of Scientific Vocabulary Through Repeated Read-Aloud Events,Retellings, and Hands-On Science Activities

This study explored 3- and 4-year-old children's development of scientific vocabulary from participation in repeated interactive read-aloud events and retellings of three informational picture books about light and color, followed by hands-on science activities. Thirty-two children attending a YWCA preschool were matched by age and general vocabulary knowledge and randomly assigned to a retelling or no retelling condition. Repeated measures ANOVA of pretest and posttest scores on a free recall target vocabulary test revealed significantly higher scores for 4-year-olds, those with higher vocabulary knowledge, and those who retold the books. Children used significantly more target words across the three retellings.     

The Efficacy of Computer-Assisted Instruction for Advancing Literacy Skills in Kindergarten Children

We examined the benefits of computer-assisted instruction (CAI) as a supplement to a phonics-based reading curriculum for kindergartners in an urban public school system. The CAI program provides systematic exercises in phonological awareness and letter–sound correspondences. Comparisons were made between children in classes receiving a sufficient amount of CAI support and children in matched classes taught by the same teacher but without CAI. The treatment and control groups did not differ on pretest measures of preliteracy skills. There were, however, significant differences between groups on posttest measures of phonological awareness skills particularly for students with the lowest pretest scores.     

The effect of counselling programmes involving expressive activities with semi‐structured groups on self‐concealment levels of adolescents

The aim of this study was to investigate the effects of counselling programmes based on expressive activities with semi‐structured groups, on the self‐concealment levels of adolescents. The research was carried out using a pretest–posttest control group design. The dependent variable of the present research, is the psychological counselling programme conducted with a semi‐structured group based on expressive activities. The research was carried out with 18 6th and 7th grade secondary school students (10 female and 8 male). The Self‐Concealment Scale was used as data collection tool. In this research, the use of nonparametric tests was preferred, as the sample was smaller than 30. Therefore, it was determined whether there is a statistically significant difference between the arithmetic means of the experimental and control groups’ pretest, posttest and trail test scores by use of Wilcoxon signed‐rank test, to determine the effects of a psychological counselling programme with semi‐structured group based on expressive activities. Also, it was determined whether there was a statistically significant relationship between the pretest, posttest and trail intervention scores received by the intervention and control group students from the Self‐Concealment Scale, using Mann–Whitney U test. The research results revealed that the “psychological counselling programme based on expressive activities with semi‐structured groups” was significantly effective in alleviation of self‐concealment levels, and this effect was also maintained in follow‐up evaluations. The obtained results are discussed in the light of related literature findings and recommendations are proposed accordingly.     

THE EFFECTS OF DIFFERENT REINFORCEMENT CONDITIONS ON THE TEST PERFORMANCE OF MULTIHANDICAPPED DEAF CHILDREN1

The present study explored the effects of different reinforcement conditions on the number of correct responses on the Raven Progressive Matrices. Four groups of 11- to 18-year-old multihandicapped deaf children matched on the basis of mean age and pretest scores were used. The groups were randomly assigned to any of four posttest conditions: end-of-session reinforcement, noncontingent reinforcement, delayed reinforcement, and immediate reinforcement. The mean posttest score of subjects tested under the immediate-reinforcement condition was significantly higher than that of any other group. No significant differences were observed between the mean posttest scores of the three other groups. The practical implications of using reinforcement procedures for testing purposes are discussed.     

Cognitive consequences of playing brain-training games in immersive virtual reality

The goal of the present study was to examine the effects of playing an immersive virtual reality game that included a collection of gamified cognitive tasks, Cerevrum, on specific components of cognition, including perceptual attention, mental rotation, working memory, visualization, visual field of view, and visual processing speed. Participants completed a pretest of cognitive assessments, played one of the two mini-games within Cerevrum (Stardust or Heroes) for 1.5 hr over three 30-min sessions and then completed a posttest of cognitive assessments and a questionnaire about interest and engagement during the game. An inactive control group completed only the pretest and posttest. Results showed no significant differences among the Heroes group, Stardust group, and control group on the posttest scores, even when controlled for pretest scores. These findings do not support the claim that playing brain-training games for a short period results in transfer of cognitive training to nongame venues.     

Worse than feared? Failure induction modulates the electrophysiological signature of error monitoring during subsequent learning

This study examined how self-relevant failure influences error monitoring—as reflected in the error-related negativity (Ne/ERN) —and behavioral adaptation during subsequent feedback-based learning. We applied two phases (pre- and posttest) of a probabilistic learning task. Between pre- and posttest, participants were assigned to one of two groups receiving either failure feedback or no feedback during a visual search task described as diagnostic of intellectual abilities. To disentangle the effects of failure and motivational disengagement due to prolonged task performance, we linked the posttest to intelligence (Experiment 1) or described it in neutral terms (Experiment 2). Failure induction was associated with an increase in Ne/ERN amplitude at posttest in both experiments, although there were no differences in overall performance. In contrast, the Ne/ERN decreased from pre- to posttest in the no-failure-feedback group, particularly in Experiment 2. Furthermore, failure feedback affected error-related behavioral adjustments, suggesting a shift toward a reactive, error-driven mode of behavior control. These findings emphasize the importance of affective-motivational state in error processing and subsequent behavioral adaptation.     

The effects of intensive,extensive, structured study on GRE scores

Thirty-eight undergraduate students engaged in self-paced self-study using standard materials in one of five Graduate Record Examination (GRE) preparatory courses, lasting 5.5 to 7.5 weeks, offering 66 to 140 hours of study, and using attendance and participation incentives. Statistically and practically significant improvements were found from pretest to posttest verbal, quantitative, and combined scores. In one series of courses, the mean combined scores improved 96 points, increasing from 842 (pretest) to 938 (posttest). In the other, the mean improved 186 points, increasing from 888 to 1074. Earlier published research involved brief training (3 to 9 hours) and showed no benefits. The present research suggests that intensive, structured training can produce a mean improvement of at least 96 points, which is a socially significant accomplishment.     

Influence of discrimination awareness on the occupational interests of African American children

This study examined the influence of discrimination awareness on children's occupational interests. Participants included 46 African American children aged 10 to 13. Children completed pretest measures of perceptions of occupational racial discrimination, discrimination-related self-efficacy beliefs, occupational outcome expectations, and the perceived status of their aspired-to and expected occupations. Participants were then randomly assigned to a bias resistance condition or a control condition. After participating in workshops that either did or did not address bias resistance, children completed posttest measures identical to pretest measures. At pretest, the gap in status of children's aspired-to and expected occupations was positively correlated to their perceptions of occupational racial discrimination, and this relation was mediated by occupational outcome expectations. At posttest, children in the bias resistance condition had higher-status occupational expectations than children in the control condition, and this difference was due to improved occupational outcome expectations in the bias resistance condition.     

大学生德性自我意象演练的教育干预实验

以149名大学生为对象、运用三种方法进行了为期八周每周三学时的道德教育现场实验,考察了自我意象演练、榜样示范、移情训练三种方法对德性自我意象、对道德行为自我管理的影响,结果表明三种方法均能产生积极作用,但自我意象演练的效果尤为显著,进而对自我意象演练方法的有效性及其机制进行了探讨,可以认为该方法是一种能够有效提高大学生的德性自我意象之水平和道德行为自我管理之能力的道德教育方法。     

Schooling and intelligence: A Methodological and logitudinal study

A general account of the adjustment problem is given within the frame of pretest-posttest design, and analysis of covariance, change score analysis and standardized change score analysis are discussed with reference to models of selection and growth. A Norwegian longitudinal investigation of the relationship between schooling and intelligence is described, and the results, generated by several adjustment techniques, showed substantial schooling effects. Thus the difference in estimated effect between a higher and a lower educational group varied from 49 to 73 per cent of the pretest standard deviation. In addition, considerable changes from pretest to posttest were observed, the mean gain amounting to 114 per cent of the pretest standard deviation. A comparison with two Swedish studies yielded similar trends in estimated schooling effect in that one study had higher and the other lower estimates than the present one.     

The Use of Prospective Versus Retrospective Pretests with Child-Welfare Involved Families

The purpose of the current study is to examine the use of both a prospective and a retrospective pretest in evaluating the impact of the Strengthening Families Program on foster care involved families affected by parental substance abuse. Debate has existed in the literature for over 60 years regarding the use of retrospective pretests in assessing self-reported program impact, with key stakeholders often arguing strongly that, due to response-shift bias, sensitization, and the nature of many human service settings, retrospective pretesting may be more appropriate. However, program evaluators must also incorporate into the evaluation design funding mandates to collect data at specific points in time. In order to mitigate potential pretest-only biases and enhance the evaluation’s rigor, this study sought to address the pretest debate through the use of both a traditional, prospective pretest and a retrospective pretest. Using data provided by 411 caregivers, program effectiveness was measured in the areas of family, child and parent functioning. Statistical significance tests and effect sizes were analyzed to compare traditional prospective pretest to posttest scores and retrospective pretests to posttest scores. Findings indicate that overall, the Strengthening Families Program positively impacted family, child and parent functioning and that there were few differences between testing approaches when testing for statistical significance; however, relative differences between prospective pretests and retrospective pretests appeared more prominent in effect size computations. This research informs the longstanding debate, and suggests that program evaluators consider the advantages and disadvantages of using a retrospective pretest in design planning.