Improving the Generalization of Computer-Based Math Fluency Building Through the Use of Sufficient Stimulus Exemplars

Computers have become an important piece of technology in classrooms for implementing academic interventions. Often, students’ responses to these interventions are used to help make important educational decisions. Therefore, it is important to consider the effect of these interventions across multiple contexts. For example, previous research has demonstrated that when students practice math fact fluency on the computers gains did not generalize to paper–pencil performance. The current study extended this research by examining the effect of multiple stimulus exemplars on the generalization of computer-based math practice to paper–pencil performance. A total of 57 second-grade students completed fluency drills only on the computer, computer mixed with paper–pencil or with only paper–pencil. Pretest–posttest performance was evaluated using a 3 × 2 doubly multivariate repeated-measures ANOVA, with follow-up univariate analysis to determine whether the interaction between time and treatment type was similar across matched and unmatched treatment modalities. Results from previous research were replicated with a lack of generalization across modalities, but the addition of multiple stimulus exemplars resulted in increased generalization for those students receiving a mix of computer and paper–pencil practice.     

Examining Immediate and Maintenance Effects of a Reading Intervention Package on Generalization Materials: Individual Verses Group Implementation

Fluency has been identified as an important outcome of effective reading instruction, and intervention packages utilizing the method of repeated readings have been shown to improve oral reading fluency. In order to improve the efficiency of these intervention packages, more research is needed on the effectiveness of small group reading interventions. An alternating treatments design was used to compare the effects of two versions of an intervention package that included repeated readings on generalization and maintenance of oral reading rate for 6 second grade students. Both intervention packages included listening passage preview, repeated readings, error correction, and contingent reward. The intervention packages differed only in whether intervention components were provided to students individually or in groups of three. Gains in oral reading rate were observed across conditions, with individual and small group interventions resulting in comparable gains for most participants. Individual differences in performance and implications for reading instruction are discussed.     

A Comparison of the Taped Spelling Intervention and Cover,Copy, and Compare for Students with Learning Disabilities

This study evaluated the effectiveness and compared the efficiency of two interventions on the spelling performance of four fifth-grade students with learning disabilities in reading and writing. The first intervention, Cover, Copy, and Compare (CCC), is an established, evidence-based procedure, while the second, the Taped Spelling Intervention (TSI), is a new procedure with only one published article establishing its effectiveness at the time of this study. Although both interventions resulted in spelling performance gains, learning speed was higher in the CCC condition and students overwhelmingly reported preferences for CCC over TSI. Spelling gains were maintained on assessments administered approximately two weeks following the final intervention sessions for both interventions. Discussion focuses on the importance of easily implemented, socially acceptable, time- and resource-efficient interventions for increasing the academic performance of students, and the value of comparative analyses for choosing appropriate interventions.     

Relationships Among Selected Tests of Spatial Orientation Ability

The purpose of this study was to investigate the relationships between selected tests of spatial orientation ability. 2 tests which have been used as predictors of spatial ability (Guilford-Zimmerman Aptitude Survey, Parts V and VI) plus 2 newly developed tests designed to measure that ability were given to 202 Junior high school boys aged 11-15 yr. 3 of these tests were of the paper and pencil type while one was a physical performance (tumbling) test. The paper-and-pencil tests correlated significantly with each other (.62, .61, .44) but the physical performance test did not correlate significantly with any of the other tests.     

The Impact of Processing Speed Training on Working Memory in Old Adults

Cross-sectional and longitudinal studies have gotten different results as to whether processing speed can explain the aging of cognitive functions. Experimental analyses are needed to develop further evidence. To explore the relationship between speed and working memory in cognitive aging, processing speed intervention is conducted in the present study. Utilizing paper–pencil training, computer training, and a control group, 78 adults aged 58–83?years from Beijing participated in a speed intervention study. After 5?weeks of training, there were substantial training gains on two speed tests in two training groups, but no transfer effect on working memory was found. Discussion: The results fail to support the view that processing speed is a foundation of working memory for aging. The aging of working memory may depend more on an executive attention component.     

Evaluating Generalization of Addition-Fact Fluency Using the Taped-Problems Procedure in a Second-Grade Classroom

A multiple-baseline design across math-fact sets was used to evaluate the effects of a taped-problems intervention on students’ performance with addition facts and their inverses in an intact, rural, second-grade classroom. Results suggested that the procedure was effective in improving fluency on math facts as well as their inverses. Across 3 sets of math facts, 15 students increased their fluency (i.e., average increases of 5 digits correct per min or more) on math-problem sets and these improvements were maintained over time. Additionally, results provide evidence that these increases generalized to inverse-problem sets. However, 4 students showed little gains (i.e., average increases of fewer than 5 digits correct per min) on math problems. Discussion focuses on evaluating academic interventions for generalization and directions for future research.     

A comparison of three interventions for increasing oral reading performance: Application of the instructional hierarchy

The instructional hierarchy is a behavior-analytic model that links level of academic skill development (i.e., acquisition, fluency, generalization, adaptation) with appropriate instructional techniques. The present study used the instructional hierarchy to compare the effects of three instructional interventions (listening passage preview, subject passage preview, and taped words) on subjects' oral reading performance on word lists and passages. Subjects were 4 male students with learning disabilities who ranged in age from 8 years 10 months to 11 years 11 months. A multielement design was used to compare the effects of the three interventions to each other and to baseline. Results indicated that the listening passage preview intervention (which contained modeling, drill, and generalization components) produced the largest performance gains. The implications of these results for selecting academic interventions based on the instructional hierarchy are discussed.     

The reliability of a computer interview for drug use/abuse information

A computer interview was developed to collect information about drug use/abuse from high school students. Responses to this interview were compared with those obtained from an identical paper-and-pencil questionnaire given to the same students, in counterbalanced order. Results were comparable for the two methods, but missing data and logically inconsistent answers obtained with the paper-and-pencil questionnaire were not possible with the computer interview. The overwhelming majority of respondents preferred the computer interview as a means of data collection.     

Implementing scientific practices through case studies: Examples using home-school interventions and consultation

Integrating scientific methods into school psychological practice is an essential tenet of effective service. In this article, we present two related examples that demonstrate methods by which to conduct high-quality intervention and consultation case studies with integrity. The present case studies utilized home note and consultation-based interventions to improve task completion and accuracy in mathematics for primary grade students who had demonstrated performance difficulties despite adequate intellectual ability and academic skills. Two sets of case studies were conducted. One involved the use of a home-school note implemented by parents and teachers in conjunction with a self-instruction manual. The second set of case studies used the home note and manual instituted in the context of behavioral consultation with parents and teachers. Both interventions (home-note-only and home note with consultation) increased math completion and accuracy in underachievers. Levels of performance were stable and differences between baseline and treatment conditions were statistically significant for consultation students only. Treatment integrity, maintenance of treatment gains, and consumer acceptability were also stronger in the consultation case studies. The implications for practitioners and future research are explored.     

Long-Term Follow-Up of an Occupation-Based Group Addressing Occupational Performance and Satisfaction in University Freshmen

The long-term impact of a 5-week, occupation-based intervention to address the life transition of university students into young adulthood is presented here. Study participants showed statistically significant gains from baseline to long-term follow-up in both occupational performance and performance satisfaction. In post hoc analyses, occupational performance maintained a level of significance. During a time of rapid change, occupation-based interventions have the potential to provide transitioning university freshmen students the skills that they need for a successful transition into a university. Changes in the structure, timing, and focus of the intervention and studies using more diverse and larger samples are warranted.     

An Experimental Analysis of Mathematics Instructional Components: Examining the Effects of Student-Selected Versus Empirically-Selected Interventions

Within the context of an experimental analysis, the current study examined the effects of student- and empirically-selected interventions on the mathematics computational fluency of three elementary-aged students. For all of the participants, the empirically-selected intervention resulted in higher levels of computational fluency then the student-selected intervention. These results suggest that empirically-selected interventions may enhance the mathematics computational fluency of students experiencing mathematics problems. However, individual responsiveness to empirically-selected interventions suggests the importance of conducting brief experimental analyses to determine the most efficacious intervention. The implication of these results for intervention selection and incorporating choice-making opportunities into academic interventions are discussed.     

Using personal digital assistants (PDAs) for the collection of safety belt use data in the field

From 1975 through 2003, because of the use of safety belts, an estimated 180,000 deaths from traffic crashes have been prevented. In order to assess the gains that have been made in safety belt use across the U.S., in individual states, and in communities, belt use surveys are conducted at regular intervals to determine use rates. The most valid method for surveying safety belt use is through direct observation. Direct observation surveys are conducted along roadways by trained researchers looking into passing vehicles and recording safety belt use. This method of data collection has been effective in the past through the use of paper-and-pencil data recording, yet it could be improved through the use of electronic and communication technology. Reported here is a study designed to compare electronic data collection, using personal digital assistants (PDAs), with collection using the traditional paperand-pencil method during the annual statewide survey of safety belt use in Michigan that we have been conducting since 1984. The goals of the study were to develop a PDA database program for data entry in the field, to directly compare the PDA data collection process with the paper-and-pencil method on both accuracy and speed, and to assess mechanical and environmental factors, such as battery life, screen visibility, and reaction to adverse weather, that may act as limitations to the PDA method, in comparison with the paper-and-pencil method. In a direct comparison of methods, two observers collected data at the same roadway intersections, one using paper and a pencil and one using a PDA equipped with our custom software. The study showed that the PDA method was as fast and as accurate as the paper-and-pencil method. There were no adverse effects on the PDA caused by environmental conditions. The PDA was superior to the paper-and-pencil method in rainy weather and for data collector supervision. In addition, the use of the PDA obviated the need for entry of paper-recorded data into an electronic format. We conclude that the use of PDAs for safety belt field data collection is superior to the paper-and-pencil method. A Web address is given where the custom PDA safety belt data collection software can be obtained free of charge.     

Dynamic spatial performance and general intelligence

A computerized video-game-like dynamic spatial ability measure and a paper-and-pencil group test of intelligence were administered to 94 university students. The purpose of the study was threefold: (a) to evaluate the reliability of the dynamic spatial measure; (b) to investigate the extent to which the dynamic spatial measure is saturated with variance associated with traditional measures of fluid and crystallized intellectual ability; and (c) to explore the possibility that dynamic computer based performance measures define factors that are distinct from those associated with traditional paper-and-pencil measures. The number of times the participant hit a moving target served as the dependent variable for the dynamic spatial measure. Number of hits proved to be correlated with Performance IQ, but was not significantly correlated with Verbal IQ. Maximum likelihood factor analysis revealed that the dynamic spatial measure did not load substantially on a factor of general intellectual ability but provided additional evidence that it is related to a performance component.     

Computer-assisted versus paper-and-pencil administration of questionnaires

In a field experiment using 95 employees, the results of computer-assisted versus paper-and-pencil administration of questionnaires during computer training were compared. Unlike past research, the computer mode of administration was made as similar as possible, in format, to the paper mode. No differences in means or reliabilities were found between any variables across the two conditions. However, when the correlations were examined, different patterns of relationships were found across the two conditions. Specifically, subjective or affective measures (software efficacy, playfulness, and training reaction) were more highly related when the measures were gathered by computer as opposed to paper-and-pencil. The relationships between subjective measures and more objective or factual measures (experience, performance, training transfer) were not affected. Thus, it appears that, while differences in mode of administration may not be reflected in differences in means, there are differences in nomological validity.     

WEB‐BASED AND PAPER‐AND‐PENCIL TESTING OF APPLICANTS IN A PROCTORED SETTING: ARE PERSONALITY,BIODATA, AND SITUATIONAL JUDGMENT TESTS COMPARABLE?

This quasi-experimental study compares the equivalence of proctored Web-based tests to paper-and-pencil tests in a selection setting. The predictor battery was composed of measures of Conscientiousness, Agreeableness, Emotional Stability, a biodata form, and a situational judgment test. Three samples were administered the same test items, but differed with respect to context and test format: 425 incumbents completed a paper-and-pencil version, 2,544 applicants completed a paper-and-pencil version, and 2,356 applicants completed an identical Web-based version (although these individuals were not randomly assigned to test format, the job and organization were the same for all samples). The results found effects for both context and test format. Relative to the applicants completing the paper-and-pencil measures, the Web-based measures showed (a) better distributional properties, (b) lower means, (c) more variance, (d) higher internal consistency reliabilities, and (e) stronger intercorrelations. Although the applicant context resulted in higher mean responses than the incumbent context, this difference was about twice as large for the paper-and-pencil test than for the Web-based test. All such differences were greater for personality measures than for biodata and situational judgment tests. Overall, these results suggest that proctored Web-based testing has some positive benefits relative to paper-and-pencil measures, and we identify several implications of these findings for research and practice.     

Training in Self-Explanation and Self-Regulation Strategies: Investigating the Effects of Knowledge Acquisition Activities on Problem Solving

Previous research has found positive correlations between particular strategies students use while studying to explain instructional materials to themselves and student performance on associated problem-solving tasks (Chi, Bassok, Lewis, Reimann, & Glaser, 1989; Pirolli & Bielaczyc, 1989; Pirolli & Recker, 1994). In the study reported here, we investigate the causal nature of this relation. This was accomplished by identifying a set of self-explanation and self-regulation strategies used by high-performance students in our earlier studies. We used strategy training to manipulate students' application of these strategies and examined the impact of their use on student explanations and performance. Twenty-four university students with no prior programming experience worked through a sequence of programming lessons. Following introductory lessons, participants received interventions involving explicit training in the strategies (instructional group) or received a similar set of interventions but no explicit training (control group). The instructional group showed significantly greater gains than the control group in the use of self-explanation and self-regulation strategies from the pre- to postinterventions lessons. Increased strategy application was accompanied by significantly greater performance gains. The results indicate that the particular self-explanation and self-regulation strategies used in training contribute to learning and problem-solving performance.     

Promoting Generalization of Oral Reading Fluency: Providing Drill versus Practice Opportunities

Extensive evidence exists demonstrating the benefits of repeated readings (RR) interventions at increasing students’ fluency on intervention passages. Few studies however have examined the extent to which repeatedly reading one passage improves students’ reading fluency on similar passages. Using an alternating treatment design, we examined the extent to which two interventions resulted in improvements in students’ fluency on generalization passages. While both interventions incorporated RR, one intervention involved students reading one passage four times and the other involved students reading two similar passages each twice. Intervention effects were evaluated by having students read a generalization passage prior to and following intervention implementation. Results indicate that both interventions were effective in increasing students reading fluency on generalization passages. For 3 participants the RR intervention produced greater gains in fluency on the generalization passages, while data for the remaining 3 participants are inconclusive. Implications of these findings for practice and for better understanding application of the instructional hierarchy to the development of reading interventions are discussed.

Using stimulus equivalence technology to teach statistical inference in a group setting

Computerized lessons employing stimulus equivalence technology, used previously under laboratory conditions to teach inferential statistics concepts to college students, were employed in a group setting for the first time. Students showed the same directly taught and emergent learning gains as in laboratory studies. A brief paper-and-pencil examination, suitable for classroom use, captured effects demonstrated previously through laboratory tests. The results support the extension of the lessons to more naturalistic settings.     

Further explorations of perceptual speed abilities in the context of assessment methods, cognitive abilities, and individual differences during skill acquisition

Measures of perceptual speed ability have been shown to be an important part of assessment batteries for predicting performance on tasks and jobs that require a high level of speed and accuracy. However, traditional measures of perceptual speed ability sometimes have limited cost-effectiveness because of the requirements for administration and scoring of paper-and-pencil tests. There have also been concerns about the validity of previous computer approaches to administering perceptual speed tests (e.g., see Mead & Drasgow, 1993). The authors developed two sets of computerized perceptual speed tests, with touch-sensitive monitors, that were designed to parallel several paper-and-pencil tests. The reliability and validity of the tests were explored across three empirical studies (N = 167, 160, and 117, respectively). The final study included two criterion tasks with 4.67 and 10 hours of time-on-task practice, respectively. Results indicated that these new measures provide both high levels of reliability and substantial validity for performance on the two skill-learning tasks. Implications for research and application for computerized perceptual speed tests are discussed.     

New approaches to the design of computerized interviewing and testing systems

Most computer interviewing and testing systems have adopted paper-and-pencil approaches to information gathering with little modification. However, computer technology offers two fundamental advantages over paper-and-pencil technology for psychological information gathering: (1) A computer can record ancillary data such as latencies and pressure on response keys during an interviewing session, and (2) A computer can react adaptively to special events as these arise during a session. Ways to capitalize on these advantages are outlined. A pilot study of interviewee behavior during a computer problem-screening interview is described, and the implications of the results for future research in the area are discussed. Passive and active computer testing systems occupy positions on a continuum between paper-based psychological testing and the flexible, but less well controlled, technology represented by the human. With its unique capabilities, computer technology has a special role to play in the future of psychological measurement.