Curriculum-based measurement of oral reading: Passage equivalence and probe-set development

Curriculum-based measurement of reading (CBM-R) is used to estimate oral reading fluency. Unlike many traditional published tests, CBM-R materials are often comprised of 20 to 30 alternate forms/passages. Historically, CBM-R assessment materials were sampled from curricular materials. Recent research has documented the potentially deleterious effects of poorly controlled alternate forms on CBM-R outcomes. The purpose of this study was to examine alternate procedures for the selection of passages that comprise CBM-R passage-sets. The study examined four procedures for the evaluation and selection of passages, including random sampling, Spache readability formula, mean level of performance evaluation, and Euclidean Distance evaluation. The latter two procedures relied on field testing and evaluation of student performance. Each of eighty-eight students in second- and third-grade were administered 50 CBM-R passages. Generalizability and dependability studies were used to examine students' performance on these passages and evaluate CBM-R passage selection procedures. Results provide support for the use of field testing methods (i.e., calculating performance means and Euclidean Distances) for passage selection. Implications are discussed for future research and practice.     

Examiner error in curriculum-based measurement of oral reading

Although curriculum based measures of oral reading (CBM-R) have strong technical adequacy, there is still a reason to believe that student performance may be influenced by factors of the testing situation, such as errors examiners make in administering and scoring the test. This study examined the construct-irrelevant variance introduced by examiners using a cross-classified multilevel model. We sought to determine the extent of variance in student CBM-R scores attributable to examiners and, if present, the extent to which it was moderated by students' grade level and English learner (EL) status. Fit indices indicated that a cross-classified random effects model (CCREM) best fits the data with measures nested within students, students nested within schools, and examiners crossing schools. Intraclass correlations of the CCREM revealed that roughly 16% of the variance in student CBM-R scores was associated between examiners. The remaining variance was associated with the measurement level, 3.59%; between students, 75.23%; and between schools, 5.21%. Results were moderated by grade level but not by EL status. The discussion addresses the implications of this error for low-stakes and high-stakes decisions about students, teacher evaluation systems, and hypothesis testing in reading intervention research.     

How accurate are interpretations of curriculum-based measurement progress monitoring data? Visual analysis versus decision rules

Curriculum based measurement of oral reading (CBM-R) is used to monitor the effects of academic interventions for individual students. Decisions to continue, modify, or terminate these interventions are made by interpreting time series CBM-R data. Such interpretation is founded upon visual analysis or the application of decision rules. The purpose of this study was to compare the accuracy of visual analysis and decision rules. Visual analysts interpreted 108 CBM-R progress monitoring graphs one of three ways: (a) without graphic aids, (b) with a goal line, or (c) with a goal line and a trend line. Graphs differed along three dimensions, including trend magnitude, variability of observations, and duration of data collection. Automated trend line and data point decision rules were also applied to each graph. Inferential analyses permitted the estimation of the probability of a correct decision (i.e., the student is improving – continue the intervention, or the student is not improving – discontinue the intervention) for each evaluation method as a function of trend magnitude, variability of observations, and duration of data collection. All evaluation methods performed better when students made adequate progress. Visual analysis and decision rules performed similarly when observations were less variable. Results suggest that educators should collect data for more than six weeks, take steps to control measurement error, and visually analyze graphs when data are variable. Implications for practice and research are discussed.     

Comparison of progress monitoring data from general outcome measures and specific subskill mastery measures for reading

Interventionists often monitor the progress of students receiving supplemental interventions with general outcome measures (GOMs) such as curriculum-based measurement of reading (CBM-R). However, some researchers have suggested that interventionists should collect data more closely related to instructional targets, specific subskill mastery measures (SSMMs) because outcomes from GOMs such as CBM-R may not be sufficiently sensitive to gauge intervention effects. In turn, interventionists may prematurely terminate an effective intervention or continue to deliver an ineffective intervention if they do not monitor student progress with the appropriate measure. However, such recommendations are based upon expert opinion or studies with serious methodological shortcomings. We used multi-variate multilevel modeling to compare pre-intervention intercepts and intervention slopes between GOM and SSMM data collected concurrently in a sample of 96 first, 44 second, and 53 third grade students receiving tier 2 phonics interventions. Statistically significant differences were observed between slopes from SSMM consonant-vowel-consonant words and CBM-R data. Statistically significant differences in slopes were not observed for consonant blend, digraph or consonant-vowel-consonant-silent e (CVCe) SSMMs. Results suggest that using word lists to monitor student response to instruction for early struggling readers is beneficial but as students are exposed to more complex phonetic patterns, the distinction between SSMMs and CBM-R become less meaningful.     

An investigation of the psychometric evidence of CBM-R passage equivalence: Utility of readability statistics and equating for alternate forms

Claims have been made that grade appropriate curriculum-based measurement of reading (CBM-R) passages are of comparable difficulty and can be used interchangeably to monitor student progress. Empirical evidence to support claims of equivalence has been lacking. This research investigated the basis for making claims of equivalence. The use of readability statistics to justify passage equivalence was found to be lacking. Using a general measurement model for congeneric tests, CBM-R passages were found to measure a single latent factor with a high degree of reliability. However, evidence indicated that the raw scores, words read correctly per minute, across passages did not provide equivalent measurements. Statistical equating was investigated as an approach to overcome the lack of equivalence with promising results.     

A Preliminary Investigation of Supplemental Computer-Assisted Reading Instruction on the Oral Reading Fluency and Comprehension of First-Grade African American Urban Students

This preliminary investigation examined the effects of a computerized supplemental reading program on the oral reading fluency, reading growth rates, and comprehension of 8 African American first graders. Participants were selected for this study according to scores within risk categories on the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) oral reading fluency (ORF) winter benchmark, indicating the potential for reading failure. Participants engaged in a supplemental, computer-based reading program designed to improve reading fluency and comprehension. Treatment sessions were conducted 3–4 times per week, for 14–16 weeks. Following the treatment, each participant received the DIBELS ORF spring benchmark as a post-test measure. A pre-intervention/post-intervention analysis showed that all of the participants increased their reading fluency, 5 of the 8 participants reduced their risk status, and 7 of the 8 students increased their reading rate. All of the students improved their comprehension scores. The results support supplementary interventions and computer-based reading programs. The findings are discussed in the context of reading needs and instruction for African American primary-aged students.     

Examining Immediate and Maintenance Effects of a Reading Intervention Package on Generalization Materials: Individual Verses Group Implementation

Fluency has been identified as an important outcome of effective reading instruction, and intervention packages utilizing the method of repeated readings have been shown to improve oral reading fluency. In order to improve the efficiency of these intervention packages, more research is needed on the effectiveness of small group reading interventions. An alternating treatments design was used to compare the effects of two versions of an intervention package that included repeated readings on generalization and maintenance of oral reading rate for 6 second grade students. Both intervention packages included listening passage preview, repeated readings, error correction, and contingent reward. The intervention packages differed only in whether intervention components were provided to students individually or in groups of three. Gains in oral reading rate were observed across conditions, with individual and small group interventions resulting in comparable gains for most participants. Individual differences in performance and implications for reading instruction are discussed.     

Eye movements and the perceptual span in silent and oral reading

Previous research has examined parafoveal processing during silent reading, but little is known about the role of these processes in oral reading. Given that masking parafoveal information slows down silent reading, we asked whether a similar effect also occurs in oral reading. To investigate the role of parafoveal processing in silent and oral reading, we manipulated the parafoveal information available to readers by changing the size of a gaze-contingent moving window. Participants read silently and orally in a one-word window and a three-word window condition as we monitored their eye movements. The lack of parafoveal information slowed reading speed in both oral and silent reading. However, the effects of parafoveal information were larger in silent reading than in oral reading, because of different effects of preview information on both when the eyes move and how often. Parafoveal information benefitted silent reading for faster readers more than for slower readers.     

An Investigation of Student-Selected and Parent-Delivered Reading Interventions

In this study, we examined whether individualized, student-selected, parent-delivered reading interventions would produce generalized oral reading fluency improvements. Three 3rd-grade students received reading fluency interventions (repeated readings, modeling, error correction, and flashcard instruction) 1 at a time and were shown the results that each intervention produced. The students then selected the intervention strategies they wanted their parents to use. We trained parents in the student-selected reading intervention package who delivered the interventions for 4–5?weeks, and students were given the opportunity to earn rewards for generalized performance improvements at the end of each week in school. Using an alternating treatments design, results of parent tutoring were compared to a control condition that included a reinforcement contingency for generalized performance improvements to control for reinforcement effects across conditions. Generalized improvements in oral reading fluency (to high-word-overlap passages) were found for all participants. Results are discussed in terms of how educators can improve parent-tutoring outcomes by influencing students’ motivation to engage in tutoring procedures.     

Promoting Generalization of Reading: A Comparison of Two Fluency-Based Interventions for Improving General Education Student’s Oral Reading Rate

The generalization effects of empirically-supported fluency-based reading interventions have been largely ignored. The purpose of this study was to evaluate the effects of two fluency-based reading interventions, Repeated Readings and Multiple Exemplars, on elementary-aged children’s immediate and generalized oral reading fluency rate. Using a within-subjects group design, a total of 42 second (n = 25) and fourth (n = 17) grade general education students were administered both interventions and their generalized responding to passages containing high word and medium word overlap was assessed. Results indicated that children’s oral fluency on intervention passages was significantly greater during the Repeated Readings intervention. However, children’s oral reading fluency on generalization passages containing medium word overlap was significantly greater following the Multiple Exemplars intervention. This difference was however possibly due to the Multiple Exemplar medium word overlap passage being easier for students to read. No significant differences between the two interventions were observed in children’s oral reading fluency on generalization passages containing high word overlap. Implications and limitations of the study are discussed in relation to improving students’ reading fluency on generalization passages.     

Using Video Self-Modeling as a Small Group Reading Fluency Intervention for Elementary School Students

Video self-modeling (VSM) has been used widely in the literature to teach a range of prosocial behaviors. More recently, VSM has been explored as both a supplement and stand-alone option to teach academic skills, particularly oral reading fluency. However, the literature is relatively sparse pertaining to the effects of video modeling, void of any current literature exploring the effects of video modeling as a component to small group reading interventions. The present study implemented a multiple baseline design across participants, with an alternating treatment design directly comparing a small group reading intervention with and without VSM. Nonoverlap of all pairs was used to measure the effects of the small group intervention with and without VSM relative to baseline, as well as to compare the two intervention packages to one another. Nonoverlap of all pairs scores indicated moderate to strong effects for both interventions across all participants, with little difference between the two interventions.     

Improving oral reading fluency with a peer-mediated intervention

This study examined the effects of an experimentally derived, peer-delivered reading intervention on the oral reading fluency of a first-grade student who had been referred for poor reading fluency. Same-grade peers were trained to lead the target student through a structured intervention protocol based on the results of a brief experimental analysis. Results indicated that reading improvements were obtained and are discussed in terms of selecting efficient interventions for use by peers.     

The influence of trend estimation method on forecasting curriculum-based measurement of reading performance

Estimating a trend line through words read correct per minute scores collected across successive weeks is a preferred method to evaluate student response to instruction with curriculum-based measurement of reading (CBM-R). This is due in part, because the slope of that line of best fit is used to predict the trajectory of student performance if the current intervention is maintained. In turn, trend lines should predict future scores with a high degree of accuracy when an intervention is maintained. We evaluated the forecasting accuracy of a trend estimation method currently used in practice (i.e., ordinary least squares), and five alternate methods recently evaluated in CBM-R simulation studies, using actual student data. Results suggest that alternate trend estimation methods predicted future performance with a similar level of accuracy as ordinary least squares trend lines across most conditions, with the exception of slopes estimated via Bayesian analysis. Bayesian trend lines estimated using informed prior distributions yielded noticeably less biased and more precise predictions when applied to short data series relative to all other estimation methods across most conditions. Outcomes from the current study highlight the need to further explore the viability of Bayesian analysis to evaluate individual time series data.     

Promoting parent use of empirically supported reading interventions: two experimental investigations of child outcomes

Two experimental investigations of the effects of parent delivered reading interventions were conducted. Tutoring packages consisting of empirically supported intervention components were delivered by parents for at least several weeks after initial parent training. Both experiments used single‐case experimental designs and measured participants' oral reading fluency in passages. Experiment 1 used a multiple‐probe design across tasks (passages) to evaluate tutoring effects for two students with learning disabilities. Results indicate that both students increased their reading fluency and maintained those increases over time. Experiment 2 used a brief experimental analysis that included both experimenter and parent delivered instructional trials to validate the treatment package. Next, the treatment package was evaluated using an alternating treatments design. Results were uniformly positive. An interesting but not surprising correlation was also found between treatment integrity and student outcomes. Results are discussed in terms of the framework and skills that behavior analysts have for working with parents and schools to improve their children's academic responding. Copyright © 2006 John Wiley & Sons, Ltd.     

Fluent Reading as the Improvement of Stimulus Control: Additive Effects of Performance-Based Interventions to Repeated Reading on Students' Reading and Error Rates

The impact of adding performance-based interventions to a skill-based intervention on oral reading fluency was examined with 3 elementary students identified as having difficulty with reading. Two performance-based interventions were combined with the skill-based intervention Repeated Reading (RR) to create the three instructional packages investigated. The packages included Repeated Readings (RR), Repeated Readings with Performance Feedback (RR/FB), and Repeated Readings with Performance Feedback and Contingent Reward (RR/FB/REW). Results suggested that Repeated Reading (RR) alone led to the greatest increase in reading fluency for participants with the highest reading rates and lowest error rates at baseline. In contrast, the participant who exhibited the lowest reading rate and highest error rate benefited the most when Repeated Readings was offered in conjunction with Performance Feedback (RR/FB) or Performance Feedback plus Contingent Reward (RR/FB/RW). Implications for blending skill- and performance-based interventions to develop stimulus control over fluent reading and future research directions are discussed.     

Supporting Fourth-Grade Students' Word Identification Using Application Software

A quasi-experimental study examined effects of a 10-week word structure intervention with fourth-grade students. During daily 10–15-minute practice periods, students worked individually with mobile apps focused on specific aspects of word identification. Pre- and post-treatment assessments showed no differences in rate and accuracy of oral reading between groups that did and did not use the apps, with little effect on motivation to read. Some differences were found in level of texts read and in spelling favoring the intervention students. However, practical significance was found in gains in percentile rank scores on standardized vocabulary and comprehension measures.     

A BRIEF EXPERIMENTAL ANALYSIS FOR IDENTIFYING INSTRUCTIONAL COMPONENTS NEEDED TO IMPROVE ORAL READING FLUENCY

Brief experimental analyses of oral reading fluency were conducted with 4 participants who had been referred by teachers and parents for reading problems. The procedures involved the sequential application of reading interventions to improve students' oral reading fluency. Following a baseline condition, instructional treatments were combined with prior conditions until there was improvement in oral reading fluency in the instructional passages and passages with high content overlap. Differentiated response patterns, assessed via a multielement design, were obtained for all participants. Results are discussed in terms of the potential benefits and limitations of conducting brief experimental analyses for selecting reading interventions.     

Reading Comprehension: Effects of Individualized,Integrated Language Arts as a Reading Approach with Struggling Readers

This study examined the effects of individualized, integrated language arts as a reading approach on struggling readers' comprehension scores obtained from oral narrative, silent narrative, and silent expository passages at three levels: below-grade, on-grade, and above-grade levels. Students (N = 93) in grades four through eight, who were reading below grade level, participated in the study. Treatment group students (n = 51) received individualized, integrated language arts as a reading approach once a week in place of basal reading instruction. Comparison group students (n = 42) received basal reading instruction for the duration of the study. Multivariate analysis of covariance was used to analyze posttest Analytical Reading Inventory (ARI) comprehension scores. Several statistically significant (p < .001) differences in comprehension performance were found for on-grade-level scores and for above-grade-level scores, but few differences were found between treatment and comparison groups on below-grade-level scores. All statistically significant differences favored students in the treatment group. The findings of the study strongly suggest that the use of individualized, integrated language arts as a method for teaching reading is an effective approach for improving the reading comprehension performance of struggling readers.     

Prescriptive Analysis: Further Individualization of Hypothesis Testing in Brief Experimental Analysis of Reading Fluency

Brief experimental analyses (BEA) have been used to identify effective individualized interventions for improving reading fluency with school-age children. Interventions involving incentives, modeling, repeated reading, and error correction are most often tested in a BEA. However, these interventions are rarely modified according to individual student needs. The current study examined the effects of varied levels of modeling (passage, sentence, word) and varied contingencies (tracking and fluency) on fluency in non-treated high-word overlap passages and general outcome measure passages. BEA-identified interventions were effective for improving reading fluency on non-treated passages for both second grade students and the third grade student. Notably, the effects reversed when intervention was withdrawn with one of the second grade students. Results are discussed in terms of considerations in the application of BEA technology in the schools.     

A procedure for increasing oral reading rate in hard-of-hearing children

This study investigated the effects of systematic training on the oral reading rate of four hard-of-hearing children. In training, systematic increases in pulsing rate of a vibrotactile stimulus were arranged. The children received points, exchangeable for money when their reading rate equalled the pulse rate of the stimulus. The training procedure was effective in increasing the oral reading rate in all children. Generalized increases in reading rate on untrained word lists, sentences, and paragraphs were also observed.