Examining the core: Relations among reading curricula, poverty, and first through third grade reading achievement

Policy changes at the federal and state level are endeavoring to improve student achievement at schools serving children from lower-SES homes. One important strategy is the focus on using evidence-based core reading curricula to provide a consistent framework for instruction across schools. However, rarely have these curricula undergone rigorous comparative testing. Therefore, the purpose of this study was to compare the effects of six core reading curricula on oral reading fluency growth, while appraising whether these effects differ by grade level and for children living in lower socioeconomic (SES) households. Over 30,000 students in first through third grade Florida Reading First classrooms comprise this academically and economically diverse cross-sectional. Hierarchical Linear Modeling was used to model latent growth curves for students' reading fluency scores over the school year. Growth curves revealed differences across curricula as well as between students of lower and higher SES, suggesting that reading fluency growth trajectories for curricula varied depending on student SES and grade level. Findings indicate that while there are similarities among curricula, they sometimes differ in their ability to promote reading skill growth. Differences by grade level and SES were also detected. However, many of these differences were small. Implications for the use of curriculum as a conduit for improving reading instruction are discussed.     

Curriculum-based measurement of oral reading: Passage equivalence and probe-set development

Curriculum-based measurement of reading (CBM-R) is used to estimate oral reading fluency. Unlike many traditional published tests, CBM-R materials are often comprised of 20 to 30 alternate forms/passages. Historically, CBM-R assessment materials were sampled from curricular materials. Recent research has documented the potentially deleterious effects of poorly controlled alternate forms on CBM-R outcomes. The purpose of this study was to examine alternate procedures for the selection of passages that comprise CBM-R passage-sets. The study examined four procedures for the evaluation and selection of passages, including random sampling, Spache readability formula, mean level of performance evaluation, and Euclidean Distance evaluation. The latter two procedures relied on field testing and evaluation of student performance. Each of eighty-eight students in second- and third-grade were administered 50 CBM-R passages. Generalizability and dependability studies were used to examine students' performance on these passages and evaluate CBM-R passage selection procedures. Results provide support for the use of field testing methods (i.e., calculating performance means and Euclidean Distances) for passage selection. Implications are discussed for future research and practice.     

Test bias and ability level testing

The average grade equivalent reading comprehension scores of students in 50 black schools were compared to those of students in 50 white schools under two forms of test administration. Traditional grade level testing resulted in smaller group differences between blacks' and whites' scores than the apparently more valid ability level testing technique. It was concluded that the use of grade level testing with the Iowa Tests of Basic Skills is biased in favor of low scoring subgroups.     

The Impact of Inappropriate Modeling of Cross-Classified Data Structures

Cross-classified random effects modeling (CCREM) is used to model multilevel data from nonhierarchical contexts. These models are widely discussed but infrequently used in social science research. Because little research exists assessing when it is necessary to use CCREM, 2 studies were conducted. A real data set with a cross-classified structure was analyzed by comparing parameter estimates when ignoring versus modeling the cross-classified data structure. A follow-up simulation study investigated potential factors affecting the need to use CCREM. Results indicated that when the structure is ignored, fixed-effect estimates were unaffected, but standard error estimates associated with the variables modeled incorrectly were biased. Estimates of the variance components also displayed bias, which was related to several study factors.     

Precision of Curriculum-Based Measurement Reading Data: Considerations for Multiple-Baseline Designs

Single-case designs provide an established technology for evaluating the effects of academic interventions. Researchers interested in studying the long-term effects of reading interventions often use curriculum-based measures of reading (CBM-R) as they possess many of the desirable characteristics for use in a time-series design. The reliability of CBM-R scores is often supported by research from group designs, but making idiographic interpretations regarding the change in a student’s oral reading rate requires attention to the precision of static scores and growth estimates. The purpose of this paper is twofold. First, we discuss how recent empirical work on the technical adequacy of CBM-R scores has revealed multiple threats to the data-evaluation validity when CBM-R passages are used to measure oral reading rate. Second, we identify pertinent considerations for conducting a visual analysis of intervention effects based on CBM-R data. We conclude with a brief discussion of implications for researchers considering the use of CBM-R within multiple-baseline designs.     

Differential Competencies Contributing to Children's Comprehension of Narrative and Expository Texts

The purpose of this study was to examine the connections between the oral reading abilities and the spelling behaviors of third and fifth grade students. Seventy-two third graders and sixty fifth graders from two different schools (one urban and one suburban) were the subjects of the study. Each subject read a selection one level above his/her current grade placement, spelled the words on the appropriate grade level list of the Qualitative Inventory of Word Knowledge and took the appropriate level of the Gates-MacGinitie Reading Tests. Oral readings were scored for accuracy, rate and phrasing. Spellings were scored for accuracy, phonetic quality and stage of spelling development. These scores were then analyzed using correlations, partial correlations and multiple regression techniques. At both grade levels there were high, significant correlations between spelling and reading variables, with spelling variables accounting for from 40% to 60% of the variance in oral reading measures and a smaller, but still significant percentage of the variance when standardized test scores were used as a control. These results confirm a strong relationship between spelling skill and oral reading ability, supporting the argument that a common body of conceptual word knowledge underlies both.     

Comparison of progress monitoring data from general outcome measures and specific subskill mastery measures for reading

Interventionists often monitor the progress of students receiving supplemental interventions with general outcome measures (GOMs) such as curriculum-based measurement of reading (CBM-R). However, some researchers have suggested that interventionists should collect data more closely related to instructional targets, specific subskill mastery measures (SSMMs) because outcomes from GOMs such as CBM-R may not be sufficiently sensitive to gauge intervention effects. In turn, interventionists may prematurely terminate an effective intervention or continue to deliver an ineffective intervention if they do not monitor student progress with the appropriate measure. However, such recommendations are based upon expert opinion or studies with serious methodological shortcomings. We used multi-variate multilevel modeling to compare pre-intervention intercepts and intervention slopes between GOM and SSMM data collected concurrently in a sample of 96 first, 44 second, and 53 third grade students receiving tier 2 phonics interventions. Statistically significant differences were observed between slopes from SSMM consonant-vowel-consonant words and CBM-R data. Statistically significant differences in slopes were not observed for consonant blend, digraph or consonant-vowel-consonant-silent e (CVCe) SSMMs. Results suggest that using word lists to monitor student response to instruction for early struggling readers is beneficial but as students are exposed to more complex phonetic patterns, the distinction between SSMMs and CBM-R become less meaningful.     

The effects of anxiety on the perception of word phrases

This study was designed to determine if anxiety in intermediate grade school children would influence their perceptual reading pattern of word phrases. The 107 fifth- and sixth-grade students had phrase reading scores significantly associated with age, sex, and grade level in the expected direction. Higher anxiety scores significantly limited student perception among 3-, 4-, and 5-word length phrases (r's from ?.37 to ?.40, p < .01). Anxiety test data increased by 10 to 13% the amount of predictive variance accounted for beyond predictions obtained established by age, sex, and grade level in combination (p < .01). R's ranged from .56 to .58.     

An investigation of the psychometric evidence of CBM-R passage equivalence: Utility of readability statistics and equating for alternate forms

Claims have been made that grade appropriate curriculum-based measurement of reading (CBM-R) passages are of comparable difficulty and can be used interchangeably to monitor student progress. Empirical evidence to support claims of equivalence has been lacking. This research investigated the basis for making claims of equivalence. The use of readability statistics to justify passage equivalence was found to be lacking. Using a general measurement model for congeneric tests, CBM-R passages were found to measure a single latent factor with a high degree of reliability. However, evidence indicated that the raw scores, words read correctly per minute, across passages did not provide equivalent measurements. Statistical equating was investigated as an approach to overcome the lack of equivalence with promising results.     

It Takes a Village: The Effects of 10th Grade College-Going Expectations of Students, Parents, and Teachers Four Years Later

Adolescents are surrounded by people who have expectations about their college-going potential. Yet, few studies have examined the link between these multiple sources of college-going expectations and the actual status of students in postsecondary education years later. The study draws on data collected in the 2002–2006 Educational Longitudinal Study and employs an underutilized statistical technique (cross-classified multilevel modeling) to account for teacher reports on overlapping groups of students (typical of high school research). Results showed that positive expectations of students, parents, English, and mathematics teachers in the 10th grade each uniquely predicted postsecondary status 4 years later. As a group, the four sources of expectations explained greater variance in postsecondary education than student characteristics such as socioeconomic status and academic performance. This suggests positive expectations are additive and promotive for students regardless of their risk status. Teacher expectations were also found to be protective for low income students. Implications for future expectancy research and equity-focused interventions are discussed.     

Estimation of a Latent Variable Regression Growth Curve Model for Individuals Cross-Classified by Clusters

The cross-classified multiple membership latent variable regression (CCMM-LVR) model is a recent extension to the three-level latent variable regression (HM3-LVR) model which can be utilized for longitudinal data that contains individuals who changed clusters over time (for instance, student mobility across schools). The HM3-LVR model can include the initial status on growth effect as varying across those clusters and allows testing of more flexible hypotheses about the influence of initial status on growth and of factors that might impact that relationship, but only in the presence of pure clustering of participants within higher-level units. This Monte Carlo study was conducted to evaluate model estimation under a variety of conditions and to measure the impact of ignoring cross-classified data when estimating the incorrectly specified HM3-LVR model in a scenario in which true values for parameters are known. Furthermore, results from a real-data analysis were used to inform the design of the simulation. Overall, it would be recommended for researchers to utilize the CCMM-LVR model over the HM3-LVR model when individuals are cross-classified, and to use a bare minimum of more than 100 clustering units in order to avoid overestimation of the level-3 variance component estimates.     

The influence of trend estimation method on forecasting curriculum-based measurement of reading performance

Estimating a trend line through words read correct per minute scores collected across successive weeks is a preferred method to evaluate student response to instruction with curriculum-based measurement of reading (CBM-R). This is due in part, because the slope of that line of best fit is used to predict the trajectory of student performance if the current intervention is maintained. In turn, trend lines should predict future scores with a high degree of accuracy when an intervention is maintained. We evaluated the forecasting accuracy of a trend estimation method currently used in practice (i.e., ordinary least squares), and five alternate methods recently evaluated in CBM-R simulation studies, using actual student data. Results suggest that alternate trend estimation methods predicted future performance with a similar level of accuracy as ordinary least squares trend lines across most conditions, with the exception of slopes estimated via Bayesian analysis. Bayesian trend lines estimated using informed prior distributions yielded noticeably less biased and more precise predictions when applied to short data series relative to all other estimation methods across most conditions. Outcomes from the current study highlight the need to further explore the viability of Bayesian analysis to evaluate individual time series data.     

学校心理环境对小学4～6年级学生学业表现的作用及条件

本研究基于具有良好全国代表性的中国儿童青少年心理发育特征数据库,使用多水平建模,分析了来自全国100个区县421所学校的心理环境特征对10826名小学4~6年级学生学业表现的影响途径及其发挥作用的条件。研究结果表明:(1)小学4~6年级学生学习动机和态度及学业成绩中分别有10.0%和33.3%的变异来自学生个体以外的学校因素。(2)在控制学生年级、性别、家庭背景和学校所在地、师资、学校学生总体家庭社会经济地位(SES)等基本特征后,学校总体和学生个体知觉两个层次的学校心理环境对学生学习动机和态度的影响均显著大于其对学生学业成绩的影响;与个体知觉到的心理环境相比,学校总体心理环境对学生学业成绩的作用相对较强。(3)学校总体心理环境对学生学业表现的影响受到学校所在地、学校师资水平、学校学生总体社会经济地位的显著调节。学校总体心理环境对处于师资水平较低、平均家庭年收入较低学校学生的学习动机和态度具有相对更强的保护作用;对处于农村、师资水平较低、平均家长文化水平较低学校学生的学业成绩具有相对更强的保护性作用。上述结果表明,学校总体心理环境对学生学业表现具有重要影响,个体知觉到的心理环境仅部分中介其作用;且学校总体心理环境对学生学业表现的作用强度受到学校所在地、师资水平和学校学生总体社会经济地位的调节。     

In the eye of the examinee: likable examiners interfere with performance

We investigated effects of examiners’ ascribed likability and examiners’ gender on test performance during a standardized face-to-face testing situation assessing self-estimated and de facto verbal knowledge. One hundred fourteen nonpsychology students were individually tested by one of 22 examiners. A moderated regression analysis revealed a significant three-way interaction of test taker’s gender, examiner’s gender, and examiner’s likability on de facto knowledge: Men and women showed lower scores on de facto knowledge with a same-gender examiner rated as likable compared to their performance with a likable opposite-gender examiner or in interaction with a nonlikable examiner.     

Parallel reading and mathematics growth for English learners: Does timing of reclassification matter?

This study compared reading and mathematics growth trajectories in a statewide dataset of 33,715 students across third through fifth grades. Specifically, we examined growth for English Learners (ELs) who were reclassified as no longer needing English Language services at different grade levels as compared to their never-EL peers. Overall, EL students performed significantly below never-EL students on reading and mathematics assessments at Grade 3, with EL students making greater academic gains across time points than never-EL students. Students who were reclassified after third grade and after fourth grade closed, or began to close, the academic opportunity gap by the end of fifth grade, providing promising evidence suggesting that reclassification policies are adequate for identifying those students who no longer need EL services. Students who continued to be classified as EL from third through fifth grades continued to score significantly lower than never-ELs, and in many cases other EL groups, indicating that there exists a group of EL students who continue to make inadequate reading and mathematics gains across late elementary school. Implications for policy and practices that support EL students' mathematics and reading growth are discussed.     

School profiles of at-risk student concentration: Differential growth in oral reading fluency

The present study provides a data-driven approach to identifying groups of schools based on the concentration of at-risk students the school serves. The percentage of English language learners, minority students, and students eligible for free or reduced priced lunch were used as indicators in a latent profile analysis of 569 schools. The goal of the present study was to determine whether school-level average student reading performance varied as a function of the groups identified in the latent profile analysis. To do so, groups extracted by the latent profile analysis were used as school-level predictors of growth in oral reading fluency, which was modeled at the within-student level of a three-level hierarchical growth curve model. Oral reading fluency was measured at four points during the year in a large cross-sectional sample of first-, second-, and third-grade students. Results indicated that schools were able to be classified into four distinct groups based on their concentrations and types of at-risk students. Further, in all three grades, there were significant differences between the four identified groups observed in average reading fluency scores at the beginning of the year, the end of the year, and growth during the year indicating that groups based on school-concentration of at-risk students were significantly related to average student achievement in reading ability.     

How accurate are interpretations of curriculum-based measurement progress monitoring data? Visual analysis versus decision rules

Curriculum based measurement of oral reading (CBM-R) is used to monitor the effects of academic interventions for individual students. Decisions to continue, modify, or terminate these interventions are made by interpreting time series CBM-R data. Such interpretation is founded upon visual analysis or the application of decision rules. The purpose of this study was to compare the accuracy of visual analysis and decision rules. Visual analysts interpreted 108 CBM-R progress monitoring graphs one of three ways: (a) without graphic aids, (b) with a goal line, or (c) with a goal line and a trend line. Graphs differed along three dimensions, including trend magnitude, variability of observations, and duration of data collection. Automated trend line and data point decision rules were also applied to each graph. Inferential analyses permitted the estimation of the probability of a correct decision (i.e., the student is improving – continue the intervention, or the student is not improving – discontinue the intervention) for each evaluation method as a function of trend magnitude, variability of observations, and duration of data collection. All evaluation methods performed better when students made adequate progress. Visual analysis and decision rules performed similarly when observations were less variable. Results suggest that educators should collect data for more than six weeks, take steps to control measurement error, and visually analyze graphs when data are variable. Implications for practice and research are discussed.     

The impact of omitting the interaction between crossed factors in cross‐classified random effects modelling

Cross‐classified random effects modelling (CCREM) is a special case of multi‐level modelling where the units of one level are nested within two cross‐classified factors. Typically, CCREM analyses omit the random interaction effect of the cross‐classified factors. We investigate the impact of the omission of the interaction effect on parameter estimates and standard errors. Results from a Monte Carlo simulation study indicate that, for fixed effects, both coefficients estimates and accompanied standard error estimates are not biased. For random effects, results are affected at level 2 but not at level 1 by the presence of an interaction variance and/or a correlation between the residual of level two factors. Results from the analysis of the Early Childhood Longitudinal Study and the National Educational Longitudinal Study agree with those obtained from simulated data. We recommend that researchers attempt to include interaction effects of cross‐classified factors in their models.     

A Social Disorganization Perspective on Bullying-Related Attitudes and Behaviors: The Influence of School Context

Social disorganization theory suggests that certain school-level indictors of disorder may be important predictors of bullying-related attitudes and behaviors. Multilevel analyses were conducted on bullying-related attitudes and experiences among 22,178 students in 95 elementary and middle schools. The intraclass correlation coefficients indicated that 0.6–2% of the variance in victimization, 5–10% of the variance in retaliatory attitudes, 5–6% of the variance in perceptions of safety, and 0.9% of the variance in perpetration of bullying was associated with the clustering of students within schools. Although the specific associations varied somewhat for elementary schools as compared to middle schools, the hierarchical linear modeling analyses generally suggested that school-level indicators of disorder (e.g., student–teacher ratio, concentration of student poverty, suspension rate, and student mobility) were significant predictors of bullying-related attitudes and experiences. Student-level characteristics (i.e., sex, ethnicity, status in school) were also relevant to students’ retaliatory attitudes, perceptions of safety, and involvement in bullying. Implications for school-based research and violence prevention are provided.     

Effect of initial status and growth in pseudoword reading on Spanish reading comprehension at the end of first grade

This study analyzes the effect of initial status and growth in Spanish pseudoword reading in kindergarten and first grade on reading comprehension in Spanish at the end of first grade. One hundred and sixty-eight Spanish-speaking English learners who were learning to read in Spanish and English in the United States participated in the study. Results of hierarchical linear modeling indicate that students grew, on average, by 95 letter-sounds in Spanish from the middle of kindergarten to the end of first grade. Structural equation modeling indicated that 53% of the variance in Spanish reading comprehension at the end of first grade was explained by Spanish initial status on pseudoword reading, Spanish kindergarten overall reading performance, and growth in Spanish pseudoword reading. Findings are important in a Response to Intervention approach where screening and progress monitoring of pseudoword reading helps educators determine the level of support beginning readers need to acquire the alphabetic principle, an important skill that contributes substantially to Spanish reading comprehension.