Automatic and controlled processes in the first- and second-language reading of fluent bilinguals

Many fluent bilinguals read their two languages with equal levels of comprehension but read their second language at a slower rate. The present study examined whether, compared with first language reading, slower second-language reading is associated with reduced involvement of automatic processing during lexical access. Subjects were bilinguals with fluent speaking and listening skills under ordinary conditions of communication and with equivalent comprehension of their first and second languages when reading and listening under speeded conditions. Half these subjects, however, read their first and second languages equally fast, and half read the second language more slowly than the first. Subjects were tested on a lexical decision task that manipulated expectations about the semantic relatedness of prime and target words and the stimulus onset asynchrony between them. Bilinguals with equal first- and second-language reading rates produced in each language a pattern of reaction times suggesting automatic processing, whereas bilinguals with a slower second-language reading rate did so in their first language but not in their second.     

Reporting subscores for institutions

Recently, there has been an increasing level of interest in reporting subscores for components of larger assessments. This paper examines the issue of reporting subscores at an aggregate level, especially at the level of institutions to which the examinees belong. A new statistical approach based on classical test theory is proposed to assess when subscores at the institutional level have any added value over the total scores. The methods are applied to two operational data sets. For the data under study, the observed results provide little support in favour of reporting subscores for either examinees or institutions.     

Does subgroup membership information lead to better estimation of true subscores?

Haberman (2008) suggested a method to determine if subtest scores have added value over the total score. The method is based on classical test theory and considers the estimation of the true subscores. Performance of subgroups, for example, those based on gender or ethnicity, on subtests is often of interest. Researchers such as Stricker (1993) and Livingston and Rupp (2004) found that the difference in performance between the subgroups often varies over the different subtests. We suggest a method to examine whether the knowledge of the subgroup membership of the examinees leads to a better estimation of the true subscores. We apply our suggested method to data from two operational testing programmes. The knowledge of the subgroup membership of the examinees does not lead to a better estimation of the true subscore for the data sets.     

Reporting Diagnostic Scores in Educational Testing: Temptations,Pitfalls, and Some Solutions

Diagnostic scores are of increasing interest in educational testing due to their potential remedial and instructional benefit. Naturally, the number of educational tests that report diagnostic scores is on the rise, as are the number of research publications on such scores. This article provides a critical evaluation of diagnostic score reporting in educational testing. The existing methods for diagnostic score reporting are discussed. A recent method (Haberman, 2008a Haberman, S. J. 2008a. When can subscores have value?. Journal of Educational and Behavioral Statistics, 33: 204–229. [Crossref], [Web of Science ®] , [Google Scholar]) that examines if diagnostic scores are worth reporting is reviewed. It is demonstrated, using results from operational and simulated data, that diagnostic scores have to be based on a sufficient number of items and have to be sufficiently distinct from each other to be worth reporting and that several operationally reported subscores are actually not worth reporting. Several recommendations are made for those interested to report diagnostic scores for educational tests.     

Race Differences on a Reading Comprehension Test With and Without Passages

Previous research has found that test takers can score above chance level on reading comprehension tests even when the passages are omitted. The present research investigated whether the effect would vary as a function of race. In Study 1, 386 participants completed a reading test with the passages omitted. General mental ability and race were significantly correlated with test performance. In Study 2, 827 job applicants completed the test as part of an entry-level selection battery. Eliminating items for which large race differences existed in Study 1 had no real effect on the size of the Black-White mean difference.     

Estimating Between-Person and Within-Person Subscore Reliability with Profile Analysis

Subscores are of increasing interest in educational and psychological testing due to their diagnostic function for evaluating examinees' strengths and weaknesses within particular domains of knowledge. Previous studies about the utility of subscores have mostly focused on the overall reliability of individual subscores and ignored the fact that subscores should be distinct and have added value over the total score. This study introduces a profile reliability approach that partitions the overall subscore reliability into within-person and between-person subscore reliability. The estimation of between-person reliability and within-person reliability coefficients is demonstrated using subscores from number-correct scoring, unidimensional and multidimensional item response theory scoring, and augmented scoring approaches via a simulation study and a real data study. The effects of various testing conditions, such as subtest length, correlations among subscores, and the number of subtests, are examined. Results indicate that there is a substantial trade-off between within-person and between-person reliability of subscores. Profile reliability coefficients can be useful in determining the extent to which subscores provide distinct and reliable information under various testing conditions.     

Reporting of Subscores Using Multidimensional Item Response Theory

Recently, there has been increasing interest in reporting subscores. This paper examines reporting of subscores using multidimensional item response theory (MIRT) models (e.g., Reckase in Appl. Psychol. Meas. 21:25–36, 1997; C.R. Rao and S. Sinharay (Eds), Handbook of Statistics, vol. 26, pp. 607–642, North-Holland, Amsterdam, 2007; Beguin & Glas in Psychometrika, 66:471–488, 2001). A MIRT model is fitted using a stabilized Newton–Raphson algorithm (Haberman in The Analysis of Frequency Data, University of Chicago Press, Chicago, 1974; Sociol. Methodol. 18:193–211, 1988) with adaptive Gauss–Hermite quadrature (Haberman, von Davier, & Lee in ETS Research Rep. No. RR-08-45, ETS, Princeton, 2008). A new statistical approach is proposed to assess when subscores using the MIRT model have any added value over (i)  the total score or (ii)  subscores based on classical test theory (Haberman in J. Educ. Behav. Stat. 33:204–229, 2008; Haberman, Sinharay, & Puhan in Br. J. Math. Stat. Psychol. 62:79–95, 2008). The MIRT-based methods are applied to several operational data sets. The results show that the subscores based on MIRT are slightly more accurate than subscore estimates derived by classical test theory.     

An Empirical Investigation of Population Invariance in the Value of Subscores

Recently there has been an increasing level of interest in subtest scores, or subscores, for their potential diagnostic value. Haberman (2008 Haberman, S. J. 2008. When can subscores have value?. Journal of Educational and Behavioral Statistics, 33: 204–229. [Crossref], [Web of Science ®] , [Google Scholar]) suggested a method to determine if a subscore has added value over the total score. Researchers have often been interested in the performance of subgroups—for example, those based on gender or ethnicity—on subtests. Several researchers found that the difference in performance between the gender-based subgroups varied over the different subtests. In this article, we examine whether the added values of the subscores vary between subgroups using data from several operational tests, including an international English proficiency test. For these data sets, the added values of the subscores occasionally vary over the subgroups, but the added values of the augmented subscores are invariant over the subgroups.     

Exploring a Source of Uneven Score Equity across the Test Score Range

Score equity assessment (SEA) refers to an examination of population invariance of equating across two or more subpopulations of test examinees. Previous SEA studies have shown that score equity may be present for examinees scoring at particular test score ranges but absent for examinees scoring at other score ranges. No studies to date have performed research for the purpose of understanding why score equity can be inconsistent across the score range of some tests. The purpose of this study is to explore a source of uneven subpopulation score equity across the score range of a test. It is hypothesized that the difficulty of anchor items displaying differential item functioning (DIF) is directly related to the score location at which issues of score inequity are observed. The simulation study supports the hypothesis that the difficulty of DIF items has a systematic impact on the uneven nature of conditional score equity.     

Reading, hearing, understanding]

The relationship between reading comprehension, listening comprehension, and two indicators of intelligence (verbal reasoning, speed of information processing) is analyzed on the basis of a hierarchical monistic model. Two tests (reading comprehension, listening comprehension) were administered to 221 4th graders in classroom context at two sessions. Order of administration was balanced. Additionally, verbal reasoning and speed of information processing were assessed, marks in German were collected. On the whole, performance in listening comprehension exceeds reading comprehension as expected for children at that age. In spite of this mean difference a high correlation between the variables was found. In accordance with theory, the relationship between reading and listening comprehension is stronger when the influence of reading specific abilities is lower and reading material is less complex. Listening comprehension appears to be of more relevance to the prediction of reading comprehension than verbal intelligence, speed of information processing, and marks in German, even when listening comprehension is introduced as the last predictor in multiple regression analysis. It is suggested to emphasize the role of listening comprehension in the assessment and prognosis of reading disabilities.     

基于双因子模型的测验总分和维度分的合成方法

双因子模型可以同时包含一个全局因子和多个局部因子,在描述多维测验结构时有其独特优势,近些年应用越来越广泛。文章基于双因子模型,提出了4种合成总分和维度分的方法,分别是:原始分法,加和法,全局题目加权加和法和局部题目加权加和法,并采用模拟的方法,在样本量、测验长度、维度间相关变化的条件下考察了这些方法与传统多维IRT方法的表现。最后,通过实证研究对结果进行了验证。结果显示:(1)全局加权加和法和局部加权加和法,尤其是局部加权加和法合成的总分和维度分与真值最接近、信度最高。(2)在维度间相关较高,测验长度较长的条件下,局部加权加和法的结果较好,部分条件下甚至优于多维IRT法。(3)仅有局部加权加和法合成的维度分能够反应维度间真实的相关关系。     

An Examination of the Simple View of Reading Among Beginning Readers in Malay

This study examines the simple view of reading from the perspective of a language other than English, the Malay language. The aims of the study were to determine (a) the contributions of decoding and listening comprehension to reading comprehension in Malay; (b) which model, the multiplicative model or the additive model, of the simple view of reading contributes more toward the variance in reading comprehension; and (c) whether adding the speed of processing factor to the multiplicative model improves its power to predict reading comprehension among beginning readers in Malay. A sample of 117 beginning first-year readers attending Malaysian schools were assessed on a battery of reading and reading-related skills measures after 6 months of initial instruction in reading. Results indicated that reading comprehension was largely explained by the decoding component, with listening comprehension adding only a small amount to the variance accounted for. The additive model consisting of the summation of decoding and listening comprehension was found to be a better predictor of reading comprehension in Malay. Results also showed that speed of processing did not account for unique variance in reading comprehension. We propose that listening comprehension did not contribute substantially to the variance because it is artefactually reduced as a result of the level of text typically read by beginning readers.     

垂直等值的应用及最新发展述评

由于实际的需求,垂直等值方法在近些年来迅速发展。但从垂直等值方法的整个过程来看,包括垂直等值的选用、双向细目编制、发展性量尺的构建、程序的选择和结果的报告,仍存在大量有待解决的问题。同时,随着其他测量方法的发展与进步,垂直等值与之相结合从而获得了进一步的完善。综观之,垂直等值方法的发展与完善,一方面依赖于各种模型和参数估计方法的改进与创新,另一方面还依赖于研究者对学业发展本质的不断深入认识。     

大学英语四、六级考试分数等值研究

对现有的大学英语四、六级考试分数等值模式中存在的若干问题进行了深入的分析,并提出了新的解决方案——一个基于铆题设计和两参数IRT模型的解决方案。主要包括：（1）用两参数逻辑斯蒂模型替代原来的Rasch模型,以改进题目模型的适合性;（2）用共同题目的等值设计取代原来的共同被试等值设计,解决共同被试等值设计中,等值考生的动机水平难以控制的难题;（3）建立专用的等值用题库,并且一次性完成其中铆题的预测和参数标定工作,以解决原来等值模式中存在的误差累积问题。同时,由于铆题的保密工作难度较小,因此,等值专用题库对保证等值结果的可靠性也具有重大意义;（4）本文还对新的分数等值方案进行了真实的考试数据等值计算实验,并得到了一个令人满意的分数等值结果。     

Improving Listening Skills in English as a Foreign Language by Reading Rather than Listening: A Cognitive Load Perspective

This study investigated the consequences of simultaneously reading and listening to the same materials when learning English as a foreign language. During acquisition, native Arabic‐speaking university students were asked to learn some English words and sentences either by reading them or by simultaneously reading and listening to the same spoken material. Following acquisition students were given reading, writing, and listening tests. The findings from the three experiments indicated that participants exposed to reading alone performed better on listening tests than participants exposed to a reading and listening condition. No differences were found on the reading and writing tests. The results, discussed within a cognitive load theory framework, suggest that at least some categories of learners will enhance their listening skills more by reading the materials only rather than simultaneously reading and listening. Copyright © 2011 John Wiley & Sons, Ltd.     

汉语综合理解能力的测试及其初步分析

采用两个测试,多媒语言理解测试和文章的阅读理解测试,分别测量了大学生的汉语综合理解能力和阅读理解能力.多媒语言理解测试根据Gernsbacher的多媒理解量表编制而成.文章的阅读理解测试是莫雷编制的<语文阅读水平测量>中的一个分测验.测斌结果表明(1)两个测试之间有显著相关和较高的一致性,说明阅读是汉语综合理解的一个重要方面;(2)汉语综合理解能力有两个成分外部语言和内部语言理解能力;(3)书面语言的阅读理解与口头语言的听力理解显著相关,说明外部语言的理解受信息传递方式的影响较小.     

The Role of Dichotic Listening Performance and Tasks of Executive Functions in Reading Impairment: A Discriminant Function Analysis

This study focused on the role of dichotic listening performance for the identification of reading impaired subtypes. Dichotic listening (DL), using verbal stimuli, has shown to be a valid measure of language lateralization. Usually, lateralization is estimated from the proportion of right ear over left ear accuracy during a free recall test procedure. However, it has been suggested that a more accurate estimate of laterality can be obtained by using a directed attention procedure. A sample of 43 reading disabled children of whom 18 showed signs of impaired language comprehension skills and 25 without language comprehension impairments, were compared to 20 age, and gender, matched controls on dichotic listening performance in both an unbiased free recall task and in a directed attention task using consonant-vowel syllables as dichotic stimuli. A laterality index was calculated for left and right ear stimuli reported during both the free recall condition and the attended ear-scores for the two directed attention conditions. Although both DL procedures yielded significant group differences, with a lower laterality score for the reading disabled compared to the controls, there was no main effect of DL-procedure or group × procedure interaction. Taken alone, DL performance could correctly classify 42% of the reading impaired samples, but together with other measures of executive functions, discriminant function analyses yielded 90.74% accuracy in classifying reading impaired children. The result indicates that DL together with tests of executive functions are valuable tools for assessment of reading impaired subjects.     

A cognitive tool to diagnose predominantly inattentive ADHD behavior

Poor performance on tests of reading comprehension could be the result of weak word-recognition skills, inconsistent attention (ADD), or a combination of the two. Identifying the source of the reading disability (RD) reliably has been difficult because inconsistent attention interferes with reading and weak word recognition skill makes attention wander. The situation is further complicated by the fact that there are no objective diagnostic tests for ADD (Breggin, 1998; Diller, 1998). We proposed a new model of differential diagnosis of ADHD-I/RD and field-tested its utility in two studies. The new diagnostic procedure utilizes intra-individual differences seen in the performance of at-risk learners on tasks related to reading that vary in the degree of sustained attention required for successful performance. The hypothesis is that children whose attention is inconsistent would perform more poorly on tests such as listening comprehension, which require sustained attention, than on tests such as reading comprehension, which are more tolerant of inattention. Such differences will not be seen in the test scores of children who have only a reading disability because their performance on reading tests is determined more by the difficulty level of the tests than by the sensitivity of the tests to attention. The validity of this new model was evaluated by determining the relationship between differences seen in the scores of tests that differ in their attention requirement and the degree of inconsistency in sustained attention as measured by Conners' CPT. The results of the two studies indicate this to be a viable approach. The results of the second study are presented in this report.     

The role of dichotic listening performance and tasks of executive functions in reading impairment: a discriminant function analysis.

This study focused on the role of dichotic listening performance for the identification of reading impaired subtypes. Dichotic listening (DL), using verbal stimuli, has shown to be a valid measure of language lateralization. Usually, lateralization is estimated from the proportion of right ear over left ear accuracy during a free recall test procedure. However, it has been suggested that a more accurate estimate of laterality can be obtained by using a directed attention procedure. A sample of 43 reading disabled children of whom 18 showed signs of impaired language comprehension skills and 25 without language comprehension impairments, were compared to 20 age, and gender, matched controls on dichotic listening performance in both an unbiased free recall task and in a directed attention task using consonant-vowel syllables as dichotic stimuli. A laterality index was calculated for left and right ear stimuli reported during both the free recall condition and the attended ear-scores for the two directed attention conditions. Although both DL procedures yielded significant group differences, with a lower laterality score for the reading disabled compared to the controls, there was no main effect of DL-procedure or group x procedure interaction. Taken alone, DL performance could correctly classify 42% of the reading impaired samples, but together with other measures of executive functions, discriminant function analyses yielded 90.74% accuracy in classifying reading impaired children. The result indicates that DL together with tests of executive functions are valuable tools for assessment of reading impaired subjects.