International Semiotics: Item Difficulty and the Complexity of Science Item Illustrations in the PISA-2009 International Test Comparison

We examined multimodality (the representation of information in multiple semiotic modes) in the context of international test comparisons. Using Program of International Student Assessment (PISA)-2009 data, we examined the correlation of the difficulty of science items and the complexity of their illustrations. We observed statistically significant mean correlation coefficient differences favoring higher-ranking over lower-ranking jurisdictions and non-Western over Western jurisdictions. We examined this correlation as a function of the variety of forms of representation of information used in illustrations. For the highest-ranking jurisdictions, the correlation tended to be positive and its magnitude tended to increase with the number of forms of representation. For the lowest-ranking jurisdictions, the correlation tended to be negative and its magnitude tended to increase with the number of forms of representation. Increased illustration complexity appears to be an affordance in making sense of science items for students from the highest-ranking jurisdictions and a challenge for students from the lowest-ranking jurisdictions. Our findings support the notion that integrating information provided in multiple semiotic modes is critical to science achievement and underscore the importance for science standards and international assessment frameworks to address this skill as intrinsic to (rather than an aspect of) science proficiency.     

Concurrent cognitive processes in rat serial pattern learning: II. Discrimination learning,rule learning,chunk length,and multiple‐item memories

The current experiment examined the factors that determine acquisition for elements of highly structured serial patterns. Three groups of rats were trained on three patterns with parallel rule‐based hierarchical structure, but with 3‐, 4‐, or 5‐element chunks, each with a final violation element. Once rats mastered their patterns, probe patterns were introduced to answer several questions. To assess the extent to which the learned response pattern depended on intrachamber location cues for anticipating different element types, Spatial Shift Probes shifted the starting lever of patterns to locations that positioned chunk boundaries where they had never been experienced during training. To assess the extent to which a phrasing cue is necessary for rats to perform a chunk‐boundary response, a Cue Removal Probe tested whether rats would produce a chunk‐boundary response in the correct serial position if the phrasing cue was omitted. To assess the extent to which cues from multiple trials leading up to the violation element are required to anticipate the violation element, Multiple‐Item Memory Probes required rats to make an unexpected response on one of the elements in the last two chunks of the pattern prior to the violation element. The results indicated that rats used multiple concurrent learning and memory processes to master serial patterns, including discrimination learning, rule learning, encoding of chunk length, and multiple‐item memories.     

Consistent Differential Discrimination Model Estimation

A novel factor-analytic model—the differential discrimination model—for assessing individual differences in scale use has been recently introduced, together with a three-stage estimation approach for model fitting. Unfortunately, the second-stage estimator and, as a consequence, the third-stage estimator of this procedure are not consistent. In this article we show that (a) the differential discrimination model can be expressed in a structural equation model framework, and (b) consistent and simultaneous estimation of all model parameters can be achieved using standard SEM software.     

Limited‐information goodness‐of‐fit testing of diagnostic classification item response models

Despite the growing popularity of diagnostic classification models (e.g., Rupp et al., 2010, Diagnostic measurement: theory, methods, and applications, Guilford Press, New York, NY) in educational and psychological measurement, methods for testing their absolute goodness of fit to real data remain relatively underdeveloped. For tests of reasonable length and for realistic sample size, full‐information test statistics such as Pearson's X² and the likelihood ratio statistic G² suffer from sparseness in the underlying contingency table from which they are computed. Recently, limited‐information fit statistics such as Maydeu‐Olivares and Joe's (2006, Psychometrika, 71, 713) M₂ have been found to be quite useful in testing the overall goodness of fit of item response theory models. In this study, we applied Maydeu‐Olivares and Joe's (2006, Psychometrika, 71, 713) M₂ statistic to diagnostic classification models. Through a series of simulation studies, we found that M₂ is well calibrated across a wide range of diagnostic model structures and was sensitive to certain misspecifications of the item model (e.g., fitting disjunctive models to data generated according to a conjunctive model), errors in the Q‐matrix (adding or omitting paths, omitting a latent variable), and violations of local item independence due to unmodelled testlet effects. On the other hand, M₂ was largely insensitive to misspecifications in the distribution of higher‐order latent dimensions and to the specification of an extraneous attribute. To complement the analyses of the overall model goodness of fit using M₂, we investigated the utility of the Chen and Thissen (1997, J. Educ. Behav. Stat., 22, 265) local dependence statistic $X_{\mathrm{LD}}^2$ for characterizing sources of misfit, an important aspect of model appraisal often overlooked in favour of overall statements. The $X_{\mathrm{LD}}^2$ statistic was found to be slightly conservative (with Type I error rates consistently below the nominal level) but still useful in pinpointing the sources of misfit. Patterns of local dependence arising due to specific model misspecifications are illustrated. Finally, we used the M₂ and $X_{\mathrm{LD}}^2$ statistics to evaluate a diagnostic model fit to data from the Trends in Mathematics and Science Study, drawing upon analyses previously conducted by Lee et al., (2011, IJT, 11, 144).     

An unusual presentation of probable dementia: Rhyming,associations, and verbal disinhibition

We report a case of probable Alzheimer's disease who presented with the unusual feature of disinhibited rhyming. Core language skills were largely intact but generative language was characterized by semantic‐based associations, evident in tangential and associative content, and phonology‐based associations, evident in rhyming, in the context of prominent executive dysfunction. We suggest this pattern is underpinned by a failure to terminate or inhibit verbal associations resulting in a ‘loosening’ of associations at the level of conceptual preparation for spoken language.     

On the asymptotic standard error of a class of robust estimators of ability in dichotomous item response models

In item response theory, the classical estimators of ability are highly sensitive to response disturbances and can return strongly biased estimates of the true underlying ability level. Robust methods were introduced to lessen the impact of such aberrant responses on the estimation process. The computation of asymptotic (i.e., large‐sample) standard errors (ASE) for these robust estimators, however, has not yet been fully considered. This paper focuses on a broad class of robust ability estimators, defined by an appropriate selection of the weight function and the residual measure, for which the ASE is derived from the theory of estimating equations. The maximum likelihood (ML) and the robust estimators, together with their estimated ASEs, are then compared in a simulation study by generating random guessing disturbances. It is concluded that both the estimators and their ASE perform similarly in the absence of random guessing, while the robust estimator and its estimated ASE are less biased and outperform their ML counterparts in the presence of random guessing with large impact on the item response process.     

义务教育阶段学业质量监测题库建设的理论与实践

义务教育阶段学业质量监测是学生学业质量评价的重要方式,在当前国内外教育质量评价领域中发挥着极为重要的作用.加强学业质量监测的题库建设,则是促进学业质量监测工具的命制与实施从经验型向科学化迈进的必由之路.本文在对义务教育阶段学业质量监测题库的性质和功能进行阐述的基础上,重点探讨了学业质量监测题库建设的理论基础、基本流程和关键环节,并进一步分析指出了当前题库建设中有待深入研究的问题.