International Semiotics: Item Difficulty and the Complexity of Science Item Illustrations in the PISA-2009 International Test Comparison

We examined multimodality (the representation of information in multiple semiotic modes) in the context of international test comparisons. Using Program of International Student Assessment (PISA)-2009 data, we examined the correlation of the difficulty of science items and the complexity of their illustrations. We observed statistically significant mean correlation coefficient differences favoring higher-ranking over lower-ranking jurisdictions and non-Western over Western jurisdictions. We examined this correlation as a function of the variety of forms of representation of information used in illustrations. For the highest-ranking jurisdictions, the correlation tended to be positive and its magnitude tended to increase with the number of forms of representation. For the lowest-ranking jurisdictions, the correlation tended to be negative and its magnitude tended to increase with the number of forms of representation. Increased illustration complexity appears to be an affordance in making sense of science items for students from the highest-ranking jurisdictions and a challenge for students from the lowest-ranking jurisdictions. Our findings support the notion that integrating information provided in multiple semiotic modes is critical to science achievement and underscore the importance for science standards and international assessment frameworks to address this skill as intrinsic to (rather than an aspect of) science proficiency.