Comparing OECD PISA Reading in English to Other Languages: Identifying Potential Sources of Non-Invariance

The use of the Programme for International Student Assessment (PISA) across nations, cultures, and languages has been criticized. The key criticisms point to the linguistic and cultural biases potentially underlying the design of reading comprehension tests, raising doubts about the legitimacy of comparisons across economies. Our research focused on the type and magnitude of invariance or non-invariance in the PISA Reading Comprehension test by language, culture, and economic development relative to performance of the Australian English-speaking reference group used to develop the tests. Multi-Group Confirmatory Factor Analysis based on means and covariance structure (MACS) modeling was used to establish a d_MACS effect size index for each economy for the degree of non-invariance. Only three wealthy, English-speaking countries had scalar invariance with Australia. Moderate or large effects were observed in just 31% of the comparisons. PISA index of economic, social and cultural status had a moderate inverse correlation with dMACS suggesting that socioeconomic resourcing of education played a significant role in measurement invariance, while educational practice and language factors seemed to play a further small role in non-invariance. Alternative approaches to reporting PISA results consistent with non-invariance are suggested.     

Les limites de l’approche fondée sur le risque de la loi du 5 mars 2012 relative aux recherches impliquant la personne humaine,dite « loi Jardé »: Lecture critique des arrêtés du 12 avril 2018

Thanks to the publication of the two decrees of 12 April 2018, and six years after the promulgation of the Jardé law, the scope of the law of 5 March 2012 as well as the different categories of research involving the human person finally appear clearly. Beside interventional research that includes intervention on the person that is not justified by usual care, the contours of two categories of research have been precised: interventional research with minimal risk and minimal burden and “non-interventional” research, whose name is questionable. If the latest decrees have managed to reach a more coherent set of rules, the French legal and regulatory provisions remain complex and incomplete. Complex, in view of the multiplicity of statutory texts adopted since the promulgation of the Jardé law. Incomplete, because the needs for ethical evaluation provided for by the international ethical principles of research are not systematically covered by French law.     

学校氛围对学生数学成绩影响的跨文化比较:基于多水平分析的结果

学校氛围是学校中被成员所体验并对其行为产生影响的、相对持久而稳定的学校环境特征。它以成员的共同行为感知为基础,对学校和学生的发展具有重要作用。以PISA2003测试的数据为基础,采用多水平线性模型,考察了中国香港、日本、美国和土耳其的15岁学生及其所在学校的校长知觉的学校氛围对学生数学成绩的影响。结果显示:(1)学生知觉的师生关系、数学课堂秩序,校长知觉的学生士气、学校消极行为对数学成绩的显著预测作用具有较高的跨文化一致性;(2)数学课堂上的教师支持、教师负面状态、数学教师间的共识能显著预测部分国家/地区的学生数学成绩;(3)学生知觉的对学校的看法、校长知觉的教师士气对四个国家/地区的学生数学成绩均无显著预测作用。     

多维对数正态作答时间模型：对潜在加工速度多维性的探究

在心理与教育测量中, 潜在加工速度反映学生运用潜在能力解决问题的效率。为在多维测验中探究潜在加工速度的多维性并实现参数估计, 本研究提出多维对数正态作答时间模型。实证数据分析及模拟研究结果表明：(1)潜在加工速度具有与潜在能力相匹配的多维结构; (2)新模型可精确估计个体水平的多维潜在加工速度及与作答时间有关的题目参数; (3)冗余指定潜在加工速度具有多维性带来的负面影响低于忽略其多维性所带来的。     

Explaining Social Disparities in Mathematical Achievement: The Role of Motivation

This study examined the role of motivation as a mediator of the relationship between parents' socio‐economic status (SES) and children's standardized test achievement in math. We employed a one‐year longitudinal approach using Programme for International Student Assessment (PISA) 2003 and a follow‐up exam in 2004. The sample consisted of N = 6020 German students (mean age 15.5 years, SD = .55) who continued school after Grade 9 (PISA 2003) and were in Grade 10 at the time of PISA 2004. Children completed measures related to their parents' SES, math‐specific self‐concept, task‐specific and global self‐efficacy, and interest, intelligence and mathematical competence. We found a small to moderate correlation between parents' SES and children's achievement. All motivational constructs partially mediated the relationship between father's SES as well as a family index for SES (economic, social, and cultural status) and children's mathematical competence, but only math‐specific self‐concept and self‐efficacy were significant mediators for mother's SES. Even when simultaneously considering the mediating effect of children's intelligence and prior achievement, the mediation effects of motivation remained significant. These results are important for our understanding of educational equality. Copyright © 2016 European Association of Personality Psychology     

Item Calibration Samples and the Stability of Achievement Estimates and System Rankings: Another Look at the PISA Model

Using an empirically-based simulation study, we show that typically used methods of choosing an item calibration sample have significant impacts on achievement bias and system rankings. We examine whether recent PISA accommodations, especially for lower performing participants, can mitigate some of this bias. Our findings indicate that standard operational methods, while not ideal, recover underlying proficiency reasonably well and generally outperform methods that more completely include all participants. Translating results onto the PISA scale, the calibration sample can induce bias of up to 12.49 points, which is important given that standard errors are around three points. Although ranking correlations are at least.95, we note the policy implications of slight ranking changes. Our findings indicate that limited accommodations targeted at low achieving educational systems do not outperform either of the other methods considered. Research that further explores accommodations for heterogeneous populations is recommended.     

阅读投入对阅读素养影响的跨文化比较研究——以PISA 2009为例

基于PISA2009的数据,从国际比较的视角出发,使用两水平线性模型,考察阅读投入对15岁学生阅读素养成绩的影响,并分析该影响模式在中国上海、韩国、芬兰和美国之间的异同。结果发现:(1)总体上,在四种不同文化背景下,学生的阅读投入均与阅读素养表现关系密切。(2)具体到构成阅读投入的各个变量,阅读兴趣、控制策略、理解和识记元认知策略以及总结元认知策略均对阅读素养表现有正向预测作用,这在不同文化背景中表现出基本一致的模式;阅读时间量、阅读材料的多样性、网上阅读活动的多样性、校内阅读变量、记忆以及精细化加工策略对阅读素养表现的影响在不同国家/地区表现出不同的关系模式。此外,本研究探讨了从增加个性化阅读时间、增加阅读文本的多样性以及进行有效学习策略的指导三个方面培养学生的阅读投入。     

Relationships of Cognitive and Metacognitive Learning Strategies to Mathematics Achievement in Four High-Performing East Asian Education Systems

The authors examined the relationships of cognitive (i.e., memorization and elaboration) and metacognitive learning strategies (i.e., control strategies) to mathematics achievement among 15-year-old students in 4 high-performing East Asian education systems: Shanghai-China, Hong Kong-China, Korea, and Singapore. In all 4 East Asian education systems, memorization strategies were negatively associated with mathematics achievement, whereas control strategies were positively associated with mathematics achievement. However, the association between elaboration strategies and mathematics achievement was a mixed bag. In Shanghai-China and Korea, elaboration strategies were not associated with mathematics achievement. In Hong Kong-China and Singapore, on the other hand, elaboration strategies were negatively associated with mathematics achievement. Implications of these findings are briefly discussed.     

Are cognitive differences between countries diminishing? Evidence from TIMSS and PISA

Cognitive ability differences between countries can be large, with average IQs ranging from approximately 70 in sub-Saharan Africa to 105 in the countries of north-east Asia. A likely reason for the great magnitude of these differences is the Flynn effect, which massively raised average IQs in economically advanced countries during the 20th century. The present study tests the prediction that international IQ differences are diminishing again because substantial Flynn effects are now under way in the less developed “low-IQ countries” while intelligence is stagnating in the economically advanced “high-IQ countries.” The hypothesis is examined with two periodically administered scholastic assessment programs. TIMSS has tested 8th-grade students periodically between 1995 and 2011 in mathematics and science, and PISA has administered tests of mathematics, science and reading between 2000 and 2009. In both TIMSS and PISA, low-scoring countries tend to show a rising trend relative to higher-scoring countries. Despite the short time series of only 9 and 16 years, the results indicate that differences between high-scoring and low-scoring countries are diminishing on these scholastic achievement tests. The results support the prediction that through a combination of substantial Flynn effects in low-scoring countries and diminished (or even negative) Flynn effects in high-scoring countries, cognitive differences between countries are getting smaller on a worldwide scale.     

Extreme response style as a cultural response to climato‐economic deprivation

We investigated the effects of climato‐economic harshness on extreme response style. Climato‐economic theorising postulates that a more threatening climate in poorer countries, in contrast to countries with a more comforting climate and richer countries with a more challenging climate, triggers intolerance of ambiguity and uncertainty avoidance inherent to conservatism, in‐group favouritism and autocracy. Scores of extreme response style at country level, a proxy of this cluster of cultural characteristics, were extracted from students' responses in the Programme for International Student Assessment to test the hypothesis. In a series of hierarchical regression analysis across 64 countries, cold demands, heat demands and GDP per capita showed a highly significant interaction effect on extreme response style, predicting in total 30.7% of the variance. Extreme response style was highest in poorer countries with higher climatic demands, and lowest in richer countries with lower climate demands. Implications are discussed.