Comparing OECD PISA reading in English to other languages

Mustafa Asil, Gavin Brown

Research output: Contribution to conferenceAbstractResearchpeer-review

1 Downloads (Pure)



The OECD PISA results have been used as a catalyst for educational reform, even though the evidence for invariance across languages is weak. Scores obtained from tests that are subsequently adapted cannot be assumed comparable unless scalar equivalence across languages is present.

The use of PISA across nations, cultures and languages has been criticized. The key criticisms point to the linguistic and cultural biases potentially underlying the design of reading comprehension tests and which raise doubts about the legitimacy of comparisons across language boundaries.


Our research question was: What level of invariance is seen in the PISA Reading Comprehension test by language and culture relative to performance by the Australian English speaking reference group?


To ensure equivalent sample sizes 500 students were randomly selected from PISA 2009 dataset for each country. MG-CFA based on MACS was used to examine levels of measurement invariance (MI). Changes in CFI ≤.01 were used to determine invariance. A scatterplot of ∆CFI against ∆Chi-square was used to identify patterns with respect to language and country.


Measurement model had good fit for each country. Only Ireland, New Zealand, UK, English Canada and USA were invariant to Australia. Scatterplot with respect to language clarifies that language alone is not sufficient for invariance. For example, Trinidad and Tobago uses English but is not a high-wealth society. Spanish in Spain is relatively close to Australia compared to poorer countries of Latin America.


Results indicate that complex factors to do with educational practice and socio-economic resourcing of education, rather than language per se do interfere with the MI of the PISA reading comprehension results. These results suggest that PISA reading comprehension tests are very much a product of a wealthy, Anglo-Commonwealth society and educational approaches to reading instruction.
Original languageEnglish
Number of pages1
Publication statusPublished - 2014
Externally publishedYes
EventThe 9th Conference of the International Test Commission:Global and Local Challenges for Best Practices in Assessment - Miramar Palace in San Sebastian, Sen Sebastian, Spain
Duration: 2 Jul 20145 Jul 2014


ConferenceThe 9th Conference of the International Test Commission:Global and Local Challenges for Best Practices in Assessment
CitySen Sebastian
OtherThe 9th Conference of the International Test Commission which was organized in San Sebastian, Spain, 2-5 July 2014, has just come to an end. Excellently organized and a biiiig success: congratulations to the local organizer, Prof. Paula Elosua, and thank you to all attendants and sponsors who made such a great event possible.

Under the theme Global and Local Challenges for Best Practices in Assessment, ITC2014 has showcased new frontiers of assessment as a means for improving and developing psychological and educational tests and test uses. ITC2014 was held at the Miramar Palace in San Sebastian, Spain. The Miramar Palace was built in 1893, and is located on the beautiful La Concha Bay. Miramar offers one of the most spectacular views of San Sebastian, which itself is an amazing city in the North-East part of Spain.
Internet address


Dive into the research topics of 'Comparing OECD PISA reading in English to other languages'. Together they form a unique fingerprint.

Cite this