Here’s how the rules were changed: to pass Sweden’s PISA results

This summer, Expressen was able to reveal that large numbers of foreign-born students were incorrectly withdrawn from participating in the latest PISA study, in violation of OECD regulations.

If the rules had been followed, all the increase praised by Swedish politicians in the results of the latest test would have apparently been replaced by a decrease, Expressen showed.

The information prompted Education Minister Anna Ekström (S) to immediately request that the OECD, which is responsible for PISA, review the Swedish results again.

Recently, Expressen was able to reveal how the National Agency for Education was allowed to influence the OECD review, among other things, by determining the “content, time frame and cost” of the work.

The National Audit Office decided shortly thereafter to initiate its own review of the Swedish PISA study.

The report approves the result

And now the OECD review report has been submitted to the National Agency for Education and the government.

The report approves Sweden’s results in PISA 2018, with the reservation that exact data is lacking in several areas.

But it is also clear from the report that the OECD has now chosen to change its own regulations, to the benefit of Sweden. According to previous rules, Swedish schools could only exclude foreign-born students who had been taught Swedish for less than a year.

The OECD now writes that Sweden was allowed in PISA 2018 to exempt students who then attended Swedish school for 3.5 years. The organization refers to the fact that newcomers to Sweden attend preparatory classes for up to 2.5 years before starting to receive regular tuition.

All official OECD results reports, published since the first study, state that newcomer students can be exempted according to the following criteria:

“Limited knowledge of the language of instruction: the student does not have any of the languages ​​of instruction of the country as a mother tongue and has lived in the country for less than a year.”

The Swedish National Education Agency’s own instructions for schools

The National Education Agency has repeatedly told Expressen that its own instruction to schools has been that only students who have been taught Swedish for less than a year can be exempt from PISA. Statistics from the National Education Agency have assumed, in their own calculations, that only students who have been in Sweden for less than one year and three months can be excluded from PISA.

According to the school ordinance, which has been decided by the government, asylum-seeking students must start school within a month after arriving in Sweden.

Expressen has sent a survey to those responsible for PISA in all OECD countries, to find out how they have interpreted the rules. Of 36 countries, 17 responded to the survey.

16 of those surveyed say that foreign-born students who were excluded from PISA 2018 had generally been enrolled in the school system for less than 12 months. Three of these countries state that the answer to the question is based on their instruction to schools, but they have no data to prove it. The seventeenth country to respond to the survey, New Zealand, writes that the question is impossible to answer because there is no data, but that they have let the schools make the decisions.

Finland has complied with the rules

Finland is one of the countries that has replied that only pupils who have been enrolled in the school system for less than 12 months have been excluded.

– We were quite strict, we said that if these are the rules, they should be followed, says Arto Ahonen, PISA project manager in Finland.

He says he has previously spoken to the Swedish National Education Agency about the exclusions.

– I talked about this with my Swedish colleagues and it gave me the impression that they gave the schools too many options on which students can be excluded, says Arto Ahonen.

You think Finland tried to include as many students as possible.

– We want as much information about the school system as possible, especially about the weakest students, so that they can get more support.

Ahonen says he also reacted to the PISA results from China, where only schools in certain regions participated and, in principle, only high-achieving students are included in the sample. He believes there may be other shortcomings in the international comparison.

– But if, instead, you mainly use the PISA study to obtain information on how the country’s own school system works, it is Sweden’s loss if the result is misleading, says Ari Ahonen.

In Germany, the EU country after Sweden that during the refugee crisis granted the most per capita residence permits to asylum seekers, only a very small proportion of newcomer students were removed from PISA 2018.

Germany’s reaction: “Strange”

One of the people responsible for PISA in Germany says that only foreign-born students who have been in the school system for less than 12 months have been excluded.

When the person is allowed to participate in the new OECD review report and realizes that Sweden has been allowed to exempt students who have attended school for up to 3.5 years, they say:

– Did the OECD give Sweden these rules? Are these the rules the OECD gave Sweden before the study?

I didn’t know they had different rules for different countries.

No, Sweden had the same instructions as Germany before the test. This report has already been published.

– Now? That’s a bit weird. We only know the usual rules, because the OECD gave these rules to all countries, as I understand it. I didn’t know they had different rules for different countries.

– One of the most important things about a study like PISA, where you make an international comparison between different countries, is that you use the same method in all countries. If you don’t, you won’t be able to compare the results.

Expressen explains that the OECD now believes that foreign-born students must have received regular instruction in all subjects in Swedish for one year to have to write the exam.

– Well, but I can’t read it in the usual OECD standards, which we have used.

Don’t you interpret it that way?

– You do? I can only say that I wonder why these rules were only released after the study. This should be explained before countries make the selection.

The new audit report also states the following:

If Sweden is allowed to exclude students who at the time of the test had been enrolled in the school system for 3.5 years, which is in conflict with the OECD’s own regulations and how it has been interpreted by at least others 16 countries, it still seems that Sweden has withdrawn too many foreign-born students.

Sweden’s results had fallen so far

A footnote on the last page of the OECD report states that Sweden’s results would have fallen from 506 to 501 points if this group of students had participated in PISA, provided they had performed as the participating students who scored. worst performance. This decrease corresponds to almost the entire increase in Swedish reading comprehension since the last test.

Therefore, this calculation still assumes that Sweden has been allowed to exclude foreign-born students who have attended Swedish schools for up to 3.5 years, unlike other OECD countries with which Expressen has been in contact. Contact.

Expressen sought out Yuri Belfali, the division manager responsible for the PISA review at the OECD, who declined an interview. In an email to Belfali, Expressen refers to the writing of OECD performance reports and contacts with other participating countries, and asks why the OECD has now changed its rules.

Yuri Belfali responds with a passage from the report, in Swedish translation:

“‘… Has received less than one year of instruction in the language of instruction’ means less than one year of instruction in Swedish in a Swedish context (and is different from having been enrolled in the Swedish school system for less than one year ) “.

When Expressen again refers to the writings in all OECD performance reports and the statements of 16 other OECD countries, there is no answer.