Late on Tuesday evening, the results of the 2012 Programme for International Student Assessment (PISA) tests were officially released. The resulting rhetoric was very predictable.
So New Zealand education is failing on the international stage? So this means the government needs to crack down harder on errant schools and lazy/incompetent teachers? That even more National Standards are needed?
Or is there an election coming within the next twelve months? Preying on parental fears and anxieties for their children’s future is a sure vote getter, if somewhat immoral.
However all is not as it seems.
This so-called failure of New Zealand education can be dismantled as many ways as you’d wish.
[At this point I need to highlight some points.
First, the obvious one - of course all children need to be competent in reading and mathematics. No one is suggesting otherwise. The contentious issue is the way to achieve this, and the great bulk of national and international evidence tells us that the present government’s approach is completely wrong.
Also, the PISA results also include Science, which is ironical, given that successive New Zealand governments have gutted the Science Advisory service to schools.
One more thing: as Hekia Parata has correctly pointed out, the students tested started school in 2001 and 2002, thus completely missing out on the ‘benefits’ of National Standards. So who’s to blame? Well actually the 1990 to 1999 National led governments who introduced the previous curriculum that was found, by 2001, to be failing. The students tested in the 2012 PISA tests spent their entire primary education under that failed curriculum framework.
About 2001, incoming Minister of Education Trevor Mallard instigated a major curriculum review, that resulted in the highly acclaimed New Zealand Curriculum of 2007, to be implemented from 2008 onwards. This implementation had barely started before the 2008 election and the introduction of National Standards which reintroduced the problems of the old curriculum. And then there’s the perennial problem of the Education Review Office, which is another story altogether...]
So why is the PISA test accepted as the benchmark for all schooling systems around the world? As the listed references show, there’s a substantial body of concern about PISA. This includes Professor Stephen Heppell, a frequent visitor to New Zealand, and admirer of our pre-National Standards education system.
Sadly for politicians, these academics are independent of the school system and so can’t be accused of protecting their own turf. Not that this stops politicians from disparaging any evidence that conflicts with their ideology….
I digress: Waikato University Professor Martin Thrupp’s research (published last week) on the effects of National Standards and accompanying policies, that revealed the damaging effect of National Standards on children’s learning, was summarily dismissed by Minister of Education Hekia Parata because Thrupp was engaged by the NZEI (primary teachers union) to carry out this research. To the contrary, Hekia argued, there are many researches that show the benefits of National Standards.
‘Parata dismissed the data as biased.
“First of all, it’s a three-year study of six schools only, [and] it’s paid for by the NZEI, who have a public position of opposing National Standards.
“So it’s hardly unbiased and balanced research, and its sample is extremely small.”
She said there were many “more authoritative” studies that showed National Standards had lifted achievement and targeted areas of slow learning.’
And which ones are these, Hekia? How about tabling them so we all can see them?
[Reminds me of the time when Nikki Kaye promised, several times, to send Dianne Khan and me evidence proving the effectiveness of charter schools. Never arrived....]
Back to PISA:
So are New Zealand children failing? Compared to?
Finland, commonly acknowledged as having the world’s best education system, has also slipped. Does this mean that the Finns have been wrong all the time? Not according to US education academic Yong Zhao ( a visitor to New Zealand earlier this year.)
‘Finland’s slip in the PISA ranking has little to do with what Finland has or has not done. It has been pushed down by others. In other words, Finland’s education quality as measured by the PISA may have not changed at all and remains strong, but the introduction of other education systems that are even better at taking tests has made Finland appear worse than it really is. In 2000 and 2003 when Finland was number one, only two East Asian education systems were included: Korea and Japan. In 2009, there were seven from Eastern Asia: Shanghai, Hong Kong, Taiwan, Singapore, South Korea, Japan, and Macao.’
Why is this? What have these Asian countries discovered that is so wonderful?
Or maybe the question should be “Do we really want to follow their education systems?”
Yong Zhao again:
‘The recipe for the East Asian success is actually not that magical. It includes all the elements that have been identified as the symptoms of the GERM (Global Education Reform Movement) by the great Finnish education scholar Pasi Sahlberg: Competition, Standardization, Frequent Testing, and Privatization. In East Asian high PISA performing systems, these ingredients are more effectively combined and carried out to an extreme to result in entire societies devoted to ensure that their youngsters become excellent test takers.’
And:
‘The East Asian education systems may have a lot to offer to those who want a compliant and homogenous test takers. For those who are looking for true high quality education, Finland would still be a better place. But for an education that can truly cultivate creative, entrepreneurial and globally competent citizens needed in the 21st century, you will have to invent it. Global benchmarking can only give you the best of the past. For the best of the future, you will have do the invention yourself.’
I could quote much more from Yong Zhao’s article but this posting is too long as it is. I recommend you read it for yourself.
This article makes similar points: China’s Schools Teaches Kids to Take Tests, Obey the State, and Not Much More
‘In China, memorization and (consequently) the ability to perform on tests are the keys to academic success, rather than the ability to think or question.’
There is another issue with China’s involvement in PISA that is conveniently ignored by the politicians and their tame media hacks. China, as a country, does not take the PISA tests.
Yes, you read that correctly. A number of provinces and the two administrative regions of Macao and Hong Kong do take the tests, with the results from Shanghai being taken as representative of the whole country – this is a Chinese government dictate. In fact the 2012 results listed Shanghai, Hong Kong and Macao as separate entities, comparing these to the national results of other countries. There is no way that the students of these three cities can be seen as representative of the country as a whole.
‘Shanghai has an economically and culturally elite population with systems in place to make sure that students who may perform poorly are not allowed into public schools. Second, the media should not present Shanghai’s scores as if they are indicative of China’s national performance in education. They aren’t, and no one will know how well China can perform on an international test until it participates, as a nation, under the same rules as all other nations.’ Source.
Hekia Parata, however has been so impressed by the Asian results that she recently led a ‘fact finding’ tour, which sends ominous signals about her plans for New Zealand education.
I wonder if she is aware that Singapore has moved away from such a narrow test focussed system because of the lack of innovation and creativity being shown by its high test performing schools and students?
That China (including Shanghai, in spite of topping the league tables) is also reviewing its education system for the same reasons?
‘The recognition that raising scores in high-stakes tests and exams alone is NOT the real purpose of education – this is what is happening in Shanghai and Singapore. Co-production of a flexible and personalised curriculum IS happening in these countries. The development of social, spiritual and personal intelligence IS happening in these countries.’ Source.
[Note: the now demoted New Zealand Curriculum encouraged similar outcomes as these countries.]
That Korean and Japanese students are amongst the most (if not the most) highly stressed students in the world, and that they are forced to undergo many hours of after schooling tuition in order to achieve? In fact, one analysis of the 2009 PISA results shows that Korean students are at the top of the school unhappiness scale. Is this the price we are prepared to pay so we can have ‘top of the PISA test’ bragging rights?
I wonder if Hekia has asked herself this question: “Why are countries that are heavily into raising achievement through national testing regimes that are far more rigorous than national standards also bemoaning that they are falling behind?”
Possibly the most test orientated country (and including all the other associated evils such as league tables, performance pay, school closures, very restricted curriculum and so on) is the United States with firstly their ‘No Child Left Behind programme (George W Bush’s legacy) and now Obama’s “Race to the Top.’ Following not far behind is the English testing regime, which has led to this: Schools Stuck at Back of the Class According to New PISA International.
Both countries are finding that they are slipping down the PISA rankings, in spite of the intense focus on raising achievement. Similar questions are being asked in Australia (Australian students slipping behind in maths, reading: OECD report) in spite of ex-Prime Minister Julia Gillard’s goal of ‘top 5 by 2025’ and the introduction of the NAPLAN testing regime in 2008. Did you know Hekia has just been to Australia looking for ideas? Have you noticed that New Zealand is not alone in bemoaning so-called failings in education?
Yet the answer according to Hekia is to focus even more on National Standards and assessment to raise achievement?
How about taking note of the PISA data which clearly shows that socioeconomic factors (i.e. inequality) are a major influence on children’s learning?
‘There is a test score gap between socioeconomically advantaged and disadvantaged students in every country. Although the size of the gap varies somewhat from country to country, countries’ gaps are more similar to each other than they are different. Countries’ average scores are affected by the relative numbers of advantaged and disadvantaged students in their schools.’ Source.
Ring any bells for you?
Let’s have a closer look at the PISA examinations:
There are many issues with both the development and implementation of these. For a start, there’s the language and cultural issues.
‘For an international test to work, all students have to answer the same questions, or at least questions of similar difficulty. In one obvious sense, they don’t: the questions are translated into different languages which, according to one Norwegian academic, “results in rather strange prose” in his country. Besides, “literacy” in Finnish or Korean, where words are consistently written as they are spoken, is different when compared with literacy in English.’
Further:
‘Danish academics, when they analysed the 2006 Pisa tests, found that eight of the 28 reading questions were deleted from the final analysis in some countries. Moreover, about half the students participating that year weren’t tested on reading at all. The OECD, which runs Pisa, says it calculates “plausible values” for the missing scores, and this is a standard statistical device. But it’s a hard idea for most of us to get our heads round, and many statisticians dispute its validity, suggesting that the results are nonsensical and meaningless.’
And:
‘How can the same set of questions work in the same way for a 15 year old child in a remote village in Peru for example and a 15 year old from Berlin, or Kerry, or Vietnam or Shanghai?’
Get the drift?
But wait, there’s more:
The PISA results provide a wealth of data, which to be fair, can be mined to extract valuable information to inform the development of education policies. However, this requires detailed analysis and interpretation, whereas the simple league tables that have excited politicians and media are very shallow but for ‘spin’ purposes, are immediately available.
‘In actuality, international data are complex, and even a day or two’s advance look at a summary report would be insufficient to make an intelligent evaluation. It takes many months for careful scholars to analyze the data. Sometimes, this analysis requires examination of more detailed data, including disaggregated scores by social class, gender or race. These are eventually available on the testing organization’s website, but often considerably after the initial public release of a government summary report. Careful analyses of these detailed data can often undermine early assertions.’
And:
‘People like to take international results like this and focus on high performers and pick out areas of policy that support the policies that they support. I never expect tests like these to tell us what works in education. That’s like taking a thermometer to explain why it’s cold outside.’ Source
You will, of course, have noted that identical criticisms are made about the publishing of schools’ National Standards league tables. There’s another equally valid comparison that can be made here. To their credit, the government has (so far) avoided resorting to one-off national tests (such as NAPLAN in Australia) to assess ‘achievement,’ admitting that these are not a particularly valid or fair means of assessing children.
’First of all, judging just about anything important on the sole basis of test scores is never a good idea. That’s not just me talking; assessment experts say it over and over and over.’ Valerie Strauss, Washington Post.
Yet, here we have a ‘one-off’ international testing regime, which is being used to rate the performance of New Zealand children against international peers. Inconsistency?
‘First, comparing nations on average scores is a pretty silly idea. It’s like ranking runners based on average shoe size or evaluating the high school football team on the basis of how fast the average senior can run the 40-yard dash. Not much link to reality. What is likely much more important is how many high performers you have.’
And:
‘It should be noted that these rankings are determined by nations’ average scores. Some researchers have suggested, however, that average score comparisons are not useful: even presuming that the tests have some meaning for future accomplishment, average students are not likely to be the leaders in fields of mathematics and science.’
Also:
‘But is it really possible to compare student performance in this way? And are there dangers associated with attempting to do so? A growing number of educationalists internationally are voicing concerns around both the methodology of PISA and the consequences the tables can have.’
Yet more:
‘Rather, the PISA data, along with other international tests, have most often been selectively raided to defend and promote the preferred ideologies and preferences of those who make and seek to influence educational policies.’
The concerns keep coming:
‘But what if there are “serious problems” with the Pisa data? What if the statistical techniques used to compile it are “utterly wrong” and based on a “profound conceptual error”? Suppose the whole idea of being able to accurately rank such diverse education systems is “meaningless”, “madness”? What if you learned that Pisa’s comparisons are not based on a common test, but on different students answering different questions? And what if switching these questions around leads to huge variations in the all- important Pisa rankings, with the UK finishing anywhere between 14th and 30th and Denmark between fifth and 37th? What if these rankings – that so many reputations and billions of pounds depend on, that have so much impact on students and teachers around the world – are in fact “useless”? Source.
Furthermore, ponder this: the intent behind PISA is the development of a global education system, detailed in this book ‘PISA, Power, and Policy the emergence of global educational governance,’ published by Oxford Studies In Comparative Education.
From the website:
‘About the book
Over the past ten years the PISA assessment has risen to strategic prominence in the international education policy discourse. Sponsored, organized and administered by the Organization for Economic Cooperation and Development (OECD), PISA seems well on its way to being institutionalized as the main engine in the global accountability regime.
The goal of this book is to problematize this development and PISA as an institution-building force in global education. It scrutinizes the role of PISA in the emerging regime of global educational governance and questions the presumption that the quality of a nation’s school system can be evaluated through a standardized assessment that is insensitive to the world’s vast cultural and institutional diversity. The book raises the question of whether PISA’s dominance in the global educational discourse runs the risk of engendering an unprecedented process of worldwide educational standardization for the sake of hitching schools more tightly to the bandwagon of economic efficiency, while sacrificing their role to prepare students for independent thinking and civic participation.’
Do we really want to sacrifice our children’s holistic education in order to jump on the economic efficiency bandwagon?
Do we want to train workers for the workforce or develop the full potential of all New Zealanders?
‘For fear of falling behind we must all adopt “best practice” as revealed by the OECD. By focusing on economic imperatives, schools risk losing sight of their roles in nurturing social solidarity, passing on cultural heritage and promoting civic engagement. Might justice, social harmony and a clean environment be just as important for our children’s future as economic prosperity?’
The PISA testing programme is neoliberalism masquerading as sound education policy.
But then again, read Kelvin Smythe’s latest article “This meeting happened’.
‘This is now the Treasury education system, it is their education system – it has been for a quarter of a century, but still they are blaming teachers, still acting as if they have the answers, and we, minus significant power, forced to be participant witnesses of the decline of a once great education system.’
The agenda is clear. Parata is implementing Treasury’s plan for education, and we all know where Treasury’s loyalties lie. This brings us back to PISA….
References.
A Reporter from Ireland Previews the Release of PISA Test Results http://www.artofteachingscience.org/an-reporter-in-ireland-previews-the-release-of-pisa-test-results/
How Pisa became the world’s most important exam http://www.bbc.co.uk/news/business-24988343
Educational performance comparison study of 15-year-olds across OECD members released http://www.rte.ie/news/special-reports/2013/1201/490321-pisa-and-its-league-table-pitfalls/
“PISA Day”—An Ideological and Hyperventilated Exercise – http://www.epi.org/blog/pisa-day-ideological-hyperventilated-exercise/
The fetishization of international test scores http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/11/14/the-fetishization-of-international-test-scores/
Don’t let dubious Pisa league tables dictate how we educate our children http://www.theguardian.com/commentisfree/2013/dec/01/dont-let-pisa-league-tables-dictate-schooling?CMP=twt_gu
PISA’s China Problem http://www.brookings.edu/blogs/brown-center-chalkboard/posts/2013/10/09-pisa-china-problem-loveless
China’s 10 new and surprising school reform rules http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/10/30/chinas-10-new-school-reform-rules-reduce-standardized-testing-homework/
Do international test comparisons make sense? http://voices.washingtonpost.com/answer-sheet/standardized-tests/so-what-if-the-us-is-not-no-1.html
Reading the PISA Tea Leaves: Who Is Responsible for Finland’s Decline and the Asian Magic http://zhaolearning.com/2013/12/02/reading-the-pisa-tea-leaves-who-is-responsible-for-finland’s-decline-and-the-asian-magic
PISA 2012: welcome to the Education Circus: http://ioelondonblog.wordpress.com/2013/12/02/pisa-2012-welcome-to-the-education-circus/
Is Pisa fundamentally flawed? http://www.tes.co.uk/article.aspx?storycode=6344672#
OECD, PISA, Shanghai, Singapore – Lessons to be Learned in Education http://3diassociates.wordpress.com/2013/12/03/oecd-pisa-shanghai-singapore-lessons-to-be-learned-in-education/