Locking disadvantage into the system

Tweet

Last week’s ‘A’ Level results fiasco reminded me of my own experience of ‘O’ Level results back in 1967. Shortly after we got the results, my French teacher stood in front of the whole class and said that she really didn’t understand how I’d managed to get a grade 3 in French whilst a fellow pupil (whom she also named) only got a grade 6 pass. “I’d have understood it better if the results had been the other way round”, she told us. Being charitable, she was probably trying to boost the confidence of my disappointed peer (and it may well have been fair comment anyway!) but my gratitude for her faith in my ability wasn’t exactly unbounded. Perhaps my fellow student had a bad day and I had a good one; perhaps the questions on the day were simply a better match for what I’d remembered than what he’d remembered; perhaps I was just better at sitting exams – I’ll never know how it happened, only that had my grade been based on teacher assessment rather than examination it would have been lower. (And, had that been repeated in other subjects, life could have turned out very differently.) The point is that teacher assessment is no more perfect a method of assessing ability in a subject than an examination. Both can create anomalies and the two methods will always produce different results for at least some of the pupils. Which result is the fairest is an open question, and our faith in the reliability and accuracy of the system as it impacts individuals is seriously misplaced even in a normal year.

We know for certain that examination performance varies between schools, and if results based on teacher assessment reduce or eliminate those differences, then they are not reflecting accurately what would have happened had the exams been held. (I should note, in passing, the implicit assumption in that statement that the exam results are the accurate ones, itself an assumption open to serious question.) That, in effect, is the justification for making ‘adjustments’ to teacher assessments. They are an attempt to reflect historic differences in performance between schools and, in fairness, they may well have more-or-less achieved that at an overall level. But, by being based on a statistical approach, we can be absolutely certain that the individual pupils whose scores were thus ‘adjusted’ from teacher assessments would not always be the same pupils whose scores would have differed from those same assessments had the exams been held. Reducing it to a mathematical exercise might produce the ‘right’ averages, but it can never produce the ‘right’ results for individuals. It’s an approach which is fundamentally flawed.

We also know the main reason that some schools regularly see lower results on average: they serve poorer communities. We have known for decades that there is a very strong correlation between academic performance as measured by examinations on the one hand and parental income on the other, and none of the actions taken to try and address that have been particularly successful. (I suspect that is primarily because they all set out to address the symptoms rather than the cause in an attempt at some sort of short-term fix, but that’s a subject for another time.) That difference in performance may be well-known and well-established but it isn’t, and never has been, fair. There is something particularly grotesque about a Labour government here in Wales trying to ensure that historical disadvantage based on relative affluence is properly reflected in this year’s results. They are effectively locking in that household income based disadvantage for another whole cohort of young people based on historical results for the schools they attend without the individual members of that cohort having even the limited opportunity to buck the trend which the examination system provides, and from which at least some would have benefited had the exams gone ahead.

There is no perfect solution to this year’s issues (and even less is there an easy fix for the real underlying long-term problem), but applying an algorithm specifically designed to perpetuate the injustices of the past is about the worst solution which anyone could devise. The (eventual and belated) Scottish decision to simply accept the teacher assessments isn’t a perfect one either (the idea that hundreds of teachers in hundreds of schools could ever be grading students precisely and consistently is laughable), but, coupled with a robust appeals process, it’s probably the least worst option in the circumstances. I really don’t understand why ‘Welsh’ Labour prefers to follow an only slightly adapted version of the approach of the English Tories instead.