The problem of averages

Tweet

Averages and aggregated data, in general, are a useful way of comparing and presenting data, but they can often hide an important level of detail which makes them dangerous as a basis for setting policy, particularly when they end up being interpreted as though all the individual items are at the ‘average’ level. One example which immediately springs to mind is the regular comparison between average spend per pupil in Welsh schools and average spend per pupil in English schools, a comparison which reflects poorly on Welsh schools, usually indicating a ‘deficit’ of around £600 per pupil per year. Now there are a variety of reasons why there might be a gap, but what such ‘averages’ hide is that there are some Welsh schools which receive more than the English average and some English schools which receive less than the Welsh average. More importantly, whilst the gap between the English and Welsh averages can be ‘solved’ by simply giving £600 per pupil per year extra to all schools in Wales (as some politicians have called for), that won’t address the differences between funding levels when comparing schools within Wales (or within England). It’s addressing the data rather than the problem, but numeracy does not seem to be a particularly strong trait amongst politicians.

We’re seeing another example of this at the moment in relation to the pandemic, with claims by English government ministers that the UK is now ‘past the peak’. It’s one of those statements which is ‘true’ but which isn’t the whole truth. The fact that the UK at an aggregate level is now seeing reducing numbers doesn’t mean that all parts of the UK are seeing reducing numbers, either geographically or in terms of sectors (such as care homes), and taking UK-wide decisions on the easing of the lockdown on the basis of an aggregated data set means that lockdown could be eased at a time which is still too early for any areas or sectors which have yet to pass the peak. It could exacerbate the rate of infection in those areas as a result. Yet that is exactly what the PM’s rhetoric suggests he wants to do. The annoyance with which more cautious statements from the First Ministers of Wales and Scotland are greeted is another indication of the way in which the overall situation is being wrongly interpreted. The First Ministers are right to make it clear that they reserve the right to act differently if the situation in their respective countries is different; I hope that they will also understand that the situation might vary within their countries as well as between them. The same distinction also applies in England – just because London’s figures are falling fairly consistently does not mean that all parts of England are past the peak yet either; they just don’t have First Ministers to stand up for their interests.

When the rate of transmission varies across a country it inevitably causes problems in taking decisions – should we wait to ease the lockdown until all parts are on a clear downward trend, accepting that that will be later than necessary for some; should we ease the lockdown as soon as the overall picture improves sufficiently, accepting that it will be too early for some; or should we vary the decision according to circumstances in different areas? None of these are ideal – all bring real problems, both economically and socially, to say nothing of the impact in terms of sickness and death. The PM’s instinct seems to be to use the aggregate figures as his guide and apply policy across the board; I just wish I could be confident that he and his government were doing that on the basis of an understanding of the compromises involved rather than simply an inability to understand what averages and aggregates are telling us.