Float like a bee

Tweet Myth has always been a powerful element of human culture, and that's fine in its place; but it should surely not displace science when it comes to matters of public policy. Mathematics is something that not everyone finds easy; some of us are very comfortable with numbers, others less so. But when it comes to taking decisions, it's surely important that numbers are properly interpreted, rather than completely misinterpreted.

It is in that context that I return again to the magic magnets which Carmarthenshire County Council has decided to fit to its fleet of vehicles "where appropriate". The subject came up again at the full council meeting last week, where one of Plaid's team challenged the decision, given the complete lack of scientific evidence for their efficacy.

The response he got was, basically, that it doesn't matter a jot whether anyone understands how they work; the only important thing is that they do. One councillor suggested that sometimes things happen and we only understand why much later; another memorably trotted out the hoary old urban myth that scientists have 'proved' that bumble bees can't fly, so what do they know about anything. (That particular myth has been well and truly debunked on a number of occasions – here's an example – but still seems to have an amazing degree of traction in the public mind).

In principle, "as long as it works" is a good response – but it does rather assume that the magic has been proved to work. And that's where we move from the unscientific to the unmathematical.

The council actually tried the devices on 22 vehicles, and after analysing the results, came to the conclusion that the devices worked on "some" of them, without having the foggiest notion why that should be.

What their figures actually showed was that 5 of the 22 results had to be disregarded because they were 'outliers' - i.e. so far removed from the sorts of results being achieved on the majority as to be not credible. As for the rest, they showed that in 9 there was a 'negligible' effect, 1 actually showed an increase in fuel consumption, and the remaining 7 showed an apparent improvement of between 3% and 9%.

They also noted that "some results have been skewed by the effect of operating conditions". This was intended to explain why some of the results were so low (after all, they 'know' that the devices work, and were only trying to measure how well). They don't seem to have realised that it could equally be the explanation for all of the apparent improvements. A more corrrect interpretation of the words would be "we haven't been able to exclude an effect from other factors so can't really be certain that any of these figures prove anything".

And in fact, on the basis of their own trial, they have proved precisely nothing. They haven't a clue how the devices might work, or why they might appear to work on some vehicles and not on others. But they have concluded that urban myth and scientific bunkum carries more weight than objective analysis and research.

On that flimsy basis, they are going to fit more of the devices – at taxpayers' expense – on other vehicles in their fleet, presumably based on sheer guesswork as to where they will 'work', since they have no objective basis on which to make such a decision.

And the council's endorsement of the product, showing a saving of 14% (which bears no relation whatsoever to any figures which they have published), remains on the company's website, where it will help to encourage more people to buy into the magic. The particular result on which this wild claim is based actually shows that the vehicle in question achieved fewer mpg with the magnets fitted than did other trucks of the same type without the magnets - a point which no-one seems to have questioned.

I understand that the salesman for invisible clothing is eagerly anticipating his visit to the council.