Building in resilience

Tweet

 

We may never discover the full details of what went wrong at Facebook and associated sites a few days ago; companies which suffer embarrassing IT failures are generally reluctant to admit to the causes of those failures. This story, however, gives us a few clues as to why it took so long to fix – suggesting that the company ran its own internal processes, including building access control, on the same systems, meaning that the technicians charged with fixing the system couldn’t even get into their own offices let alone use their computers to fix the problem. It highlights the essential fact that, in IT, building resilient systems necessarily means incorporating a degree of redundancy into the design, and that redundancy carries a cost.

That simple rule doesn’t just apply to IT, though, and when the UK government said the other day that it wants the UK to be the “most resilient country in the world”, it raised some obvious questions about whether they understand the costs involved. The cynical response would be to say that of course they don’t; it’s just another of their many vacuous and meaningless slogans which they don’t know how to implement even if they wanted to. And the cynics would probably be right, but let’s pretend for a moment that the government are serious.

If we take energy supplies as an example, back in the days pre-privatisation, the approach of the old CEGB to running a highly resilient grid was to have a significant amount of extra generating capacity available, at a significant cost. Post-privatisation, those ‘unnecessary’ costs could be (and were) reduced by running the system closer to full capacity. Similarly, pre-privatisation, the gas boards had a huge storage capacity, meaning that any disruption to production was unlikely to impact on the consumer. Today, as a result of the pursuit of profit, the UK has one of the lowest levels of reserve stocks of gas in Europe. That move to a ‘just-in-time’ approach isn’t restricted to the energy sector. For decades, business schools and consultants have been pushing companies towards the model as a means of improving their ‘efficiency’. Not holding stock on the site of a factory means a reduced requirement for space and a reduction in the need for working capital, both leading to an improved return on investment. Profit, again, has been king.

It all worked rather well on the whole, just as long as the system operated smoothly and reliably. It was always, though, going to be more vulnerable to a shock event than the previous model and the system has suffered two major shocks recently, whilst a third looms large on the horizon. One of those two was unplanned and the other entirely intentional. That a major pandemic such as Covid would happen at some point was both foreseeable and foreseen; that Brexit would have a similarly disruptive effect on supply chains was also both foreseeable and foreseen; and that climate change will provoke another large shock to economic systems is again both foreseeable and foreseen. In the first two cases, however, neither government nor businesses considered the risk level sufficiently high to warrant the expenditure involved in building in the necessary contingency, and there’s little sign to date that they are any closer to preparing for the third. One might, perhaps, excuse businesses, to an extent at least, for not spending large amounts of money on preparing for a type of Brexit which they probably believed no rational government would pursue, but they can’t escape the blame for failing to allow for the possibility that we no longer had a rational government. And in both cases, a responsible government would have led the planning and preparation work.

The lesson we should be learning is that long global supply chains with little or no built-in redundancy are highly vulnerable to shock, and the corollary is that secure and reliable supplies of essentials are easier to guarantee with a more localised approach and a greater level of redundancy and stock-holding. If that’s what the government mean by increasing resilience, then it’s something to be welcomed. It is, though, likely to compromise both international competitiveness and productivity the way the latter is usually measured. It seems extremely doubtful, to say the least, that the current regime is really intending to put security of essential supplies ahead of the ability of their friends, cronies, and donors to make profit. The more cynical assumption expressed earlier is a much better fit with the observable facts. We should treat it as the vacuous slogan which it is.