The efficiency trap: why optimising everything can backfire, badly

We have all been there: packed shoulder to shoulder on the train while the person beside us coughs and sneezes. A protective flu jab offers some reassurance, but so does the thought that there will be a hospital bed for you, should you need it. Yet, in England, about 95% of adult hospital beds were occupied at the height of flu season last year.

If hospitals were hotels, that level of occupancy would be good business, and we would commend them for their efficiency. But do we really want a hospital to have so few beds in reserve that it’s just one bad flu week away from having to treat patients in corridors?

This is the subtle thing about efficiency. We usually assume that more of it must be better. But we can become so focused on improving one thing that we lose sight of the others. You can try to arrange your calendar so that every minute is accounted for, but you’ll soon start running over and end up behind. A gap between meetings might seem like inefficiency, but it’s what allows your schedule to recover when something unexpected happens.

Mathematicians have a name for the pursuit of efficiency. We call it optimisation. Give us something to minimise or maximise – cost, time, fuel, profit – and we will try to find the best possible answer. For most of human history, people optimised by common sense. But computers have enormously expanded what can be effectively tackled, from airline schedules to supply chains. The gains for society have been extraordinary. But there are at least three traps to beware of in the relentless pursuit of optimality.

The hospital bed story illustrated the first one: weakening a system’s robustness by squeezing out too much slack. An empty bed can be redundant on a quiet Tuesday night, but it’s somewhere precious to put the next patient when flu cases surge. If you judge a system by its performance under ordinary conditions, you can lose the ability to cope with extraordinarily challenging ones.

A second trap is having a too-narrow definition of “best”. Take the city of Milton Keynes. Designed by urban planners in the late 1960s, it has an American-style grid system. As the city began to be built out, roundabouts rather than traffic lights became a defining feature of its road junctions; today there are about 130 of them. The roundabouts were great for traffic flow, but they came with a now-notorious cost. Have you ever taken the coach from Oxford to Cambridge? An hour in, you get great views of Milton Keynes, but a busload of queasy passengers, as the coach negotiates one roundabout after another. Slow down, swing left, curve right, accelerate. It’s hard to appreciate the seamless merging of vehicles when your stomach is churning.

Traffic flow is important, to be sure. Engineers find it convenient to optimise because it’s relatively easy to measure. Passenger comfort, on the other hand, is harder to quantify. In a case like this, optimisation gives you the best answer to the question you asked, but can deliver unwelcome answers to questions you failed to consider.

A third trap appears when attempts to optimise collide. Even if every part of a system is acting optimally, the system as a whole may not. On 6 May 2010, US financial markets gave a dramatic demonstration of the effect. In the middle of an already falling market, the S&P 500 suddenly plunged another 5% in just five minutes. The whipsaw on the upside was even wilder; within 10 minutes, the market had recovered. Hundreds of individual securities briefly traded at absurdly low prices.

One trigger for these violent fluctuations was an automated program selling 75,000 futures contracts, worth more than $4bn. Other automated traders reacted nearly instantly. Then their responses fed back on one another, and liquidity evaporated. The algorithms’ actions may have been individually sensible in isolation, but they combined to produce collective turmoil.

Which brings us to artificial intelligence. AI is one of the most spectacular triumphs of optimisation in history. During training, enormous numbers of parameters are repeatedly adjusted so that a model gets better at predicting language, writing computer programs, answering scientific questions and giving legal advice. Unfortunately, AI now has the potential to make all three optimisation traps more treacherous than ever. The first trap – the urge to remove the slack – could tempt organisations to dispense with the human oversight that looks redundant in the age of quicker-witted algorithms. Why retain experienced staff when a machine can handle the routine chores? But in cutting that human capacity, you may lose the ability to spot and handle exceptional circumstances.

The second trap – a blinkered view of what “best” means – arises if we regard intelligence as whatever scores highest on a leaderboard. We are remarkably good at building AI systems that do well on measures such as user satisfaction, but we are finding it much harder to optimise them for judgment or empathy. Indeed, researchers have found that training algorithms on human preferences can encourage AI to sycophantically agree with us rather than tell us uncomfortable truths.

Then there is the third trap. AI systems are increasingly interacting with one another, often at blinding speeds and in vast numbers – especially in the era of agentic AI – and we worry that their interactions could generate unpredictable, dangerous behaviour. Recently, an OpenAI agent broke into a non-public Medicare statistics portal in Australia, while public logs suggested that hundreds of other agents were coordinating attempts to gain access to Australian government health data. Although those attempts have not been shown to be part of the Medicare breach, they certainly revealed something disturbing: hundreds of agents apparently working together in unauthorised ways. The concern here goes beyond what any given AI may or may not do; it’s about what happens when swarms of them join forces.

As optimisation becomes more powerful, avoiding these traps becomes more important. The challenge is not simply finding the best answer. It is deciding what “best” really means – and knowing what not to optimise away.

Steven Strogatz and Alex Townsend are mathematics professors at Cornell University and the authors of Big Math (John Murray).

Further reading

The Efficiency Paradox: What Big Data Can’t Do by Edward Tenner (Vintage, £16.99)

The Unaccountability Machine: Why Big Systems Make Terrible Decisions (Profile, £11.99)

Optimal Illusions: The False Promise of Optimization by Coco Krumme (Bantam, £25.99)