In the autumn of 2020, Britain’s pandemic-tracking system stumbled over a number programmers know by heart and officials had probably seen for the first time. This is the story of how nearly sixteen thousand infected people fell out of epidemiologists’ sight for several days because a spreadsheet ran out.
The end of September 2020 in England was a time when the second wave of COVID-19 was gathering pace. Daily case numbers were creeping upward, the government was discussing local lockdowns, and commercial laboratories handling the so-called Pillar 2 — mass testing of the public outside hospitals — were producing results in an ever-denser stream. Every positive test had to pass through a chain: the lab records the result, the data goes to the central systems of Public Health England (PHE), from there to dashboards for epidemiologists and, crucially, into the NHS Test and Trace system, where contact tracers begin calling sick people to find out whom they had seen in recent days. The speed of that chain — literally a matter of hours — determined whether people could be warned before they carried the virus further.
Patients received their results as normal in those days. Someone who gave a swab on 27 September would learn they had tested positive and be told to self-isolate. Their personal path did not break down. But their data — name, contacts, test date — got stuck halfway to the people whose job it was to call their colleagues, their mates from the pub, their relatives, and say: isolate, you’re at risk. And nobody noticed, because from the outside everything looked normal: the numbers in the daily reports were rising, but rising a little more slowly than they should have, and you could not see that gap without checking against the laboratories’ raw data.
To understand what happened, we need to pause for a second and recall the history of a file format. The .xls extension is the binary Microsoft Excel format, introduced in 1987, when a personal computer with a megabyte of RAM was considered a powerful machine. The limit of 65,536 rows per sheet was built into the architecture: rows were addressed with a sixteen-bit number, 2 to the sixteenth power — there’s your ceiling. In 2007, Microsoft released the .xlsx format, where the limit rose to 1,048,576 rows — more than a million. By 2020, the .xls format was thirty-three years old, and in the development world it was regarded much like a rotary phone: it works, but why?
PHE’s automated process received results from commercial laboratories as CSV files — simple text tables with no length limit. The data was then transferred into .xls templates for further upload into the central systems. And here is the detail that gives me chills every time: every test result generated several rows in the spreadsheet — contact details, specimen collection date, identifiers. That meant the 65,536-row ceiling was reached after roughly 1,400 cases. Fourteen hundred. In a country of nearly sixty million people, at the height of the second wave, one template had a capacity of fifteen hundred people — and that was it; beyond that, the data was simply cut off, like water spilling over the rim of a bucket. No error, no warning, no red alert on the screen. Excel does not cry out when rows do not fit into an old format. It silently discards them.
The failure was discovered overnight into Friday, 2 October 2020. Public reports are sparse on who exactly noticed the discrepancy and how, but the very fact that it took a week to surface — data had been disappearing since at least 25 September — says a great deal. The monitoring system had no built-in completeness check: nobody was automatically comparing the number of results sent by laboratories with the number of records that made it to the dashboards. It was as if a post office counted only the letters placed in mailboxes, without checking that number against what the mail carrier had brought in.
Once the scale of the loss became clear, emergency work began. By 1:00 a.m. on 3 October, all 15,841 missing positive results had been passed into the NHS Test and Trace system. More than 75% of them — 11,968 cases — belonged to the last three days, from 30 September to 2 October. The rest were spread across the preceding week. On 3 October, the data was added to the official statistics all at once, and the daily increase in cases shot up so sharply that the charts looked like a wall. Newspaper headlines screamed about records, social-media commentators suspected manipulation, and epidemiologists clutched their heads, because the distorted figures from the previous week had already affected models and forecasts.
Michael Brodie, PHE’s acting chief executive, confirmed the number in an official statement: a technical failure in the data-upload process had left all 15,841 cases out of the reporting. Susan Hopkins, medical adviser to NHS Test and Trace and PHE, explained that the test results had not been lost — they existed in the laboratories’ systems, and patients had been notified. But contact tracers, the very people meant to break chains of infection with calls and letters, were working from incomplete lists. How many contacts were not warned in time remains a question that never received a precise public answer.
The line that instantly spread across the internet was irresistible: “Excel broke British healthcare.” It was elegant, compact, and almost entirely wrong. Excel did exactly what it was supposed to do: it opened a file in .xls format and held as many rows as the format allows. Blaming the program is like blaming a pot because the soup boiled over: the pot did not choose how much water would be poured into it.
The real problem lay several layers deeper. First, the choice of format. At some point — perhaps months earlier, in the rush of the spring wave — someone had created a .xls template, and nobody had revisited that decision since, even though testing volumes had grown many times over. Second, the architecture of the pipeline. Data travelled along the route CSV → .xls → central system, and the intermediate Excel step was essentially a bottleneck inserted into the flow without any validation whatsoever. No script checked whether every row from the source CSV had made it into the final file. Third, and this was perhaps the most painful part: the whole construction was a temporary solution that became permanent. The pandemic demanded speed, and in spring 2020 PHE engineers assembled processes from whatever was at hand. Excel spreadsheets were always at hand. The problem with temporary fixes is that they outlive their creators: the person who put the pipeline together in March may have left by summer, but the template kept running.
Matt Hancock, then health secretary, told Parliament that this “should not have happened” and reported that the team had responded swiftly; by Monday, though, contact tracers had managed to reach only 51% of the patients caught in the backlog. Jonathan Ashworth, Labour’s shadow health secretary, called it “chaos” and demanded an explanation for why critical infrastructure depended on outdated software. Professor John Croft of the University of Cambridge pointed to a systemic problem: this was not about one particular bug, but about the absence of an engineering culture for handling data at a level appropriate to the scale of the task.
It is easy to say: well, the data was restored, the patients knew their results — what is the tragedy? The tragedy lies in the nature of contact tracing. Its effectiveness is measured in hours. If a person learns of a positive test on Monday but their colleague is warned only on Friday, that colleague has spent four days going to work, riding the Tube, meeting friends. Every lost day is a branching tree of potential infections, and by the time you try to prune it after the fact, it is already too late.
NHS Test and Trace contact tracers were already working with delays: the system was overloaded, they could not always get through, people did not answer their phones or could not remember whom they had seen. But there is one thing about a delay of several hours caused by a queue of calls, and quite another about data being entirely absent from the system for days. For those 15,841 cases that existed for a week only in laboratory files and in patients’ personal notifications, contact tracing simply never began. What gets me is the arithmetic: if every infected person had an average of just five or six close contacts, we are talking about tens of thousands of people who did not receive a warning in time. PHE did not publish an exact figure, and perhaps it is impossible to calculate, because we will never know which of those left unwarned became infected and whom they infected in turn.
There was another layer to the problem — the distortion of the epidemiological picture. The models on which the government based its lockdown and restriction decisions fed on daily data. A week of underestimated figures followed by a sharp spike is noise that makes it difficult to distinguish a real trend from an artefact. Epidemiologists had to recalculate the curves after the fact, reconstructing the distribution of cases by specimen-collection dates rather than reporting dates. It was painstaking, thankless work, because by the time it was finished, political decisions based on inaccurate data had already been made.
This episode is not about Excel. It is not even specifically about PHE. It is about the architecture of systems governments build in a crisis, and what happens when the temporary becomes load-bearing. In every large organisation — from a bank to a ministry — there are dozens of “invisible” spreadsheets, scripts, and macros that someone once wrote for their own use, only for them to become embedded in a critical process. Nobody documents them, tests them under peak loads, or checks whether they still match current data volumes. They work — and that is enough, until one day they do not.
In PHE’s case, several factors converged. The pandemic scaled up testing faster than processing systems could scale. Commercial laboratories connected to Pillar 2 sent data in a format that was not itself a problem — CSV has no row limit. The problem arose at the junction — when data was poured from one format into another, like water from a wide pipe into a narrow one. And nobody monitored that junction because it seemed trivial: come on, copying data from one file to another — what could possibly go wrong?
What stays with me most, frankly, is precisely this: not the failure itself, but the week of silence. For seven days, the system was losing data, and not one automated test, not one integrity check raised the alarm. That means the pipeline did not even have the most basic control — a row count at input and output. Any second-year computer science student knows that when migrating data, you need to check: how many records there were, how many there are now, whether they match. But PHE at that moment was not doing conventional software development — it was patching pipes on the fly, under pressure from politicians, the press, and rising infection numbers. And in that rush, the basic principles of data engineering became a luxury for which there was not enough time, people, or attention.
After the incident, the quick patch was to split large files into chunks so that each one fit into the old template, while PHE launched a full review of all processing systems — from laboratory to dashboard. Later, in 2021, PHE was reorganised: its functions passed to a new agency, the UK Health Security Agency (UKHSA), though the reasons for the reorganisation went beyond a single spreadsheet incident. NHS Test and Trace continued to operate, but the episode of the missing data became one of the arguments cited by critics pointing to systemic problems in Britain’s pandemic response.
The case now lives in data-governance textbooks and in presentations at data-engineering conferences as a textbook example. But it deserves a slightly more uncomfortable question than “why didn’t they update the file format?” The question is this: why was a system whose speed determined whether chains of infection would be broken — literally a matter of life and death — assembled in such a way that the only barrier between laboratory data and epidemiologists was a row limit set by Microsoft engineers three years before the fall of the Berlin Wall? The answer is probably neither malice nor stupidity. It lies in the way large bureaucratic systems work: they grow by accretion, each new layer settling over the last, and nobody looks underneath until the floor gives way. In October 2020, the floor gave way to a depth of exactly 65,536 rows.