On the morning of September 24, 2026, Endcrawl, whose service prepares cast lists and credits for Hollywood films, lost its production server for half a day. Company co-founder Alan Grow merely wanted to cancel a test render and typed the classic command: sudo kill -TERM -1490947. The minus sign before the number did not mean “kill the process with this number,” but “kill the entire process group” — a trick as old as job control. Instead of the render group, everything went dark: databases, web servers, monitoring.
Only init — process number 1, immune under an old Unix rule — survived the massacre. Grow first blamed himself: a typo in the command, or a broken script. He was wrong in another direction — the bug was inside the kill utility itself, a contemporary of Unix that the whole world invokes billions of times a day. And digging deeper, he found an entire series of bugs that had lived in Ubuntu LTS for more than a decade.
It started with a Lobsters digest: the headline “A Decade of Kill Bugs,” one command in the opening line, and the promise that the root of it lay far down. The digest gave only the event — the server died. The point remained offstage: how could a half-century-old utility with the simplest possible specification take down an entire machine over one unlucky hyphen?
The excavation was worth it for three reasons. First, the mechanics were beautiful: inside the offending getopt library, a negative PID collapses into a single character, and a giant group number turns into “minus one” — kernel code for “the entire machine.” Second, the history: this was already the fourth such bug, and all of them had spread through Ubuntu LTS releases, the branch kept on servers precisely because it “doesn’t change.” Third, the archaeology: the very right to kill processes with a minus sign was born in 1985 thanks to Sun, and ever since, the command-line parser has never learned to read unambiguously a string where the hyphen means both a signal and a process group.
The investigation uncovered something unexpected: no integer overflow, no exotic behavior involving a giant PID. The code in /bin/kill — the utility from procps, Linux’s standard collection of system tools — read the optopt variable: the standard “error variable” of the C function getopt, which parses command-line arguments. getopt is strict: when it encounters an option not present in the list, it returns an “unknown option” code and places in optopt the first character after the hyphen. Not the entire argument — just the first byte. From the string -1490947, the function extracted the character 1, and then a line of code calculated the PID as the difference between character codes: '0' - '1'. In ASCII, that is minus one.
Then the kernel takes the stage. The kill(2) system call’s man page says it plainly about minus one: the signal is delivered to every process that the caller has permission to signal, except process number one. The only one allowed to survive is init, the system’s first process. Thus the developers’ string logic divided the machine into “everyone who can be killed” and one untouchable. The command ran through sudo, and sudo runs as root — root has permission to signal everything.
The second layer of the discovery: the bug was not alone. Grow ran an experiment on kill parsers from different Ubuntu LTS releases and got a collection worthy of a bad-code museum:
| Ubuntu | procps | What the parser did with a negative PID | What it led to |
|---|---|---|---|
| 14.04 | 3.3.9 | Switched to getopt: kill -HUP -123 got split into the letters -1 -2 -3 |
-12345 worked, -1 silently did nothing |
| 16.04 | 3.3.10 | The hack pid = '0' - optopt |
-12345 turned into -1: kill everything |
| 18.04 | 3.3.12 | atoi(argv[optind]) after the argument had already been consumed |
kill itself segfaulted |
| 22.04 | 3.3.17 | atoi(argv[optind-1]) when the argument had NOT been consumed |
kill -TERM -123 killed the parent shell’s processes |
| 24.04 | 4.0.4 | The resurrection of the 16.04 hack | The Endcrawl scenario |
The mechanics of the first bug are a Grand Guignol of their own. getopt supports “bundled” short options: tar -xvf is simply -x -v -f. For signals, -HUP is bundled too — -H -U -P. The argument is consumed through the optind index only on the final character of the bundle, but procps code jumped out of the getopt loop on the very first “?” error — before the index had reached the end.
The result looked schizophrenic: a long PID, -12345, was handled correctly (the argument remained untouched and reached the strtol parser), while short -1 meant “nothing happened,” because getopt had already consumed it as an option. They fixed the short case — using the optopt trick — and the price was giant group numbers turning into minus one.
The thing is, every subsequent release “fixed” the previous method — and broke a new one. Five versions, five different ways to shoot yourself in the foot, and all of them could remain in use for years. Responsibility for the 22.04 regression, moreover, fell on the parent shell rather than kill itself, which made the failure look like “my session crashed for no apparent reason.”
The Endcrawl outage was not the first case. Launchpad bug report #1637026 was opened as far back as October 27, 2016: dann frazier — a well-known kernel developer — documented kill taking down production systems during Hadoop deployments, with the wording “basically killing everything on the system.” In other words, a series of dead machines was documented, fixed in 16.04 ... and then code carrying the same sin quietly came back to life in 24.04 in 2026.
And dann frazier’s trace with kill(4294967295, SIGUSR2) looks like a chess problem: strace prints the argument as unsigned, but in reality it is that same minus one — “kill everyone.”
The moral is twofold. First: a bug tracker is the industry’s working memory, not its long-term memory; fix Released in one release does not guarantee that the idea behind the code will not return in another. Second: bugs of this class live for decades because their trigger is so rare that years pass between two hits — and every time, it seems like “that only happens to other people.”
The very right to “kill with a minus sign” is a lovely detective story. In 1985, Sun supplied BSD kernel code for negative PIDs (the man page says “for compatibility with System V”); in 1986, it made its way into userland kill and appeared in Unix V9 at the same time. Where the idea came from exactly — System V or SunOS — remains impossible to establish: System V archives from the eighties are not publicly available, which for a Unix historian is like a lost scene from a silent film. Tony Finch (fanf from Lobsters) dug this up in the CSRG archives right in the thread below the article.
The Linux lineage is a chapter of its own. In the nineties, userland was rewritten from scratch for legal reasons (BSD code), and kill ended up with Albert Cahalan — who, according to an LWN legend, once DoS’d himself with five forks while testing it. Support was later picked up by Debian maintainer Craig Small — and he still carries it today.
The practical minimum worth taking away:
-- delimiter. kill -TERM -- -1490947 parses correctly even on all five broken versions — getopt stops treating the argument as an option. The GNU-savvy reflex of “after a hyphen — always --” literally saves everything here.sudo -u render kill ... would have killed only render processes after the same slip, not the entire machine. Root means having the right to signal everything, including yourself by accident.-s/--signal — but process groups still have no explicit flag after forty years. The two-layer interpretation of a hyphen is a tax the system collects once a decade.Grow closes the article with a line you want to hang above the ops desk: “An outage not understood is an outage that happens again”. In the thread, he also offered up a working title for a future talk: “Getopt Considered Harmful. Well okay, at least in this specific case”.