It started with an anecdote. In a fresh feed of tech news, I came across an analysis: a person who'd worked with intelligence analysts for a decade and a half took the Cuban Missile Crisis methodology — scenarios, indicators, probability reassessment — and applied it to the University of Illinois football season. After a loss, the model became more pessimistic about the playoffs and more optimistic about the offensive line, meaning it produced a contradiction that the method was supposed to surface.
Commenters reacted predictably: show that this is a methodology, not a language model's hallucinations dressed up in terminology. And here's where I got interested not in whether the author was right, but in something else: why should the phrase "intelligence assesses it's probable" prove anything in the first place? Where did the confidence come from that such words represent a measurable quantity?
The trail led to 1964, to Sherman Kent's essay "Words of Estimative Probability," declassified and posted on the CIA website. I read it expecting "a dry methodological text with a table," but got forty pages about how the agency couldn't agree on the meaning of "possible" for twenty years, and how the analyst's psyche resists this for entirely self-interested reasons.
In March 1951, the latest National Intelligence Estimate — a consolidated document from the U.S. intelligence community — came out titled "Probability of an Invasion of Yugoslavia in 1951." The key judgment read: an attack "should be considered a serious possibility." A few days later, Kent was talking with the chairman of the State Department's Policy Planning Staff, who asked directly: what odds did you mean by "serious possibility"? Kent answered that personally he put it at roughly 65 to 35 in favor of attack. His interlocutor was puzzled: he and his colleagues had read the formulation as significantly lower probability.
Gripped by a bad feeling, Kent went around to his colleagues on the Board of National Estimates and asked what numbers they'd had in mind when agreeing to that formulation. The range turned out to be 20 to 80 percent. The same text, signed by the same group of people, meant things that differed fourfold to them. What shocked Kent wasn't so much that they hadn't communicated with the reader, but that they hadn't communicated with each other.
This episode gave birth to his proposal. First he built a scale with eleven gradations, assigning each verbal formulation its own odds, then admitted the whole thing was foolish: the distinctions between adjacent steps were sharper than the data allowed, and the verbal equivalent still wouldn't convey precision. In the second attempt, five steps remained. One hundred percent certainty, "almost certain" — 93 percent plus or minus six, "probable" — 75 plus or minus twelve, "chances about even" — 50 plus or minus ten, "probably not" — 30, "almost certainly not" — seven, and impossibility. The key thought Kent hammered home separately: the word "unlikely" should mean odds of roughly three to one against, meaning event probability around a quarter, not "improbable" in the everyday sense.
The rule about "possible" became a cardinal error for Kent. He saw it in the fact that they'd taken a word at the very bottom of probabilities, attached an intensifying adjective, and pushed it to the top. The word "possible" and its relatives are not subject to modification — no "quite possible," "barely possible," "serious possibility." It should have exactly one job: denote something for which no numerical assessment can be given.
Kent divided his colleagues into two parties and didn't hide his contempt for the first. "Poets" believed that in speculative matters, language can in principle convey only the most general meaning, and if the message arrived distorted, well — such is life. "Mathematicians," to whom Kent counted himself, demanded numbers. The poets didn't object stupidly: there are no fewer poet-readers than poet-writers, and they won't respond to a careful numerical formulation, whereas a living, fuzzy phrase will get through much better.
Then comes the part of the essay that made me sit down to write this at all. Kent lists the expressions analysts use to replace assessment: "apparently," "it would seem," "reportedly," "suggests," "indicates." And he gives their purpose a formulation I'll quote verbatim, because paraphrase weakens it: "what we consciously or unconsciously seek is an expression that conveys a certain meaning and at the same time either relieves us of all responsibility or removes the estimate to a sufficient distance that we cannot be entangled in it."
Then he describes the mechanics of pressure from above. The customer isn't satisfied with "cannot assess," and forces a number anyway, because "you're the experts" and "otherwise I'll go to people who know less than you but won't refuse." The agency invented workarounds: call the document "intelligence assumptions for planning," publish ranges instead of points, and print tables with calculation assumptions in red with a warning that without accompanying text they cannot be used for anything.
History, however, ended as it should have. A few people hung a copy of the scale on their office wall, someone proposed printing it on the inside cover of every document as a glossary, someone used it as a lecture slide. The idea wasn't adopted. Kent writes with bitterness: "from the vantage point of almost fourteen years of experience, the difficulties seem virtually insuperable."
In the 1990s, acting CIA director John McLaughlin tried to push numerical assessments — they also "didn't take." A senior CIA officer explained the reason to a journalist: analysts "prefer words to numbers when it comes to how confident we are," and intelligence consumers "like words and pictures."
Kent was irritated by the second part of this explanation until his dying day. Not that the analyst evades, but that it works.
Sixty years passed, the scale was finally adopted — more precisely, three different ones were adopted. And if you put them side by side, it becomes clear that agreement was never reached, just now each agency has its own.
American directive ICD 203 prescribes seven terms: "remote chance" — one to five percent, "very unlikely" — five-twenty, "unlikely" — twenty-forty-five, "roughly even chance" — forty-five-fifty-five, "likely" — fifty-five-eighty, "very likely" — eighty-ninety-five, "almost certain" — ninety-five-ninety-nine. The bands don't overlap, and this is a deliberate decision. But the extreme points of the axis — zero to one percent and ninety-nine to one hundred — aren't covered by any term: intelligence, by its own standard, has no words for absolute certainty and absolute impossibility.
The British scale, updated in March 2025 by the Professional Head of Assessment (Probability Yardstick), is structured differently. There are also seven bands, but gaps are left between them: "remote chance" — up to five percent, "highly unlikely" — ten-twenty, "unlikely" — twenty-five-thirty-five, "realistic possibility" — forty-less than fifty, "likely" — fifty-five-seventy-five, "highly likely" — eighty-ninety, "almost certain" — ninety-five and above.
The document states that the scale "divides the probability scale into seven ranges." If an event is assessed at seven percent, twenty-two, or thirty-seven, no term corresponds to it, and the document doesn't say what the analyst should do. This is perhaps the most honest detail of the entire British paper: the scale admits that between "highly unlikely" and "unlikely" lies a band for which language simply has no word.
The motive, however, is stated directly: terms are used instead of numbers like "55%" so the reader won't take the judgment as overly precise, since most intelligence assessments aren't based on quantitative data. And a second, less advertised function: a rigid set of words forces analysts themselves to formulate judgment rather than write "if X happens, then Y might occur."
The third scale is climatic, from the Intergovernmental Panel on Climate Change. Here it's arranged differently still: "virtually certain" — 99–100 percent, "very likely" — 90–100, "likely" — 66–100. The bands are anchored to the edge of the scale and therefore overlap, and the authors themselves write about "fuzzy boundaries."
This immediately raises a practical question that conscientious readers have been asking for a decade and a half: is "likely" from 66 to 90 or from 66 to 100? Formally the second. Intuitively the first.
The logical move — attach a number to the word. Not replace, but duplicate: "very unlikely (five-twenty percent)." Seems like what could go wrong.
In 2019, a research group tested four presentation methods on 924 participants and compared how much the understood matched the intended. Range in parentheses directly in text gave 66 percent agreement. Separate table with all terms, opening via link — 39. Pop-up tooltip on cursor hover — 40. Control group without any guidance — 32.
Only the format with number in parentheses significantly improved the situation, and the probability of the reader's best estimate landing inside the official band rose from 59 to 82 percent. Along the way, it turned out people follow the link to the table in about one case out of eight.
Then comes the most unpleasant part for the standardization idea. In the same study, they separately asked 809 people what numbers they themselves put into the seven official terms — without any context. Agreement turned out acceptable only for "roughly even chance." Worst of all were the words "unlikely" and "likely."
And an asymmetry emerged that the manuals don't mention: the spread of assessments for terms with negation is noticeably wider than for terms without it. Standard deviation for "very unlikely" — 21.1, for "unlikely" — 20.7, and for "likely" — 13.0 and "very likely" — 12.9. A word built through negation is understood worse than a word built through assertion. Probably because a person has to first imagine the event, then mentally cancel it.
The most radical result belongs to another group that published an experiment on 1,202 participants in 2021. Numerical format and "word plus range" format surpassed purely verbal and additionally better distinguished low probability from high. The combined format, however, gave no advantage over purely numerical.
The authors' conclusion sounds like a verdict on the entire enterprise: numerically bounded verbal probability schemes are an ineffective means of conveying probability information to other people. And this directly contradicts the position of the previous research group, which recommended the combined format as the solution. Two respected teams, twenty years of work, mutually exclusive conclusions.
A separate observation I like more than any averages. In the verbal condition, participants' share of answers at exactly "50 percent" jumped sharply. Moreover, medians for "unlikely" and "likely" practically stuck together around this mark. A fifty-percent answer isn't a probability assessment, but a way to say "I don't know." The phenomenon even got a name — "fifty-fifty blip" — and it means a simple thing: a person given a scale of seven words, when stumped, chooses the one in the exact middle, and their answer slides into the indeterminacy they were trying to resolve.
Reports from the same climate group are published in six languages. Until 2013, no one systematically checked whether terms read the same in different translations, and when they did check, the result was rougher than expected.
They compared British and Chinese readers — first freshmen, then random passersby in London and Jinan. Chinese interpretations turned out shifted toward the middle of the scale noticeably more than British: mean deviation from fifty percent was 19.6 points versus 31.9. Chinese participants distinguished terms worse from each other and gave greater spread. Terms at the upper end of the scale dropped sharply for them: "virtually certain" — median 86.5 percent, "very likely" — 70, "likely" — exactly 50. That is, high probability in Chinese reading meant the same as "roughly even chance."
But the most interesting thing in this comparison is that the discrepancy was partially explained by translation error. English "unlikely" in the official Chinese edition of the report was rendered with a word literally meaning "impossible." Fifty-one percent of Chinese participants read this term as zero percent. Not as low probability — as absence of probability.
Along the way, a detail emerged that calls into question the very idea of a tidy dictionary. In English, the set of terms for describing probabilities is much larger than in Chinese, and according to earlier research data, Chinese "keneng" — an ordinary word meaning "probable" — covers a range from 20 to 95 percent. It's pointless to assign such a word a narrow band: it's simply wider than the entire proposed scale. A number doesn't have this problem — 70 percent remains 70 percent in any language, and that's precisely why the authors of the work end with an almost political demand: if terms are assigned numerical ranges anyway, why not always indicate ranges.
There's another layer that numbers don't convey at all. Psychologists call it directionality: part of verbal expressions point to an event's occurrence ("possible," "likely"), part to its non-occurrence ("doubtful," "not entirely clear"). The number "seventy percent" lacks this directionality: it equally communicates both seventy for and thirty against.
And words carry a trace of the speaker's attitude. "Internal" confidence — "I'm 70 percent confident" — and "external" — "this is 60 percent reliable" — are consistently separated by numbers when applied to the same event, with internal always higher. The formulation isn't decoration on top of the assessment, it's part of the assessment.
If the problem were limited to confusion within the intelligence community, the story would be parochial. But exactly the same arithmetic unfolds in three areas where the stakes are measurable in money and life.
Rain forecast is the world's most massive experience of reading probability, and also the most failed. Probability of precipitation is defined as the chance that at a given point more than a quarter millimeter of precipitation will fall, averaged over the forecast area. In practice, it's the forecaster's confidence multiplied by the area fraction. Three most common mistaken interpretations: that such-and-such a fraction of the district will get wet, that rain will fall for such-and-such a fraction of time, and that the forecaster is that confident in rain. Wrong definitions are given by 35 to 74 percent of respondents, with meteorologists landing in the upper part of this range. The Canadian service doesn't publish probability of exactly 50 percent at all, considering it too uncertain.
Meanwhile, professional forecasters shift the scale deliberately — the phenomenon is called "wet bias." According to a project collecting forecasts since 2002, the U.S. national service works without bias, while commercial channels overstate. When one popular channel said "20 percent," rain historically fell in five percent of cases. Local TV stations overstate across the entire scale and frequently issue 100 percent where rain happens in seventy. The opposite edge of the same shift: probability of 90 and higher was rounded to one hundred.
And with a forecast of "zero percent," precipitation still fell in roughly one case out of ten. Formally this is a lie in the forecast, but motivated: an overstated chance of rain prompts taking an umbrella, and a taken umbrella is more pleasant than an untaken one in rain.
In medicine, the shift is organized differently — through legal liability. Word ranking among doctors and patients is coordinated, while absolute numbers diverge: a compilation of three independent studies gives means of 70, 75, and 77 percent for "likely," 25, 20, and 47 for "possible," 13, 11, and 20 for "unlikely." The researcher compiling the summary noted that agreement in medicine is higher than in meteorology, and suggested a quite earthly reason: the law requires disclosing risk information to the patient regardless of whether they asked about it.
In finance, everything repeats with a third attempt. In a 1988 survey, financial advisors and clients were asked to assess a set of probability phrases with numbers: ranking matched for both sides, but absolute values diverged so that the phrase "rates will likely rise" in the minds of two-thirds of clients meant a range from 37 to 95 percent. Even "highly unlikely" — one of the words with the smallest spread — two-thirds of clients placed between zero and 28 percent. The authors' conclusion from that study was direct: advisors should resist the desire to use imprecise probability words.
It remains to explain why all this matters at all beyond laboratories. Here I have to return to intelligence, because that's where the price of verbal carelessness is posted as a casualty list. True, I'll stipulate immediately: I'm not going to prove that the Iraq war was unleashed by an unfortunate phrase. The scale doesn't work as a cause, it works as an amplifier, and in conjunction with political will that's enough for a maximum to turn into a typical value.
In fall 2002, the British dossier on Iraq featured a claim about forty-five minutes. Let's trace its path. On August 29, MI6 receives an intelligence report that, according to the Butler Commission's findings, came third-hand: through a primary reliable source and a second link in the chain, originally from an Iraqi military source, with the middle link subsequently deemed unreliable.
In the first draft assessment from the Joint Intelligence Committee on September 5, it says that chemical and biological munitions "could be with military units and ready for use within 45 minutes." The next day, the relevant unit clarifies: intelligence speaks of a maximum time of 45 minutes, average — 20. In the new draft on September 9, "within 20–45 minutes" appears. And in the dossier draft dated September 10–11, the maximum disappears and only "can deploy such weapons within 45 minutes of an order" remains.
The maximum turned into a typical value. Then people get involved. On September 17, a Defense Intelligence Staff officer writes to the committee that the formulation is "quite strong since it's based on a single source," and suggests noting it's intelligence data. The same day, the Downing Street media chief tells the committee chairman that the verb "can" in the main text is weaker than in the summary. On September 18, the chairman replies: language in the main text "tightened up." On September 24, the dossier is published.
In the Prime Minister's foreword, it says that military planning allows "some of these weapons to be ready within 45 minutes of an order to use them." Then the agencies work, then the newsrooms. On September 25, the Evening Standard runs with the headline "45 Minutes from Attack," The Sun — "Brits 45 Mins from Doom," The Star — "Mad Saddam Ready to Attack: 45 Minutes from Chemical War." In the nine months between the dossier's publication and the first journalistic exposés, this claim was mentioned in the House of Commons once in passing and twice among more than thirty-eight thousand written questions.
In February 2004, the Prime Minister stated in Parliament that he hadn't known before the war: it was only about battlefield munitions, not missile warheads. The Defense Secretary said he learned this distinction when he asked his subordinates out of curiosity.
He also explained to the commission why he didn't correct the papers: hard to make the press print retractions. On October 12, 2004, the head of MI6 officially withdrew the claim.
A separate line — the phrase "slam dunk." It was spoken by the CIA director at a White House meeting in 2002, answering the question whether these were the best data on Iraqi weapons. The phrase leaked into Bob Woodward's book and became proof that intelligence was certain. Only in 2007, in an interview, did the director first explain what he'd meant: he was talking about being able to assemble a more convincing case for public presentation, not about the intelligence assessment. By this point, his phrase had served as a political argument for five years, and he called the leak "the most despicable thing" that had happened to him.
Now worth juxtaposing two numbers. According to an analysis of key judgments in one hundred twenty consolidated intelligence documents over fifty-eight years, the most frequent estimative word turned out to be not "likely" and not "possible," but "will." Seven hundred seventeen uses, by a huge margin, with distribution across decades stable.
The CIA director in 2008 formulated it this way: "anything above seven is not on our scale. If we're at eight, nine, or ten, it means we're not being asked questions that can be answered with such confidence." The word with absolute meaning is the most popular in documents whose author by definition can never be completely certain.
And here we return to Kent, who described this forty-four years before that analysis: the agency seeks a formulation that means something and at the same time removes responsibility. The word "will" works perfectly for this. It doesn't leave the reader a chance to understand that the assessment is probabilistic — meaning the analyst won't be asked later why it didn't come true.
True, this story has a detail that prevents me from putting a period. That same frequency analysis was done by a master's student, not a group of methodologists, and she counted words in key judgments, not the meaning of those words in context. So "717 times will" is a fact about the language of documents, not a verdict on the agency's thinking. What exactly the analyst backed up their "will" with in each specific case isn't visible with such a methodology. I don't know what to do with this correction: it doesn't cancel the irony, but doesn't let you press it all the way either.