A recent Habr digest featured an article about "the evolution of Ludwig Boltzmann's views"—a story about how a man spent thirty years trying to derive irreversibility from reversible mechanical laws, was broken three times by paradoxes (Loschmidt, Zermelo), put forward a desperate cosmological hypothesis "the entire world is a random fluctuation," and ended with suicide in 1906, never seeing Einstein prove atoms were real. It's a powerful story, but biography is biography: it's been written, rewritten, and retold before us.
I latched onto something else. At the end of that journey, in 1877, Boltzmann counterattacked the paradoxes with a statement that today is carved on his tombstone: entropy is the logarithm of the probability of a state. But what happened to this formula after that—where it went after the author's death—usually stays off-camera. I dug in to check and discovered that this very exponential exp(−E/kT) is a rare case in the history of science: a formula that didn't stay in its domain but over a century and a half burned through physics, chemistry, computing, and finally settled into the interface of every chatbot. The "temperature" option in neural network settings isn't a marketing metaphor or borrowed terminology: it's mathematically the same bracket Boltzmann wrote for gas—keeping the letter T and all its properties.
That's the plot: the life of one formula across six epochs. From a dispute about the existence of atoms to you sliding a dial between "creative" and "precise" mode.
We have to start with what Boltzmann didn't do—this matters more than it seems. He never wrote the formula S = k log W in the form carved on his Viennese grave. Having formulated the idea between 1872 and 1875, he gave its rigorous statistical version in an 1877 paper, but the modern notation with a separate constant k was introduced by Max Planck around 1900—who honestly noted that "the logarithmic connection between entropy and probability was first indicated by L. Boltzmann." So the most famous epitaph in physics history—on the tombstone at Zentralfriedhof, next to the bust—is, strictly speaking, the formula in Planck's edition about the man who earned it.
The surrounding drama wasn't academic vanity. Atoms in the 1870s were considered a convenient hypothesis, not fact, and the second law of thermodynamics—"heat flows from hot to cold on its own"—stubbornly refused to be derived from Newton's mechanics, where everything is symmetric with respect to time. Loschmidt showed: reverse the velocities of all molecules—and the gas runs backward, violating the "law." Boltzmann's 1877 answer was a brilliant knight's move: irreversibility isn't a prohibition but statistics; a system transitions from less probable macrostates to more probable ones, almost always, but not by compulsion. Within this logic the exponential was born: high-energy states simply occur less frequently, and less frequently according to a specific law—exp(−E/kT), where T is temperature and k is a constant, a bridge between the world of molecules and the world of thermometers.
Fate delivered an irony rich enough for a separate novel. For Boltzmann's sixtieth birthday in 1904, colleagues prepared a jubilee volume, and 117 scientists from around the world sent contributions—including his ideological opponents Mach and Duhem. This doesn't look like "persecuted heretic": his career succeeded, recognition came. The man who trusted atoms (that's the title of his biography, written by Cercignani) died in 1906 at the peak of fame—a couple years before Brownian motion and Einstein's molecular count closed the question forever. But the formula remained.
Before following the formula into the twentieth century, it's worth assessing the scale of its power in the nineteenth. The exponential exp(−E/kT) isn't "one of the formulas of statistical physics," it's the way nature assigns probabilities to everything hot. James Clerk Maxwell in 1860 found its first incarnation: gas molecules are distributed by velocity exactly this way—most unhurried, a few frantic. Boltzmann in 1868–1871 generalized the distribution to cases with external forces. And the same bracket emerged everywhere there's heat and choice.
Chemistry is the most obvious example. The Arrhenius equation of 1889, k = A·exp(−Ea/kB·T), tells at what rate any reaction proceeds—from fuel combustion to metabolism in your cells. The activation energy Ea sits in the exponent exactly where Boltzmann had the energy of state; the difference between the "chemical" form with gas constant R and the "physical" with kB is only in units, per mole or per molecule. Everything that lives and burns, lives and burns on Boltzmann's schedule. Even Planck, deriving quanta in 1900, started from Boltzmann's 1877 cell method—the formula didn't just survive to quantum mechanics, it was one of its foundations.
The meaning of this universality is worth stating explicitly, because it will become plot-driving later. exp(−E/kT) is a compromise between energy and randomness: the higher the temperature, the more readily the system pays energy for variety; the lower, the more strictly it chooses cheap states. "Hot means more random" isn't a figure of speech but an exact property of this bracket. Remember this property. In a hundred years chips will control it.
The first transition happened in a war with numbers. At Los Alamos, calculating the equation of state for the hydrogen bomb, they hit a classical wall: to compute liquid properties requires taking an integral over configuration space with hundreds of dimensions. Any honest numerical scheme drowned: if you choose molecular configurations by uniform random sampling, almost all get negligible weight exp(−E/kT)—the calculation runs but results don't move. The MANIAC computer pounded in vain.
In the June 1953 issue of Journal of Chemical Physics, Metropolis, Arianna and Marshall Rosenbluth, Augusta and Edward Teller published the way out. The formulation from the article deserves word-for-word quotation, because this is one of the great inversions in computing history: "instead of choosing configurations at random and weighting them with exp(−E/kT), we choose configurations with probability exp(−E/kT) and weight them equally." Read twice: the Boltzmann exponential moved from the right side of the equation into the selection procedure itself. Physical law became algorithm. On a two-dimensional hard-sphere gas of a few hundred particles, MANIAC first calculated equilibrium properties of such a model by Monte Carlo method.
The most beautiful thing in this story—nobody in 1953 was thinking about "sampling from distributions" in the modern sense. They were calculating bombs and liquids. But in passing they created a universal recipe: if you have a target probability of the form exp(−E/kT), you can construct a random walk that in the limit walks exactly according to it. All modern statistics—from Bayesian MCMC chains to text generation—stands on this technique. And in thirty years someone will ask: what if we don't wait for temperature to settle, but turn it ourselves?
The question was asked by IBM physicists. Scott Kirkpatrick, Charles Gelatt, and Mario Vecchi from the Thomas J. Watson Research Center in May 1983 published in Science an article with an opening sentence hard to beat for sheer density of intent: "There is a deep and useful connection between statistical mechanics... and combinatorial optimization."
This is the deep and useful connection between statistical physics and the problem of finding the minimum of a function of many variables. Physicists rediscovered for computers what metallurgists always knew: how a substance cools determines what it becomes.
Annealing is ancient technology: metal is heated red-hot, then slowly cooled so atoms have time to find the right places in the crystal lattice. Cool quickly (quenching)—you get fine-grained scrap with trapped defects; cool gradually—large proper crystals. Kirkpatrick and co-authors translated this into optimization language: element placement on a chip is the "energy" of a configuration, local minima are "defects," and the knob T controls the algorithm's willingness to accept worsening. At high temperature the solver happily climbs out of wells—but doesn't get stuck in the first one it finds; during cooling it increasingly locks into what's found, until it crystallizes into a solution.
Simulated annealing became a classic and thrives to this day including D-Wave quantum annealers. But for our story something else matters more: the 1983 article physically sat on the desk of a man who at that moment was thinking about how to make a neural network learn. What followed is almost a comedy of moments.
Geoffrey Hinton told this scene in his 2025 Nobel lecture so thoroughly it could be shot as a chronicle. Small workshop in Rochester: John Hopfield presents his 1982 network—a system of binary neurons with symmetric connections that stores memories as energy minima. In the audience sits Terry Sejnowski, who "had recently read the fresh article by Kirkpatrick, Gelatt and Vecchi about a technique called simulated annealing." The two look at each other: annealing can be built into the Hopfield network. All it takes is replacing deterministic neurons with noisy ones—deciding whether to be one or zero by probability 1/(1+exp(−Δ/T)), where Δ is the energy difference and T is temperature.
Look at this formula carefully. It's the Boltzmann factor wrapped in a logistic sigmoid; at T→0 noise disappears leaving pure Hopfield network, while at high T the neuron nearly flips a fair coin. "Terry and I were thrilled by the idea of crossing Hopfield networks with simulated annealing," Hinton recalled. So in 1983–1985, together with Sejnowski, the Boltzmann machine was born: a network whose states are distributed—without any caveats or metaphors—according to the canonical distribution P(s) ∝ exp(−E/T). The Nobel Committee in its official 2024 scientific background calls T in this formula a "fictitious temperature"—fictitious because there's no thermometer, but the mathematics is identical to the last decimal.
The most elegant part—how it learned. Two phases: wake and sleep. In waking, data is shown to the network—and it reinforces connections between jointly active neurons; in sleep the network fantasizes according to its own distribution—and weakens connections guilty of hallucinations. The brain analogy of sleep as memory cleanup was floating in the air then (defended by Crick and Mitchison), and the mathematics fit it suspiciously well.
The machine's fate was paradoxical. For large networks it proved too slow and lost to backpropagation—Hinton speaks about this without sentimentality: "it was an interesting application of statistical physics to neural networks, but it wasn't the answer to how the brain learns." But seventeen years later he'd return to a simplified version—the restricted Boltzmann machine, RBM—and it's precisely this, as the Nobel committee recorded, that would become one of the pillars of 2000s deep learning. And in 2024 Hopfield and Hinton received the Nobel Prize in Physics "for foundational discoveries and inventions that enable machine learning with artificial neural networks." Neural networks grown from the Boltzmann exponential received a physics Nobel—and this isn't a curiosity but the most honest statement of kinship.
One last jump remained: from training networks to generating text. It happened in 2015, and again Hinton. In the paper "Distilling the Knowledge in a Neural Network" (with Oriol Vinyals and Jeff Dean) he solved the distillation problem: how to pack knowledge from a huge ensemble of models into a compact one suitable for production. The answer was: make the big model output not just the right answer but the full picture of its doubts—"soft" probabilities of all classes from which the small model learns how the big one generalizes. An image of a BMW might only with tiny probability be confused with a garbage truck, but that mistake is still a million times more likely than confusing it with a carrot—and such "almosts" carry knowledge.
The tool for softening doubts was ready: softmax with temperature, qᵢ = exp(zᵢ/T)/Σⱼ exp(zⱼ/T), "where T is a temperature, normally set to 1." The eighty-year-old Boltzmann bracket at that point, in which state energy was replaced with neuron logits and physical T with a dimensionless multiplier: higher T—distribution softer and more honestly shows the model's "second thoughts," lower—harder and more confident. Later Holtzman and co-authors in the ICLR 2020 work "The Curious Case of Neural Text Degeneration" would show the entire dilemma in sequence: under hard regimes generation becomes endlessly repetitive, at high temperatures—incoherent, because the tail of unlikely tokens can't be tamed by temperature alone; and they'd propose nucleus sampling, dynamically trimming the tail. Curiously, as the source of the "sampling with temperature" idea itself their bibliography cites—not nineteenth-century physics but Ackley, Hinton and Sejnowski's 1985 paper on the Boltzmann machine. The chain closed in writing: modern LLM literature traces the temperature knob's genealogy directly to the Boltzmann machine.
So the Boltzmann bracket reached your screen. In any chatbot API the temperature parameter is set by number—at OpenAI, for example, from 0 to 2: lower—the model is "focused and deterministic," higher—"more random." The slider you move choosing whether the answer will be a dry summary or lively improvisation turns the same quantity T that in 1877 determined what fraction of air molecules has energy sufficient to fly out of a teacup. And when people today argue whether this parameter sets a "real temperature"—physicists get nervous: in logits there are no joules, no Boltzmann constant, only T and the property "hot means more random." But this is a dispute about the knob's ontology, not its genealogy: Boltzmann left the formula's mathematics intact.
Look at the arc's chronology and it stops looking like a collection of curiosities. 1877: dispute about atom reality, desperate statistical maneuver. 1953: military calculations at Los Alamos, exponential first becomes algorithm. 1983: annealing as optimization method at IBM. 1985: neural network living by canonical distribution—and paper where "temperature" definitively enters computer science. 2015: distillation and legalization of temperature as user parameter. 2024–2025: Nobel Prize in Physics for neural networks and Nobel lecture where the author firsthand describes how from an article about cooling chips the Boltzmann machine was born.
Thirty-two years between "heating metal" and "heating logits"—and not a single link that had to be guesswork: all transitions documented, with dates, names, and formulas.
This is a rare case of rigorous—not rhetorical—continuity of ideas. Science loves beautiful words about "transferring methods between disciplines," but usually it's metaphor: bio-inspired algorithms resemble ant behavior. Here the transfer is literal: same exponential, same letter T, same "energy versus randomness" compromise reproduced in gas, chemical reaction, chip, neural network, and text generation. The Boltzmann distribution is possibly the most portable formula in science history, and it's not about fashion for physics terms but that it solves one and the same basic problem: how to honestly choose between options when there's a cost of error and an acceptable level of chaos. Every time this problem arises, the exponential appears—regardless of what exactly is burning, crystallizing, or printing text.
And the last irony, hard to believe without citations. Atom reality—the question from which the formula grew—won after the author's death: Boltzmann left in 1906, and Einstein's 1905 Brownian motion and molecular count closed the debate by 1908. And a bit over a hundred years later the same formula turns probabilities inside devices people use to argue about chatbot free will—and the world of fluctuations Boltzmann worked on in his final desperate hope literally became a factory of randomness on demand. The slab at Zentralfriedhof promises S = k log W. It didn't promise the logarithm would apply to such volume.