Probability Distributions
The Stories Behind the Math

Abraham de Moivre was studying what happens when you flip a coin not ten times, not a hundred times, but thousands of times. He noticed that the histogram of results — how many heads, how many tails — started taking a shape. A bell shape. The same curve Gauss would later derive from telescope errors and claim as his own.

De Moivre died poor and largely forgotten. Gauss became one of the most celebrated mathematicians in history. Science is not always fair.

But de Moivre left us something important. By studying coin flips, he gave us the binomial distribution — and showed that inside it, if you look long enough, the normal distribution is hiding.

The mathematics came from coins. But the philosophy came from Jacob Bernoulli, whose book Ars Conjectandi — The Art of Conjecture — was published in 1713, eight years after his death. Bernoulli was not interested in gambling. He was interested in justice. His question was moral: how many observations do you need before you can be confident in a conclusion? How many witnesses before you convict? How many experiments before you publish? His answer was the binomial theorem — and the insight that more trials always bring you closer to truth.

It sounds morbid. It is. But Ladislaus Bortkiewicz was not morbid — he was pedantic. And in those numbers he saw something nobody else had: perfect order inside chaos. The death counts were random — but their distribution followed a mathematical formula to almost the decimal place.

That formula had been written sixty years earlier by Siméon Denis Poisson — and nobody had believed him. Poisson was a mathematician, not a statistician. His distribution was considered a theoretical curiosity. Bortkiewicz took it, applied it to horses and soldiers, and suddenly — everyone wanted to use it.

The Poisson distribution has one single parameter: lambda (λ) — the average number of events in your time period. If the average number of cardiac arrests per day is four — λ = 4. That is all you need to know. From that one number, the distribution tells you: what is the probability of zero arrests today? Of ten? Of exactly four?

And here is something strange: the mean and the variance are always equal. Both are λ. If your data show a variance much larger than the mean — Poisson does not fit. Your data have more chaos than the distribution assumes. That is called overdispersion — and it is a signal that you need a different model.

In the early 1900s, Ernest Rutherford and Frederick Soddy were studying radioactive decay in Montreal. Particles were being emitted — but not at perfectly regular intervals. Sometimes two in a second, sometimes none for three seconds. Rutherford noticed that if the number of particles per second followed a Poisson distribution, then the waiting time between particles followed a precise mathematical pattern. That pattern is the exponential distribution.

The exponential distribution has one parameter — lambda (λ), the same rate as Poisson. The mean waiting time is 1/λ. If particles arrive at 3 per second on average, you wait an average of 1/3 of a second between them.

But the exponential has a strange property that took physicists a while to accept: it is memoryless. The machine does not remember how long it has been running. A radioactive atom that has existed for a thousand years is no more likely to decay in the next second than one created a moment ago. The past is irrelevant. Only the rate matters.

Take any dataset — any shape, any distribution. Take a random sample of n observations and calculate their mean. Now repeat that a thousand times. Plot all those means.

The distribution of those means will be normal.

Always. Regardless of what the original data looked like. This is the Central Limit Theorem, formally proved by Aleksandr Lyapunov in 1901 — though Gauss, Laplace, and de Moivre had all glimpsed it before him.

This is why CRP can be log-normal and you can still use a t-test on it with a large enough sample. This is why income can be wildly skewed and economists still report confidence intervals. The test does not care about your raw data. It cares about the mean of your sample — and the mean is always normal.

There is a catch: n must be large enough. The rule of thumb is n ≥ 30. For very skewed data, you may need more. For roughly symmetric data, less.

John von Neumann and Stanislaw Ulam called it the Monte Carlo method — named after the casino, because it relied on random chance. The idea was simple: instead of solving the equations, simulate thousands of random scenarios and average the results. To generate randomness, you start with uniform numbers — every value equally likely — and transform them into whatever distribution you need.

Ulam reportedly got the idea while playing solitaire, wondering about the probability of a successful game. From solitaire to nuclear physics to clinical trial power calculations — the uniform distribution is the foundation of every simulation ever run on a computer.

The uniform distribution itself is simple: every value in the range [a, b] is equally likely. No value is more probable than any other. It is the flattest possible distribution — maximum uncertainty, no preference, no bias.

Gosset's problem was practical: he had small samples — a few batches of barley, a few fermentation tanks — and he needed to make decisions about quality. The normal distribution gave wrong answers with small n, and he knew it. So in 1908, he derived the correct distribution for small-sample means.

The t-distribution looks like the normal — bell-shaped, symmetric — but with heavier tails. Those heavier tails reflect a simple truth: when your sample is small, you are less certain. Your estimate of the standard deviation is itself uncertain, estimated from the same small data you are trying to analyse. The t-distribution widens the tails to account for that extra layer of uncertainty.

The parameter that controls the heaviness of the tails is degrees of freedom (df). For a one-sample t-test, df = n − 1. For two independent groups, df = n₁ + n₂ − 2. As df increases — as your sample grows — the t-distribution converges to the normal. Move the slider below and watch it happen.

Karl Pearson was not trying to expose Mendel. He was trying to solve a general problem: how do you know if an observed frequency distribution matches what your theory predicts? How far is "too far"? He needed a way to quantify the gap between observed and expected counts — and his answer, in 1900, was the chi-squared statistic.

The chi-squared distribution is always non-negative and right-skewed for small degrees of freedom — because it is built from squared values, which can never be negative. As degrees of freedom increase, it becomes more symmetric, eventually approaching normal.

It has two main uses in medicine. First: goodness-of-fit — do your observed frequencies match expected ones? Second: test of independence — are two categorical variables associated? Is smoking independent of lung cancer? Is blood type independent of disease risk? These are chi-squared questions.

Francis Galton — Darwin's cousin, founder of eugenics, and one of the most brilliant and controversial scientists of the 19th century — noticed something that Quetelet had missed. Quetelet believed everything followed the normal distribution. Galton was sceptical. He pointed out that many biological measurements — income, lifespan, species sizes, the strength of inherited traits — were not symmetric. They were right-skewed, with a long tail of large values.

His explanation in 1879 was elegant: some phenomena are governed by additive processes — many small independent influences adding up, producing a normal distribution. But others are governed by multiplicative processes — influences that compound on each other. When you multiply many small random factors together, the result is not normally distributed. But its logarithm is.

This matters enormously in medicine. CRP does not increase by a fixed amount when inflammation rises — it doubles, triples, multiplies. Viral load is not additive — it grows exponentially. Drug concentrations in the body follow multiplicative pharmacokinetics. All of these are log-normal. And the key implication: log-transform them before analysis, and report the geometric mean — not the arithmetic mean, which is pulled far into the right tail by extreme values.