Bayes Rule¶

Definition¶

Let $A$ and $B$ be events with $\mathbb{P}(A)>0$ and $\mathbb{P}(B)>0$ $0$ . Bayes' rule states that

\mathbb{P}(A\mid B)=\frac{\mathbb{P}(B\mid A)\mathbb{P}(A)}{\mathbb{P}(B)}.

Interpretation¶

Bayes' rule explains how to update a belief after new evidence is observed. It is useful when one conditional probability is easier to understand than the reverse conditional probability.

A frequency tree for 10,000 people: 100 sick and 9,900 healthy, then positive and negative tests in each group; of the 585 positive tests only 90 come from sick people, about 15 percent.

In the illustrated screening model, $\mathbb{P}(D)=0.01$ $0.01$ , $\mathbb{P}(+\mid D)=0.9$ and $\mathbb{P}(+\mid D^c)=0.05$ $0.05$ . Bayes' rule gives $\mathbb{P}(D\mid+)=\frac{0.9\cdot0.01}{0.9\cdot0.01+0.05\cdot0.99}=\frac{90}{90+495}\approx0.15$ $0.15$ . Thus only about $15\%$ of positive tests correspond to disease, because healthy people are much more numerous.

When $A$ is one block of a partition, the law of total probability computes the denominator $\mathbb{P}(B)$ by summing the joint contributions from all blocks. Bayes' rule then expresses the share of that total contributed by $A$ .

Key Results¶

Bayes' rule is obtained by writing $\mathbb{P}(A\cap B)$ in two ways using conditional probability.
With a partition $A_1,\ldots,A_n$ satisfying $\mathbb{P}(A_j)>0$ for every $j$ , and with $\mathbb{P}(B)>0$ $0$ , Bayes' rule becomes $\displaystyle \mathbb{P}(A_i\mid B)=\frac{\mathbb{P}(B\mid A_i)\mathbb{P}(A_i)}{\sum_j \mathbb{P}(B\mid A_j)\mathbb{P}(A_j)}$ $\frac{P ( B ∣ A _{i} ) P ( A _{i} )}{\sum _{j} P ( B ∣ A _{j} ) P ( A _{j} )}$ . Zero-probability blocks must be omitted from terms that condition on them.
Base rates enter through the prior probabilities $\mathbb{P}(A_i)$ , assigned before $B$ is observed, and can strongly affect the posterior probabilities $\mathbb{P}(A_i\mid B)$ $B)$ .

Examples¶

A diagnostic test with high sensitivity (it detects most people who have the disease) can still have a low positive predictive value (few of its positive results are true cases) when the disease is rare.
In spam filtering, the probability that a message is spam is updated after observing words in the message.
In reliability analysis, a component type can be inferred from an observed failure pattern using prior component frequencies.

Prerequisites¶

Sources¶

Probability Theory for EOR · Week 2

Aliases: Bayes' theorem, Bayes' formula, Bayes' rule, Bayes theorem, Bayes formula, Bayes's theorem, Bayes's rule, theorem of Bayes Symbols: intersection, sum

Last updated: 27 October 2026 11:52