Strong Law of Large Numbers¶

Theorem¶

Let $X_1,X_2,\ldots$ be i.i.d. with $\mathbb{E}|X_1|<\infty$ and mean $\mu=\mathbb{E}X_1$ $E X_{1}$ . Then the sample mean $\displaystyle \bar X_n=\frac{1}{n}\sum_{i=1}^n X_i$ satisfies

\begin{aligned} \bar X_n\xrightarrow{a.s.}\mu. \end{aligned}

The i.i.d. assumption gives one common integrable law and independence. The conclusion is Almost-Sure Convergence: outside one probability-zero exceptional set, the entire sequence of sample averages tends to $\mu$ .

Interpretation¶

The running average of an integrable i.i.d. sequence settles at the population mean with probability one. Fix one outcome and follow the whole sequence of averages $\bar X_1,\bar X_2,\ldots$ : for almost every outcome this sequence of numbers converges to $\mu$ .

Running averages of three simulated sequences of fair die rolls over 400 rolls, each wandering early on and then settling near the horizontal line at the mean 3.5.

The figure shows three simulated sequences of fair die rolls, so $\mu=3.5$ $3.5$ . Each line is one sample path $n\mapsto\bar X_n$ $\overset{ˉ}{X}_{n}$ . Early on the paths wander, because a single roll still moves the average noticeably: after the first roll the three averages are $1$ , $3$ and $4$ . After a few hundred rolls each path stays close to $3.5$ . The strong law says that on almost every path the average does not only come close but converges to $3.5$ .

Each path is one sequence of draws. Almost every path eventually enters the band $\mu\pm\varepsilon$ and stays there; show more steps or narrow the band to test it.

The hypotheses are an i.i.d. sequence and $\mathbb{E}|X_1|<\infty$ $\infty$ ; finite variance is not needed, as the second Example shows. Independence cannot simply be dropped: if every $X_i$ equals the first die roll $X_1$ , the variables are identically distributed but $\bar X_n=X_1$ for every $n$ , which never approaches $3.5$ . Without a finite mean there is no $\mu$ to converge to.

Almost-sure convergence implies convergence in probability, so the strong law implies the weak law. Applied to indicators $\indicator_A$ of independent repetitions of an experiment, it says that the relative frequency of $A$ converges to $\mathbb{P}(A)$ almost surely. This justifies reading probabilities as long-run frequencies.

Key Results¶

Finite $\mathbb{E}|X_1|$ is sufficient; finite variance is not required.
The almost-sure conclusion also implies $\bar X_n\xrightarrow{P}\mu$ $μ$ .

Examples¶

For i.i.d. fair die rolls, the running average converges almost surely to $3.5$ .
For i.i.d. variables with density $f(x)=\frac{3}{2}x^{-5/2}$ for $x\ge1$ and zero elsewhere, the mean is $3$ and the second moment is infinite; the running average still converges almost surely to $3$ .

Prerequisites¶

Sources¶

Nicky van Foreest · Probability Distributions · Week 7

Aliases: SLLN, strong law, Kolmogorov's strong law Symbols: epsilon, infinity, mu, sum

Last updated: 20 October 2026 03:02