Independent and Identically Distributed Random Variables¶

Definition¶

A sequence $X_1,X_2,\ldots$ is independent and identically distributed (i.i.d.) if it is mutually independent and all its variables have the same distribution.

Interpretation¶

Being i.i.d. means that the observations share one distribution and do not influence each other probabilistically. Both requirements are needed in the course’s weak and strong laws of large numbers and CLT.

Each variable in the diagram has the PMF of a fair die. Independence gives $\mathbb{P}(X_1=a,X_2=b,X_3=c)=\mathbb{P}(X_1=a)\mathbb{P}(X_2=b)\mathbb{P}(X_3=c)$$c)$, so these rolls form an i.i.d. sample.

A random variable assigns numerical values to outcomes. Saying that several such variables are identically distributed concerns their individual laws, while independence concerns their joint law; neither condition alone implies the other.

Key Results¶

• For i.i.d. $X_i$, $\mathbb{P}(X_1\in A_1,\ldots,X_n\in A_n)=\prod_{i=1}^n\mathbb{P}(X_i\in A_i)$ for every finite collection.
• Identical marginal distributions alone do not imply independence; pairwise independence alone does not establish mutual independence.

Examples¶

• Repeated independent fair die rolls are i.i.d., all with the discrete uniform law on $\{1,\ldots,6\}$.
• If $X_1=X_2=Z$ with $Z\sim\operatorname{Bern}(1/2)$$\mathrm{Bern}(1/2)$, the margins match but the observations are not i.i.d.: $\mathbb{P}(X_1=1,X_2=0)=0$ rather than $1/4$.

Prerequisites¶

• Random Variable
• Distribution
• Independent Random Variables

Related Concepts¶

• Bernoulli Distribution
• Exponential Distribution
• Binomial Distribution
• Order Statistic

Sources¶

• Probability Theory for EOR · Week 3
• Probability Distributions · supporting concept

Aliases: IID random variables, i.i.d. random variables, independent and identically distributed random variables, independent and identically distributed, i.i.d., iid, independent identically distributed Symbols: element of, product

Last updated: 17 October 2026 21:53