Order Statistic¶

Definition¶

Let $X_1,\ldots,X_n$ be random variables and let $X_{(1)}\le\cdots\le X_{(n)}$ be the ordered values. The random variable $X_{(k)}$ is called the $k$ th order statistic.

Interpretation¶

Order statistics describe a sample by rank rather than by the order in which the values were drawn: $X_{(k)}$ is the $k$ th smallest value. The most used ones are the minimum $X_{(1)}$ , the maximum $X_{(n)}$ and, for odd $n$ , the sample median.

Five values drawn on a number line in random order, and below them the same values sorted from smallest to largest, with the minimum, median and maximum labelled.

The upper row shows five values in the order in which they were drawn; the lower row shows the same five points renamed by rank. No point moves: the minimum $X_{(1)}$ is $X_2$ , the median $X_{(3)}$ is $X_4$ and the maximum $X_{(5)}$ is $X_3$ . Which draw gets which rank varies from run to run, so each $X_{(k)}$ is a random variable.

Sort a uniform sample and pick the $k$ th smallest. Over many samples $X_{(k)}$ clusters around $k/(n+1)$ $1)$ : the minimum near $0$ , the maximum near $1$ .

Even when $X_1,\ldots,X_n$ are i.i.d., the order statistics are neither independent nor identically distributed, since $X_{(1)}\le X_{(2)}$ always holds. The extremes are still easy to handle through the CDF $F$ of one observation: the maximum is at most $x$ exactly when every observation is, and the minimum exceeds $x$ exactly when every observation does, so independence gives

\mathbb{P}(X_{(n)}\le x)=F(x)^n,\qquad \mathbb{P}(X_{(1)}>x)=(1-F(x))^n.

A middle order statistic needs a counting argument instead: $X_{(k)}\le x$ means that at least $k$ of the $n$ observations are at most $x$ , and the number of such observations has the binomial distribution with parameters $n$ and $F(x)$ .

The brackets matter: $X_{(k)}$ is the $k$ th smallest value, while $X_k$ is the $k$ th draw. For exponential observations the minimum formula returns as the minimum of independent exponentials.

Key Results¶

The first order statistic is the sample minimum and the last is the sample maximum.
Order statistics are functions of a sample.
They are central in quantile estimation and reliability calculations.

Examples¶

The smallest of five component lifetimes is the first order statistic.
The median of a sample of odd size $n$ is the middle order statistic $X_{((n+1)/2)}$ ; for $n=5$ it is $X_{(3)}$ .
The maximum daily demand in a month is the largest order statistic of the daily demands.

Prerequisites¶

Sources¶

Probability Theory for EOR · Week 5
Game Theory · Week 6 (revisited)

Aliases: order statistics Keywords: maximum of iid uniforms, sample maximum

Last updated: 22 October 2026 19:04