Orthogonal Projection¶

Definition¶

Let $W$ be a subspace of $\mathbb{R}^n$ (or of a finite-dimensional inner product space $V$ ). Every $\boldsymbol{b}$ can be written uniquely as

\boldsymbol{b}=\boldsymbol{p}+\boldsymbol{e},\qquad \boldsymbol{p}\in W,\quad \boldsymbol{e}\in W^\perp,

where $W^\perp$ is the orthogonal complement of $W$ . The vector $\boldsymbol{p}=\operatorname{proj}_W\boldsymbol{b}$ is the orthogonal projection of $\boldsymbol{b}$ onto $W$ , and $\boldsymbol{e}=\boldsymbol{b}-\boldsymbol{p}$ is the error. The map $\boldsymbol{b}\mapsto\operatorname{proj}_W\boldsymbol{b}$ is a linear map; its matrix $P$ is the orthogonal projection matrix onto $W$ .

Interpretation¶

Drop a perpendicular from $\boldsymbol{b}$ to $W$ : its foot is $\boldsymbol{p}$ . The error is orthogonal to every vector of $W$ , not just to $\boldsymbol{p}$ ; checking $\boldsymbol{e}\perp\boldsymbol{p}$ alone is not enough.

A plane W through the origin with a vector b above it, its projection p in W, the error e = b - p perpendicular to W, and another point w of W forming a right triangle with b and p.

In the figure, $\boldsymbol{e}$ stands perpendicular to the whole plane $W$ , not only to $\boldsymbol{p}$ . For any other $\boldsymbol{w}\in W$ the points $\boldsymbol{b}$ , $\boldsymbol{p}$ , $\boldsymbol{w}$ form a right triangle with the right angle at $\boldsymbol{p}$ , so the hypotenuse $\|\boldsymbol{b}-\boldsymbol{w}\|$ is longer than $\|\boldsymbol{e}\|$ . Pythagoras makes this exact: $\|\boldsymbol{b}-\boldsymbol{w}\|^2=\|\boldsymbol{e}\|^2+\|\boldsymbol{p}-\boldsymbol{w}\|^2$ $w ∥^{2}$ . So orthogonality is what makes $\boldsymbol{p}$ the closest point of $W$ to $\boldsymbol{b}$ in the Euclidean norm; this best-approximation property is the basis of least squares.

Move the entries of $\boldsymbol{b}$ and switch between a line and a plane $W$ : the error $\boldsymbol{e}=\boldsymbol{b}-\boldsymbol{p}$ stays perpendicular to $W$ . Switch on the comparison to see that every other $\boldsymbol{w}\in W$ lies farther from $\boldsymbol{b}$ than $\boldsymbol{p}$ .

Key Results¶

Best approximation: $\|\boldsymbol{b}-\operatorname{proj}_W\boldsymbol{b}\|<\|\boldsymbol{b}-\boldsymbol{w}\|$ for every $\boldsymbol{w}\in W$ with $\boldsymbol{w}\neq\operatorname{proj}_W\boldsymbol{b}$ $proj_{W} b$ .
Onto a line spanned by $\boldsymbol{a}\neq\boldsymbol{0}$ $0$ : $\boldsymbol{p}=\dfrac{\boldsymbol{a}^{\top}\boldsymbol{b}}{\boldsymbol{a}^{\top}\boldsymbol{a}}\,\boldsymbol{a}$ and $P=\dfrac{\boldsymbol{a}\boldsymbol{a}^{\top}}{\boldsymbol{a}^{\top}\boldsymbol{a}}$ $\frac{a a ^{⊤}}{a ^{⊤} a}$ .
With an orthonormal basis $\boldsymbol{q}_1,\ldots,\boldsymbol{q}_k$ of $W$ as columns of $Q$ : $\displaystyle \boldsymbol{p}=\sum_i(\boldsymbol{q}_i^{\top}\boldsymbol{b})\boldsymbol{q}_i$ and $P=QQ^{\top}$ $Q Q^{⊤}$ .
If $W=\operatorname{Col}(A)$ $Col (A)$ , the column space of an $n\times k$ matrix $A$ of full column rank: $P=A(A^{\top}A)^{-1}A^{\top}$ $A (A^{⊤} A)^{- 1} A^{⊤}$ .
A square matrix $P$ is an orthogonal projection matrix if and only if $P^2=P$ and $P^{\top}=P$ $P$ ; it then projects onto its column space.
Every orthogonal projection matrix is idempotent, but oblique projections are idempotent without being symmetric, so not every idempotent matrix is an orthogonal projection.
$I-P$ is the orthogonal projection onto $W^\perp$ , and $\|P\boldsymbol{b}\|\le\|\boldsymbol{b}\|$ $∥ b ∥$ .
The eigenvalues of $P$ lie in $\{0,1\}$ , and $\operatorname{rank}P=\operatorname{tr}P=\dim W$ (trace).

Examples¶

Onto a line. Project $\boldsymbol{b}=(1,2,3)^{\top}$ onto the line spanned by $\boldsymbol{a}=(1,1,1)^{\top}$ $(1, 1, 1)^{⊤}$ : $\boldsymbol{a}^{\top}\boldsymbol{b}=6$ and $\boldsymbol{a}^{\top}\boldsymbol{a}=3$ $3$ , so $\boldsymbol{p}=(2,2,2)^{\top}$ and $\boldsymbol{e}=(-1,0,1)^{\top}$ $(- 1, 0, 1)^{⊤}$ . Check $\boldsymbol{a}^{\top}\boldsymbol{e}=0$ $0$ . The distance from $\boldsymbol{b}$ to the line is $\|\boldsymbol{e}\|=\sqrt{2}$ $2$ .

Onto a plane. Let $W=\operatorname{Col}(A)$ with $A=\begin{pmatrix}1&0\\0&1\\1&0\end{pmatrix}$ and $\boldsymbol{b}=(1,2,5)^{\top}$ $(1, 2, 5)^{⊤}$ . Then $A^{\top}A=\operatorname{diag}(2,1)$ $diag (2, 1)$ , $A^{\top}\boldsymbol{b}=(6,2)^{\top}$ $(6, 2)^{⊤}$ , so the coefficients are $(3,2)^{\top}$ and

\boldsymbol{p}=(3,2,3)^{\top},\qquad \boldsymbol{e}=(-2,0,2)^{\top},\qquad P=\begin{pmatrix}\tfrac{1}{2}&0&\tfrac{1}{2}\\0&1&0\\\tfrac{1}{2}&0&\tfrac{1}{2}\end{pmatrix}.

The error is orthogonal to both columns, $P$ is symmetric with $P^2=P$ $P$ , and $\operatorname{tr}P=2=\dim W$ $dim W$ .

Prerequisites¶

Sources¶

Math V: Advanced Linear Algebra · Week 5

Aliases: orthogonal projector, projection onto a subspace, perpendicular projection Keywords: proj_W b, P = A(A^T A)^{-1} A^T, P = Q Q^T, (a^T b / a^T a) a, best approximation, projection matrix Symbols: element of, real numbers, square root, sum

Last updated: 22 October 2026 19:04