Singular Value¶

Definition¶

Let $A$ be a real $m\times n$ matrix and $p=\min(m,n)$$\min (m,n)$. The matrix $A^{\top}A$ is symmetric and positive semidefinite, so its eigenvalues are real and nonnegative; order them $\lambda_1\ge\cdots\ge\lambda_n\ge0$$0$. The singular values of $A$ are

They are the diagonal entries of $\Sigma$ in the singular value decomposition $A=U\Sigma V^{\top}$$U\Sigma V^{\top }$. Some texts list only the positive ones, or all $n$ values; the positive singular values are the same under every convention.

Interpretation¶

Singular values measure stretching. A matrix maps the unit sphere onto an ellipsoid whose semi-axis lengths are the singular values: $\sigma_1$ is the largest factor by which $A$ lengthens a unit vector, and zero singular values mark directions that are collapsed.

The figure uses the first example below, $A=\begin{pmatrix}1&1\\0&1\end{pmatrix}$$\left(\begin{array}{cc}1& 1\\ 0& 1\end{array}\right)$. Both eigenvalues are $1$, yet the unit circle is stretched into an ellipse with semi-axes $\sigma_1\approx1.618$ and $\sigma_2\approx0.618$$0.618$. The blue and red unit vectors on the left are the directions that land on these semi-axes (eigenvectors of $A^{\top}A$), and every length $\|A\boldsymbol{x}\|$ with $\|\boldsymbol{x}\|=1$ lies between $\sigma_2$ and $\sigma_1$.

Nonnegativity of $\lambda_i$ comes from $\boldsymbol{x}^{\top}A^{\top}A\boldsymbol{x}=\|A\boldsymbol{x}\|^2\ge0$$0$, with $\|\cdot\|$ the Euclidean norm.

Singular values are not the absolute values of the eigenvalues, except for special matrices such as symmetric ones. They exist for rectangular matrices, are always real and nonnegative, and do not change under orthogonal transformations of either side, whereas eigenvalues can be complex and are defined only for square matrices.

Key Results¶

• $A^{\top}A$ and $AA^{\top}$ have the same nonzero eigenvalues, so $A$ and $A^{\top}$ have the same singular values; either product may be used, whichever is smaller.
• The number of positive singular values equals the rank of $A$.
• $\sigma_1=\max_{\|\boldsymbol{x}\|=1}\|A\boldsymbol{x}\|$${\max }_{\Vert \mathbf{x}\Vert =1}\Vert A\mathbf{x}\Vert$, the spectral (operator) norm $\|A\|_2$. If $m\ge n$$n$, $\sigma_n=\min_{\|\boldsymbol{x}\|=1}\|A\boldsymbol{x}\|$${\min }_{\Vert \mathbf{x}\Vert =1}\Vert A\mathbf{x}\Vert$.
• Via the trace: $\displaystyle \sigma_1^2+\cdots+\sigma_p^2=\operatorname{tr}(A^{\top}A)=\sum_{i,j}a_{ij}^2$ (squared Frobenius norm).
• For square $A$, the determinant satisfies $|\det A|=\sigma_1\cdots\sigma_n$${\sigma }_1\cdots {\sigma }_n$, so $A$ is invertible if and only if $\sigma_n>0$$0$.
• Symmetric $A$: $\sigma_i=|\lambda_i(A)|$ after reordering. Orthogonal $Q$: all singular values equal $1$. Scaling: the singular values of $cA$ are $|c|\sigma_i$.

Examples¶

Eigenvalues versus singular values. $A=\begin{pmatrix}1&1\\0&1\end{pmatrix}$ has both eigenvalues equal to $1$. But $A^{\top}A=\begin{pmatrix}1&1\\1&2\end{pmatrix}$ has trace $3$ and determinant $1$, so eigenvalues $\tfrac{3\pm\sqrt{5}}{2}$ and

Checks: $\sigma_1\sigma_2=1=|\det A|$ and $\sigma_1^2+\sigma_2^2=3=1^2+1^2+0^2+1^2$$1^2$.

A $2\times3$ matrix. For $A=\begin{pmatrix}1&0&1\\0&1&0\end{pmatrix}$ the smaller product is $AA^{\top}=\begin{pmatrix}2&0\\0&1\end{pmatrix}$$\left(\begin{array}{cc}2& 0\\ 0& 1\end{array}\right)$, so $\sigma_1=\sqrt{2}$$\sqrt{2}$, $\sigma_2=1$$1$; the $3\times3$ matrix $A^{\top}A$ has eigenvalues $2,1,0$. The maximal stretch is attained at $\boldsymbol{x}=\tfrac{1}{\sqrt{2}}(1,0,1)^{\top}$$\frac{1}{\sqrt{2}}(1,0,1{)}^{\top }$: $A\boldsymbol{x}=(\sqrt{2},0)^{\top}$$(\sqrt{2},0{)}^{\top }$, of length $\sqrt{2}=\sigma_1$${\sigma }_1$. The rank is $2$, the number of positive singular values.

Prerequisites¶

• Eigenvalue and Eigenvector
• Symmetric Matrix
• Definite Matrix
• Euclidean Norm

Related Concepts¶

• Singular Value Decomposition
• Moore-Penrose Inverse
• Norm and Normed Vector Space
• Determinant

Sources¶

• Math V: Advanced Linear Algebra · Week 7

Aliases: singular values Keywords: σ_i, σ_max, eigenvalues of A^T A, spectral norm, ||A||_2, Frobenius norm Symbols: lambda, sigma, square root, sum

Last updated: 22 October 2026 19:04