Spectral Norm

• categories: Linear algebra, Norm

Definition

The spectral norm of a matrix $A\in {\mathbb{R}}^{m\times n}$ is defined as the largest singular value of $A$, denoted by ${\sigma }_{\max }(A)$. It is equivalent to the Norm of an operator induced by the vector 2-norm:

$$\Vert A{\Vert }_2=\underset{\mathbf{x}\ne 0}{\max }\frac{\Vert A\mathbf{x}{\Vert }_2}{\Vert \mathbf{x}{\Vert }_2}.$$

Alternatively:

$$\Vert A{\Vert }_2=\underset{\Vert \mathbf{x}{\Vert }_2=1}{\max }\Vert A\mathbf{x}{\Vert }_2.$$

For square matrices, $\Vert A{\Vert }_2$ is also the square root of the largest eigenvalue of $A^{T}A$:

$$\Vert A{\Vert }_2=\sqrt{{\lambda }_{\max }(A^{T}A)}.$$

---

Properties

1. Non-Negativity:
   $\Vert A{\Vert }_2\ge 0$.

2. Zero Matrix:
   $\Vert A{\Vert }_2=0$ if and only if $A=0$.

3. Submultiplicativity:
   For compatible matrices $A$ and $B$:

   $$\Vert AB{\Vert }_2\le \Vert A{\Vert }_2\Vert B{\Vert }_2.$$

4. Unitary Invariance:
   The spectral norm is invariant under orthogonal or unitary transformations:

   $$\Vert A{\Vert }_2=\Vert UA{\Vert }_2=\Vert AV{\Vert }_2,$$

   where $U$ and $V$ are orthogonal (or unitary) matrices.

5. Relationship to Singular Values:
   The spectral norm is the largest singular value ${\sigma }_{\max }(A)$:

   $$\Vert A{\Vert }_2={\sigma }_{\max }(A).$$

6. Relation to Vector Norms:
   The spectral norm satisfies:

   $$\Vert A\mathbf{x}{\Vert }_2\le \Vert A{\Vert }_2\Vert \mathbf{x}{\Vert }_2\forall \mathbf{x}.$$

---

Computation

1. Singular Value Decomposition (SVD): The spectral norm is the largest singular value obtained from the SVD of $A$:

   $$A=U\Sigma V^T,\Vert A{\Vert }_2={\sigma }_{\max }.$$

2. Power Iteration Method (Approximation):
   Iteratively compute the dominant eigenvalue of $A^{T}A$ to estimate $\Vert A{\Vert }_2$:

   • Start with a random vector $\mathbf{x}$.
   • Iterate: $\mathbf{y}=A^{T}A\mathbf{x}$, $\mathbf{x}=\frac{\mathbf{y}}{\Vert \mathbf{y}{\Vert }_2}$.
   • Converges to the eigenvector associated with ${\lambda }_{\max }(A^{T}A)$.