Matrix norm


In mathematics, a matrix norm is a vector norm in a vector space whose elements are matrices.

Definition

In what follows, will denote a field of either real or complex numbers.
Let denote the vector space of all matrices of size with entries in the field.
A matrix norm is a norm on the vector space. Thus, the matrix norm is a function that must satisfy the following properties:
For all scalars and for all matrices,
Additionally, in the case of square matrices, some matrix norms satisfy the following condition, which is related to the fact that matrices are more than just vectors:
A matrix norm that satisfies this additional property is called a submultiplicative norm. The set of all matrices, together with such a submultiplicative norm, is an example of a Banach algebra.
The definition of submultiplicativity is sometimes extended to non-square matrices, for instance in the case of the induced p-norm, where for and holds that. Here and are the norms induced from and, respectively, and.
There are three types of matrix norms which will be discussed below:
Suppose a vector norm on is given. Any matrix induces a linear operator from to with respect to the standard basis, and one defines the corresponding induced norm or operator norm on the space of all matrices as follows:
In particular, if the p-norm for vectors is used for both spaces and, then the corresponding induced operator norm is:
These induced norms are different from the [|"entrywise"] p-norms and the Schatten p-norms for matrices treated below, which are also usually denoted by
Any induced operator norm is a submultiplicative matrix norm: this follows from
and
Moreover, any induced norm satisfies the inequality
where is the spectral radius of. For symmetric or hermitian, we have equality in for the 2-norm, since in this case the 2-norm is precisely the spectral radius of. For an arbitrary matrix, we may not have equality for any norm; a counterexample being given by
which has vanishing spectral radius. In any case, for square matrices we have the spectral radius formula:

Special cases

In the special cases of the induced matrix norms can be computed or estimated by
which is simply the maximum absolute column sum of the matrix;
which is simply the maximum absolute row sum of the matrix;
where represents the largest singular value of matrix. There is an important inequality for the case :
where is the Frobenius norm. Equality holds if and only if the matrix is a rank-one matrix or a zero matrix. This inequality can be derived from the fact that the trace of a matrix is equal to the sum of its eigenvalues.
When we have an equivalent definition for as. It can be shown to be equivalent to the above definitions using the Cauchy–Schwarz inequality.
For example, for
we have
In the special case of , the induced matrix norm is the spectral norm. The spectral norm of a matrix is the largest singular value of i.e. the square root of the largest eigenvalue of the matrix where denotes the conjugate transpose of :

"Entrywise" matrix norms

These norms treat an matrix as a vector of size, and use one of the familiar vector norms. For example, using the p-norm for vectors,, we get:
This is a different norm from the induced p-norm and the Schatten p-norm, but the notation is the same.
The special case p = 2 is the Frobenius norm, and p = ∞ yields the maximum norm.

and norms

Let be the columns of matrix. The norm is the sum of the Euclidean norms of the columns of the matrix:
The norm as an error function is more robust since the error for each data point is not squared. It is used in robust data analysis and sparse coding.
The norm can be generalized to the norm,, defined by

Frobenius norm

When for the norm, it is called the Frobenius norm or the Hilbert–Schmidt norm, though the latter term is used more frequently in the context of operators on Hilbert space. This norm can be defined in various ways:
where are the singular values of. Recall that the trace function returns the sum of diagonal entries of a square matrix.
The Frobenius norm is an extension of the Euclidean norm to and comes from the Frobenius inner product on the space of all matrices.
The Frobenius norm is submultiplicative and is very useful for numerical linear algebra. The submultiplicativity of Frobenius norm can be proved using Cauchy–Schwarz inequality.
Frobenius norm is often easier to compute than induced norms and has the useful property of being invariant under rotations and, more generally, under unitary operations, that is, for any unitary matrix. This property follows from the cyclic nature of the trace :
and analogously
where we have used the unitary nature of .
It also satisfies
and
where is the Frobenius inner product.

Max norm

The max norm is the elementwise norm with p = q = ∞:
This norm is not submultiplicative.
Note that in some literature an alternative definition of max-norm, also called the -norm, refers to the factorization norm:

Schatten norms

The Schatten p-norms arise when applying the p-norm to the vector of singular values of a matrix. If the singular values of the matrix are denoted by σi, then the Schatten p-norm is defined by
These norms again share the notation with the induced and entrywise p-norms, but they are different.
All Schatten norms are submultiplicative. They are also unitarily invariant, which means that for all matrices and all unitary matrices and.
The most familiar cases are p = 1, 2, ∞. The case p = 2 yields the Frobenius norm, introduced before. The case p = ∞ yields the spectral norm, which is the operator norm induced by the vector 2-norm. Finally, p = 1 yields the nuclear norm, defined as
where denotes a positive semidefinite matrix such that. More precisely, since is a positive semidefinite matrix, its square root is well-defined. The nuclear norm is a convex envelope of the rank function, so it is often used in mathematical optimization to search for low rank matrices.

Consistent norms

A matrix norm on is called consistent with a vector norm on and a vector norm on if:
for all. All induced norms are consistent by definition.

Compatible norms

A matrix norm on is called compatible with a vector norm on if:
for all. Induced norms are compatible with the inducing vector norm by definition.

Equivalence of norms

For any two matrix norms and, we have
for some positive numbers r and s, for all matrices. In other words, all norms on are equivalent; they induce the same topology on. This is true because the vector space has the finite dimension.
Moreover, for every vector norm on, there exists a unique positive real number such that is a submultiplicative matrix norm for every.
A submultiplicative matrix norm is said to be minimal if there exists no other submultiplicative matrix norm satisfying.

Examples of norm equivalence

Let once again refer to the norm induced by the vector p-norm.
For matrix of rank, the following inequalities hold:
Another useful inequality between matrix norms is
which is a special case of Hölder's inequality.