Let be an m-by-n matrix over a field, where, is either the field, of real numbers or the field, of complex numbers. There is a unique n-by-m matrix over, that satisfies all of the following four criteria, known as the Moore-Penrose conditions:
,
,
,
.
is called the Moore-Penrose inverse of. Notice that is also the Moore-Penrose inverse of. That is,.
Useful lemmas
These results are used in the proofs below. In the following lemmas, A is a matrix with complex elements and n columns, B is a matrix with complex elements and n rows.
Lemma 1: ''A''''A'' = 0 ⇒ ''A'' = 0
The assumption says that all elements of A*A are zero. Therefore, Therefore, all equal 0 i.e..
Lemma 2: ''A''''AB'' = 0 ⇒ ''AB'' = 0
Lemma 3: ''ABB'' = 0 ⇒ ''AB'' = 0
This is proved in a manner similar to the argument of Lemma 2.
Existence and uniqueness
Proof of uniqueness
Let be a matrix over or. Suppose that and are Moore-Penrose inverses of. Observe then that Analogously we conclude that . The proof is completed by observing that then
Proof of existence
The proof proceeds in stages.
1-by-1 matrices
For any, we define: It is easy to see that is a pseudoinverse of .
Let be an n-by-n matrix over with zeros off the diagonal. We define as an n-by-n matrix over with as defined above. We write simply for. Notice that is also a matrix with zeros off the diagonal. We now show that is a pseudoinverse of :
General non-square diagonal matrices
Let be an m-by-n matrix over with zeros off the main diagonal, where m and n are unequal. That is, for some when and otherwise. Consider the case where. Then we can rewrite by stacking where is a square diagonal m-by-m matrix, and is the m-by- zero matrix. We define as an n-by-m matrix over, with the pseudoinverse of defined above, and the -by-m zero matrix. We now show that is a pseudoinverse of :
By multiplication of block matrices, so by property 1 for square diagonal matrices proven in the previous section,.
Similarly,, so
By 1 and property 3 for square diagonal matrices,.
By 2 and property 4 for square diagonal matrices,
Existence for such that follows by swapping the roles of and in the case and using the fact that.
The proof works by showing that satisfies the four criteria for the pseudoinverse of. Since this amounts to just substitution, it is not shown here. The proof of this relation is given as Exercise 1.18c in.
The results of this section show that the computation of the pseudoinverse is reducible to its construction in the Hermitian case. It suffices to show that the putative constructions satisfy the defining criteria.
''A''+ = ''A'' (''A A'')+
This relation is given as exercise 18 in, for the reader to prove, "for every matrix ". Write. Observe that Similarly, implies that i.e.. Additionally, so. Finally, implies that. Therefore,.
''A''+ = (''A'' ''A'')+''A''
This is proved in an analogous manner to the case above, using instead of Lemma 3.
Products
For the first three proofs, we consider products C = AB.
''A'' has orthonormal columns
If has orthonormal columns i.e. then. Write. We show that satisfies the Moore-Penrose criteria. Therefore,.
''B'' has orthonormal rows
If B has orthonormal rows i.e. then. Write. We show that satisfies the Moore-Penrose criteria. Therefore,
Since has full column rank, is invertible so. Similarly, since has full row rank, is invertible so. Write. We show that satisfies the Moore-Penrose criteria. Therefore,.
Conjugate transpose
Here,, and thus and. We show that indeed satisfies the four Moore-Penrose criteria. Therefore,. In other words: and, since
Projectors and subspaces
Define and. Observe that. Similarly, and finally, and. Thus and are orthogonal projection operators. Orthogonality follows from the relations and. Indeed, consider the operator : any vector decomposes as and for all vectors and satisfying and, we have It follows that and. Similarly, and. The orthogonal components are now readily identified. If belongs to the range of then for some, and. Conversely, if then so that belongs to the range of. It follows that is the orthogonal projector onto the range of. is then the orthogonal projector onto the orthogonal complement of the range of, which equals the kernel of. A similar argument using the relation establishes that is the orthogonal projector onto the range of and is the orthogonal projector onto the kernel of. Using the relations and it follows that the range of P equals the range of, which in turn implies that the range of equals the kernel of. Similarly implies that the range of equals the range of. Therefore, we find,
Additional properties
Least-squares minimization
In the general case, it is shown here for any matrix that where. This lower bound need not be zero as the system may not have a solution. To prove this, we first note that, using the fact that satisfies and, we have so that as claimed. If is injective i.e. one-to-one, then the bound is attained uniquely at.
The proof above also shows that if the system is satisfiable i.e. has a solution, then necessarily is a solution. We show here that is the smallest such solution. To see this, note first, with, that and that. Therefore, assuming that, we have Thus with equality if and only if, as was to be shown.
