$$ \newcommand{\qed}{\tag*{$\square$}} \newcommand{\span}{\operatorname{span}} \newcommand{\dim}{\operatorname{dim}} \newcommand{\rank}{\operatorname{rank}} \newcommand{\norm}[1]{\|#1\|} \newcommand{\grad}{\nabla} \newcommand{\prox}[1]{\operatorname{prox}_{#1}} \newcommand{\inner}[2]{\langle{#1}, {#2}\rangle} \newcommand{\mat}[1]{\mathcal{M}[#1]} \newcommand{\null}[1]{\operatorname{null} \left(#1\right)} \newcommand{\range}[1]{\operatorname{range} \left(#1\right)} \newcommand{\rowvec}[1]{\begin{bmatrix} #1 \end{bmatrix}^T} \newcommand{\Reals}{\mathbf{R}} \newcommand{\RR}{\mathbf{R}} \newcommand{\Complex}{\mathbf{C}} \newcommand{\Field}{\mathbf{F}} \newcommand{\Pb}{\operatorname{Pr}} \newcommand{\E}[1]{\operatorname{E}[#1]} \newcommand{\Var}[1]{\operatorname{Var}[#1]} \newcommand{\argmin}[2]{\underset{#1}{\operatorname{argmin}} {#2}} \newcommand{\optmin}[3]{ \begin{align*} & \underset{#1}{\text{minimize}} & & #2 \\ & \text{subject to} & & #3 \end{align*} } \newcommand{\optmax}[3]{ \begin{align*} & \underset{#1}{\text{maximize}} & & #2 \\ & \text{subject to} & & #3 \end{align*} } \newcommand{\optfind}[2]{ \begin{align*} & {\text{find}} & & #1 \\ & \text{subject to} & & #2 \end{align*} } $$
In this section, we introduce the inverse of a linear map, and we use the notions of injectivity and surjectivity to characterize it. Intuitively, a linear map is invertible if there exists another linear map such that the composition of the two yields the identity map; the existence of an invertible map between two vector spaces tells us that the two spaces are in some sense equivalent, an idea that we’ll make precise shortly.
Definition 7.1 A linear map is invertible if there exists another linear map such that equals the identity map on (i.e., for all ) and equals the identity map on (i.e., for all ). Such a map is called the inverse of .
Note that in the above definition we say that is the inverse of ; implicit in our definition is the fact that an invertible linear map has a unique inverse. To see why this is true, let and be inverses of . Then
i.e., . Uniqueness allows us to denote the inverse of with a single symbol; by convention, this symbol is .
Theorem 7.2.A map is invertible if and only if it is both injective and surjective.
Theorem 7.2 should be intuitive, because a map that is both injective and surjective looks something like this:
As the picture illustrates, an invertible linear map from to can be thought of as a reversible relabeling of all the points in V to all the points in W. You should take the time to prove theorem 7.2, even if you find it intuitive.
Definition 7.3 Two vector spaces and are said to be isomorphic if there exists an invertible linear map between them, and an invertible linear map, i.e., a map that is both injective and surjective, is said to be an isomorphism between and .
When two vector spaces are isomorphic, we often say that they are “equivalent, up to relabeling.” The map illustrated in the previous section is an informal example of this: each letter in is relabeled to a unique letter in , and likewise each letter in is relabeled to a unique letter in .
For an analogy of what it means for two vector spaces to be isomorphic, imagine that and are two different natural languages such that both languages are capable of expressing exactly the same set of concepts and such that each language has exactly one word for every concept (i.e., words don’t have synonyms). Then every word in can be translated to a unique word in (injectivity), and every word in can be translated back to a unique word in (surjectivity). We might refer to this correspondence between concepts expressable in and concepts expressable in by saying that the two languages are equivalent, up to a relabeling of the concepts (the two languages use different words to refer to the same concept).
Theorem 7.4. Two vector spaces are isomorphic if and only if they have the same dimension.
The proof of theorem 7.4 is fairly straightforward; as a hint, you might find the Rank-Nullity theorem helpful in proving it.
A linear map from one vector space back to itself is called a linear operator.
Theorem 7.5 For a finite-dimensional vector space and a linear operator , the following are equivalent:
(a) T is invertible; (b) T is injective; (c) T is surjective.
Note that theorem 7.5 is not true for infinite-dimensional vector spaces.
Linear Algebra Done Right, by Sheldon Axler.