Lecture Notes

11. Eigenvectors and Diagonalization

Part of the Series on Linear Algebra.

By Akshay Agrawal. Last updated Dec. 20, 2018.

Previous entry: Isometries ; Next entry: Symmetric Matrices

In this section, we introduce the concepts of eigenvectors and eigenvalues of square matrices. The eigenvectors and eigenvalues of a matrix provide insight into its behavior as an operator; for this reason, the list of eigenvalues of a matrix is referred to as its spectrum. An important result that we will present is that if a matrix has a basis of eigenvectors, then it is diagonal with respect to that basis.

Definition

A nonzero vector $v$ is an eigenvector of a matrix $A \in \RR^{n \times n}$ if there exists a scalar $\lambda \in \RR$ such that

$$Av = \lambda v.$$

The scalar $\lambda$ is referred to as the eigenvalue corresponding to the eigenvector $v$. Notice that $v$ is an eigenvalue of $A$ if and only if $A - \lambda I$ is not injective, or, equivalently, $A - \lambda I$ is not invertible.

It is a fact that eigenvectors corresponding to distinct eigenvalues are linearly independent; for a proof, see 5.10 of Axler. Notice that this means that $A \in \RR^{n \times n}$ has at most $n$ distinct eigenvalues.

Diagonalizable matrices

Let $v_1, \ldots, v_n$ be a basis of $\RR^n$ consisting of eigenvectors of $A \in \RR^{n \times n}$, with corresponding eigenvalues $\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_n$ (note that such a basis need not exist). Then $A$ has a diagonal matrix $\Lambda$ with respect to the basis $(v_1, \ldots, v_n)$:

$$\Lambda = \begin{bmatrix} \lambda_1 & & \\ & \ddots & \\ & & \lambda_n \end{bmatrix}.$$

Whenever $A$ has a diagonal matrix with respect to some basis, we say that $A$ is diagonalizable. Concretely, letting $V = \begin{bmatrix} v_1 \cdots v_n \end{bmatrix}$, we have that

$$AV = V \Lambda,$$

or equivalently that

$$\Lambda = V^{-1} A V, \quad A = V \Lambda V^{-1}$$

The matrix $V \Lambda V^{-1}$ is referred to as an eigendecomposition of $A$.

Diagonalizability of $A \in \RR^{n \times n}$ is equivalent to $A$ having $n$ linearly independent eigenvectors. We can also give a sufficient (but not necessary) condition for diagonalizability: any $n \times n$ matrix with $n$ distinct eigenvalues is diagonalizable, since eigenvectors corresponding to distinct eigenvalues are linearly independent.

Notice that it is easy to compute powers of diagonalizable matrices:

$$A^k = V \Lambda^k V^{-1}.$$

Relationship between eigenvalues and rank

The rank of an $n \times n$ matrix is equal to the number of nonzero eigenvalues; in other words, if the eigenvalue $0$ is repeated $k$ times, the rank of the matrix is $n - k$. The eigendecomposition of a diagonalizable matrix $A$ of rank $p$ is of the form

$$A = \begin{bmatrix} v_1 \cdots v_n \end{bmatrix} \begin{bmatrix} \lambda_1 & & & \\ & \ddots & & \\ & & \lambda_p & \\ & & & 0 \end{bmatrix} \begin{bmatrix} v_1 \cdots v_n \end{bmatrix}^{-1}.$$

References

1. Sheldon Axler. Linear Algebra Done Right.
2. Stephen Boyd and Sanjay Lall. EE 263 Course Notes.