Lecture Notes

3. Basis and Dimension

Part of the Series on Linear Algebra.

By Akshay Agrawal. Last updated Sept. 8, 2017.

Previous entry: Subspaces; Next entry: Linear Maps and Matrices

Until now, the role of vectors was limited to their membership in vector spaces. But the vectors inside a vector space play roles more expressive than membership — the contents of vector spaces can in some sense be generated by lists of vectors, if the vectors in the lists are chosen appropriately. It is our endeavor in this section to make the previous statement precise. We begin by defining vocabulary needed to do so. Note that whenever we refer to a list of vectors in this section, we make implicit the assumption that the vectors all belong to the same vector space.

3.1 Linear Combinations and Spans

Definition 3.1 A weighted sum of a list of vectors $v_1, \ldots, v_m$ is called a linear combination. Such weighted sums have the form $a_1v_1 + a_2v_2 + \ldots + a_mv_m$, where the coefficients $a_i$ are drawn from the field over which the vector space is defined; for our purposes, simply think of the $a_i$ as real or complex numbers.

If the vectors $v_1, v_2, \ldots, v_n$ belong to the same vector space, then every linear combination of them also belongs to the vector space; if this is not immediately obvious to you, revisit the section on vector spaces and prove to yourself, in no more than a sentence, why this is the case.

Definition 3.2 The set generated by taking all linear combinations of the vectors $v_1, v_2, \ldots, v_n$ is called the span of the list of vectors, and we write

$$\operatorname{span} (v_1, v_2, \ldots, v_m) = \left\{ \sum_{i=1}^{m} a_i v_i \mid a_i \in \mathbb{F}\right\}.$$

The span of the empty list is defined to be the singleton set $\{0\}$.

Spans connect lists of vectors to vector spaces: The span of a list of vectors is a vector space. In fact, the generated vector space is the the smallest subspace containing all the vectors in the list.

Proof. Let $U = \operatorname{span}(v_1, v_2, \ldots, v_m)$, where all the vectors $v_i$ belong to the same vector space. That $U$ is a subspace of $V$ is clear, for $0 \in U$, $U$ is closed under addition, and $U$ is closed under scalar multiplication. That each of the $v_i$ belong to $U$ is also clear (simply take all but one of the coefficients in a linear combination of the vectors to be $0$). And $U$ is the smallest subspace containing $v_1, v_2, \ldots, v_m$ because every vector space containing them must contain their every linear combination, for vector spaces are closed under addition and scalar multiplication. $\qed$

Linear algebra is the study of linear maps on finite-dimensional vector spaces, whereas the study of infinite-dimensional is relegated to functional analysis (a subject for another year). We are now positioned to make precise what we mean by finitude.

Definition 3.3 A vector space is finite-dimensional if it is the span of a finite list of vectors.

3.2 Linear Independence

Combining spans with the notion of linear independence will allow us define the concept of a basis, which will prove to be one of our most useful tools.

Definition 3.4 A list of vectors $v_1, \ldots, v_m$ is linearly indepedent if the only way to write $0$ as a linear combination of them is to take every coefficient to be $0$.

The definition implies that $0$ cannot be in any linearly independent list. Moreover no vector in a linearly independent list can be written as a linear combination of the others, i.e., no vector in such a list is in the span of the others. These implications are instances of the following theorem: A list of vectors is linearly independent if and only if every vector in its span can be written uniquely as a linear combination of the vectors in it.

Proof. If a list of vectors is linearly independent, then each vector in its span has a unique representation as a linear combination of the vectors in it: taking two linear-combination representations of a vector in the span and subtracting one from the other shows that the coefficients in both representations must be equal if the list is to be linearly independent. And if each vector in the span of a list of vectors has a unique linear-combination representation, then in particular the $0$ vector does as well and the list is therefore linearly independent. $\qed$

Definition 3.5 A list of vectors is linearly depedent if it is not linearly independent.

In any linearly dependent list $v_1, v_2, \ldots, v_m$ at least one vector is redundant: There is some vector $v_j$ for $j \in \{1, 2, \ldots, m\}$ that is in the span of $v_1, v_2, \ldots, v_{j-1}$, and the span of the list is unchanged by removing $v_j$. (If you do not immediately see why this is true, stop here and verify this fact with approximately three lines of proof.) This fact implies that every list of linearly independent vectors in a vector space $V$ is no longer than any list of vectors spanning $V$.

3.3 Bases and Dimension

We can now define the concept of a basis.

Definition 3.6 A list of vectors in a vector space $V$ is a basis of $V$ if (1) it is linearly independent and (2) it spans $V$; the vectors in a basis are called basis vectors.

A basis can be thought of as a compact description of a vector space: Every vector in $V$ by definition can be written uniquely as a linear combination of basis vectors. It does not make sense to speak of the basis for a vector space, for every vector space has infinitely many bases.

Here are some important facts about bases, the veracity of which you may prove using material already presented in this section: All bases of a vector space have the same length, every linearly independent list in $V$ can be extended to a basis for $V$, and all lists spanning $V$ can be converted to a basis for $V$ by iteratively removing redundant vectors.

These facts allow us to define concretely the dimension of a vector space.

Definition 3.7 The dimension of a vector space $V$, denoted by $\dim V$, is the length of any basis for it.

This definition agrees with our intuitive understanding of dimension: $\Reals$ is of dimension $1$, for the scalar $1$ is a basis for it, and $\Reals^2$ is of dimension $2$, for the vectors $(1, 0)$, and $(0, 1)$ form a basis for it, and more generally $\Reals^n$ is of dimension $n$, for the list of vectors

$$(1, 0, \ldots, 0), (0, 1, \ldots, 0), \ldots, (0, \ldots, 0, 1)$$

is a basis for it. This basis is referred to as the standard basis for $\Reals^n$, and the indicator vector for the $i$-th coordinate is by convention referred to as $e_i$.

The dimension of a vector space is analagous to the cardinality of a set: If $U$ is a subspace of $V$, then $\dim U \leq \dim V$, and $\dim (U_1 + U_2) = \dim U_1 + \dim U_2 - \dim (U_1 - U_2)$ for subspaces $U_1$ and $U_2$.

3.4 Exercises

Prove some of the facts presented in sections 3.2 and 3.3.

3.5 References

Linear Algebra Done Right, by Sheldon Axler.