Up to this point in the chapter, we have been concerned with solving systems of linear equations, and we have used the tool of matrices to great effect. In this section we will connect linear systems to some basic geometric concepts, and this will result in alternate ways of writing linear systems which, at times, will be more helpful.
An \(n\)-dimensional vector over \(\rr\) is an ordered list of \(n\) real numbers. We will adopt the convention that, unless stated otherwise, vectors are column vectors written in this form:
where \(v_i \in \rr\) for all \(i\text{.}\) (Column vectors are therefore matrices with a single column.) The set of all \(n\)-dimensional vectors over \(\rr\) is denoted \(\rr^n\text{.}\)
We say that two vectors \(\bfu\) and \(\bfv\) in \(\rr^n\) are equal if \(u_i=v_i\) for each \(i=1,\ldots,n\text{.}\) This means that an equation involving vectors in \(\rr^n\) captures the same information as \(n\) equations involving real numbers.
and let \(c \in \rr\text{.}\) Then the scalar multiple of \(\bfv\) by \(c\) is the vector \(c\bfv\) in \(\rr^n\) obtained by multiplying each entry of \(\bfv\) by \(c\text{;}\) that is,
then the sum of \(\bfu\) and \(\bfv\) is the vector \(\bfu + \bfv\) in \(\rr^n\) obtained by adding the corresponding entries in \(\bfu\) and \(\bfv\text{.}\) That is,
We will use the notation \(\mathbf{0}\) to denote the zero vector—that is, the vector in \(\rr^n\) whose entries are all \(0\text{.}\) We will also use the notation \(-\bfu\) to indicate the scalar multiple \((-1)\bfu\text{.}\)
When combining vectors, we are limited to adding vectors of the same dimension—that is, vectors with the same number of entries. However, we are not limited to either addition or scalar multiplication; we can certainly do both at once. Nor are we limited to adding only two vectors at a time. The following definition provides the correct generalization.
Let \(c_1, c_2, \ldots, c_m\) be real numbers and let \({\bfv_1, \bfv_2, \ldots, \bfv_m \in \rr^n}\text{.}\) Then the linear combination of the vectors \(\bfv_1, \bfv_2, \ldots, \bfv_m \) with weights \(c_1, c_2, \ldots, c_m\) is
The span of the vectors \(\bfv_1, \bfv_2, \ldots, \bfv_m\) is the set of all possible linear combinations of \(\bfv_1, \bfv_2, \ldots, \bfv_m\) and is written \(\spn\{\bfv_1, \ldots, \bfv_m\}\text{.}\) In other words, the span is defined to be
We have defined addition and scalar multiplication for vectors here, but these concepts also make sense for matrices. We will set the stage briefly here for a return in Section 2.3.
If we think of an \(m\times n\) matrix \(A\) in terms of its columns, then the \(n\) columns of \(A\) are all vectors in \(\rr^m\text{.}\) For two \(m\times n\) matrices \(A\) and \(B\text{,}\) we define the sum \(A + B\) in this way: column \(j\) of \(A + B\) is the sum of the two vectors in \(\rr^m\) which are the \(j\)th columns of \(A\) and \(B\text{.}\) Similarly, if \(c \in \rr\text{,}\) then we can define the scalar multiple \(cA\) in terms of its columns: column \(j\) of \(cA\) is the scalar multiple of the \(j\)th column of \(A\) by \(c\text{.}\) In this way these algebraic notions for matrices are built upon the corresponding notions for vectors.
We finally note that, as with vectors, the sum (and thus the linear combination) of two or more matrices only makes sense if all involved matrices are of the same size.
Before we explore the concept of span, we need to discuss the connection between systems of linear equations and vector equations. We saw a glimpse of this connection in Note 1.4.2, and we will develop it further here.
Solving this linear system involves (as always) investigating whether there are values of \(x_1\) and \(x_2\) which satisfy both of these equations simultaneously. We will now arrive at this same question from a different angle.
Since all of these vectors are in \(\rr^2\text{,}\) we can ask this question: Can \(\bfw\) be written as a linear combination of \(\bfu\) and \(\bfv\text{?}\) (This question is equivalent to asking if \(\bfw\) belongs to \(\spn\{\bfu,\bfv\}\text{.}\)) In other words, do there exist scalars \(x_1\) and \(x_2\) such that \(x_1\bfu + x_2\bfv = \bfw\text{?}\)
We have written this question in terms of a vector equation, but by the previous discussion we should be able to translate this question to a more familiar one about solutions to linear systems and answer the question using Algorithm 1.3.8.
Since there is a pivot in the final column of this matrix, we conclude (by invoking Theorem 1.3.5) that the linear system is inconsistent. This means that the answer to the original question is no, \(\bfv_3\) is not in \(\spn\{\bfv_1,\bfv_2\}\text{.}\)
Example 1.4.9 illustrates a general workflow for questions like this. There is no need to explicitly write out the intermediate step involving linear systems; instead, we can form a matrix using the given vectors as the appropriate columns, row reduce the matrix, and answer the question.
Combining vectors in \(\rr^n\) is already (or soon will be) fairly natural for readers of this chapter. Under the operations of scalar multiplication and addition, vectors in \(\rr^n\) have some useful properties, which we record in the following fact. We will not spend time with these properties now, but we will look at them intently in Section 2.3. These properties can all be verified using the corresponding properties of addition and multiplication of real numbers. (And the fact that real numbers have these properties is essential!)
For all vectors \(\bfu\text{,}\)\(\bfv\text{,}\) and \(\bfw\) in \(\rr^n\text{,}\) and for all real numbers \(c\) and \(d\text{,}\) the following properties hold.
Thinking about linear systems through the lens of vectors also brings some structure to the solutions to consistent linear systems. When a linear system is consistent and the solution is unique, we have recorded this in terms of the variables involved. For example, we recorded the solution to the linear system in Example 1.3.9 as \(x = -1/9\text{,}\)\(y = 4/3\text{,}\) and \(z = 2/9\text{.}\) However, if we think of these variables as forming a vector in \(\rr^3\text{,}\) then we can record the solution this way:
\begin{equation*}
\bfx = \begin{bmatrix} x \\ y \\ z \end{bmatrix} = \begin{bmatrix} -1/9 \\ 4/3 \\ 2/9 \end{bmatrix}\text{.}
\end{equation*}
Similarly, when a consistent system does not have a unique solution, we can again use vector notation. (This would replace the parametric description of the solution set we presented in Example 1.3.4.) In Example 1.3.4 we considered such a linear system. The solution set can now be written this way:
then all solutions to this system can be written as \(\bfv + t\bfw\text{,}\) where \(t\) can be any real number. As we learn more about the geometric interpretations of the solutions to linear systems, this phrasing in terms of vectors will be useful.
Before we leave this chapter, we need to introduce the connection between vectors and geometry. Readers who have taken multivariable calculus are likely aware of some of these concepts. For the sake of simplicity, we will restrict most of our discussion (and all of our illustrations) to \(\rr^2\) in this section.
We will visualize a vector \(\bfv = \left[\begin{array}{@{}c@{}}
a \\ b
\end{array}\right]\) by identifying it with the point \((a,b)\) in the plane. We will draw an arrow from the origin to \((a,b)\) to aid this visualization. So, for example, the vectors \(\bfu = \left[\begin{array}{@{}c@{}}
1 \\ -2
\end{array}\right]\) and \(\bfv = \left[\begin{array}{@{}c@{}}
-3 \\ 1
\end{array}\right]\) would be visualized in the following picture.
The graph of two vectors plotted on coordinate axes. The horizontal axis ranges from \(-3\) to \(3\) and the vertical axis ranges from \(-2\) to \(2\text{.}\) The vector \(\bfv\) is drawn in red and begins at the origin extending to \((-3,1)\text{.}\) The vector \(\bfu\) is drawn in blue and begins at the origin extending to the point \((1,-2)\text{.}\)
Scalar multiplication of a vector can be seen as stretching or contracting the vector, if the scalar is positive. If the scalar is negative, the vector will be pointing in the opposite direction, then stretched or contracted. Here is an example using the vector \(\bfv\) previously defined.
The graph of three vectors plotted on coordinate axes. The horizontal axis ranges from \(-3\) to \(3\) and the vertical axis ranges from \(-2\) to \(2\text{.}\) The vector \(\bfv\) is drawn in red and begins at the origin extending to \((-3,1)\text{.}\) The vector \(\frac{1}{2}\bfv\) is drawn in blue and begins at the origin extending to the point \((-1.5,0.5)\text{.}\) The vector \(-\bfv\) is drawn in green and begins at the origin extending to the point \((3,-1)\text{.}\)
Addition of vectors is also easy to visualize. To add two vectors, move the second vector from the origin so that its beginning coincides with the end of the first vector. The vector sum can be represented by the arrow from the origin to the end of this relocated second vector. (The dashed arrow in the following diagram is the relocated vector \(\bfv\text{.}\))
The graph of four vectors plotted on coordinate axes. The horizontal axis ranges from \(-3\) to \(3\) and the vertical axis ranges from \(-2\) to \(2\text{.}\) The vector \(\bfv\) is drawn in red and begins at the origin extending to \((-3,1)\text{.}\) The vector \(\bfu\) is drawn in blue and begins at the origin extending to the point \((1,-2)\text{.}\) A vector drawn in dashed red extends from \((1,-2)\) to \((-2,-1)\text{;}\) this vector is labeled \(\bfv\text{.}\) The vector \(\bfu + \bfv\) is drawn in black begins at the origin and extends to \((-2,-1)\text{.}\)
We can put these previous two ideas together to explain the way to visualize linear combinations (and spans). Let’s consider the vectors \(\bfu\) and \(\bfv\) as previously defined, and we will also define \(\bfw\) as \(\bfw = \left[\begin{array}{@{}c@{}}
2 \\ 2 \end{array}\right]\text{.}\)
When we ask whether \(\bfw\) is in \(\spn\{\bfu,\bfv\}\text{,}\) this is the same as asking whether we can form \(\bfw\) as a linear combination of \(\bfu\) and \(\bfv\text{.}\) We answer this question by forming a matrix with the appropriate columns and row reducing. We find the matrix
The graph of four vectors plotted on coordinate axes. The horizontal axis ranges from \(-3\) to \(3\) and the vertical axis ranges from \(-1\) to \(3\text{.}\) The vector \(-1.2\bfv\) is drawn in red and begins at the origin extending to \((3.6,-1.2)\text{.}\) The vector \(-1.6\bfu\) is drawn in blue and begins at the origin extending to the point \((-1.6,3.2)\text{.}\) A vector drawn in dashed red extends from \((-1.6,3.2)\) to \((2,2)\text{;}\) this vector is labeled \(-1.2\bfv\text{.}\) The vector \(\bfw\) is drawn in black begins at the origin and extends to \((2,2)\text{.}\)
The reader may be able to see that no matter what vector \(\bfw\) in \(\rr^2\) was chosen in this last instance, that vector would be in \(\spn\{\bfu,\bfv\}\text{.}\) (This means that \(\spn\{\bfu,\bfv\}=\rr^2\text{.}\)) This has everything to do with the relationship between \(\bfu\) and \(\bfv\text{.}\)
We will discuss this more carefully in Chapter 5, but this brief comment will suffice for now. As long as \(\bfu \neq \mathbf{0}\) in \(\rr^2\text{,}\) then \(\spn\{\bfu\}\) is a line through the origin. (Remember that the span in this case consists of all multiples of \(\bfu\text{.}\)) Similarly, as long as neither \(\bfu\) nor \(\bfv\) is a multiple of the other vector, then \(\spn\{\bfu,\bfv\}\) is all of\(\rr^2\text{.}\) (If we were talking about vectors in \(\rr^3\) instead of \(\rr^2\) this span would be a plane through the origin.)
List five vectors in \(\mathrm{Span}\{\bfu, \bfv\}\text{.}\) For each vector, write down the specific linear combination of \(\bfu\) and \(\bfv\) used to generate the vector.
There are an infinite number of correct answers! Three vectors that are in the span are \(\bfu = 1\bfu + 0\bfv\text{,}\)\(\bfv = 0\bfu + 1\bfv\text{,}\) and \(\bfo = 0\bfu + 0\bfv\text{.}\) Here are two additional vectors in the span:
Describe all possible ways of writing \(\begin{bmatrix} -1 \\ 1 \end{bmatrix}\) as a linear combination of \(\begin{bmatrix} 2 \\ 3 \end{bmatrix}\text{,}\)\(\begin{bmatrix} -1 \\ 0 \end{bmatrix}\text{,}\) and \(\begin{bmatrix} 4 \\ 1 \end{bmatrix}\text{.}\)
If \(\bfx = \begin{bmatrix} -1 \\ 1 \end{bmatrix}\text{,}\)\(\bfu = \begin{bmatrix} 2 \\ 3 \end{bmatrix}\text{,}\)\(\bfv = \begin{bmatrix} -1 \\ 0 \end{bmatrix}\text{,}\) and \(\bfw = \begin{bmatrix} 4 \\ 1 \end{bmatrix}\text{,}\) then \(\bfx\) can be written as
\begin{equation*}
\bfx = a \bfu + b \bfv + c \bfw\text{,}
\end{equation*}
where \(a = \frac{1}{3} - \frac{1}{3}c\text{,}\)\(b = \frac{5}{3} + \frac{10}{3}c\text{,}\) and \(c\) can be any real number.
For each of the following, determine whether \(\mathbf{b}\) is a linear combination of \(\bfu\text{,}\)\(\bfv\text{,}\) and \(\bfw\text{.}\) Explain your answer.
Under what conditions is \(\mathbf{b}\) in \(\mathrm{Span}\{\bfu, \bfv\}\text{?}\) This will be an equation (or equations) satisfied by the \(b_i\text{.}\) Explain your answer.
Let \(\bfu = \begin{bmatrix} 2 \\ 3 \end{bmatrix}\) and \(\bfv = \begin{bmatrix} -1 \\ 4 \end{bmatrix}\text{.}\) Show that \(\begin{bmatrix} h \\ k \end{bmatrix}\) is in \(\mathrm{Span}\{\bfu, \bfv\}\) for all \(h\) and \(k\text{.}\) Explain your answer.
Two vectors \(\bfu\) and \(\bfv\) in \(\rr^n\) are called collinear if \(\bfu = k\bfv\) or \(\bfv= k \bfu\) for some \(k \in \rr\text{.}\) Show that the span of any two nonzero vectors in \(\rr^2\) which are not collinear is all of \(\rr^2\text{.}\)
One might try to prove that \(\bfu = \begin{bmatrix} a \\ c \end{bmatrix}\) and \(\bfv = \begin{bmatrix} b \\ d \end{bmatrix}\) are collinear if and only if \(ad-bc = 0\text{.}\) Then an exercise from section 1.3 can be invoked.
Give an example of three vectors \(\bfu\text{,}\)\(\bfv\text{,}\)\(\bfw\) in \(\rr^3\) such that no two of them are collinear but \(\mathrm{Span}\{\bfu, \bfv, \bfw\} \neq \rr^3\text{.}\) Explain why your example works.
State criteria on vectors \(\bfv_1, \bfv_2, \ldots, \bfv_n\) in \(\rr^3\) such that \({\mathrm{Span}\{\bfv_1,\ldots,\bfv_n\} = \rr^3}\text{.}\) Explain your answer.
