Orthonormal Bases

Section 7.2 Orthonormal Bases

While the previous section extended the dot product in \(\rr^n\) to the notion of an inner product in \(V\text{,}\) in this section we focus on bases for a vector space. One remarkably nice feature of the standard basis in \(\rr^n\) is that, with respect to the dot product, it is an orthogonal set in which every vector is a unit vector. In this section we will see that such a basis exists in every inner product space.

🔗

Subsection 7.2.1 Orthonormality

Our first definition is the most important in the section, as we generalize the important characteristics of the standard basis in \(\rr^n\text{.}\)

🔗

Definition 7.2.1.

Let \(V\) be an inner product space, and let \(V' = \{\bfv_1, \ldots, \bfv_n\}\) be a subset of \(V\text{.}\) Then \(V'\) is an orthonormal set if it is an orthogonal set and \(\vnorm{\bfv_i}=1\) for all \(\bfv_i \in V'\text{.}\) If \(V'\) is a basis for \(V\text{,}\) then it is called an orthonormal basis.

🔗

Example 7.2.2.

The standard basis is an orthonormal basis of either \(\rr^n\) or \(\cc^n\text{.}\)

🔗

Example 7.2.3.

The following is an orthonormal basis of either \(\rr^2\) or \(\cc^2\text{:}\)

\begin{equation*} \mcb = \left\{ \begin{bmatrix} \frac{1}{\sqrt{2}} \\[6pt] \frac{1}{\sqrt{2}} \end{bmatrix}, \begin{bmatrix} \frac{1}{\sqrt{2}} \\[6pt] -\frac{1}{\sqrt{2}} \end{bmatrix} \right\}\text{.} \end{equation*}

🔗

Example 7.2.4.

For any fixed \(\theta \in \rr\text{,}\) the following is an orthonormal basis of \(\rr^2\) or \(\cc^2\text{:}\)

\begin{equation*} \mcb = \left\{ \begin{bmatrix} \cos(\theta) \\ \sin(\theta) \end{bmatrix}, \begin{bmatrix} -\sin(\theta) \\ \cos(\theta) \end{bmatrix} \right\}\text{.} \end{equation*}

🔗

Example 7.2.5.

Because of the requirement that each vector be a unit vector, the vectors in an orthonormal basis aren’t the prettiest to behold. Here is an orthonormal basis of \(\rr^3\text{:}\)

\begin{equation*} \mcb = \left\{ \begin{bmatrix} \frac{1}{\sqrt{5}} \\[6pt] \frac{2}{\sqrt{5}} \\[6pt] 0 \end{bmatrix}, \begin{bmatrix} -\frac{\sqrt{2}}{\sqrt{15}} \\[6pt] \frac{1}{\sqrt{30}} \\[6pt] -\frac{\sqrt{5}}{\sqrt{6}} \end{bmatrix}, \begin{bmatrix} -\frac{\sqrt{2}}{\sqrt{3}} \\[6pt] \frac{1}{\sqrt{6}} \\[6pt] \frac{1}{\sqrt{6}} \end{bmatrix} \right\}\text{.} \end{equation*}

🔗

Example 7.2.6.

Consider the following inner product on the space \(P_1\text{:}\)

\begin{equation*} \ip{p,q} = p(0)q(0) + p(1)q(1)\text{.} \end{equation*}

The set \(\mcb = \{t, 1-t \}\) is an orthonormal basis for \(P_1\text{.}\) We first verify that this is an orthogonal set:

\begin{equation*} \ip{t, 1-t} = (0)(1) + (1)(0) = 0\text{.} \end{equation*}

We can also see that each of these are unit vectors:

\begin{align*} \ip{t,t} \amp = 0^2 + 1^2 = 1\\ \ip{1-t, 1-t} \amp = 1^2 + 0^2 = 1\text{.} \end{align*}

Thus, \(\mcb\) is an orthonormal basis for \(P_1\) with this inner product.

🔗

Subsection 7.2.2 Coordinates in Orthonormal Bases

Having an orthonormal basis makes some tasks easier than they would be otherwise. In particular, when we need to find coordinates of vectors with respect to an orthonormal basis, the path is fairly gentle to walk.

🔗

Theorem 7.2.7.

Let \(V\) and \(W\) be inner product spaces, and let \(\mcb = \{\bfv_1, \ldots, \bfv_n \}\) be an orthonormal basis for \(V\) and \(\mcc = \{ \bfw_1, \ldots, \bfw_m \}\) be an orthonormal basis for \(W\text{.}\) For every \(\bfv \in V\) and every \(T \in L(V,W)\text{,}\) we have

\begin{equation} [\bfv]_{\mcb} = \begin{bmatrix} \ip{\bfv, \bfv_1} \\ \vdots \\ \ip{\bfv,\bfv_n} \end{bmatrix}\tag{7.2} \end{equation}

and

\begin{equation} [[T]_{\mcb,\mcc}]_{jk} = \ip{T(\bfv_k),\bfw_j}\text{.}\tag{7.3} \end{equation}

🔗

Proof.

For \(\bfv \in V\text{,}\) we have

\begin{equation*} \bfv = \sum_{i=1}^n c_i\bfv_i \end{equation*}

for some \(c_i \in \ff\) since \(\mcb\) is a basis of \(V\text{.}\) If we take the inner product of both sides of this equation with \(\bfv_k\) and use both the linearity of the inner product and the orthonormality of \(\mcb\text{,}\) we have

\begin{equation*} \ip{\bfv, \bfv_k} = \ip{\sum_{i=1}^n c_i\bfv_i, \bfv_k} = \sum_{i=1}^n c_i \ip{\bfv_i, \bfv_k} = c_k\text{.} \end{equation*}

This means that

\begin{equation*} \bfv = \sum_{i=1}^n \ip{\bfv, \bfv_i} \bfv_i\text{,} \end{equation*}

which is the same as (7.2).

🔗

We proceed similarly for the second statement in the theorem. For any \(\bfw \in W\text{,}\) we have

\begin{equation*} \bfw = \sum_{i=1}^m \ip{\bfw,\bfw_i}\bfw_i\text{.} \end{equation*}

Since \(T(\bfv_i) \in W\text{,}\) this must also be true for each \(T(\bfv_i)\text{:}\)

\begin{equation*} T(\bfv_i) = \sum_{i=1}^m \ip{T(\bfv_i),\bfw_i}\bfw_i\text{.} \end{equation*}

This is the same as (7.3).

🔗

We can illustrate one part of this theorem with an example.

🔗

Example 7.2.8.

In \(\rr^2\) with the dot product, we consider the basis \(\mcb = \{\bfv_1, \bfv_2\}\) from Example 7.2.3. Then the coordinate vector of \(\bfv = \begin{bmatrix} 2 \\ -5 \end{bmatrix}\) with respect to \(\mcb\) is

\begin{equation*} [\bfv]_{\mcb} = \begin{bmatrix} \ip{\bfv, \bfv_1} \\ \ip{\bfv, \bfv_2} \end{bmatrix} = \begin{bmatrix} -\frac{3}{\sqrt{2}} \\[6pt] \frac{7}{\sqrt{2}} \end{bmatrix}\text{.} \end{equation*}

🔗

The previous theorem has an important consequence. The inner product of any two vectors in an inner product space is the same as the usual inner product of their coordinate vectors in \(\cc^n\text{.}\)

🔗

Theorem 7.2.9.

Let \(\mcb = \{ \bfv_1, \ldots, \bfv_n \}\) be an orthonormal basis for an inner product space \(V\text{.}\) Then, for any vectors \(\bfu, \bfv \in V\text{,}\) we have

\begin{equation*} \ip{\bfu, \bfv} = \sum_{i=1}^n \ip{\bfu, \bfv_i} \overline{\ip{\bfv,\bfv_i}} \end{equation*}

and

\begin{equation*} \vnorm{\bfv}^2 = \sum_{i=1}^n |\ip{\bfv, \bfv_i}|^2\text{.} \end{equation*}

🔗

Proof.

By Theorem 7.2.7, we have

\begin{equation*} \bfu = \sum_{i=1}^n \ip{\bfu, \bfv_i}\bfv_i \hspace{6pt} \text{and} \hspace{6pt} \bfv = \sum_{i=1}^n \ip{\bfv, \bfv_i}\bfv_i\text{.} \end{equation*}

So, using the properties of the inner product and the orthonormality of \(\mcb\text{,}\) we have

\begin{align*} \ip{\bfu, \bfv} \amp = \ip{\sum_{i=1}^n \ip{\bfu, \bfv_i}\bfv_i, \sum_{j=1}^n \ip{\bfv, \bfv_j}\bfv_j}\\ \amp = \sum_{i=1}^n \sum_{j=1}^n \ip{\bfu,\bfv_i} \overline{\ip{\bfv,\bfv_j}} \ip{\bfv_i, \bfv_j}\\ \amp = \sum_{i=1}^n \ip{\bfu, \bfv_i} \overline{\ip{\bfv, \bfv_i}}\text{.} \end{align*}

🔗

Subsection 7.2.3 The Gram-Schmidt Process

Orthonormal bases are quite useful, but they do not appear around every corner. In this section we will describe a reliable process to produce an orthonormal basis of an inner product space.

🔗

Algorithm 7.2.10. The Gram-Schmidt Process.

This process results in an orthonormal basis for any finite-dimensional inner product space \(V\text{.}\)

Start with a basis \(\{ \bfv_1, \ldots, \bfv_n \}\) of \(V\text{.}\)
🔗

🔗
Define \(\bfe_1 = \frac{1}{\vnorm{\bfv_1}} \bfv_1\text{.}\)
🔗

🔗
For \(j = 2, \ldots, n\text{,}\) define \(\bfe_j\) recursively by

\begin{equation*} \bfe_j' = \bfv_j - \sum_{k=1}^{j-1} \ip{\bfv_j,\bfe_k} \bfe_k \end{equation*}

and

\begin{equation*} \bfe_j = \frac{1}{\vnorm{\bfe_j'}}\bfe_j'\text{.} \end{equation*}

🔗

🔗

Then \(\{ \bfe_1, \ldots, \bfe_n \}\) is an orthonormal basis of \(V\text{.}\) Additionally, for each \({j=1, \ldots, n}\text{,}\) we have \(\spn\{\bfe_1, \ldots, \bfe_j \} = \spn\{\bfv_1, \ldots, \bfv_j \}\text{.}\)

🔗

Note 7.2.11.

We omit the proof that the Gram-Schmidt process does what it claims to do, as the proof is on the long and technical side. However, it may be helpful for readers to have a non-technical description of the process. Starting with the original basis, for each vector we strip away the parts of the vector that point in the direction of previous basis vectors. (We understand that “direction” only makes geometric sense in \(\rr^n\text{,}\) but perhaps the reader will allow this use of analogy.) What remains gets normalized to be a unit vector and then added to the growing orthonormal basis.

🔗

To get a good handle on this algorithm, we now present some examples.

🔗

Example 7.2.12.

Consider the vector space \(P_1\) with the inner product introduced in Example 7.2.6. The standard basis for \(P_1\) is \(\{1, t\}\text{,}\) but this is not an orthonormal basis. We will apply the Gram-Schmidt process to this basis.

🔗

First, we calculate that \(\ip{1,1} = \sqrt{2}\text{,}\) so our first vector in the orthonormal basis should be \(\bfe_1 = \frac{1}{\sqrt{2}}\bfv_1 = \frac{1}{\sqrt{2}}\text{.}\) Then, for the second calculation we need

\begin{equation*} \ip{t, \bfe_1} = (0)\left(\frac{1}{\sqrt{2}}\right) + (1)\left(\frac{1}{\sqrt{2}}\right) = \frac{1}{\sqrt{2}}\text{.} \end{equation*}

So, we have

\begin{equation*} \bfe_2' = t - \frac{1}{\sqrt{2}}\frac{1}{\sqrt{2}} = t- \frac{1}{2}\text{.} \end{equation*}

Our final step is to normalize \(\bfe_2'\text{.}\) We find that \(\vnorm{\bfe_2'} = \frac{1}{\sqrt{2}}\text{,}\) meaning that

\begin{equation*} \bfe_2 = \sqrt{2}\bfe_2' = \sqrt{2}t - \frac{\sqrt{2}}{2}\text{.} \end{equation*}

The reader may want to verify that \(\{ \bfe_1. \bfe_2\}\) is indeed an orthonormal set in \(P_1\) with this inner product.

🔗

Example 7.2.13.

We will carry out the Gram-Schmidt process on a basis for \(\rr^3\) where we will use the standard dot product in \(\rr^3\text{.}\) Consider the set \(\mcb = \{\bfv_1, \bfv_2, \bfv_3 \}\text{,}\) where

\begin{equation*} \bfv_1 = \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}, \hspace{6pt} \bfv_2 = \begin{bmatrix} 2 \\ 1 \\ 0 \end{bmatrix}, \hspace{6pt} \bfv_3 = \begin{bmatrix} -1 \\ 1 \\ -1 \end{bmatrix}\text{.} \end{equation*}

It is easily checked that \(\mcb\) is a basis for \(\rr^3\text{.}\) Now \(\vnorm{\bfv_1} = \sqrt{2}\text{,}\) so we have

\begin{equation*} \bfe_1 = \begin{bmatrix} \frac{1}{\sqrt{2}} \\ 0 \\ \frac{1}{\sqrt{2}} \end{bmatrix}\text{.} \end{equation*}

We need \(\bfv_2 \cdot \bfe_1 = \frac{2}{\sqrt{2}}\) for the next calculation:

\begin{equation*} \bfe_2' = \bfv_2 - (\bfv_2 \cdot \bfe_1)\bfe_1 = \begin{bmatrix} 1 \\ 1 \\ -1 \end{bmatrix}\text{.} \end{equation*}

Since \(\vnorm{\bfe_2'} = \sqrt{3}\text{,}\) we have

\begin{equation*} \bfe_2 = \begin{bmatrix} \frac{1}{\sqrt{3}} \\[6pt] \frac{1}{\sqrt{3}} \\[6pt] -\frac{1}{\sqrt{3}} \end{bmatrix}\text{.} \end{equation*}

Finally, we will calculate \(\bfe_3'\) and \(\bfe_3\text{.}\) We need these two dot product calculations first:

\begin{equation*} \bfv_3 \cdot \bfe_1 = -\frac{2}{\sqrt{2}}, \hspace{6pt} \bfv_3 \cdot \bfe_2 = \frac{1}{\sqrt{3}}\text{.} \end{equation*}

Then we have

\begin{align*} \bfe_3' \amp = \bfv_3 - (\bfv_3 \cdot \bfe_1)\bfe_1 - (\bfv_3 \cdot \bfe_2)\bfe_2\\ \amp = \begin{bmatrix} -1 \\[6pt] 1 \\[6pt] -1 \end{bmatrix} + \begin{bmatrix} 1 \\[6pt] 0 \\[6pt] 1 \end{bmatrix} - \begin{bmatrix} \frac{1}{3} \\[6pt] \frac{1}{3} \\[6pt] -\frac{1}{3} \end{bmatrix}\\ \amp = \begin{bmatrix} -\frac{1}{3} \\[6pt] \frac{2}{3} \\[6pt] \frac{1}{3} \end{bmatrix}\text{.} \end{align*}

Since \(\vnorm{\bfe_3'} = \frac{\sqrt{6}}{3}\text{,}\) we have

\begin{equation*} \bfe_3 = \frac{3}{\sqrt{6}} \bfe_3' = \begin{bmatrix} -\frac{1}{\sqrt{6}} \\[6pt] \frac{2}{\sqrt{6}} \\[6pt] \frac{1}{\sqrt{6}} \end{bmatrix}\text{.} \end{equation*}

The set \(\{ \bfe_1, \bfe_2, \bfe_3 \}\) is an orthonormal basis of \(\rr^3\text{.}\)

🔗

Example 7.2.14.

In our final example we will consider \(\rr^2\) with the alternative inner product introduced in Example 7.1.29. We consider a basis \(\{\bfv_1, \bfv_2 \}\) for \(\rr^2\text{,}\) where

\begin{equation*} \bfv_1 = \begin{bmatrix} 1 \\ 1 \end{bmatrix}, \hspace{12pt} \bfv_2 = \begin{bmatrix} 3 \\ 0 \end{bmatrix}\text{.} \end{equation*}

For the Gram-Schmidt process, we first need to calculate \(\vnorm{\bfv_1} = \sqrt{3}\text{.}\) So, we have

\begin{equation*} \bfe_1 = \frac{1}{\sqrt{3}} \bfv_1 = \begin{bmatrix} \frac{1}{\sqrt{3}} \\[6pt] \frac{1}{\sqrt{3}} \end{bmatrix}\text{.} \end{equation*}

Then we need \(\ip{\bfv_2, \bfe_1} = \frac{6}{\sqrt{3}}\) for the next step of the process:

\begin{equation*} \bfe_2' = \bfv_2 - \ip{\bfv_2,\bfe_1}\bfe_1 = \begin{bmatrix} 1 \\ -2 \end{bmatrix}\text{.} \end{equation*}

Finally, we normalize \(\bfe_2'\) in order to get \(\bfe_2\text{.}\) We find that \(\vnorm{\bfe_2'} = \sqrt{6}\text{,}\) so

\begin{equation*} \bfe_2 = \begin{bmatrix} \frac{1}{\sqrt{6}} \\[6pt] -\frac{2}{\sqrt{6}} \end{bmatrix}\text{.} \end{equation*}

🔗

While the Gram-Schmidt process has obvious computational implications, it also has some theoretical consequences.

🔗

Corollary 7.2.15.

Every finite-dimensional inner product space \(V\) has an orthonormal basis.

🔗

Proof.

Since we know that every finite-dimensional vector space has a basis (Corollary 5.2.13), we can apply the Gram-Schmidt process to that basis. This proves that such a \(V\) always has an orthonormal basis.

🔗

Corollary 7.2.16.

Suppose that \(V' = \{\bfv_1, \ldots, \bfv_k \}\) is an orthonormal set in a finite-dimensional inner product space \(V\text{.}\) Then \(V'\) can be extended to an orthonormal basis of \(V\text{.}\)

🔗

Proof.

By Theorem 5.3.16, \(V'\) can be extended to a basis \({\mcb = \{ \bfv_1, \ldots, \bfv_n \}}\) of \(V\text{.}\) We can apply the Gram-Schmidt process to \(\mcb\text{,}\) producing an orthonormal basis \(\{ \bfe_1, \ldots, \bfe_n \}\) of \(V\text{.}\) Since \(V'\) is an orthonormal set, the Gram-Schmidt process will produce vectors \(\bfe_i\) such that \(\bfe_j = \bfv_j\) for \(j=1, \ldots, k\text{,}\) meaning that \(V'\) is a subset of \(\mcb\text{.}\) This justifies the claim that \(\mcb\) is an extension of \(V'\text{.}\)

🔗

Reading Questions 7.2.4 Reading Questions

1.

Define the following inner product on \(P_1\text{:}\)

\begin{equation*} \ip{p,q} = p(-2)q(-2) + p(2)q(2)\text{.} \end{equation*}

(You do not need to prove that this is an inner product.) Let \(p_1 = \frac{1}{4}t+\frac{1}{2}\) and \(p_2 = \frac{1}{4}t - \frac{1}{2}\text{.}\) Prove or disprove that \(\{ p_1, p_2 \}\) is an orthonormal basis of \(P_1\text{.}\) Show your work.

🔗

2.

Consider the set \(\{ \bfv_1, \bfv_2 \}\) of vectors in \(\rr^2\text{,}\) where

\begin{equation*} \bfv_1 = \begin{bmatrix} 1 \\ -2 \end{bmatrix}, \hspace{12pt} \bfv_2 = \begin{bmatrix} -1 \\ 1 \end{bmatrix}\text{.} \end{equation*}

Using the standard dot product in \(\rr^2\text{,}\) use the Gram-Schmidt process on this basis to produce an orthonormal basis for \(\rr^2\text{.}\)

🔗

Exercises 7.2.5 Exercises

1.

Consider the following basis for \(\rr^2\text{:}\)

\begin{equation*} \mcb = \left\{ \begin{bmatrix} 2 \\ -3 \end{bmatrix}, \begin{bmatrix} -1 \\ 2 \end{bmatrix} \right\}\text{.} \end{equation*}

Use the Gram-Schmidt process to create an orthonormal basis from \(\mcb\text{.}\) (Use the standard dot product on \(\rr^2\text{.}\))
🔗

🔗
Let \(\bfv = \begin{bmatrix} 3 \\ 3 \end{bmatrix}\text{.}\) Use Theorem 7.2.7 to find the coordinate vector of \(\bfv\) with respect to the orthonormal basis of \(\rr^2\) you created in part (a).
🔗

🔗

🔗

2.

Consider the basis \(\mcb = \{p_1, p_2 \}\) for \(P_1\text{,}\) where

\begin{equation*} p_1 = 2+t, \hspace{12pt} p_2 = 1-2t\text{.} \end{equation*}

Consider the inner product on \(P_1\) defined by

\begin{equation*} \ip{p, q} = 2p(0)q(0) + p(1)q(1)\text{.} \end{equation*}

(You do not need to prove that this is an inner product.)

Use the Gram-Schmidt process to create an orthonormal basis from \(\mcb\text{.}\)
🔗

🔗
Let \(p = 1 + t\text{.}\) Use Theorem 7.2.7 to find the coordinate vector of \(p\) with respect to the orthonormal basis of \(P_1\) you created in part (a).
🔗

🔗

🔗

Answer.

The orthonormal basis we obtain is

\begin{equation*} \left\{ \frac{1}{\sqrt{17}}(2+t), \frac{1}{\sqrt{34}}(3-7t)\right\}\text{.} \end{equation*}

🔗

🔗
The coordinate vector is

\begin{equation*} \begin{bmatrix} 10/ \sqrt{17} \\ -2/\sqrt{34} \end{bmatrix}\text{.} \end{equation*}

🔗

🔗

🔗

3.

Carry out the Gram-Schmidt process on the following set of vectors in \(\cc^3\text{.}\) Use the standard dot product in \(\cc^3\text{:}\)

\begin{equation*} \left\{ \begin{bmatrix} 1 \\ i \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ i \end{bmatrix}, \begin{bmatrix} i \\ 0 \\ 1 \end{bmatrix} \right\}\text{.} \end{equation*}

🔗

Answer.

The orthonormal basis we obtain is

\begin{equation*} \left\{ \begin{bmatrix} 1 / \sqrt{2} \\ i/\sqrt{2} \\ 0 \end{bmatrix}, \begin{bmatrix} i/\sqrt{6} \\ 1/\sqrt{6} \\ 2i / \sqrt{6} \end{bmatrix}, \frac{\sqrt{3}}{2\sqrt{2}}\begin{bmatrix} -\tfrac{1}{3} + \tfrac{4}{3}i \\ -\tfrac{2}{3} + \tfrac{1}{3}i \\ \tfrac{1}{3} - \tfrac{1}{3}i \end{bmatrix} \right\}\text{.} \end{equation*}

🔗

4.

Consider the standard basis of the vector space \(P_2\text{.}\) Carry out the Gram-Schmidt process on this basis with respect to the following inner product on \(P_2\text{:}\)

\begin{equation*} \ip{p,q} = \int_0^1 p(t)q(t)\; dt\text{.} \end{equation*}

🔗

5.

Let \(A\) be the following matrix in \(M_3(\rr)\text{:}\)

\begin{equation*} A = \begin{bmatrix} 1 \amp 0 \amp 2 \\ -1 \amp 1 \amp 0 \\ 0 \amp -2 \amp 1 \end{bmatrix}\text{.} \end{equation*}

Consider the inner product on \(\rr^3\) defined by

\begin{equation*} \ip{\bfu, \bfv} = (A\bfu) \cdot (A\bfv)\text{,} \end{equation*}

where the standard dot product is in view on the right side of this equation. Find an orthonormal basis of \(\rr^3\) with respect to this inner product.

🔗

Writing Exercises

6.

Suppose that the matrix of \(T \in L(V)\) with respect to some basis \(\mcb\) is upper triangular. Show that if \(\mcc\) is the orthonormal basis obtained by applying the Gram-Schmidt process to \(\mcb\text{,}\) then \([T]_{\mcc}\) is also upper triangular.

🔗

Prev Top Next