6.5 Projection

The last aspect I want to touch related with the inner product is the so-called projections. The idea we need to consider is the orthogonal projection of a vector \(\mathbf{y}\) onto another vector \(\mathbf{x}\).

The basic notion of projection requires two ingredients: two vectors \(\mathbf{x}\) and \(\mathbf{y}\). To obtain the projection of \(\mathbf{y}\) onto \(\mathbf{x}\), we need to express \(\mathbf{x}\) in unit norm. The obtained projection \(\hat{\mathbf{y}}\) is expressed as \(a \mathbf{x}\). This means that a projection implies multiplying \(\mathbf{x}\) by some number \(a\), such that \(\hat{\mathbf{y}} = a \mathbf{x}\) is a stretched version of \(\mathbf{x}\). This is better appreciated in the following figure.

Having two nonzero vectors \(\mathbf{x}\) and \(\mathbf{y}\), we can project \(\mathbf{y}\) on \(\mathbf{x}\):

\[ \text{projection } \mathbf{\hat{y}} = \mathbf{x} \left( \frac{\mathbf{y^\mathsf{T} x}}{\mathbf{x^\mathsf{T} x}} \right) \]

\[ = \mathbf{x} \left( \frac{\mathbf{y^\mathsf{T} x}}{\| \mathbf{x} \|^2} \right) \]

Likewise, we can project \(\mathbf{x}\) on \(\mathbf{y}\):

\[ \text{proj}_{x} y \mathbf{\hat{x}} = \mathbf{y} \left( \frac{\mathbf{x^\mathsf{T} y}}{\mathbf{y^\mathsf{T} y}} \right) \]

\[ = \mathbf{y} \left( \frac{\mathbf{x^\mathsf{T} y}}{\| \mathbf{y} \|^2} \right) \]