6.1 Inner Product

The concept of an inner product is one of the most important matrix algebra concepts, also commonly referred to as the dot product. The inner product is a special operation defined on two vectors \(\mathbf{x}\) and \(\mathbf{y}\) that, as its name indicates, allows us to multiply \(\mathbf{x}\) and \(\mathbf{y}\) in a certain way.

The inner product of two vectors \(\mathbf{x}\) and \(\mathbf{y}\)—of the same size— is defined as:

\[ \mathbf{x \cdot y} = \sum_{i = 1}^{n} x_i y_i \]

basically the inner product consists of the element-by-element product of \(\mathbf{x}\) and \(\mathbf{y}\), and then adding everything up. The result is not another vector but a single number, a scalar. We can also write the inner product \(\mathbf{x \cdot y}\) in vector notation as \(\mathbf{x^\mathsf{T} y}\) since

\[ \mathbf{x^\mathsf{T} y} = (x_1 \dots x_n) \begin{pmatrix} y_1 \\ \vdots \\ y_n \end{pmatrix} = \sum_{i = 1}^{n} x_i y_i \]

Consider the data about Leia and Luke used in the last chapter:

     weight height
Leia    150     49
Luke    172     77

For example, the inner product of weight and height in \(\mathbf{X}\) is

\[ \texttt{weight}^\mathsf{T} \hspace{1mm} \texttt{height} = (150 \times 49) + (172 \times 77) = 20594 \]

What does this value mean? To answer this question we need to discuss three other concepts that are directly derived from having an inner product:

  1. Length of a vector

  2. Angle between vectors

  3. Projection of vectors

All these aspects play a very important role for multivariate methods. But not only that, we’ll see in a moment how many statistical summaries can be obtained through inner products.