6.1 Inner Product
The concept of an inner product is one of the most important matrix algebra concepts, also commonly referred to as the dot product. The inner product is a special operation defined on two vectors \(\mathbf{x}\) and \(\mathbf{y}\) that, as its name indicates, allows us to multiply \(\mathbf{x}\) and \(\mathbf{y}\) in a certain way.
The inner product of two vectors \(\mathbf{x}\) and \(\mathbf{y}\)—of the same size— is defined as:
\[ \mathbf{x \cdot y} = \sum_{i = 1}^{n} x_i y_i \]
basically the inner product consists of the element-by-element product of \(\mathbf{x}\) and \(\mathbf{y}\), and then adding everything up. The result is not another vector but a single number, a scalar. We can also write the inner product \(\mathbf{x \cdot y}\) in vector notation as \(\mathbf{x^\mathsf{T} y}\) since
\[ \mathbf{x^\mathsf{T} y} = (x_1 \dots x_n) \begin{pmatrix} y_1 \\ \vdots \\ y_n \end{pmatrix} = \sum_{i = 1}^{n} x_i y_i \]
Consider the data about Leia and Luke used in the last chapter:
weight height
Leia 150 49
Luke 172 77
For example, the inner product of weight
and height
in \(\mathbf{X}\) is
\[ \texttt{weight}^\mathsf{T} \hspace{1mm} \texttt{height} = (150 \times 49) + (172 \times 77) = 20594 \]
What does this value mean? To answer this question we need to discuss three other concepts that are directly derived from having an inner product:
Length of a vector
Angle between vectors
Projection of vectors
All these aspects play a very important role for multivariate methods. But not only that, we’ll see in a moment how many statistical summaries can be obtained through inner products.