10.2 Cross-Products
There are two fundamental matrix products that play a crucial role when the data is in an \(n \times p\) matrix \(X\) with objects in rows, and variables in columns (assume \(n > p\)):
\[ \mathbf{X^\mathsf{T} X} \quad \& \quad \mathbf{X X^\mathsf{T}} \]
\(\mathbf{X^\mathsf{T} X}\) is also known as the minor product moment because it is of size \(p \times p\) (assuming \(n > p\)).
- sum-of-squares and cross-products (SSCP) of columns
- made of inner products of the columns of \(\mathbf{X}\)
- association matrix for the variables
\(\mathbf{X X^\mathsf{T}}\) is also known as the major product moment because is of size \(n \times n\) (assuming \(n > p\)).
- sum-of-squares and cross-products of rows
- made of inner products of the rows of \(\mathbf{X}\)
- association matrix for the objects
Covariance Matrix
If \(\mathbf{X}\) is mean-centered, i.e. \(\mathbf{X} = \mathbf{X_C}\), then
\[ \frac{1}{n} \mathbf{X^\mathsf{T} X} \qquad \text{and} \qquad \frac{1}{n-1} \mathbf{X^\mathsf{T} X} \]
are the covariance matrices (population and sample flavors).
Correlation Matrix
If \(\mathbf{X}\) is standardized, i.e. \(\mathbf{X} = \mathbf{X_S}\), then
\[ \frac{1}{n} \mathbf{X^\mathsf{T} X} \qquad \text{and} \qquad \frac{1}{n-1} \mathbf{X^\mathsf{T} X} \]
are the correlation matrices (population and sample flavors).