9.1 Two Variables

Let us start by considering the most simple scenario when we have two variables \(X\) and \(Y\), represented by vectors \(\mathbf{x}\) and \(\mathbf{y}\), respectively.

Different types of relations between \(\mathbf{x}\) and \(\mathbf{y}\) are possible, as shown in the figure below containing several scatter-plots. There are linear relations of both type positive and negative. There are also two types of nonlinear relations, one monotone and the other one non-montone. In addition, the last two plots show absence of relations.

Different relations between two variables

Figure 9.1: Different relations between two variables


  • Positive linear relation: \(\mathbf{x}\) and \(\mathbf{y}\) vary simultaneously in the same direction; an increase in \(\mathbf{x}\) is accompanied by an increase in \(\mathbf{y}\) in constant proportion.

  • Negative linear relation: \(\mathbf{x}\) and \(\mathbf{y}\) vary in opposite directions; an increase in \(\mathbf{x}\) is accompanied by a decrease in \(\mathbf{y}\).

  • Nonlinear monotone relation: \(\mathbf{x}\) and \(\mathbf{y}\) vary in the same direction but the slope is not constant; changes in \(\mathbf{y}\) are different dependening on the values of \(\mathbf{x}\).

  • Nonlinear non-monotone relation: although there is a functional relation between \(\mathbf{x}\) and \(\mathbf{y}\), the relation is not monotone; \(\mathbf{y}\) increases and decrases accordingly to \(\mathbf{x}\).

  • No relation: regardless of the values of \(\mathbf{x}\), two things may happen. One is that knowing \(\mathbf{x}\), nothing can be said about \(\mathbf{y}\); the other is that \(\mathbf{y}\) remains constant.

For convenience sake most multivariate techniques focus on linear relations. Even though two variables \(\mathbf{x}\) and \(\mathbf{y}\) may show a nonlinear relation, it is often possible to apply some transformation to one of the them in order to have a more linear association.

The general approach to determine if \(\mathbf{x}\) and \(\mathbf{y}\) are related is to assess whether there is simultaneous variation between them. The idea is to define a measure for quantifying the strength of the relation. When \(\mathbf{x}\) and \(\mathbf{y}\) are in a quantitative scale, the most common way to quantify and assess the relationship between them is with the covariance and correlation.