Empirical correlation and regression

Ask the tutor about Empirical correlation and regression Open the graphing workbench

The statistical evaluation of a sequence of measurements bounds the mean and standard deviation of one series. Experiments often produce two series at once. Write them as

$x_{1}, \dots, x_{n_{1}} and y_{1}, \dots, y_{n_{2}} .$

Two basic questions then arise.

Is there a dependence between the two sequences?
Is there a significant difference between the two random variables?

Dependence is measured by a correlation coefficient. Significant differences are checked later with the $F$ -test, the $t$ -test, and the Wilcoxon test. This page answers only the first question.

Same length gives one number ρ

When the two sequences have the same length $n_{1} = n_{2} = n$ , the empirical correlation coefficient is

$ρ = \frac{\sum _{j = 1}^{n} ( x _{j} - x ˉ ) ( y _{j} - y ˉ )}{( n - 1 ) Δ x Δ y} .$

Here $x ˉ, y ˉ$ and $Δ x, Δ y$ are the empirical means and empirical standard deviations of each series. Always $- 1 \leq ρ \leq 1$ . If $ρ = 0$ , the sequences show no dependence. The dependence is stronger the larger $ρ^{2}$ is.

The closest line leaves $(Δ y)^{2} (1 - ρ^{2})$

Plot the pairs $(x_{j}, y_{j})$ in the plane. The straight line closest to those points, in the least-squares sense

$j = 1 \sum n (y_{j} - a - b x_{j})^{2} =! min,$

is the regression line

$y = y ˉ + ρ \frac{Δ y}{Δ x} (x - x ˉ) .$

The minimal sum of squared residuals equals $(Δ y)^{2} (1 - ρ^{2})$ . The fit is optimal when $ρ^{2} = 1$ : every point then lies on the line.

Example. The eight height pairs in the table below give $x ˉ = 175$ , $Δ x = 5$ , $y ˉ = 165$ , $Δ y = 5$ , and

$ρ = 0.96.$

Because $Δ y = Δ x$ , the regression line collapses to

$y = y ˉ + 0.96 (x - x ˉ) .$

The points hug that line, so the two height sequences are strongly linked.

	$1$	$2$	$3$	$4$	$5$	$6$	$7$	$8$	mean	$Δ$
$x$	$168$	$170$	$172$	$175$	$176$	$177$	$180$	$182$	$175$	$5$
$y$	$157$	$160$	$163$	$165$	$167$	$167$	$168$	$173$	$165$	$5$

Drag $ρ$ . The green points stay fixed at the eight pairs. The blue line is always $y = y ˉ + ρ (x - x ˉ)$ . At $ρ = 0.96$ it is the regression line above; near $ρ = \pm 1$ the residual factor $1 - ρ^{2}$ vanishes and the line would fit a perfectly linked series.

The eight solid points are the height pairs from the table above (x̄ = 175, ȳ = 165, Δx = Δy = 5). Drag the correlation ρ. The blue line is y = ȳ + ρ(x − x̄). At ρ = 0.96 it is the regression line above.

Comparing the means of two series, and asking whether their standard deviations match, is the next step — Comparing two means with t and F tests.

Empirical correlation and regression

Same length gives one number ρ

The closest line leaves (Δy)2(1−ρ2)

The closest line leaves $(Δ y)^{2} (1 - ρ^{2})$