Concept map / Correlation

Share

Covariance and correlation

Use paired test scores to compare covariance and Pearson correlation, and see how outliers and nonlinear patterns affect them.

1Forty students, two forms, one number

Every dot is one student’s scores on two forms of a test, and the dashed lines mark the two means. Each student’s rectangle runs from the dot to the point where those lines cross, and its area is the product of the student’s two distances from the means. The area counts as positive (orange) when both scores fall on the same side of their means and negative (blue) when they fall on opposite sides.

Add up the areas and divide by n − 1 to get the covariance. Divide that by the two standard deviations to get Pearson’s correlation r. Four of the datasets below share r = 0.80, and they look nothing alike.

The first four share r = 0.80
020406080100020406080100Score on form B (y)Score on form A (x)r = 0.60mean x 50.0mean y 50.0
Same side of both meansOpposite sidesOne SD either side of each mean

Point at a dot to see its rectangle, or drag it. With a keyboard, Tab to a dot and use the arrow keys.

For these 40 students r = 0.60, a strong upward straight-line association. 31 of them are above average on both forms or below average on both (orange), and 9 are above on one and below on the other (blue). The covariance is 48.6 square points.

More controls

2One new student at the far corner

Thirty students took two forms of a test, and their scores form a round cloud with r = 0.00. A 31st student scored 95 on both forms (the dashed ring). How much does that one student change r?

020406080100020406080100Score on form BScore on form A30 students: r = 0.00

3Adding up the rectangles

The covariance is the average signed area: the orange total minus the blue total, divided by n − 1. It comes in square points, so its size depends on how each form is marked. Dividing by both standard deviations measures every rectangle's sides in standard deviations instead of points, and that keeps r between −1 and 1.

Orange outweighs blue, so the covariance is positive.

cov⁡(x,y)  =  ∑(xi−xˉ)(yi−yˉ)n−1\htmlClass{term-cov}{\operatorname{cov}(x,y)} \;=\; \frac{\htmlClass{term-cov}{\sum (x_i-\bar{x})(y_i-\bar{y})}}{n-1}

With your numbers: (2,148 − 253) / (40 − 1) = 48.6. x̄ and ȳ are the two means, the dashed lines.

r  =  cov⁡(x,y)sx×syr \;=\; \frac{\htmlClass{term-cov}{\operatorname{cov}(x,y)}}{\htmlClass{term-sx}{s_x} \times \htmlClass{term-sy}{s_y}}

With your numbers: 48.6 / (9.00 × 9.00) = 0.60. The standard deviations sx and sy are the gold bars on the axes. The scale slider changes the covariance and sy by the same factor, so r stays where it is.

4Check yourself

The tasks use the chart and controls in section 1 and tick themselves when you meet them.

  • Get a strong correlation with a small covariance: r above 0.80 and a covariance below 20. The scale slider is under More controls.Now: r = 0.60, covariance = 48.6
  • Set the target r to 0.80 or higher, add an outlier, and drag it (or use the arrow keys) until r for everyone falls below 0.30.Now: target r 0.60, no outlier yet
  • Find a dataset where the form A score fixes the form B score exactly, yet r is within 0.10 of zero.Now: the adjustable cloud, r = 0.60

5The same numbers in R

The snippet builds a cloud like the page’s starting one: 40 students, both forms with mean 50 and standard deviation 9, and r of exactly 0.60, so the covariance is 48.6. Stretching form B by 1.5 raises the covariance to 72.9 and leaves r at 0.60. One student at (90, 90) lifts r to 0.73, and the U shape gives r = 0.

Covariance, correlation and an outlierShow the code
# 40 students, both forms with mean 50, SD 9 and r = 0.60
set.seed(7)
z1 <- scale(rnorm(40))[, 1]
z2 <- residuals(lm(rnorm(40) ~ z1))
z2 <- z2 / sd(z2)
a <- 50 + 9 * z1
b <- 50 + 9 * (0.6 * z1 + 0.8 * z2)
round(c(cov = cov(a, b), r = cor(a, b)), 2)

# Stretch form B by 1.5: the covariance grows, r does not
b2 <- 50 + 1.5 * (b - 50)
round(c(cov = cov(a, b2), r = cor(a, b2)), 2)

# Add one student at (90, 90), as the outlier button does
round(cor(c(a, 90), c(b, 90)), 2)

# The U shape: form B is fixed by form A, yet r is zero
u <- seq(-1, 1, length.out = 40)
round(cor(50 + 30 * u, 22 + 56 * u^2), 2)

Uses base R only.

What covariance and r tell you