Concept map / Correlation
Share
Covariance and correlation
Use paired test scores to compare covariance and Pearson correlation, and see how outliers and nonlinear patterns affect them.
1Forty students, two forms, one number
Every dot is one student’s scores on two forms of a test, and the dashed lines mark the two means. Each student’s rectangle runs from the dot to the point where those lines cross, and its area is the product of the student’s two distances from the means. The area counts as positive (orange) when both scores fall on the same side of their means and negative (blue) when they fall on opposite sides.
Add up the areas and divide by n − 1 to get the covariance. Divide that by the two standard deviations to get Pearson’s correlation r. Four of the datasets below share r = 0.80, and they look nothing alike.
Point at a dot to see its rectangle, or drag it. With a keyboard, Tab to a dot and use the arrow keys.
For these 40 students r = 0.60, a strong upward straight-line association. 31 of them are above average on both forms or below average on both (orange), and 9 are above on one and below on the other (blue). The covariance is 48.6 square points.
More controls
2One new student at the far corner
Thirty students took two forms of a test, and their scores form a round cloud with r = 0.00. A 31st student scored 95 on both forms (the dashed ring). How much does that one student change r?
3Adding up the rectangles
The covariance is the average signed area: the orange total minus the blue total, divided by n − 1. It comes in square points, so its size depends on how each form is marked. Dividing by both standard deviations measures every rectangle's sides in standard deviations instead of points, and that keeps r between −1 and 1.
Orange outweighs blue, so the covariance is positive.
With your numbers: (2,148 − 253) / (40 − 1) = 48.6. x̄ and ȳ are the two means, the dashed lines.
With your numbers: 48.6 / (9.00 × 9.00) = 0.60. The standard deviations sx and sy are the gold bars on the axes. The scale slider changes the covariance and sy by the same factor, so r stays where it is.
4Check yourself
The tasks use the chart and controls in section 1 and tick themselves when you meet them.
- Get a strong correlation with a small covariance: r above 0.80 and a covariance below 20. The scale slider is under More controls.Now: r = 0.60, covariance = 48.6
- Set the target r to 0.80 or higher, add an outlier, and drag it (or use the arrow keys) until r for everyone falls below 0.30.Now: target r 0.60, no outlier yet
- Find a dataset where the form A score fixes the form B score exactly, yet r is within 0.10 of zero.Now: the adjustable cloud, r = 0.60
5The same numbers in R
The snippet builds a cloud like the page’s starting one: 40 students, both forms with mean 50 and standard deviation 9, and r of exactly 0.60, so the covariance is 48.6. Stretching form B by 1.5 raises the covariance to 72.9 and leaves r at 0.60. One student at (90, 90) lifts r to 0.73, and the U shape gives r = 0.
Covariance, correlation and an outlierShow the code
# 40 students, both forms with mean 50, SD 9 and r = 0.60
set.seed(7)
z1 <- scale(rnorm(40))[, 1]
z2 <- residuals(lm(rnorm(40) ~ z1))
z2 <- z2 / sd(z2)
a <- 50 + 9 * z1
b <- 50 + 9 * (0.6 * z1 + 0.8 * z2)
round(c(cov = cov(a, b), r = cor(a, b)), 2)
# Stretch form B by 1.5: the covariance grows, r does not
b2 <- 50 + 1.5 * (b - 50)
round(c(cov = cov(a, b2), r = cor(a, b2)), 2)
# Add one student at (90, 90), as the outlier button does
round(cor(c(a, 90), c(b, 90)), 2)
# The U shape: form B is fixed by form A, yet r is zero
u <- seq(-1, 1, length.out = 40)
round(cor(50 + 30 * u, 22 + 56 * u^2), 2)Uses base R only.
What covariance and r tell you
- Covariance comes in the product of the two units, so rescoring a form changes it. Pearson r has no units and stays the same when either score is multiplied by a positive number.
- r measures straight-line association only. The U shape has r = 0 even though form A fixes form B exactly.
- One student far from both means can make r or break it. The clump gets its whole correlation from one student, and one stray student drags the line from 1.00 down to 0.80.
- The four datasets with r = 0.80 look nothing alike, so plot the scores before you report r.
- The correlation between two parallel forms is one way to estimate reliability, and the pattern of correlations among items is where factor analysis starts.
Recommended reading
- Introductory Statistics 2eBarbara Illowsky and Susan DeanChapter 12, correlation and linear regression