Concept map / Exploratory factor analysis
Share
Exploratory factor analysis
Choose the number of factors with a scree plot and parallel analysis, then compare unrotated, varimax and oblimin loadings for the same solution.
1Eight items: how many factors are behind them?
A factor model starts from a structure you already believe in. Exploratory factor analysis (EFA) starts from the correlation matrix and asks how many factors it needs and which items belong to which.
The data below are simulated from two factors: x1 to x4 measure F1, and x5 to x8 measure F2. Each eigenvalue of the correlation matrix is the variance one more component would explain. The usual rule of thumb keeps every eigenvalue above 1. Parallel analysis keeps only those above what random data of the same size produce. Which rule finds two?
set.seed(16).In this sample of 200, 3 eigenvalues are above 1 (2.35, 1.45, 1.06), so the Kaiser rule keeps 3. Parallel analysis keeps 2: the first 2 beat the random-data line, and eigenvalue 3, 1.06, falls short of 1.15. The data have two factors. Without sampling noise the third eigenvalue would be 0.86; noise lifted it to 1.06.
2Which rotation fits better?
Take the lab's starting sample of 200 people and extract two factors. Varimax rotates them and keeps them uncorrelated. Oblimin rotates them and lets them correlate; here it estimates a factor correlation of 0.29. Which rotated solution reproduces the 28 correlations among the eight items more closely?
3The same two factors, turned three ways
Two factors are extracted from the sample above by principal-axis factoring. The unrotated solution is one answer among infinitely many that fit equally well. A rotation picks one that is easier to read, with each item loading mostly on one factor.
| Item | F1 | F2 | h² |
|---|---|---|---|
| x1 | 0.88 | 0.09 | 0.78 |
| x2 | 0.63 | 0.14 | 0.42 |
| x3 | 0.47 | 0.06 | 0.23 |
| x4 | 0.37 | 0.10 | 0.15 |
| x5 | 0.13 | 0.76 | 0.60 |
| x6 | 0.12 | 0.56 | 0.33 |
| x7 | 0.02 | 0.38 | 0.14 |
| x8 | 0.08 | 0.23 | 0.06 |
Varimax turns the two axes, keeping them at right angles, until each item loads mostly on one factor. The largest secondary loading is 0.14, for x2. With correlated factors, an orthogonal rotation has to show the correlation as secondary loadings. The communalities run from 0.06 to 0.78 in all three views, and RMSR stays at 0.0397.
A common reading error is to treat a varimax secondary loading as a second construct in the item. When the factors correlate, part of every secondary loading is that correlation, and oblimin moves it into φ.
4Why rotation cannot change the fit
A rotation multiplies the loadings by a matrix T and adjusts the factor correlations to match. The two changes cancel in the matrix the model implies, so the residuals, the fit and each item's communality stay put. Point at a symbol to find its part of the table.
Point at the loadings, Φ or h². The matching part of the table lights up.
For x1: unrotated, h² = 0.795² + (−0.380)² = 0.775. With oblimin, h² = 0.887² + (−0.023)² + 2 × 0.887 × (−0.023) × 0.294 = 0.775.
5Check yourself
The tasks use the controls in sections 1 and 3 and tick themselves when you meet them.
- Make the sample large enough that the largest eigenvalue of pure noise, the first point on the blue line, falls below 1.20.Now: N = 200: the largest random-data eigenvalue is 1.36
- Raise the factor correlation until parallel analysis keeps only one factor, although the data still have two.Now: φ = 0.30: parallel analysis keeps 2
- Find a setting where varimax gives some item a secondary loading of 0.25 or more while oblimin puts the same loading within 0.05 of zero.Now: x2: varimax 0.14, oblimin 0.06
6Parallel analysis and rotation in psych
The snippet draws the lab’s starting sample with set.seed(16), so its eigenvalues are the ones on the scree plot: 2.35, 1.45, 1.06, 0.84 and so on. fa.parallel() makes its own random data, so its cutoffs differ a little from the lab’s, but it also keeps 2 components. The varimax and oblimin loadings, the factor correlation of 0.29 and the communalities match section 3.
Parallel analysis and two rotations with psychShow the code
library(psych)
library(GPArotation)
# Two factors correlated 0.3: x1 to x4 measure F1, x5 to x8 F2
lam <- c(0.8, 0.65, 0.5, 0.4, 0.7, 0.55, 0.45, 0.35)
L <- cbind(c(lam[1:4], rep(0, 4)), c(rep(0, 4), lam[5:8]))
Phi <- matrix(c(1, 0.3, 0.3, 1), 2)
R <- L %*% Phi %*% t(L)
diag(R) <- 1
# The lab's starting sample: 200 people from set.seed(16)
set.seed(16)
x <- matrix(rnorm(200 * 8), 200) %*% chol(R)
colnames(x) <- paste0("x", 1:8)
# Eigenvalues, then parallel analysis against random data
round(eigen(cor(x))$values, 2)
pa <- fa.parallel(x, fa = "pc", plot = FALSE)
# Two principal-axis factors, rotated two ways
vm <- fa(x, nfactors = 2, fm = "pa", rotate = "varimax")
ob <- fa(x, nfactors = 2, fm = "pa", rotate = "oblimin")
round(cbind(unclass(vm$loadings), unclass(ob$loadings)), 2)
round(ob$Phi, 2)
# The communalities do not depend on the rotation
round(rbind(varimax = vm$communality,
oblimin = ob$communality), 3)Uses psych, GPArotation. Install once with install.packages(c("psych", "GPArotation")).
What the eigenvalues and rotations tell you
- In the population, a two-factor model cannot produce a third eigenvalue above 1. When a sample shows one, as the starting sample does with 1.06, it is sampling noise, and the eigenvalue-above-1 rule keeps it because the rule ignores the sample size.
- Parallel analysis compares each eigenvalue with random data of the same size, which is why its line falls toward 1 as N grows. It can still miss a weak factor: with φ at 0.50 the starting sample’s second eigenvalue falls below the line.
- A rotation chooses among solutions that fit identically. Varimax keeps the factors uncorrelated, so a real factor correlation shows up as secondary loadings; oblimin reports it as a correlation instead, 0.29 here against a true 0.30.
- Communalities, residuals and RMSR are the same under every rotation, so fit cannot tell you which rotation to use. That choice rests on whether you expect the traits to correlate.
- EFA lets the data suggest a structure. Confirmatory factor analysis fixes the structure in advance, sets the cross-loadings to zero, and tests how well that fits.
Recommended reading
- Evaluating the use of exploratory factor analysis in psychological researchLeandre R. Fabrigar, Duane T. Wegener, Robert C. MacCallum, and Erin J. StrahanPaper · Psychological Methods, 4(3), 1999
- A rationale and test for the number of factors in factor analysisJohn L. HornPaper · Psychometrika, 30(2), 1965