Concept map / Exploratory factor analysis

Share

Exploratory factor analysis

Choose the number of factors with a scree plot and parallel analysis, then compare unrotated, varimax and oblimin loadings for the same solution.

Prerequisites Factor analysis

1Eight items: how many factors are behind them?

A factor model starts from a structure you already believe in. Exploratory factor analysis (EFA) starts from the correlation matrix and asks how many factors it needs and which items belong to which.

The data below are simulated from two factors: x1 to x4 measure F1, and x5 to x8 measure F2. Each eigenvalue of the correlation matrix is the variance one more component would explain. The usual rule of thumb keeps every eigenvalue above 1. Parallel analysis keeps only those above what random data of the same size produce. Which rule finds two?

0.00.51.01.52.02.53.012345678EigenvalueComponenteigenvalue 1
Eigenvalues of this sample (filled: kept by parallel analysis)Random data, 95th percentileKaiser cutoff
Sample 16, the one R draws after set.seed(16).

In this sample of 200, 3 eigenvalues are above 1 (2.35, 1.45, 1.06), so the Kaiser rule keeps 3. Parallel analysis keeps 2: the first 2 beat the random-data line, and eigenvalue 3, 1.06, falls short of 1.15. The data have two factors. Without sampling noise the third eigenvalue would be 0.86; noise lifted it to 1.06.

2Which rotation fits better?

Take the lab's starting sample of 200 people and extract two factors. Varimax rotates them and keeps them uncorrelated. Oblimin rotates them and lets them correlate; here it estimates a factor correlation of 0.29. Which rotated solution reproduces the 28 correlations among the eight items more closely?

3The same two factors, turned three ways

Two factors are extracted from the sample above by principal-axis factoring. The unrotated solution is one answer among infinitely many that fit equally well. A rotation picks one that is easier to read, with each item loading mostly on one factor.

Varimax loadings for the sample of 200. Green is positive, blue negative; darker is larger.
ItemF1F2h²
x10.880.090.78
x20.630.140.42
x30.470.060.23
x40.370.100.15
x50.130.760.60
x60.120.560.33
x70.020.380.14
x80.080.230.06
Factor correlation: 0 (fixed)RMSR: 0.0397

Varimax turns the two axes, keeping them at right angles, until each item loads mostly on one factor. The largest secondary loading is 0.14, for x2. With correlated factors, an orthogonal rotation has to show the correlation as secondary loadings. The communalities run from 0.06 to 0.78 in all three views, and RMSR stays at 0.0397.

A common reading error is to treat a varimax secondary loading as a second construct in the item. When the factors correlate, part of every secondary loading is that correlation, and oblimin moves it into φ.

4Why rotation cannot change the fit

A rotation multiplies the loadings by a matrix T and adjusts the factor correlations to match. The two changes cancel in the matrix the model implies, so the residuals, the fit and each item's communality stay put. Point at a symbol to find its part of the table.

Λrot=Λ (T⊤)−1,Φ=T⊤T\htmlClass{term-load}{\Lambda_{\text{rot}}} = \Lambda\,(T^{\top})^{-1}, \qquad \htmlClass{term-phi}{\Phi} = T^{\top}T
Λrot Φ Λrot⊤=ΛΛ⊤,hi2=∑j,kλij ϕjk λik\htmlClass{term-load}{\Lambda_{\text{rot}}}\,\htmlClass{term-phi}{\Phi}\,\htmlClass{term-load}{\Lambda_{\text{rot}}^{\top}} = \Lambda\Lambda^{\top}, \qquad \htmlClass{term-h}{h_i^2} = \sum_{j,k}\htmlClass{term-load}{\lambda_{ij}}\,\htmlClass{term-phi}{\phi_{jk}}\,\htmlClass{term-load}{\lambda_{ik}}
Λrot Φ Λrot⊤=ΛΛ⊤hi2=∑j,kλij ϕjk λik\begin{aligned}\htmlClass{term-load}{\Lambda_{\text{rot}}}\,\htmlClass{term-phi}{\Phi}\,\htmlClass{term-load}{\Lambda_{\text{rot}}^{\top}} &= \Lambda\Lambda^{\top} \\ \htmlClass{term-h}{h_i^2} &= \textstyle\sum_{j,k}\htmlClass{term-load}{\lambda_{ij}}\,\htmlClass{term-phi}{\phi_{jk}}\,\htmlClass{term-load}{\lambda_{ik}}\end{aligned}

Point at the loadings, Φ or h². The matching part of the table lights up.

For x1: unrotated, h² = 0.795² + (−0.380)² = 0.775. With oblimin, h² = 0.887² + (−0.023)² + 2 × 0.887 × (−0.023) × 0.294 = 0.775.

5Check yourself

The tasks use the controls in sections 1 and 3 and tick themselves when you meet them.

  • Make the sample large enough that the largest eigenvalue of pure noise, the first point on the blue line, falls below 1.20.Now: N = 200: the largest random-data eigenvalue is 1.36
  • Raise the factor correlation until parallel analysis keeps only one factor, although the data still have two.Now: φ = 0.30: parallel analysis keeps 2
  • Find a setting where varimax gives some item a secondary loading of 0.25 or more while oblimin puts the same loading within 0.05 of zero.Now: x2: varimax 0.14, oblimin 0.06

6Parallel analysis and rotation in psych

The snippet draws the lab’s starting sample with set.seed(16), so its eigenvalues are the ones on the scree plot: 2.35, 1.45, 1.06, 0.84 and so on. fa.parallel() makes its own random data, so its cutoffs differ a little from the lab’s, but it also keeps 2 components. The varimax and oblimin loadings, the factor correlation of 0.29 and the communalities match section 3.

Parallel analysis and two rotations with psychShow the code
library(psych)
library(GPArotation)
# Two factors correlated 0.3: x1 to x4 measure F1, x5 to x8 F2
lam <- c(0.8, 0.65, 0.5, 0.4, 0.7, 0.55, 0.45, 0.35)
L <- cbind(c(lam[1:4], rep(0, 4)), c(rep(0, 4), lam[5:8]))
Phi <- matrix(c(1, 0.3, 0.3, 1), 2)
R <- L %*% Phi %*% t(L)
diag(R) <- 1
# The lab's starting sample: 200 people from set.seed(16)
set.seed(16)
x <- matrix(rnorm(200 * 8), 200) %*% chol(R)
colnames(x) <- paste0("x", 1:8)

# Eigenvalues, then parallel analysis against random data
round(eigen(cor(x))$values, 2)
pa <- fa.parallel(x, fa = "pc", plot = FALSE)

# Two principal-axis factors, rotated two ways
vm <- fa(x, nfactors = 2, fm = "pa", rotate = "varimax")
ob <- fa(x, nfactors = 2, fm = "pa", rotate = "oblimin")
round(cbind(unclass(vm$loadings), unclass(ob$loadings)), 2)
round(ob$Phi, 2)
# The communalities do not depend on the rotation
round(rbind(varimax = vm$communality,
            oblimin = ob$communality), 3)

Uses psych, GPArotation. Install once with install.packages(c("psych", "GPArotation")).

What the eigenvalues and rotations tell you