Concept map / CFA model fit
Share
CFA model fit
Fit a one-factor model to a covariance matrix from a true model you can perturb, and compare the chi-square test, RMSEA, CFI, and SRMR as the sample size changes.
1One factor fitted to data that need a little more
A factor model implies a covariance matrix for the observed scores. A confirmatory factor analysis (CFA) fixes the structure in advance, here one factor behind six items, estimates the loadings and uniquenesses, and measures how far the implied matrix is from the data. The chi-square test compares that distance with the chi-square distribution of a correct model. RMSEA, CFI and SRMR describe the size of the misfit.
The true model below adds something the fit leaves out: a correlated error between two items, or a second factor. Keep that omission fixed and raise the sample size. Which of the four numbers move?
The true model adds a residual correlation of 0.20 between x5 and x6, which the fitted one-factor model leaves out. With the population matrix standing in for 300 people, the chi-square test does not reject the model (p = 0.346). RMSEA, CFI and SRMR all meet the Hu and Bentler cutoffs. The largest residual correlation, 0.11, is between x5 and x6.
More controls
The lab treats the true covariance matrix as the data, so there is no sampling noise.
2Ten times the people, the same omission
Two studies fit the same one-factor model to the same population: x5 and x6 share a residual correlation of 0.50 that the model leaves out. Study A has 200 people and study B has 2000. Both see the population matrix exactly, with no sampling noise. How do study B's chi-square and RMSEA compare with study A's?
3One discrepancy, multiplied by N
All four indices start from FML, the maximum likelihood discrepancy between the data matrix S and the matrix the one-factor fit implies. The chi-square multiplies it by N, RMSEA and CFI rescale the chi-square, and SRMR averages the squared residual correlations in the heat map. Point at a symbol to see what it stands for.
Point at F, N, df or the baseline term and the matching line or tile lights up.
With your numbers: FML = 0.033517, so χ² = 300 × 0.033517 = 10.06. RMSEA = √(max(10.06 − 9, 0) / (9 × 300)) = 0.020. The independence model gives χ²B(15) = 538.57, so CFI = 1 − 1.06 / 523.57 = 0.998.
For the same discrepancy the chi-square grows in proportion to N, while RMSEA flattens out at √(F/df). At small N RMSEA sits lower because subtracting df takes away the part of the chi-square that noise alone would produce. A population matrix has no noise, so here the subtraction removes some real misfit too.
N or N − 1?
lavaan multiplies F by N by default. LISREL, EQS and lavaan with likelihood = "wishart" use N − 1, which at N = 300 changes the chi-square by 0.3%.
4Check yourself
The tasks use the controls in section 1. The sample switch for the third is under More controls.
- Keep the omission small and raise N until the chi-square test rejects the model at .05 while RMSEA is still 0.06 or below.Now: N = 300: χ² p = 0.346, RMSEA 0.020
- Find a true model for which CFI meets its cutoff (0.95 or above) while RMSEA misses its own (above 0.06).Now: CFI 0.998, RMSEA 0.020
- Make the one-factor model true, switch to one sample with N of 150 or fewer, and draw samples until RMSEA is above 0.05.Now: Model misspecified, population matrix, RMSEA 0.020
5Fit indices in lavaan
The code builds the lab’s starting covariance matrix, with a residual correlation of 0.20 between x5 and x6, and fits one factor with lavaan at N = 300 and N = 3000. The chi-square goes from 10.055 to 100.550, RMSEA from 0.020 to 0.058, and SRMR stays at 0.026. These are the numbers the lab shows at those two sample sizes.
Fit one factor and read the fit indices with lavaanShow the code
library(lavaan)
# True model: one factor, plus a residual correlation of 0.20
# between x5 and x6 that the fitted model leaves out
lam <- c(0.8, 0.7, 0.6, 0.7, 0.6, 0.5)
S <- lam %o% lam
diag(S) <- 1
S[5, 6] <- S[6, 5] <- S[5, 6] + 0.20 * sqrt(0.64 * 0.75)
dimnames(S) <- list(paste0("x", 1:6), paste0("x", 1:6))
# Fit one factor as if the matrix came from N people
model <- "f =~ x1 + x2 + x3 + x4 + x5 + x6"
idx <- c("chisq", "df", "pvalue", "rmsea", "cfi", "srmr")
for (N in c(300, 3000)) {
fit <- cfa(model, sample.cov = S, sample.nobs = N)
print(round(fitMeasures(fit, idx), 3))
}
# lavaan's chisq is N times F_ML; likelihood = "wishart"
# in cfa() switches to N - 1Uses lavaan. Install once with install.packages(c("lavaan")).
How the four fit indices behave
- Every index starts from one discrepancy, FML = log|| − log|S| + tr(S⁻¹) − p. The chi-square multiplies it by N, so for a fixed misspecification it grows in proportion to N, and a large sample rejects even a trivial omission.
- RMSEA = √(max(χ² − df, 0) / (df × N)) levels off at √(F/df) as N grows. In small samples a true model can still show a nonzero RMSEA, because sampling noise alone pushes the chi-square above df.
- CFI is the share of the independence model’s excess chi-square that the fitted model removes. High loadings make that baseline fit badly, so CFI can look good while RMSEA does not.
- SRMR is the root mean square of the residual correlations in the heat map, and it does not depend on N. The heat map also shows where the misfit is, which no single index can. Hu and Bentler’s cutoffs (RMSEA 0.06 or below, CFI 0.95 or above, SRMR 0.08 or below) came from simulations of particular models and are not significance tests.
- Measurement invariance fits the same factor model in several groups and uses changes in these indices to decide whether loadings and intercepts can be treated as equal.
Recommended reading
- Confirmatory Factor Analysis for Applied ResearchTimothy A. Brown2nd ed., chapter 3, evaluating the fit of a confirmatory factor model
- Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternativesLi-tze Hu and Peter M. BentlerPaper · Structural Equation Modeling, 6(1), 1999