Concept map / CFA model fit

Share

CFA model fit

Fit a one-factor model to a covariance matrix from a true model you can perturb, and compare the chi-square test, RMSEA, CFI, and SRMR as the sample size changes.

1One factor fitted to data that need a little more

A factor model implies a covariance matrix for the observed scores. A confirmatory factor analysis (CFA) fixes the structure in advance, here one factor behind six items, estimates the loadings and uniquenesses, and measures how far the implied matrix is from the data. The chi-square test compares that distance with the chi-square distribution of a correct model. RMSEA, CFI and SRMR describe the size of the misfit.

The true model below adds something the fit leaves out: a correlated error between two items, or a second factor. Keep that omission fixed and raise the sample size. Which of the four numbers move?

True model0.80.70.60.70.60.5Fx1x2x3x4x5x6r = 0.20Residual correlations: data minus fitx2x3x4x5x6x1x2x3x4x50.010.010.000.010.010.00−0.01−0.01−0.01−0.01−0.02−0.02−0.02−0.020.11
Path the one-factor fit leaves outData correlation above the fitBelow the fit
Chi-square testχ²(9) = 10.06p = 0.346: not rejected at .05
RMSEA0.020Cutoff 0.06 or below: meets
CFI0.998Cutoff 0.95 or above: meets
SRMR0.026Cutoff 0.08 or below: meets

The true model adds a residual correlation of 0.20 between x5 and x6, which the fitted one-factor model leaves out. With the population matrix standing in for 300 people, the chi-square test does not reject the model (p = 0.346). RMSEA, CFI and SRMR all meet the Hu and Bentler cutoffs. The largest residual correlation, 0.11, is between x5 and x6.

More controls

The lab treats the true covariance matrix as the data, so there is no sampling noise.

2Ten times the people, the same omission

Two studies fit the same one-factor model to the same population: x5 and x6 share a residual correlation of 0.50 that the model leaves out. Study A has 200 people and study B has 2000. Both see the population matrix exactly, with no sampling noise. How do study B's chi-square and RMSEA compare with study A's?

3One discrepancy, multiplied by N

All four indices start from FML, the maximum likelihood discrepancy between the data matrix S and the matrix the one-factor fit implies. The chi-square multiplies it by N, RMSEA and CFI rescale the chi-square, and SRMR averages the squared residual correlations in the heat map. Point at a symbol to see what it stands for.

FML=log⁡∣Σ^∣−log⁡∣S∣+tr⁡(SΣ^−1)−pχ2=N×FMLRMSEA=max⁡(χ2−df, 0)df×NCFI=1−max⁡(χ2−df, 0)max⁡(χB2−dfB,  χ2−df,  0)\begin{aligned} \htmlClass{term-f}{F_{\text{ML}}} &= \log|\hat\Sigma| - \log|S| + \operatorname{tr}(S\hat\Sigma^{-1}) - p \\[2pt] \chi^2 &= \htmlClass{term-n}{N}\times\htmlClass{term-f}{F_{\text{ML}}} \qquad \text{RMSEA} = \sqrt{\frac{\max(\chi^2 - \htmlClass{term-df}{\mathit{df}},\,0)}{\htmlClass{term-df}{\mathit{df}}\times \htmlClass{term-n}{N}}} \\[2pt] \text{CFI} &= 1 - \frac{\max(\chi^2 - \htmlClass{term-df}{\mathit{df}},\,0)}{\max(\htmlClass{term-base}{\chi^2_B - \mathit{df}_B},\;\chi^2 - \htmlClass{term-df}{\mathit{df}},\;0)} \end{aligned}
FML=log⁡∣Σ^∣−log⁡∣S∣+tr⁡(SΣ^−1)−pχ2=N×FMLRMSEA=max⁡(χ2−df, 0)df×NCFI=1−max⁡(χ2−df, 0)max⁡(χB2−dfB,  χ2−df,  0)\begin{aligned} \htmlClass{term-f}{F_{\text{ML}}} &= \log|\hat\Sigma| - \log|S| + \operatorname{tr}(S\hat\Sigma^{-1}) - p \\[2pt] \chi^2 &= \htmlClass{term-n}{N}\times\htmlClass{term-f}{F_{\text{ML}}} \\[2pt] \text{RMSEA} &= \sqrt{\frac{\max(\chi^2 - \htmlClass{term-df}{\mathit{df}},\,0)}{\htmlClass{term-df}{\mathit{df}}\times \htmlClass{term-n}{N}}} \\[2pt] \text{CFI} &= 1 - \frac{\max(\chi^2 - \htmlClass{term-df}{\mathit{df}},\,0)}{\max(\htmlClass{term-base}{\chi^2_B - \mathit{df}_B},\;\chi^2 - \htmlClass{term-df}{\mathit{df}},\;0)} \end{aligned}

Point at F, N, df or the baseline term and the matching line or tile lights up.

With your numbers: FML = 0.033517, so χ² = 300 × 0.033517 = 10.06. RMSEA = √(max(10.06 − 9, 0) / (9 × 300)) = 0.020. The independence model gives χ²B(15) = 538.57, so CFI = 1 − 1.06 / 523.57 = 0.998.

Chi-square05010015050100200500100020005000Sample size N (log scale).05 criticalvalue 16.9RMSEA0.000.020.040.060.080.1050100200500100020005000Sample size N (log scale)cutoff 0.06
Value for this F at each NRMSEA limit √(F/df) = 0.061Gold line: your N

For the same discrepancy the chi-square grows in proportion to N, while RMSEA flattens out at √(F/df). At small N RMSEA sits lower because subtracting df takes away the part of the chi-square that noise alone would produce. A population matrix has no noise, so here the subtraction removes some real misfit too.

N or N − 1?

lavaan multiplies F by N by default. LISREL, EQS and lavaan with likelihood = "wishart" use N − 1, which at N = 300 changes the chi-square by 0.3%.

4Check yourself

The tasks use the controls in section 1. The sample switch for the third is under More controls.

  • Keep the omission small and raise N until the chi-square test rejects the model at .05 while RMSEA is still 0.06 or below.Now: N = 300: χ² p = 0.346, RMSEA 0.020
  • Find a true model for which CFI meets its cutoff (0.95 or above) while RMSEA misses its own (above 0.06).Now: CFI 0.998, RMSEA 0.020
  • Make the one-factor model true, switch to one sample with N of 150 or fewer, and draw samples until RMSEA is above 0.05.Now: Model misspecified, population matrix, RMSEA 0.020

5Fit indices in lavaan

The code builds the lab’s starting covariance matrix, with a residual correlation of 0.20 between x5 and x6, and fits one factor with lavaan at N = 300 and N = 3000. The chi-square goes from 10.055 to 100.550, RMSEA from 0.020 to 0.058, and SRMR stays at 0.026. These are the numbers the lab shows at those two sample sizes.

Fit one factor and read the fit indices with lavaanShow the code
library(lavaan)
# True model: one factor, plus a residual correlation of 0.20
# between x5 and x6 that the fitted model leaves out
lam <- c(0.8, 0.7, 0.6, 0.7, 0.6, 0.5)
S <- lam %o% lam
diag(S) <- 1
S[5, 6] <- S[6, 5] <- S[5, 6] + 0.20 * sqrt(0.64 * 0.75)
dimnames(S) <- list(paste0("x", 1:6), paste0("x", 1:6))

# Fit one factor as if the matrix came from N people
model <- "f =~ x1 + x2 + x3 + x4 + x5 + x6"
idx <- c("chisq", "df", "pvalue", "rmsea", "cfi", "srmr")
for (N in c(300, 3000)) {
  fit <- cfa(model, sample.cov = S, sample.nobs = N)
  print(round(fitMeasures(fit, idx), 3))
}
# lavaan's chisq is N times F_ML; likelihood = "wishart"
# in cfa() switches to N - 1

Uses lavaan. Install once with install.packages(c("lavaan")).

How the four fit indices behave