Concept map / Coefficient alpha and omega
Share
Coefficient alpha and omega
Compute coefficient alpha and omega total from a one-factor model and see when alpha understates or overstates reliability.
1Six items, one sitting, two coefficients
Classical test theory estimates reliability by correlating two parallel forms. Most tests are given once, so reliability has to come from the items themselves. Coefficient alpha uses only the item covariances. Omega uses a factor model, in which each item has a loading λ on one common factor and an error variance θ.
The lab builds a one-factor model with six items whose loadings run from 0.30 to 0.80. Alpha comes out at 0.72 and omega at 0.73. Change the items and see when the two agree, and which one moves.
With 6 unequally loading (congeneric) items, the common factor accounts for 73% of the total-score variance, so omega is 0.73. Alpha is 0.01 below omega, at 0.72. With uncorrelated errors alpha is a lower bound.
More controls
2Two items share a phrase: what happens to alpha?
Six items each load 0.70 on one factor with error variance 0.51, so alpha and omega are both 0.85. Items 1 and 2 are then reworded so that they share a phrase, and their errors now correlate 0.60. Nothing else changes. What happens to coefficient alpha?
3Two formulas from the same covariance matrix
The model gives item i a variance of λi2 + θi, and two items a covariance of λiλj, plus ψ12 for items 1 and 2 when their errors correlate. Alpha reads two numbers off that matrix: the sum of its diagonal and the sum of every entry, which is the variance of the total. Omega divides the factor's part of the total, (Σλ)², by the total. Point at a colored term to find it in the charts.
Alpha: 6/5 × (1 − 6.000 / 14.900) = 0.717.
Omega: Σλ = 3.300, so 10.890 / (10.890 + 4.010) = 0.731.
With unequal loadings and uncorrelated errors, alpha falls short of omega, here by 0.014. The wider the spread of the loadings, the bigger the gap.
Show alpha against test length
The curve holds the average inter-item correlation at 0.30 and adds items: kr̄ / (1 + (k − 1)r̄). It gives 0.72 at 6 items and 0.89 at 20, though the items hang together no better than before. It equals the alpha above when every item is identical.
4Check yourself
The tasks use the controls in section 1. The shortcut buttons are under More controls.
- With at least 4 items and no correlated error, make every loading equal (the error variances may differ). Alpha should now equal omega.Now: 6 items, loadings unequal, alpha 0.72, omega 0.73
- Make every item identical, with an average inter-item correlation below 0.30 and no correlated error, then add items until alpha reaches 0.80. Copy item to every item does the first part.Now: Different items, 6 of them, average r 0.30, alpha 0.72
- Correlate the errors of items 1 and 2 until alpha is at least 0.02 above omega.Now: Error correlation 0.00, alpha 0.72, omega 0.73
5Alpha with psych, omega with lavaan
The code first computes alpha and omega for the lab’s starting model, 0.717 and 0.731, from the implied covariance matrix and the loadings. Then it simulates 2,000 people from that model, estimates alpha with psych::alpha, fits a one-factor model with lavaan::cfa, and computes omega from the estimated loadings and error variances. The sample estimates, 0.738 and 0.752, sit close to the population values, with alpha again a little below omega.
Alpha and omega for a one-factor modelShow the code
library(psych)
library(lavaan)
set.seed(1)
# The lab's starting model: six congeneric items
lambda <- c(.8, .4, .7, .3, .6, .5)
theta <- 1 - lambda^2
# Population alpha from the implied covariance matrix
Sigma <- lambda %o% lambda + diag(theta)
k <- length(lambda)
k / (k - 1) * (1 - sum(diag(Sigma)) / sum(Sigma))
# Population omega total
sum(lambda)^2 / (sum(lambda)^2 + sum(theta))
# Simulate 2000 people from the model
n <- 2000
eta <- rnorm(n)
e <- sapply(theta, function(t) rnorm(n, 0, sqrt(t)))
x <- eta %o% lambda + e
colnames(x) <- paste0("x", 1:k)
alpha(x, warnings = FALSE)$total$raw_alpha
# Fit one factor and compute omega from its estimates
fit <- cfa("f =~ x1 + x2 + x3 + x4 + x5 + x6", data = x,
std.lv = TRUE)
est <- parameterEstimates(fit)
l <- est$est[est$op == "=~"]
th <- est$est[est$op == "~~" & est$lhs != "f"]
sum(l)^2 / (sum(l)^2 + sum(th))Uses psych, lavaan. Install once with install.packages(c("psych", "lavaan")).
When alpha and omega part ways
- Alpha compares the sum of the item variances with the variance of the total score. It needs no model, which makes it easy to compute and easy to misread.
- Omega total is (Σλ)² / ((Σλ)² + Σθ), the share of total-score variance that comes from the common factor. Correlated errors add twice each error covariance to the denominator.
- When every loading is the same (tau-equivalent items), alpha equals omega exactly, whatever the error variances. With unequal loadings alpha is lower, often by only a hundredth or two: 0.717 against 0.731 at the start of the lab.
- Alpha reads every covariance as true score. Two items that share wording can push alpha above omega, and then it is no longer a lower bound. A high alpha on a long test also says little about how strongly the items are related, because alpha rises with length alone.
- Omega is only as good as the one-factor model behind it. CFA model fit tests that model, and item information shows how precision changes across abilities instead of summing it up in one number.
Recommended reading
- Coefficient alpha and the internal structure of testsLee J. CronbachPaper · Psychometrika, 16(3), 1951
- Thanks coefficient alpha, we’ll take it from hereDaniel McNeishPaper · Psychological Methods, 23(3), 2018