Concept map / Coefficient alpha and omega

Share

Coefficient alpha and omega

Compute coefficient alpha and omega total from a one-factor model and see when alpha understates or overstates reliability.

1Six items, one sitting, two coefficients

Classical test theory estimates reliability by correlating two parallel forms. Most tests are given once, so reliability has to come from the items themselves. Coefficient alpha uses only the item covariances. Omega uses a factor model, in which each item has a loading λ on one common factor and an error variance θ.

The lab builds a one-factor model with six items whose loadings run from 0.30 to 0.80. Alpha comes out at 0.72 and omega at 0.73. Change the items and see when the two agree, and which one moves.

0.00.51.0123456Item varianceItem0.800.400.700.300.600.50
Common variance λ²Error variance θNumbers above the bars: loadings
Variance of the total score, 14.90ω 0.73α 0.7200.250.50.751Variance of the total score, 14.90ω 0.73α 0.7200.250.50.751
From the factor (Σλ)²Error Σθ

With 6 unequally loading (congeneric) items, the common factor accounts for 73% of the total-score variance, so omega is 0.73. Alpha is 0.01 below omega, at 0.72. With uncorrelated errors alpha is a lower bound.

More controls

2Two items share a phrase: what happens to alpha?

Six items each load 0.70 on one factor with error variance 0.51, so alpha and omega are both 0.85. Items 1 and 2 are then reworded so that they share a phrase, and their errors now correlate 0.60. Nothing else changes. What happens to coefficient alpha?

3Two formulas from the same covariance matrix

The model gives item i a variance of λi2 + θi, and two items a covariance of λiλj, plus ψ12 for items 1 and 2 when their errors correlate. Alpha reads two numbers off that matrix: the sum of its diagonal and the sum of every entry, which is the variance of the total. Omega divides the factor's part of the total, (Σλ)², by the total. Point at a colored term to find it in the charts.

α  =  kk−1(1−∑iσi2σX2)\alpha \;=\; \frac{k}{k-1}\left(1 - \frac{\htmlClass{term-item}{\textstyle\sum_i \sigma_i^2}}{\htmlClass{term-tot}{\sigma_X^2}}\right)
ω  =  (∑iλi)2(∑iλi)2+∑iθi+2ψ12\omega \;=\; \frac{\htmlClass{term-com}{\left(\sum_i \lambda_i\right)^2}}{\htmlClass{term-com}{\left(\sum_i \lambda_i\right)^2} + \htmlClass{term-err}{\sum_i \theta_i} + \htmlClass{term-psi}{2\psi_{12}}}

Alpha: 6/5 × (1 − 6.000 / 14.900) = 0.717.

Omega: Σλ = 3.300, so 10.890 / (10.890 + 4.010) = 0.731.

With unequal loadings and uncorrelated errors, alpha falls short of omega, here by 0.014. The wider the spread of the loadings, the bigger the gap.

Show alpha against test length0.000.250.500.751.00148121620Standardized alpha at average r = 0.30Number of items6 items: 0.72

The curve holds the average inter-item correlation at 0.30 and adds items: kr̄ / (1 + (k − 1)r̄). It gives 0.72 at 6 items and 0.89 at 20, though the items hang together no better than before. It equals the alpha above when every item is identical.

4Check yourself

The tasks use the controls in section 1. The shortcut buttons are under More controls.

  • With at least 4 items and no correlated error, make every loading equal (the error variances may differ). Alpha should now equal omega.Now: 6 items, loadings unequal, alpha 0.72, omega 0.73
  • Make every item identical, with an average inter-item correlation below 0.30 and no correlated error, then add items until alpha reaches 0.80. Copy item to every item does the first part.Now: Different items, 6 of them, average r 0.30, alpha 0.72
  • Correlate the errors of items 1 and 2 until alpha is at least 0.02 above omega.Now: Error correlation 0.00, alpha 0.72, omega 0.73

5Alpha with psych, omega with lavaan

The code first computes alpha and omega for the lab’s starting model, 0.717 and 0.731, from the implied covariance matrix and the loadings. Then it simulates 2,000 people from that model, estimates alpha with psych::alpha, fits a one-factor model with lavaan::cfa, and computes omega from the estimated loadings and error variances. The sample estimates, 0.738 and 0.752, sit close to the population values, with alpha again a little below omega.

Alpha and omega for a one-factor modelShow the code
library(psych)
library(lavaan)
set.seed(1)
# The lab's starting model: six congeneric items
lambda <- c(.8, .4, .7, .3, .6, .5)
theta <- 1 - lambda^2
# Population alpha from the implied covariance matrix
Sigma <- lambda %o% lambda + diag(theta)
k <- length(lambda)
k / (k - 1) * (1 - sum(diag(Sigma)) / sum(Sigma))
# Population omega total
sum(lambda)^2 / (sum(lambda)^2 + sum(theta))
# Simulate 2000 people from the model
n <- 2000
eta <- rnorm(n)
e <- sapply(theta, function(t) rnorm(n, 0, sqrt(t)))
x <- eta %o% lambda + e
colnames(x) <- paste0("x", 1:k)
alpha(x, warnings = FALSE)$total$raw_alpha
# Fit one factor and compute omega from its estimates
fit <- cfa("f =~ x1 + x2 + x3 + x4 + x5 + x6", data = x,
           std.lv = TRUE)
est <- parameterEstimates(fit)
l <- est$est[est$op == "=~"]
th <- est$est[est$op == "~~" & est$lhs != "f"]
sum(l)^2 / (sum(l)^2 + sum(th))

Uses psych, lavaan. Install once with install.packages(c("psych", "lavaan")).

When alpha and omega part ways