The following notation will be used: x_ij is the item score for person i on item j, S_i is the sum score for person i, ξ_i is the common factor value for person i, τ_j is the factor intercept for item j, λ_j is the common factor loading for item j, _sλ_j is the completely standardised common factor loading for item j, δ_ij is the residual (or the unique factor value, respectively) of person i on item j, θ_j is the variance of these residuals, κ is the common factor mean, ϕ is the common factor variance, Λ and τ are p×1 vectors of the common factor loadings and factor intercepts, respectively, and Θ is a p×p diagonal matrix of the residual variances. The subscripts R and F denote the reference and focal group (G), respectively. Although only the case with two groups is explicitly treated, the results can be generalised to any number of groups. The parameter values are assumed to be realistic, that is, 1`Λ > 0, 1`Θ1 > 0, and ϕ > 0.

Consider the standard linear one-factor model (e.g., Kaplan, 2000, p. 68):

Let us assume that the linear one-factor model holds and that the item scores can be treated as continuous variables (for a discussion on this topic, see e.g., Rhemtulla et al., 2012). Let us further assume that scalar invariance holds, i.e. Λ_R = Λ_F and τ_R = τ_F. Following standard practice, we set the mean value of the common factor in the reference group to zero, κ_R = 0.

The sum score on the test for person i can be decomposed as follows:

and its expected value can be written as

If the factor mean is set to 0, the mean test score is simply equal to the sum of intercepts. In matrix form, the one factor model can be written as

For further developments, it is useful to note that, in the case of a perfectly fitting one-factor model, the variance of the sum score can be decomposed into the explained and the unexplained part:

The covariance between the latent trait and the j-th item score is

and the covariance between the sum score and the latent trait is

Recall that the squared correlation between the common factor and the sum score is equivalent to McDonalds’s omega²2

The formulas for the coefficient omega (e.g., McDonald, 1999, p. 89) typically do not take into account the common factor variance, as they assume the use of loadings that are standardised for the latent variable (for instance, obtained by the option std.lv in lavaan (Rosseel, 2012)).

Using Equations 8 and 10, rξS2 can also be written as the standardised covariance:

The expected sum score of an individual, conditional on the common factor ³3

To avoid notational clutter, I omit the item indices in the remainder of the paper.

Alternatively, under the standard assumption of a linear relationship between the common factor and its indicators, we can express this expectation by the basic linear regression equation:

The regression slope can be expressed as the product of the correlation and the ratio of the standard deviations. Expressing the correlation as in Equation 12, and the sum score standard deviation as in Equation 8, we obtain:

The regression constant can then be expressed using the means and the slope. After expressing E(S) as in Equation 13 and the slope as in Equation 15, it becomes:

This illustrates the well-known implication of scalar invariance: if the loadings and intercepts are invariant across groups, individuals with the same value of the common factor will have the same expected test score regardless of the group membership. In our case, however, we are interested in the expectation of the common factor given the sum score. Therefore, we need to find the parameters of the equation

We can first express the slope in the same way as before:

Since the denominator is the same as in Equation 12, we can express b as

Accordingly, ω is equal to b∑λ. Using these results and Equation 6, and noting that E(ξ)≡κ, we express the constant as:

Note that, when using the conventional approach to identification, the part κ(1 – ω) equals 0 in the reference group.

Therefore, the expected common factor value for a person with a sum score S_i is:

Or, alternatively, expressing ω as in Equation 12:

Under scalar invariance, the value of the scaled test score (S_i – Στ)/Σλ = C in Equation 21 does not depend on the group membership. Setting the value of κ_R to zero, the difference between the expected common factor values for two persons who belong to different groups and have the same test score can be expressed as:

Equation 23 is the key result of this paper. It implies that scalar invariance is not sufficient for person comparison invariance: if either the common factor means or the coefficients omega differ across the groups, individuals with the same sum score but belonging to different groups will have different expected common factor values. The value of ΔE(ξ_i|S) can be used as a measure of the non-invariance effect size.⁴4

It might seem that the value obtained by Equation 23 depends on the choice of the reference group, since in general κF(1-ωF) ≠ κR(1-ωR). Appendix A of the Supplementary Materials (see Sočan, 2026) shows that the choice of the reference group is in fact irrelevant.

If the mean factor value is equal in both groups (and therefore κ_F = κ_R = 0), this point corresponds to the mean test score. Otherwise, it can be either above or below the mean test score, depending on the value and the sign of the difference between the factor means and between the coefficients omega.

Let us now consider a special case of structural invariance, where the common factor mean and variance are equal across groups, κ_R = κ_F and ϕ_R = ϕ_F. In this case, we are free to set their values to the standardised metric (κ = 0 and ϕ = 1), and the expectation of the common factor value (see Equation 22) then simplifies to:

The expected common factor value equals the weighted deviation of the sum score from the mean sum score (note that the mean score is now equal in both groups). The weight W is invariant across groups (W_R = W_F) if and only if, in addition to scalar invariance, the coefficient omega is invariant. In this particular case, it also follows that the sums of residual variances are equal across groups, because both the factor loadings and factor variances are invariant. We shall denote this condition as “omega invariance”. With invariant factor means and variances, the strict invariance is sufficient but not necessary for the omega invariance, because it is only the sum of the residual variances that matters.

It is also evident from Equation 25 that the value of the weight W is inversely related to the sum of the residual variances. Therefore, among two individuals with the same test score, the person from the population with the lower sum of residual variances (or, equivalently, higher value of coefficient omega) has the larger absolute expected value of the common factor. In other words, the interpretation of the test scores that does not take the group membership into account will be biased against the group in which the measurement is more reliable.

Let us now consider the case where the common factor means are equal across groups (κ_F = κ_R), but the common factor variances differ (ϕ_R ≠ ϕ_F). After simplifying Equation 22, the expected value of the common factor can now be expressed as:

In this case, the person comparison invariance holds if and only if the ratio ∑θϕ is invariant across groups, in addition to scalar invariance. Note that also in this case the weight W is invariant across groups if ω_R = ω_F. The omega invariance is therefore the necessary and sufficient condition for the person comparison invariance in this case as well. In case of non-invariant omegas, a person belonging to the group with the smaller value of ∑θϕ will have a larger absolute expected value of the common factor, compared to a person with the same test score belonging to the group with the larger value of the residual-to-factor variance ratio.

Finally, the across-group comparisons of individuals’ scores are not invariant if the means differ across groups: even if omega invariance holds, the expected values of the common factor for individuals with the same sum score will still differ for a constant value of κ_F(1 – ω_F), see Equation 23. In practice it is important that the effect size of the non-invariance is small if either the mean difference κ_F – κ_R (which in fact equals κ_F) is small, or the measurement reliability as reflected in coefficient omega is high, because in both cases the value of κ_F(1 – ω_F) will be close to zero.

Person comparison invariance can also be regarded as a special case of predictive invariance (Millsap, 1995, 1997, 2007), in which the common factor is treated as the dependent variable predicted by the test score. In Appendix B of the Supplementary Material (see Sočan, 2026) it is shown that Equation 18 and Equation 20 can also be derived from the more general results presented by Millsap (1997, 2007). Technically speaking, person comparison invariance is therefore a special case of predictive invariance. Millsap’s (1995) duality theorem states that either measurement or predictive invariance can hold empirically, but they hold simultaneously only under constrained conditions. This implies that person comparison invariance can hold even if scalar invariance does not. However, unlike the general case of predictive invariance, in which an external variable is predicted, it is much less likely for person comparison invariance to hold without scalar invariance, because the regression parameters in Equation 17 are functions of the factor parameters. As shown in Appendix B of Sočan (2026), person comparison invariance without scalar invariance requires invariant factor intercepts, and a specific dependence pattern among factor-model parameters. In particular, the ratios of error-to-factor variance (ϕ-1∑θ) in both groups need to be related through a linear transformation with coefficients depending on the factor loadings in both groups:

It is unlikely that this relationship would hold in practice, therefore the combination of perfect person comparison invariance and violated scalar invariance is unlikely to be observed in real data. Note also that under scalar invariance, where ∑λR=∑λF, person comparison invariance requires the equality of error-to-factor variance ratios across groups, as also follows from Equation 18 (see also Equation B7 in Appendix B of Sočan, 2026).

From the classical test theory viewpoint, the common factor is, under certain conditions, a linear transformation of the true score (Jöreskog, 1971). The difference between the expected true scores (T) for two individuals belonging to different groups and having the same sum score S_i can be expressed using the well-known Kelley’s formula (Lord & Novick, 1968, p. 65):

The similarity with Equation 23 should not be surprising due to the syntactic similarity between factor analysis and classical test theory. However, the generality of the classical test theory approach is limited because the true score is defined with the respect to the observed scores on a particular test (namely, as its expected value for an individual), so its range and scale are determined by the number of items and the response scale; on the other hand, the existence of a common factor is, at least in principle, independent of its indicators.

After scalar invariance has been established, person comparison invariance can be tested by separately testing the equality of both factor means and coefficients omega across groups. The first hypothesis can be tested using the Wald test that is routinely reported by SEM software, or by constraining the factor means across groups and then testing the difference in the model fit. Omega invariance, on the other hand, can be tested by defining the omegas as new model parameters in the scalar invariance model and then either constrain them to equality or (in case of two groups) compute a bootstrap confidence interval for the difference between the omegas. The advantage of the first approach is the possibility of combining both hypotheses: since the usual statistical practice is to constrain the slopes first and the intercepts second, I propose to test the model with equal omegas against the scalar invariance model in the first step, and to test the model with equal latent variable means against the omega invariance model in the second step. In addition to difference tests, I recommend using the plots as described below to assess the size of the bias; (differences between) fit indices do not seem to be very useful in this respect. R scripts for testing the person comparison invariance and visualization of the effects of non-invariance are available as Supplementary Materials (files PCI_LRtest.R, PCI_boot_omega.R, and PCI_graphs.R, see Sočan, 2026).

I present a re-analysis of the data collected by Kavčič et al. (2023). Among other things, they investigated the measurement invariance of the Connor-Davidson Resilience Scale (CD-RISC) across educational groups (higher vs. lower education). The scale consists of 10 items rated on a 5-point scale; the range of sum scores is from 0 to 40. The confirmatory factor analysis with the MLMV estimator showed a good fit to the scalar invariance model (robust RMSEA = .046, SRMR = .056, robust CFI = .962). The higher education group had a higher mean resilience (standardised factor mean of .342) and a lower coefficient omega (ω_HI = .788, ω_LO = .840) than the lower education group. Figure 1 shows the effect of these differences on the expected standardised factor values. The solid and dashed lines represent the expected common factor values for individuals with a given sum score belonging to different education groups. The shaded area shows the difference between these expected values, i.e., the bias in person comparisons. The lines intersect at a sum score of 32.8 (this can also be obtained from Equation 24), about one standard deviation above the mean test score in the whole sample. At this point, person comparisons would be invariant. In the range of sum scores below this point, comparisons based on sum scores are biased against individuals with higher education, and vice versa.

In Figure 2, the bias values (i.e., the differences between the expected values) are plotted. The shaded area shows where the absolute bias values are smaller than 0.2 SD. By analogy with the commonly accepted interpretation of Cohen’s d (see also Nye et al., 2019), this is tentatively considered a negligible bias. Using this criterion, we conclude that comparisons of sum scores across groups are notably biased only among participants with the lowest scores (12 or lower). The positive bias in this case means that an individual from the higher education group has a higher expected factor value than an individual from the lower education group with the same sum score.

Figure 3 illustrates the magnitude of bias (y-axis) across standardised test scores on the reference group metric (-2 ≤ z ≤ 2; the x-axis) under scalar invariance and different parameter combinations:

The panels correspond to the ω_R×κ_F combinations, and, within each panel, regression lines correspond to different values of Δω. The shaded region again marks absolute bias values smaller than 0.2 SD. In all situations shown, κ_F ≥ κ_R = 0, and ω_F ≤ ω_R. If the values were reversed, the basic shape would remain the same, with the lines mirrored. Figures C1–C3, showing the remaining combinations, appear in the Supplementary Materials (Appendix C); see Sočan (2026). The R script to reproduce and modify the plots is also provided there (file Figure_3.R).

In accordance with Equation 23, a positive value means that a member of the focal group has a higher expected factor value than a member of the reference group who has an equal observed test score. A positive value therefore indicates bias against the focal group.

When omega invariance holds (Δω = 0), the bias lines are horizontal because the bias equals κF(1-ω) (see Equation 23), being constant across all score levels. In our range of conditions, the bias never exceeds the threshold, but it would do so with sufficiently low omega values, or sufficiently large latent-mean differences.

When omega coefficients differ across groups, bias also depends on the test score. When |Δω| ≤ .05 and κ_F ≤ 0.2, the bias is consistently negligible. When κ_F = 0.5, bias is notable in the lower score range, especially when ω is lower or Δω is larger. The bias remains negligible for standardised scores that are either close to κ_F, or somewhat higher.

Two general rules can be inferred from the figures:

The aim of this paper is to offer a new perspective on measurement invariance, focusing on comparisons between individuals rather than groups. The main conclusion is that the standard models of measurement invariance are not an optimal framework for assessing the comparability of individual scores: scalar invariance is neither necessary nor sufficient for person comparison invariance. Assuming a weakened form of scalar invariance (i.e., the sums of factor loadings and the sums of intercepts being invariant across groups), individuals with equal sum scores have equal expected factor values when the factor means are equal and omega coefficients are invariant across groups. In the case of invariant omega coefficients and non-invariant factor means, the effect of person comparison non-invariance is constant across the range of scores and depends on both the magnitude of the mean difference and the size of the omega coefficient. If the omega coefficients are not invariant, differences between their values also affect the size of the non-invariance effect. In this case, the non-invariance effect varies across the score range: it is generally higher at the extremes, and there is a point where the effect is zero (i.e., where the individual scores are comparable).

These results accord with several previous findings. Shealy and Stout (1993) noted that conditioning on observed scores distorts the estimation of bias in a studied subtest when true score means differ across groups. It is also well known that latent-trait estimates may be biased, particularly at the extremes (Feuerstahler, 2018). Finally, Meredith’s (1993) warning concerning the effect of non-invariant residual variances on person comparability was confirmed.

As the effect-size measure of bias, I propose the difference between the expected values of the measured trait (possibly standardised) for two individuals who share the same sum score but belong to different groups (see Equation 23). The calculation is easy to perform, because it requires only parameters routinely reported by SEM software (factor means, loadings, and intercepts) and the omega coefficient, which has become a standard psychometric quality indicator for tests based on factor analysis.

Two limitations of the proposed approach should be noted. First, it relies on the standard factor analysis model with homoscedastic errors, assuming linear relationships between the common factor and the item scores, and treating the item scores as numeric variables. While this reflects a common practice in applied psychometrics, it can be somewhat inaccurate, especially for items with a smaller number of rating categories. Second, I assume that the residuals in the regression equations have a mean value of zero. Although the expected value of random measurement error is zero by definition, this may not be true for the so-called specific factors (see Millsap, 2011, p. 77). However, this limitation also applies to the standard testing of scalar invariance.

Mathematically speaking, person comparison invariance is a special case of Millsap’s (1995, 1997, 2007) predictive invariance. Therefore, it could hold even without scalar invariance, but this would require restrictions unlikely to hold in practice. The added value of this paper relative to the existing literature on predictive invariance is the treatment of the special case where the underlying latent factor is “predicted”, and the derivation of an explicit expression for bias and elucidation of its determinants and their interplay in this special case. I also provide estimated bias values for various likely combinations of parameters.

From the practical viewpoint, person comparison invariance can be conceptualised as an extension of the standard measurement invariance testing in situations where specific persons are to be compared. Even for such tests, scalar invariance should be examined as an initial step in order to identify items that function differently across subpopulations. Person comparison invariance is more difficult to achieve in practice than scalar invariance because a larger set of parameters is constrained and because it partially depends on factors beyond the researcher’s control, such as the means and variances of the common factor. At first glance, this may appear discouraging to test users. Fortunately, as the presented illustrations show, the size of the bias may be small or even negligible in practice, especially if the measurement reliability is high. Indeed, the bias would vanish if coefficients omega equalled 1, regardless of other parameter values. In contrast to some liberal recommendations on reliability that can be found in the literature, these findings underscore the importance of striving for a high omega coefficient in test construction. On the other hand, they point to inherent limitations of the ubiquitously used sum scores as indicators of the measured latent variables.