Scaling Metric Measurement Invariance Models

Measurement invariance analysis has become a widely used tool in many areas of applied research, e.g., psychology, sociology, and organizational research, and has received a lot of attention in methodological research, e.g., Davidov et al. (2018), Greiff and Scherer (2018), Millsap (2011), and van de Schoot et al. (2015); see Leitgöb et al. (2023) for a recent overview of measurement invariance analysis in the social sciences. The goal of measurement invariance (MI) analysis is to investigate whether measurement properties of a latent variable’s indicators are the same across two or more groups. MI analysis has many facets, e.g., which method is the best when looking for invariant indicators. However, each of these methods requires the application of a certain scaling method. Without scaling, the respective models could not be uniquely estimated. The focus of this paper is on this specific requirement of each MI investigation, i.e. how to set scaling restrictions appropriately when investigating the invariance of factor loadings.

There are several levels of MI that are investigated consecutively (e.g., Brown, 2015, pp. 242–243). The lowest level is configural MI, where the factor structure is the same in all groups. At the next level, metric MI, the invariance condition states additionally that the loadings are equal across groups. The third level is scalar MI, which means that the intercepts are invariant across groups¹ . In this paper, we focus on the metric MI model and elaborate on the question of how to scale this model. In general, we examine whether the scaling method affects the discrepancy function, and in turn the resulting test statistic of a model. In particular, we examine whether the selection of a particular referent indicator or, more generally, the selection of certain scaling method affects the discrepancy function, and in turn the resulting test statistic, for the metric MI model.

The test statistics of both the configural and metric MI model are assumed to be ${\chi }^2$-distributed with $\mathit{df}$ degrees of freedom under the corresponding null hypothesis. The test statistic’s numeric value results from the estimation process, in which a discrepancy function F is minimized. According to the procedure in Vandenberg and Lance (2000, p. 56), testing for metric MI requires the following steps: In the first step, the configural model is tested. In the second step, the metric model is tested. If the test statistics indicate in both steps that the model fits the data, then a ${\chi }^2$-difference test is conducted in the third step. This last test evaluates whether the metric MI conditions hold. Thus, MI analysis draws on testing single models as well as on testing nested models.

In addition to the level of MI analysis, there is also a distinction between full and partial metric MI models. In the full MI model, all loadings are assumed to be invariant across groups, while in partial MI models, one or more loadings are free to vary between the groups.

As a metric MI model entails latent variables, it is a well known fact that scaling restrictions must be placed, otherwise the model would not be identified. In this paper, we use the term scaling restriction to refer to the fixation of a certain parameter to a given value imposed by the application of a certain scaling method and use the term invariance condition to refer to the imposed equality of the loadings in a metric MI model. This distinction follows Wu and Estabrook (2016) who introduced a similar distinction between scaling restriction and invariance conditions. Finally, we use the term constraints to refer to both, i.e., the combination of the invariance conditions and the scaling restriction in a model. The Fixed Marker ( $\mathrm{FM}$) scaling (cf. Kline, 2016, pp. 199–200) is the most commonly applied scaling method and it is the de facto standard in most structural equation software. According to Raykov et al. (2012, p. 968), the $\mathrm{FM}$ method restricts one of the estimated loadings of each latent variable to 1 in each group. The respective manifest variable is then called the referent indicator (RI). From that description, it is obvious that the $\mathrm{FM}$ scaling method has as many different variants as there are indicators for a latent variable. For instance, when there are 6 indicators, then each of these indicators’ estimated loadings can be restricted to 1. In this paper, if the first indicator is the RI, we use the abbreviation ${\mathrm{FM}}_1$. If the second indicator is the RI, we write ${\mathrm{FM}}_2$, and so on.² Usually, it is suggested that the indicator that best represents the latent variable should be the RI (e.g., Brown, 2015, pp. 242–243).

In the context of metric MI analysis, the use of the $\mathrm{FM}$ method has been critically discussed in textbooks.³ According to Brown (2015, p. 271; cf., Cheung and Rensvold, 1999, pp. 8–9), there may be difficulties in multiple group settings regarding the selection of an RI. For instance, Brown (2015) mentions that non-invariance of the RI may not be detected in a metric MI setting, because it is restricted to 1 in every group, and as a consequence thereof, there is the implicit assumption that the RI is invariant across groups. Similarly, Kline (2016, p. 405) argues that the selection of an RI is arbitrary and that the fixation of the RI to 1 over the groups is tantamount to an assumption of invariance of the RI. Furthermore, Kline argues that the RI is excluded from the test of metric MI, but the RI choice should not affect the overall model fit most of the time. Raykov et al. (2020) provide an example in which a metric MI model is estimated using an non-invariant RI. In a Monte Carlo simulation, the metric MI model failed to reveal the non-invariance of the RI by means of the test statistic most of the time. However, the authors did not consider choosing an invariant RI.

Johnson et al. (2009, p. 654) found in a Monte Carlo simulation for the full metric MI model that when RIs are non-invariant, this non-invariance does not distort the results of metric MI tests. The authors state that differences between groups on the RIs were transferred to other indicators via the constraints on the RI to be equal to 1 in both groups, which in turn affected the estimated loadings of all other indicators by setting a scale. Furthermore, the authors summarize that, in general, the tests of the full metric MI model were accurate in detecting non-invariance when the RI differed across groups, regardless of whether or not other indicators also differed. Given these results, Johnson et al. (2009, p. 644) argue that all estimated loadings are rescaled relative to the RI, by a magnitude of $1/{\lambda }_{\mathrm{RI}}$ (where ${\lambda }_{\mathrm{RI}}$ denotes the “true” loading of the RI) and although the RI is non-invariant across groups, the parameter estimates will be adjusted by different scaling constants and the loadings are expressed in two different metrics (Johnson et al., 2009, p. 644). This kind of rescaling is also mentioned by Raykov et al. (2012, Appendix A). These authors state that setting the RI to 1 yields a rescaling of the remaining loadings.

The concerns and findings mentioned above were in the context of the full metric MI model. For partial metric MI, Johnson et al. (2009) found simulation evidence that the choice of the RI affects the test results.⁴ Vandenberg and Lance (2000, pp. 46–47) state that in partial metric MI settings, problems may arise when an RI is in reality non-invariant because that yields different metrics for a latent variable over the groups (cf., Brown, 2015).

Up to here, we have only considered the choice of the RI in the $\mathrm{FM}$ scaling method, but there are two other methods of scaling latent variables in multiple group settings with metric MI conditions (cf. Kline, 2016, pp. 199–200). These are the Effects Coding ( $\mathrm{EC}$) method and the Reference Group ( $\mathrm{RG}$) method.

The $\mathrm{EC}$ method (Little et al., 2006) restricts the estimated loadings by enforcing that their sum equals the number of indicators. This restriction is equivalent to the restriction that the average of the estimated loadings of a latent variable equals 1.

The $\mathrm{RG}$ method (cf., Kline, 2016, p. 393, Lee et al., 2011, p. 60, Raykov et al., 2012, p. 961, Wu & Estabrook, 2016), which is the multiple group version of the Fixed Factor ( $\mathrm{FF}$) scaling, restricts the variances of the latent variables to 1 in the reference group only, while being freely estimated in the remaining groups. As noted by Raykov et al. (2012, p. 969), the identification of this model is achieved by the imposed equality constraints of the estimated loadings in the remaining groups. Put differently, the metric MI conditions propagate the scaling to the other groups. This is in contrast to the FM method, in which an RI is set to 1 in each group (see above).

Obviously, due to the lack of an RI, neither $\mathrm{EC}$ nor $\mathrm{RG}$ contain any similar invariance assumption. Additionally, the $\mathrm{FM}$ scaling method only provides estimated loadings for all indicators but not the RI, whereas the $\mathrm{EC}$ or $\mathrm{RG}$ provide estimated loadings for all indicators. Indeed, van de Schoot et al. (2012, p. 489) recommend using the $\mathrm{RG}$ method when the goal is to compare factor loadings across groups.

In a nutshell, what was said above provides an inconsistent picture on the issues involved in scaling (partial) metric MI models. These inconsistencies can be summed up in the following four questions:

The fundamental topic of the first two questions is if there are any effects of the scaling method on the value of the discrepancy function F. All four questions are not only of interest for methodological considerations, but also for applied researchers who wish to conduct a MI analysis. This paper aims at clarifying these questions and thus aims to bring consistency into the literature concerning the choice of RIs in particular or the choice of a scaling method in general. The general approach of the paper is to present examples to empirically demonstrate some basic facts and ideas, and then to turn to formal considerations. However, the formal part is kept at the minimal required level.

In the following section, we provide the general setup and notations used in this paper. Then, we first present an example showing that neither the scaling method in general nor the choice of an RI in particular affects the discrepancy function of the full metric MI model. We also use the example to introduce the concept of change of scale, which we make use of in the following section to provide a formal proof of the equivalence of all scaling methods for the full metric MI model. Additionally, we derive some corollaries about the resulting test statistic and other fit indices, showing that these are also unaffected by the scaling methods. Furthermore, we also provide a corollary about the degrees of freedom for the ${\chi }^2$-difference test between the configural and the full metric MI model, which we then extend to partial metric MI models. As a consequence, we find that a partial metric MI model with only one invariance condition per latent variable is equivalent to the configural model. Thus, it is impossible to test the invariance of only one indicator (Steenkamp & Baumgartner, 1998). In the following section, we provide a further example, in which we firstly empirically demonstrate these corollaries. Again, the example illustrates that neither the scaling method in general nor the choice of an RI in particular affect the discrepancy function of the partial metric MI model. Afterwards, we provide a formal account for the partial MI model. In particular, we explain why the (non-)invariance of an RI, or more generally the data generating process, does not affect tests of either full or partial metric MI. Afterward, in the conclusion we elaborate on the results obtained in the paper and provide the answers to the four questions. Supplementary materials containing additional information and R scripts for the examples are available in Klopp and Klößner (2022) and referred to in the appropriate places in the text.

Setup

In this section, we introduce the notation we use throughout this article. Basically, we consider measurement models or confirmatory factor analysis models in multiple group settings of the form

Let $p$ and $m$ be the number of manifest and latent variables, resp., and let $g\in 1,\dots ,G$ be an index of the group membership. Then ${\mathit{X}}_g$ is a $p\times 1$ vector of manifest variables, ${\mathbf{\Lambda }}_g$ is a $p\times m$ loading matrix, ${\mathit{\xi }}_g$ is a $m\times 1$ vector of latent variables, and ${\mathit{\delta }}_g$ is a $p\times 1$ vector of measurement residuals. Additionally, ${\lambda }_{j,i,g}$ represents the loading of the $j$-th indicator on the $i$-th latent variable in group $g$. The other model parameters are indexed analogously.

Under the standard assumptions for confirmatory factor analysis models (e.g., Bollen, 1989, pp. 233–234), this measurement model yields the following model-implied $p\times p$ covariance matrix for ${\mathit{X}}_g$:

where ${\mathbf{\Phi }}_g$ is the $m\times m$ covariance matrix of the latent variables ${\mathit{\xi }}_g$ and ${\mathbf{\Theta }}_g$ is the $p\times p$ covariance matrix of the measurement residuals ${\mathit{\delta }}_g$, which is assumed to be a diagonal matrix containing the manifest residual variances. We collect the parameter matrices in the parameter vector $\theta =({\mathbf{\Lambda }}_1,\dots ,{\mathbf{\Lambda }}_G,{\mathbf{\Phi }}_1,\dots ,{\mathbf{\Phi }}_G,{\mathbf{\Theta }}_1,\dots ,{\mathbf{\Theta }}_G)$. To simplify our considerations, we assume a fully saturated model for the mean structure.

As is common in statistics, we distinguish between population models and estimated models (cf., Klopp & Klößner, 2021, pp. 185–187, Figure 4). In particular, we refer to a model in the form of Equation (1) as the population model that contains the data generating process (dgp). If the relation

holds within a dgp, then this dgp fulfills full metric invariance. In contrast, a dgp fulfills partial metric MI if

for all groups $g_1\ne g_2$ and for some combinations of indicator $j$ and latent variable $i$ for which the corresponding nonzero loadings are invariant across groups, while some loadings are allowed to be non-invariant. We call the dgp either a full or a partial metric MI dgp.

Following the above-mentioned distinction between population and estimated models, the estimated model is stipulated by a researcher. We assume that the estimated model is also a confirmatory factor analysis model of the same structure. The estimated model implied-covariance matrix is

The parameter vector $\hat{\theta }=({\hat{\mathbf{\Lambda }}}_1,\dots ,{\hat{\mathbf{\Lambda }}}_G,{\hat{\mathbf{\Phi }}}_1,\dots ,{\hat{\mathbf{\Phi }}}_G,{\hat{\mathbf{\Theta }}}_1,\dots ,{\hat{\mathbf{\Theta }}}_G)$ collects the model matrices of the estimated model.

The invariance condition for full metric MI on the estimated loadings is

For partial metric MI, these invariance conditions are replaced by the less restrictive invariance conditions

for all groups $g_1\ne g_2$ and all combinations of indicator $j$ and latent variable $i$ for which the corresponding estimated nonzero loadings are assumed to be invariant across groups.

In the following, we will always explicitly use either the term full metric MI model or partial metric MI model. If we want to make assertions that refer to both types of models, we use the term metric MI model. Moreover, we call a model such as in Equation (5) in combination with invariance conditions as in Equation (6) or as in Equation (7) the unscaled model, cf. Klopp and Klößner (2021, pp. 185–187), as no scaling restrictions have been added yet.

To actually estimate the model, we have to apply a scaling method. It is important to note that the restrictions of the various scaling methods are applied to the estimated model (Klopp & Klößner, 2021, p. 185), but not to the dgp. We collect the scaling methods in the set

To refer to any of the elements of $\mathcal{S}$, we use the generic notation S. The estimated model-implied covariance matrix then is

We call a model as in Equation (9) a scaled model. Please notice that the estimated manifest residual variances ${\hat{\mathbf{\Theta }}}_g$ in the scaled model do not have a superscript indicating the scaling method. This is because in order to estimate residual variances, no scaling method needs to be applied and estimated residual variances do not depend on the scaling method employed, i.e., the estimated values are identical for all scaling methods S (Klößner and Klopp, 2019, p. 148; Klopp and Klößner, 2021, p. 185).

In the following, when estimating a metric MI model, we assume that configural invariance is given. Furthermore, we assume that the scaled model is estimated by minimizing a discrepancy function $F({\mathit{S}}_g,{\hat{\mathbf{\Sigma }}}_g^S)$. The discrepancy function is based on a sample covariance matrix ${\mathit{S}}_g$ and the model implied matrix ${\hat{\mathbf{\Sigma }}}_g^S$. To keep the notation simple we write $F(\hat{\theta })$ (cf., Bollen, 1989, p. 106, for the properties of a discrepancy function).

As the distinction between a dgp as well as unscaled and scaled estimated models is not common in the literature, we want to explain the relations between these models with the help of Figure 1 (cf., Klopp & Klößner, 2021, p. 186). Firstly, in the upper left corner, there is a dgp that functions in a population and from which a random sample of observations is drawn. The observations are collected in a data set that is later used in the estimation process, which is depicted on the right side of the figure. Secondly, in the lower left corner, there is a researcher who has a substantial theory about the metric MI in the population and is interested in answering the substantial hypothesis “Does metric MI hold in the population?” Importantly, the dgp is unknown to the researcher. To answer the question, the researcher specifies a statistical model like the one given in Eequation (5), with metric MI conditions of the form given in equation (6) or equation (7), i.e., the researcher specifies an unscaled model. This represents the researcher’s statistical hypothesis. To actually estimate the model, the researcher has to apply scaling restrictions, i.e., the researcher creates a scaled model. This can be done with any one of the available scaling methods, of which the two possibilities $S_1$ and $S_2$ are depicted in the figure. With the scaled model, the statistical hypothesis of whether the metric MI model fits to the data is tested⁵ and afterward, the researcher arrives at a conclusion about the substantial hypothesis. The figure illustrates two issues: Firstly, the model used by the researcher is a distinct entity from the dgp. Secondly, the researcher starts with invariance conditions in the unscaled model, to which the scaling methods are then added. Thus, the scaling of the model is not related to the dgp. We will now move on to our first example.

Full Metric Measurement Invariance: An Example

In the following example, we present a simple two-group case which demonstrates that regardless of the scaling method, the same value of the discrepancy function results when a full metric MI model is estimated. The aim of this section is to provide an intuitive and heuristic understanding, both of the equivalence of the scaling methods for the full metric MI model and of the concept of change of scale. A change of scale allows to convert the estimates obtained under a certain scaling method to those obtained under any other scaling method, without re-estimating the model. This concept is vital to the more formal account in the next section.

We consider a model with only one latent variable with six indicators in two groups, $g_0=1$ indicates the reference group and $g=2$ indicates the focal group (cf., Holland & Thayer, 1988). The example’s dgp is given by (see also Figure 2 for a graphical display of the dgp):

This is a partial metric MI dgp. The loadings of the last two indicators are non-invariant. Especially, both loadings differ to a different extent. If the concerns about non-invariant RIs in the FM scaling methods were justified, then the estimated values of the discrepancy function (on which the test statistic rests) would differ.

We now take the role of the researcher from Figure 1 and set up our unscaled model as in Equation (5) and with the full metric invariance condition ${\hat{\mathbf{\Lambda }}}_{g_1}={\hat{\mathbf{\Lambda }}}_{g_2}$. In detail, this invariance condition entails the following set of invariance conditions:

To scale the model, we apply each scaling method from the set $\mathcal{S}$ and start with the $\mathrm{FM}$ scaling method, using the first indicator as RI⁶ , i.e., ${\mathrm{FM}}_1$. The ${\mathrm{FM}}_1$ scaling restriction is ${\hat{\lambda }}_{1,1,1}=1$. Considering the other indicators as RI, we can express all variants of the $\mathrm{FM}$ scaling method as

Notably, the invariance condition in Equations (11) yields ${\hat{\lambda }}_{j,1,2}=1$ and we see that the invariance condition propagates the $\mathrm{FM}$ scaling restriction from the reference to the focal group.

For the $\mathrm{EC}$ scaling method, the scaling restriction we set in the reference group is

As stated in the introduction, this scaling restriction is equivalent to the restriction that the average of the estimated loadings equals 1, i.e.,

Again, the invariance conditions cause the propagation of the scaling restriction to the focal group.

The most important observation from these scaling examples is that the scaling restrictions are applied in one group only. As we have seen, a common feature of all scaling methods is the propagation of the scaling restrictions into the focal group via the invariance conditions. Because of its importance, we want to set up this rule in colloquial terms, which is Apply the scaling restriction in one group only, the invariance conditions do the rest!

Finally, for the $\mathrm{RG}$ scaling method, the scaling restriction in the reference group is

In contrast to the previous scaling methods, this restriction does not directly propagate to the focal group. However, the restriction on the estimated variance in the reference group scales the estimated loadings in this group indirectly. Due to the invariance conditions, this restriction is then propagated to the focal group, thus overall acting as a scaling restriction.

We now estimate the full metric MI model for ten different generated samples, using all scaling methods, i.e., the six different variants of the FM scaling as well as the EC and RG scaling method. We use the ML estimator. The sample size per group is $N=300$, thus $N=600$ in total. We chose this sample size because it reflects a number often achieved in real studies. All calculations for the data generation and model estimation were done in R (R Core Team, 2022), using the packages MASS (Venables and Ripley, 2002) and lavaan (Rosseel, 2012).

Regarding the degrees of freedom, there are $p(p+1)/2$ unique entries in the covariance matrix in each group, i.e., with $p=6$ manifest variables there are $21$ unique entries in each group and thus $42$ unique pieces of information in total. In each group, we have to estimate 6 loadings, 6 manifest residual variances and 1 latent variance, i.e., 13 parameters per group and 26 parameters in total. Because we have a full metric MI model, there are 6 invariance conditions (see Equation 11). Additionally, there is 1 scaling restriction, regardless of the scaling method; see Equations (12), (13), and (15). Thus, there are 7 restrictions in total, which gives $26-7=19$ free parameters. This results in $df=42-19=23$ degrees of freedom.

The results are presented in Tables 1 and 2. Table 1 shows the values of the ML discrepancy function resulting from the estimation, multiplied by the factor ${10}^6$, while Table 2 shows the estimated loadings and latent variances for the first generated sample. In each sample, the value of the ML discrepancy function is the same regardless of the scaling methods. In summary, we can state as a first observation that neither the specific choice of an RI nor in general the choice of the scaling method affect the full metric MI model’s resulting value of the discrepancy function. Consequently, the test statistic and all fit indices derived from the test statistic are also identical.

This result is in line with Johnson et al. (2009) and demonstrates empirically that any concerns about truly non-invariant RIs are unwarranted, at least in this example. Considering the distinction between population and specified models explains this finding. The non-invariance is in the dgp, not in the specified, scaled model, in which an invariance condition is set by the researcher. This restriction represents an invariance assumption that applies to the estimates only, it is used to test if this restriction mirrors the invariance properties of the dgp.

Now, we turn to a second observation: Parameters estimated under one scaling method can be converted to those estimated under another scaling method (Klopp and Klößner, 2021, Proposition 2; Newsom, 2015, p. 4). We illustrate this conversion and convert the estimated parameters under ${\mathrm{FM}}_1$ scaling to those obtained under $\mathrm{RG}$ and EC scaling for the first out of the ten generated samples. The estimated parameters under ${\mathrm{FM}}_1$ scaling are:

and those estimated under $\mathrm{RG}$ scaling are:

Now, we choose $\sqrt{0.104}$ as constant, which is the square root of the estimated latent variance ${\hat{\varphi }}_{1,1,1}^{{\mathrm{FM}}_1}$, and apply the following conversion:

The choice of the constant exemplifies the mechanism of the conversion: It yields an “estimated” latent variance in the reference group of $1$, which corresponds to the restriction prescribed by the RG scaling method.

To convert from ${\mathrm{FM}}_1$ to $\mathrm{EC}$, we must bear in mind that the EC scaling restriction requires $\mathrm{A}({\hat{\lambda }}_{\bullet ,1}^{\mathrm{EC}})=1$. The loadings estimated under $\mathrm{EC}$ scaling are:

Denoting the average of the estimated loadings under the ${\mathrm{FM}}_1$ scaling as $\mathrm{A}({\hat{\lambda }}_{\bullet ,1}^{{\mathrm{FM}}_1})$, then choosing the constant $1/\mathrm{A}({\hat{\lambda }}_{\bullet ,1}^{{\mathrm{FM}}_1})=1.351$ gives the conversion:⁷

Klopp and Klößner (2021, Proposition 2) provide a theoretical account that demonstrates how to convert the estimated parameters obtained under any scaling method to the estimates obtained under any other scaling method, without the need to re-estimate the model, given that the estimated model-implied covariance matrices under the two different scaling methods are identical. To start, we note that the scaling method that was actually used has no effect on the conversion, only the “target” scaling method is of relevance (Klopp & Klößner, 2021, p. 190). For our further considerations, let $S_1$, $S_2\in \mathcal{S}$ be two different scaling methods. The model was actually estimated with $S_1$ scaling and $S_2$ is the target scaling method. Additionally, let $d\in \mathbb{R}\backslash \{0\}$ be a constant. For our one latent variable example, the conversion equations according to Klopp and Klößner (2021) are

Obviously, for our example with one latent variable, a conversion according to Equation (21) and Equation (22) with a constant $d_1$ does not change the estimated model-implied covariance matrix, because

The type of relation in Equation (23) is widely known in the SEM literature (e.g., Jöreskog, 1978; Mulaik, 2010, p. 443; Yoon and Millsap, 2007); Klopp and Klößner (2021, p. 185) call this relation change of scale. Note that this relation deserves special attention in the multiple group context. Because of the invariance conditions over the two groups, only one constant must be used simultaneously for both groups.

A general version of a change of scale for the model in Equation (9), which also easily accommodates more than one latent variable, would use non-singular diagonal matrices $\mathit{D}$ containing the $d_i$ factors for each latent variable. This general form is

In the following section, we turn over to the formal side of this exemplary consideration and provide a general proof that each scaling method results in the same value of the discrepancy function. As we will see, the idea of a change of scale and the idea of the propagation of the scaling restriction over the groups via the invariance conditions will be essential in proving the equivalence of the scaling methods.

Full Metric Measurement Invariance: Theory

The considerations laid out above lead to the following proposition.

The outline of the proof is as follows: First of all, we will explain why the minimum of the discrepancy function, taken over all parameters fulfilling full metric MI, coincides, for every scaling method, with the discrepancy function’s minimum taken over all parameters simultaneously fulfilling full metric MI as well as the restrictions stemming from the corresponding scaling method. After having established this fact, it will be obvious that the resulting optimal discrepancy value does not depend on the particular scaling method, as the optimal values for different scaling methods all take the same value (namely, the discrepancy function’s minimum taken over all parameters fulfilling full metric MI). As a by-product of proving the invariance of the optimal discrepancy value, the proof will also show that the estimated model-implied covariance matrices do not depend on the method used for scaling the MI model.

To follow the outline described above, we denote by $\hat{\theta }$ an arbitrary parameter which fulfills full metric MI and minimizes the discrepancy function when no scaling restrictions are imposed. By definition, $F(\hat{\theta })\le F(\theta )$ for all parameters $\theta$ which fulfill full metric MI, with F denoting the discrepancy function. For all scaling methods, this in particular implies $F(\hat{\theta })\le F({\hat{\theta }}^S)$, with ${\hat{\theta }}^S$ denoting the estimate for the parameters for a given scaling method S. We will now show that the reverse relation holds, too, i.e. $F(\hat{\theta })\ge F({\hat{\theta }}^S)$. When this relation has been established, it is clear that we have equality, $F(\hat{\theta })=F({\hat{\theta }}^S)$, implying that all scaling methods lead to the same value for discrepancy function, $F(\hat{\theta })$, for the full metric MI model. To this end, we show that, for each scaling method S, there exists a change of scale which transforms the parameter $\hat{\theta }$ to a new parameter ${\tilde{\theta }}^S$, such that the transformed parameter ${\tilde{\theta }}^S$ fulfills the constraints associated with the scaling method S as well as full metric MI.

In all three cases, the change of scale can be described by ${\stackrel{\mathbf{~}}{\mathbf{\Lambda }}}_g^S={\hat{\mathbf{\Lambda }}}_g\mathit{D}$ and ${\stackrel{\mathbf{~}}{\mathbf{\Phi }}}_{\mathit{g}}^{\mathit{S}}={\mathit{D}}^{\mathbf{-}\mathbf{1}}{\hat{\mathbf{\Phi }}}_{\mathit{g}}{\mathit{D}}^{\mathbf{-}\mathbf{1}}$ for all groups $g$, where $\mathit{D}$ is a diagonal matrix (with entries $d_{i,i}=\frac{1}{{\hat{\lambda }}_{R_i,i,1}}$ for FM, $d_{i,i}=\frac{1}{\mathrm{A}({\hat{\lambda }}_{\bullet ,i,1})}$ for EC, and $d_{i,i}=\sqrt{{\hat{\varphi }}_{i,i,g_0}}$ for RG scaling). From

it follows that, in all groups, the estimated model-implied covariance matrices are identical for ${\tilde{\theta }}^S$ and $\hat{\theta }$. As the discrepancy function depends on the parameters only through the estimated model-implied covariance matrices, we can conclude that $F(\hat{\theta })=F({\tilde{\theta }}^S)\ge F({\hat{\theta }}^S)$, as ${\hat{\theta }}^S$ minimizes the discrepancy function among all parameters which fulfill both full metric MI and the restrictions coming with the scaling method S, a set to which ${\tilde{\theta }}^S$ belongs.⁹ Additionally, Equation (25) shows that the estimated model-implied covariance matrices do not depend on the method used for scaling the full metric MI model.

This immediately follows from Proposition 1, as all these quantities are calculated using the full metric MI model’s likelihood value, which does not depend on the particular method used for scaling the model.

This immediately follows from the preceding corollary and the well-known fact that the corresponding quantities for the configural model do not depend on the particular method used for scaling the configural model, either.

Concerning the ${\chi }^2$-difference test comparing the configural to the full metric MI model, we would also like to shed some light on how to determine its degrees of freedom. Our explanations above show that scaling the full metric MI model consists of imposing exactly $m$ restrictions, for instance by fixing all latent variables’ variances to 1 in one group, as it is the case when scaling is done using the RG method. On the other hand, it is well-known that scaling the configural model consists of imposing exactly $Gm$ restrictions, namely one per group and latent variable. Thus, there is a difference of $Gm-m=(G-1)m$ between the numbers of restrictions needed to scale the configural model and the full metric MI model, respectively. This difference needs to be taken into account when calculating the degrees of freedom of the ${\chi }^2$-difference test. Intuitively, one would think that the degrees of freedom were determined by the number of restrictions due to the full metric MI model, i.e. by the number of conditions required to ensure all loadings’ invariance. This number is given by $(G-1)p$, as, for every indicator, the loadings in the first group must match the loadings in the remaining $G-1$ groups. However, this number has to be corrected to account for the difference induced by the scaling restrictions, i.e. by $(G-1)m$. These considerations yield the following corollary:

Corollary 3 can easily be adapted to partial metric MI models. In this case, the overall number of invariant indicators will be denoted by $p^{\ast }$. To calculate the degrees of freedom for the ${\chi }^2$ difference test comparing the partial metric MI model to the configural one, we can apply exactly the same approach as above: the number of restrictions due to the partial metric MI model, intuitively given by $(G-1)p^{\ast }$, has to be corrected to account for the difference induced by the scaling restrictions, i.e. by $(G-1)m$. This results in the following corollary:

The preceding corollary implies a special case which deserves particular attention: the case of $p^{\ast }=m$, which leads to the ${\chi }^2$-difference test having zero degrees of freedom. Regarding this case, we first we need to investigate when it appears: the condition $p^{\ast }=m$ means that the number of indicators required to have invariant loadings, $p^{\ast }$, equals the number of latent variables, $m$. This happens if exactly one indicator per latent variable is presumed to be invariant. Regarding the consequences, the fact that the ${\chi }^2$-difference test has zero degrees of freedom implies that these particular partial metric MI models have as many degrees of freedom as the configural model. Even more, as partial metric MI models are covariance-nested within the configural model (cf., Bentler & Bonett, 1980, p. 592f), all these partial metric MI models are indeed equivalent to the configural model. Together, this leads to the following corollary:

Partial Metric Measurement Invariance: An Example

Up to this point, we have mainly considered full metric MI models. In this section, building on the corollaries developed in the preceding section, we want to present a further example, in which we look at partial metric MI models. The example consists of three scenarios, A, B, and C. Each of thee scenarios is further divided into two settings. The dgp stays the same as in the previous example, please see the model given in Equation (10). As before, we use the ML discrepancy function and take the data from the first sample of the previous example’s simulation.

In the first scenario A, we want to look at the special case with an invariance condition on the estimated loading of only one indicator. In this case, there is $\mathrm{\Delta }\mathit{df}=0$ and the model under consideration is equivalent to the configural model, which has $d{f}_{\mathrm{configural}}=18$ degrees of freedom (please see the supporting information for how the degrees of freedom are calculated). Estimating the configural model¹⁰ results in a value of $19404\cdot {10}^{-6}$ for the discrepancy function.

In the first setting of scenario A, we examine the case in which the estimated loading of the first indicator, which is actually invariant according to the dgp, is assumed to be invariant by the researcher. Thus, the unscaled model is identical to Equation (5) with $g=1,2$ and with the metric invariance condition ${\hat{\lambda }}_{1,1,1}={\hat{\lambda }}_{1,1,2}$. To scale this model, we start with the $\mathrm{FM}$ scaling method, see Table 3 for details.¹¹ Once again, we consider every indicator as a potential RI, i.e., we look at ${\mathrm{FM}}_1,\dots ,{\mathrm{FM}}_6$. For the ${\mathrm{FM}}_1$ scaling, the invariance condition ${\hat{\lambda }}_{1,1,1}={\hat{\lambda }}_{1,1,2}$ yields that estimated loadings equal $1$ in both groups. For the ${\mathrm{FM}}_2$ to ${\mathrm{FM}}_6$ scaling, the invariance condition regarding the first loading propagates the scaling from the reference to the focal group. Concerning the $\mathrm{EC}$ method, a first variant is to constrain the sum of only invariant indicators’ estimated loadings, which is for instance the way the semTools package (Jorgensen et al., 2021) applies this scaling method in the context of partial metric MI models.¹² In this case, this leads to restricting the estimated loading of the first indicator to $1$ ( ${\mathrm{EC}}_1$). Thus, ${\mathrm{EC}}_1$ coincides with the ${\mathrm{FM}}_1$ scaling in this case. A second possibility is to constrain the sum of the estimated loadings of indicators which are not presumed to be invariant, such that their sum equals $5$ ( ${\mathrm{EC}}_2$). As a third possibility, we might also incorporate all estimated loadings, regardless of them being invariant or not, resulting in the condition that the sum of all estimated loadings equals $6$ ( ${\mathrm{EC}}_3$). Finally, there is the $\mathrm{RG}$ scaling method ( ${\mathrm{RG}}_1$).

The second setting of scenario A entails an invariance condition on the estimated loading of the fifth indicator, i.e., ${\hat{\lambda }}_{5,1,1}={\hat{\lambda }}_{5,1,2}$, which is non-invariant in the dgp. The scaling options are basically the same as in the first setting, they differ only with respect to which indicators the researcher deems invariant or non-invariant, see Table 3 for details.

Estimating the models produces the results shown in Table 4. The table shows that in both settings, the value of the discrepancy function multiplied by the factor ${10}^6$ is $19404$. Thus, estimation leads to the same value of the discrepancy function, although there are different invariance conditions, and these partial models are indeed equivalent to the configural model. The scenario shows exemplarily that a partial metric MI condition on only one estimated loading is not feasible. Scrutinizing the various scaling options in Table 3 provides insight into this issue. For example, the ${\mathrm{FM}}_1$ scaling in the first setting as well as the ${\mathrm{FM}}_5$ scaling in the second setting correspond to the respective $\mathrm{FM}$ scaling of the configural model, where the RI’s loading is set to $1$ in both groups. The reason is the propagation of the scaling via the invariance condition. Interestingly, this way of looking at the configural model is not new: Reise et al. (1993) developed a scaling method for the configural model, in which they set the variance of the latent variable to 1 in one group and required one loading to be invariant across groups. Their scaling method for the configural model is therefore identical to the $\mathrm{RG}$ scaling method for a partial metric MI model with only one invariant indicator. Steenkamp and Baumgartner (1998, p. 81) also pointed out that testing the invariance of only one loading is not meaningful. Following from our results, it is impossible to test the invariance of only one indicator because such a model will always correspond to the configural model.

In order to get around the problem arising in Scenario A, one might be tempted to ignore the rule of applying scaling restrictions in only one group and apply them in both groups. Doing so, there would be one degree of freedom for the ${\chi }^2$-difference test, and the partial metric MI model would no longer be equivalent to the configural model. In particular, such an approach would lead to a model with $\mathit{df}=19$ degrees of freedom, but at the cost of violating the rule to apply the scaling restriction in only one group.¹³

In the following Scenario B, we will study this situation. The unscaled models in the two settings are identical to those in Scenario A. The various scaling options are shown in Table 5. Please note that in the first setting, ${\mathrm{FM}}_1$ scaling coincides with the configural model, as restricting the first loading in each group is the same as imposing an invariance condition. An analogous assertion holds for ${\mathrm{FM}}_5$ scaling in the second setting. The resulting values for the ML discrepancy function are given in Table 6. Somewhat surprisingly, almost all scaling methods yield different values for the discrepancy function (for the exception $\mathrm{EC}$ scaling we will provide hints below). Thus, the model fit as well as test statistics for the ${\chi }^2$-difference test are idiosyncratic to each scaling method, a phenomenon called constraint interaction (Klößner & Klopp, 2019; Steiger, 2002). The reason is that enforcing scaling restrictions in both groups leads to the hypothesis about the indicators’ invariance becoming dependent on the scaling method employed. The findings of Johnson et al. (2009) mentioned in the introduction are also driven by this effect, which we will now explore in more depth.

To do so, let us have a look at the situation in Setting 1 and the use of ${\mathrm{FM}}_2$ to scale the model. In this model, the first indicator’s estimated loading is invariant due to the condition ${\hat{\lambda }}_{1,1,1}={\hat{\lambda }}_{1,1,2}$, while the second indicator’s loading indirectly becomes invariant due to the scaling restrictions ${\hat{\lambda }}_{2,1,1}=1$ and ${\hat{\lambda }}_{2,1,2}=1$, which obviously imply ${\hat{\lambda }}_{2,1,1}={\hat{\lambda }}_{2,1,2}$. Therefore, this model does not presume that only the first indicator’s loading is invariant, but it implicitly stipulates that the first and second indicators’ loadings are both invariant. Indeed, as the conditions ${\hat{\lambda }}_{2,1,1}=1$ and ${\hat{\lambda }}_{2,1,2}=1$ are equivalent to the conditions ${\hat{\lambda }}_{2,1,1}={\hat{\lambda }}_{2,1,2}$ and ${\hat{\lambda }}_{2,1,1}=1$, the ${\mathrm{FM}}_2$-scaled model in the first setting is actually equivalent to a partial metric MI model where the first two indicators’ loadings are presumed to be invariant and ${\mathrm{FM}}_2$ scaling is used in the first group only. Thus, the ${\mathrm{FM}}_2$-scaled model in the first setting is a model with $p^{\ast }=2$, not $p^{\ast }=1$, invariance conditions, investigating whether the first two indicators’ loadings are invariant across groups.

Scenario B demonstrates another aspect, too. There may be conditions under which identical values of the discrepancy function emerge, even though the model is not scaled according to the rule of setting scaling restrictions in one group only, as it was the case for the $\mathrm{EC}$ scaling variants in this scenario¹⁴ .

We now consider the last scenario, Scenario C, in which we want to showcase the scaling of a partial metric MI model with $p^{\ast }=4$ indicators presumed to be invariant across groups and with obeying the rule of applying scaling restrictions in only one group. Consequently, this partial metric MI model has ${\mathrm{df}}_{\mathrm{partial}}=21$ degrees of freedom. The first setting represents a situation in which the first four estimated loadings are investigated with respect to invariance across groups, i.e., the invariance conditions stipulated by the researcher correspond exactly to the invariant indicators in the dgp. Again, the unscaled model is as given in Equation (5) with $g=1,2$, and with the metric invariance conditions ${\hat{\lambda }}_{j,1,1}={\hat{\lambda }}_{j,1,2}$ for $j=1,2,3,4$.

For scaling, we use all possible versions of the $\mathrm{FM}$, $\mathrm{EC}$, and $\mathrm{RG}$ scaling methods. For the $\mathrm{FM}$ scaling method, these are six different versions, i.e., ${\mathrm{FM}}_1,\dots ,{\mathrm{FM}}_6$. For indicators with invariance conditions, we see that if we set the $\mathrm{FM}$ scaling restriction in the reference group, the invariance conditions automatically propagate the corresponding condition to the focal group. For instance, for the ${\mathrm{FM}}_1$ scaling, the restriction ${\hat{\lambda }}_{1,1,1}=1$ is propagated to the focal group via the invariance conditions ${\hat{\lambda }}_{1,1,1}={\hat{\lambda }}_{1,1,2}$. This reasoning applies for ${\mathrm{FM}}_2$, ${\mathrm{FM}}_3$, and ${\mathrm{FM}}_4$ scaling, too. For ${\mathrm{FM}}_5$ and ${\mathrm{FM}}_6$ scaling, where the reasoning given above no longer applies, the partial metric invariance conditions on the first four loadings nevertheless propagate the scaling from the reference to the focal group.

For the $\mathrm{EC}$ scaling method, there are again several ways to set the restriction. The first way ( ${\mathrm{EC}}_1$, see Table 7), consists in restricting the invariant loadings such that their sum equals $4$, while the second way leads to restricting the two loadings without invariance conditions ( ${\mathrm{EC}}_2$). Finally, it is also possible to apply the scaling restrictions such that the sum of all loadings equals $6$ ( ${\mathrm{EC}}_3)$. As above, these restrictions are applied in the first group only, following our rule that the scaling restrictions must not be applied in both groups.

Finally, for the $\mathrm{RG}$ scaling method, we set the variance of the latent variable in the reference group to 1. In contrast to the $\mathrm{FM}$ and $\mathrm{EC}$ scaling method, it is obvious for the $\mathrm{RG}$ scaling method that the scaling restriction is only applied in one group, as this feature is part of the description of this scaling method, as outlined in the introduction. The invariance conditions placed on the first four indicators certainly propagate the scaling to the focal group also for the $\mathrm{EC}$ and $\mathrm{RG}$ scaling.

In the second setting, there are four indicators again with invariance conditions on the estimated loadings. However, in this setting, the invariance conditions are placed on the estimated loadings of the first and second indicator, which are invariant in the dgp, and on the estimated loadings of the fifth and sixth indicator, which are non-invariant in the dgp. Thus, the metric invariance conditions are ${\hat{\lambda }}_{j,1,1}={\hat{\lambda }}_{j,1,2}$ for $j=1,2,5,6$. The scaling options are basically the same as in the first setting, they differ only with respect to which indicators the researcher deems invariant or non-invariant, see Table 7 for details. We would like to draw the reader’s attention to the fact that the scaling methods follow the researcher’s choices of invariant and non-invariant indicators, irrespective of the indicators’ invariance or non-invariance in the dgp, which is unknown to the researcher.

The results are presented in Table 8, which shows the values of the ML discrepancy function multiplied by the factor ${10}^6$. Obviously, the value of the discrepancy function, and therefore the test statistic and all other fit indices are identical, irrespective of the scaling method. Additionally, the example also demonstrates that the invariance or non-invariance of certain loadings in the dgp does not interact with the specific scaling method the researcher chooses for scaling the partial metric MI model.

In the following section, we will provide the formal proof that for partial metric MI models, the scaling method does not affect the value of the discrepancy function, as long as scaling restrictions are applied in one group only.