^{*}

^{a}

^{a}

^{b}

^{a}

^{c}

^{a}

^{c}

Since factor analysis is one of the most often used techniques in psychometrics, comparing or combining solutions from different factor analyses is often needed. Several measures to compare factors exist, one of the best known is Tucker’s congruence coefficient, which is enjoying newly found popularity thanks to the recent work of Lorenzo-Seva and ten Berge (2006), who established cut-off values for factor congruence. While this coefficient is in most cases very good in comparing factors in general, it also has some disadvantages, which can cause trouble when one needs to compare or combine many analyses. In this paper, we propose a modified Tucker’s congruence coefficient to address these issues.

Tucker’s congruence coefficient (TCC) was first proposed by Cyril Burt but then became popular based on Ledyard

The Tucker’s congruence coefficient (φ(

where _{i} and _{i} are the factor loadings of variable (item)

Putting aside the problem of the subjectivity of choosing a cut-off, there are several advantages of this measure, which are summarised by

φ(

φ(

φ(_{i}, _{i}), which reflects a change in the sign of variable

φ(_{i} and _{i}.

One problem with the TCC is that it is sensitive to changes in sign of individual loadings because, when only _{i}, _{i}) is predominantly the same since then the product is positive, hence the numerator’s value increases with nearly each term while in the case of a difference in signs the product is negative and the total decreases. Vice versa, the similarity is underestimated if the signs are predominantly different.

A second, more practical, problem arises if the survey on which the factor analysis is performed contains negatively framed items. Normally, these items are reversed before analysis. If one performs only one factor analysis without reversing these items, since the factor loadings are calculated based on the correlation/covariance matrix of all items, the factor loadings for these items will not change in magnitude, but the sign will reverse. This may or may not result in a TCC indicating incongruence erroneously and one needs to pay attention when interpreting such analyses. However, when such an analysis is compared to one where the negatively framed items were reversed before the analysis, the result is an extremely low TCC indicating incongruence while the modified coefficient will be high, indicating congruence. Similarly, an erroneously coded dataset where all items have been reversed may result in a TCC close to -1. Very negative TCCs can also be caused by high cross loadings in one of the factor analyses, for example when the analysis was performed on a small sample. Regardless of the chosen cut-off point, in these situations, the TCC incorrectly declares the factors to be incongruent.

The modification we propose, which solves both problems in a factor analytical context, is to use the absolute value of the products in the numerator:

This results in losing the nice geometric interpretation of the index but has several advantages. First, all advantages of the TCC are preserved: ψ(_{i} and _{i}. Obviously, ψ(

A TCC of -1 becomes 1, which is advantageous since in practice this may only occur in case of erroneous coding. Similarly, very low values (between -1 and -0.90) normally happen when two factors are similar (in interpretation) but many signs are reversed for one factor compared to the other. In such cases, the Tucker’s congruence coefficient would erroneously reject the possibility that the two are equal, while the modified coefficient results in a value above 0.90. This also may happen because of the many nil loadings mentioned earlier (

In this section, the original (TCC) and the modified (mTCC) congruence coefficients are compared in questionnaire data. The analyses were performed on the Big Five Inventory (BFI;

On each of these 1000 datasets, factor analysis with principal component extraction was performed and the results were rotated using direct oblimin (quartimin) rotation. It should be noted, that the choice of rotation method does not influence the results in this case (see

TCCs and mTCCs were calculated for each possible pair of factors (5×5) in all pairs of subsets and with the goal to find the five congruent factor pairs from the different analyses based on TCCs and mTCCs, which could be a step forward to combine factor analyses; after the matching factors are found based on congruence and factors are correctly ordered in the two (or more) analyses, the results of the different analyses could be merged. We assume that the highest congruence coefficient (CC) is when the factors are congruent. However, matching based on the TCC suffers from the previously described problem of not recognizing high negative loadings as potential matches, albeit in a very small amount of cases. For this reason, a scatterplot of all TCCs and mTCCs is presented in

The left hand side of

As it can be seen in

The overlap in

A small simulation study to show the advantage of the mTCC, when combining a high amount of FAs, was performed. Four settings were considered: 2 and 5 factors with 2 or 10 items each. Because the congruence coefficient is calculated directly from the factor loadings, instead of generating datasets, factor analyses were generated where the primary loadings were chosen arbitrarily (means and

Factor | Range | ||||
---|---|---|---|---|---|

F2I2 | F2I10 | F5I2 | F5I10 | ||

Factor 1 | 0.75 (0.07) | 0.68 (0.08) | 0.85 (0.07) | 0.68 (0.08) | 0.6 – 0.9 |

Factor 2 | 0.65 (0.07) | 0.58 (0.08) | 0.75 (0.07) | 0.59 (0.10) | 0.5 – 0.8 |

Factor 3 | - | - | 0.70 (0.00) | 0.55 (0.08) | 0.4 – 0.7 |

Factor 4 | - | - | 0.65 (0.07) | 0.51 (0.09) | 0.4 – 0.7 |

Factor 5 | - | - | 0.60 (0.00) | 0.49 (0.09) | 0.4 – 0.6 |

Similarly, when cross-loadings become higher (approaching or exceeding primary loadings in magnitude), none of the coefficients do really well (mismatches ranging from 7.5% in

The interesting part, from the point of view of the mTCC, is when cross-loadings are not nil or close to nil but are, in general, lower than the lowest primary loadings, in other words, when ostensible primary and cross-loadings are clearly distinguishable from each other, with a certain number of negative cross-loadings. Obviously, the number of negative cross-loadings increases with the rise of the

In this paper, we presented an alternative to the Tucker’s congruence coefficient using the absolute values of the products in the numerator. This method results in much higher values for incongruent factor pairs but for pairs with a TCC ≥ 0.95 the change is minimal. We suggest the cut-off value 0.95 (as established for TCCs by

Nice potential lies in the combined use of the two coefficients where we suggest two options: an “inclusive” and an “exclusive” one. The exclusive combination only accepts congruency if both TCC and mTCC indicate factor congruence while the inclusive option would indicate congruency if at least one of the two coefficients signal factor congruence. Since the mismatches come in pairs, and either TCC or mTCC are correct while the other is incorrect, the use of either of these combinations comes with the cost of lower sensitivity or specificity in matching the factors. The exclusive combination tries to avoid including false positives but at the same time may miss out on true positives. Meanwhile, the inclusive combination may gain true positives at the cost of the inclusion of false positives in an increased number. Either of these combinations could be useful depending on the context.

Another potentially useful candidate could be the Pearson correlation coefficient (PCC). While our case study does not endorse this coefficient, the number of mismatches being 99, 97 and 1, for the TCC, PCC and mTCC, respectively, both the toy examples and the simulations including the PCC in SM H show settings where the PCC can be advantageous. Which coefficient works best, depends on many components, and therefore it has merit to add this coefficient as a reference point, especially if the number of mismatches is high with the other coefficients. Both the PCC’s role in factor matching and identifying which coefficient is best for which setting should be explored more thoroughly in the future, but this investigation is out of scope of this paper.

As a reviewer pointed out, the mainstream approach to TCCs for factor matching, based on optimally reflected and ordered pattern matrices, should be considered. The main idea behind this approach is to rotate one factor analysis to a target matrix using a predefined criterion, for example by maximising the TCC (e.g.,

Because of the lack of clear geometric interpretation and well-established cut-off values, the new coefficient cannot simply replace the TCC. However, the mTCC may be more useful to combine factor analyses of a simple factor structure or to deal with a high amount of factor analyses where it is not feasible to assess each pair manually. In a small simulation study we have shown that with the increase of the cross-loadings both in range and magnitude (especially if some of the loadings are negative) the mTCC outperforms the TCC in correctly matching factors. A high amount of mismatches, or more mismatches for mTCC than TCC, may indicate the presence of too many (too high) cross-loadings compared to the primary loadings or even the lack of a simple factor structure.

Of course, even when factors are considered to be equivalent based on congruence this does not necessarily mean the interpretation of the factor is the same. However, while no congruence coefficient could possibly make sure that two factors have the same meaning, it allows to assess whether they are likely to measure the same concept. Also, our example shows that it is quite problematic to set cut-off values that are valid in every sitution. Therefore, TCCs and mTCCs should be interpreted and used with caution.

The authors have no funding to report.

The authors have declared that no competing interests exist.

The authors have no support to report.