The MUPP-2PL model can be used in dichotomous FC blocks, where the probability of agreement with response option is modeled following a 2PL function. The model includes an invariance assumption, which states that item parameters remain constant regardless of the response format (i.e., FC vs. Likert) and the within-block context (which item is paired with another). Therefore, aj1 and aj2 in Equation (1) should be the same for 2PL items and in the FC block. The block characteristic function is:

where Yij=1 indicates that the respondent selected item j1 (the first item in the pair); θi represents a person’s position on each of the D latent traits measured by the FCQ; θj1˜i and θj2˜i are the coordinates of θi for items j1 and j2; aj1 and aj2 are the scale parameters; dj is the block intercept parameter; and bj1 and bj2are the location parameters. The sign of the scale (discrimination) parameter determines the item’s polarity. A positive scale parameter signifies a positively keyed item, where higher trait levels are associated with greater agreement. Conversely, a negative scale parameter indicates a negatively keyed item, where higher trait levels correspond to lower agreement. If both items have the same sign, either positive or negative, the block is considered homopolar or equally keyed; otherwise, it is called heteropolar or unequally keyed. In any case, it must be ensured that the resulting factor loading matrix is of full rank so that non-ipsative scores can be obtained.

One important consideration in the design of FCQs is the control of social desirability (SD). If SD matching (SDM) is not carefully considered during block construction, differences in item SDs are likely to emerge. This is especially true for heteropolar pairs. Such differences may compromise the validity of the questionnaire scores by making it easier for respondents to select socially desirable options instead of those that reflect their true traits. Matching items by their level of SD is a widely recommended strategy for reducing faking in FCQs (Cao & Drasgow, 2019; Pavlov et al., 2021).

One approach for matching items based on SD involves convening an expert committee to rate the desirability level of each item. The average desirability rating is then calculated for each item across all raters and items are matched by minimizing the absolute difference between items’ mean desirability values. Items are considered well-matched when the difference in SD falls below a predetermined cutoff. For homopolar blocks, a common cutoff is 0.50 on a 5-point scale, whereas for heteropolar blocks, the cutoff typically needs to be relaxed in order to ensure that a sufficient number of valid heteropolar pairs can be formed (Graña et al., 2025; Li et al., 2025).

The ability to obtain normative scoring from FCQ responses through IRT relies on the differential weighting of responses (Bürkner, 2022), which is driven by the scale parameters. Heteropolar blocks naturally lead to this differential weighting. However, this is a topic of debate in FCQ design since pairing them can be tricky and increase cognitive demand. On the one hand, studies (e.g., Brown & Maydeu-Olivares, 2011; Frick et al., 2023) suggest that including heteropolar blocks can enhance estimation accuracy and validity. On the other hand, recent empirical findings (Graña et al., 2025) question whether heteropolar blocks are actually necessary. These authors suggest that if FCQs are assembled with homopolar blocks that differ in scale parameters, heteropolar blocks are not necessary. This is an ongoing debate, with many authors suggesting that including approximately 15–40% heteropolar blocks can enhance the psychometric properties of FCQs, even if it means accepting a minor compromise in SD (Lee et al., 2022; Li et al., 2025). Thus, achieving faking resistance requires careful SDM, especially when heteropolar blocks are included.

Three factors were systematically manipulated in the simulation study. For clarity, we categorize these factors into two groups, block factors, which relate to the FCQ construction, and an item factor, which pertains to the construction of the item banks. The block factors are: 1) questionnaire length (40, 80), and 2) use of SDM (Yes, No). The item factor is: 3) degree of correlation between scale parameter, aj, and SDj of the positively keyed bank, which relates to the types of blocks formed (homopolar vs. heteropolar), and results in four levels (raj+−SDj+=0.20 with 25% heteropolar blocks, raj+−SDj+=0.20 with 0% heteropolar blocks, raj+−SDj+=0.50 with 0% heteropolar blocks, raj+−SDj+=0.80 with 0% heteropolar blocks). Hereafter, “+” indicates items that are positively keyed to the trait, and “-” indicates items that are negatively keyed. All factors are fully crossed, resulting in 16 conditions.

First, the questionnaire length factor determines how many pairwise blocks are formed. We established two levels, 80 and 40 blocks. Second, the SDM factor indicates whether item pairs were matched based on their SD ratings, with two levels, yes (matching applied) and no (matching not applied). When SDM is applied, the absolute difference between two items’ SDj ratings was calculated; if this difference exceeded a predefined cutoff, the items were not eligible for pairing. We established a cutoff of 0.5 for homopolar blocks to ensure a stricter level of matching. However, since it is more difficult to find heteropolar blocks that match in SD (Graña et al., 2025), the cutoff was relaxed to 0.75 for heteropolar blocks.

Since the third factor pertains to the generation of the item banks, we will describe the item bank generation process together. One five-dimensional bank of 320 SS items was generated for each condition and replication, as to imitate personality item pools, such as the International Personality Item Pool (IPIP; Goldberg, 1999). Three prototypical item banks were created. The main difference among them lies in the degree of association between the scale parameter, aj, and SDj of the positively keyed items. All banks were balanced with 64 items per trait; in the mixed keyed bank, each trait had 32 positive and negative items. Each simulation condition used a separate item bank, corresponding to one of the three bank types described below.

The item banks differed in the degree of correlation between aj and SDj, as well as in block polarity, resulting in four distinct categories (Table 1). Bank 1 was used for Categories 1 and 2 (and Levels 1 and 2 of the third simulation factor). It included both positively keyed items and negatively keyed items, with a small correlation between aj and SDj for both types (raj+−SDj+=raj−−SDj−=0.20) and a naturally high correlation across keys (raj−SDj=0.86). Using this bank, Category 1 formed a FCQ with 25% heteropolar blocks, while Category 2 included only homopolar blocks (0% heteropolar).

Bank 1, which reflects a more realistic scenario, is expected to pose less difficulty in assembling reliable tests due to the low correlation between aj and SDj. To explore more challenging conditions, we included two additional item banks. Levels 3 and 4 of the third simulation factor (and Categories 3 and 4) correspond to Banks 2 and 3, respectively; both consist of positively keyed items and therefore can form only homopolar blocks. Bank 2 was generated with a moderate correlation between the scale parameter, aj, and SDj (raj+−SDj+=0.50), while Bank 3 has a high correlation (raj+−SDj+=0.80), making optimal block matching more difficult. The SDj values were sampled this way to represent that, in real contexts, when traits are scored in the socially desirable direction (i.e., conscientiousness, emotional stability), positively keyed items tend to have higher desirability. The values in these distributions were primarily based on the empirical distributions of aj, dj, and SDj reported in the publicly available datasets from Johnson (2014) and Hughes et al. (2021), while also considering that positively keyed items tend to exhibit higher SD values than negatively keyed items (Graña et al., 2025; Li et al., 2025). Under our scoring convention, positively keyed items for each trait were defined as the socially desirable direction (e.g., for items measuring Neuroticism, a positively keyed item has a lower SD rating). These choices were made to represent realistic item parameters and SD behavior. This also includes the correlation between aj and dj (approximately 0.30 for both positively and negatively keyed items). The correlation between dj and SDj was kept at zero so that SDj would only be linked to aj, allowing us to analyze the impact of this variable.

The structure of the simulation is as follows. First, a SS item bank was generated for each condition and replication. Second, a FCQ was constructed with each algorithm using the SS parameters. Then, a binary FC response dataset of 5,000 respondents was simulated using the MUPP-2PL for each assembled FCQ. Finally, the MUPP-2PL was estimated, and trait recovery was assessed for each questionnaire. Trait estimates (θ^) were obtained as maximum a posteriori scores with the Metropolis-Hastings Robbins-Monro algorithm. These analyses were conducted using the mirt R package (Chalmers, 2012). This process was replicated 50 times. We controlled for the following content constraints: (1) block multidimensionality (i.e., each block had to include items measuring two different traits), and (2) trait balance across the selected blocks (i.e., each trait had to be represented by the same number of items). Item repetition was not allowed. During FCQ assembly, we verified that content and polarity constraints were met and that the distribution of blocks and items across traits remained generally balanced, allowing a fair comparison of the algorithms. Additionally, we conducted simulation checks to evaluate if any of the resulting scale parameter matrices after FCQ assembly were rank restricted by analyzing the least singular value of such matrices. Across all conditions, assembly algorithms, and replications, the least singular value was strictly greater than zero, indicating that none of the matrices were rank-deficient. We recorded the time spent in seconds for each assembly algorithm. In the case of BF, 100 questionnaires that met the specified constraints and SD requirements were randomly formed and the one with the highest reliability was selected. The R code used for the analysis and the empirical study can be found at Sorrel et al. (2026). The repository also includes a document detailing all algorithm specifications. All procedures were executed with a 2.50 GHz Intel Core i9-11900 CPU and 32 GB of RAM.

The GA is notably influenced by the questionnaire length, with longer questionnaires resulting in increased duration, as shown in Table S1 in the Supplementary Material (see Escudero et al., 2026). Assembling questionnaires of 80 and 40 blocks took an average of 4.90 and 3.21 minutes, respectively. In contrast, the other algorithms show minimal sensitivity to questionnaire length. Both SA methods are typically completed in a few seconds. However, SA has higher skewness and kurtosis, as we implemented an iterative process that reruns the algorithm until the target design is achieved. Across all conditions and replications, the constraint on the number of heteropolar blocks was always satisfied. In a small number of replications of the SA method, however, a few blocks did not meet the SDM constraint. Out of the 400 replications with SDM, there were 28 with 1 block affected and 2 with 2 blocks affected for the SA blueprint, and SA with Scale Parameter aj optimization showed 54 with 1 block affected and 1 with 2 blocks affected. The small number of affected blocks (1 or 2 out of 40 or 80) suggests that the impact is negligible. In any case, this implies a potential advantage in reliability and a disadvantage in ipsativity for these replications.

Table 2 presents the marginal results for the ρθ^θ2 and BiasΦ^ of the four assembly methods. GA consistently yields the highest reliability (M=.80), followed by SA with aj optimization (M=.78), BF (M=.76), and SA blueprint (M=.73). These differences indicate a large effect size (ηp2=.88). All simulation conditions are relevant, as seen in Figure 2. The most relevant factor was the questionnaire length, which can be expected, as longer questionnaires enhance reliability (ηp2=.98). Shorter tests show lower reliability overall, with the same relative patterns across correlations, heteropolar proportions, and algorithms. For instance, controlling SD at raj+−SDj+=.80 with 0% heteropolar blocks, GA’s reliability is .71 for Length 40, compared to .80 for Length 80. The next relevant factor was the degree of relation between the scale parameter, aj, and SDj in the item bank and having heteropolar blocks or not (ηp2=.95). The highest reliability is achieved with mixed banks forming questionnaires containing 25% heteropolar blocks; for example, for Length 80, controlling for SD at raj+−SDj+=.20, GA achieves .88 vs. .84 in the same condition with 0% heteropolar blocks. As the correlation between the scale parameter and SDj increases, reliability decreases. The same tendency can be seen in both test lengths. Specifically, the condition with the lowest reliability results is the positively keyed item bank with raj+−SDj+=.80 in the 40-block test length and it is where the most differences between algorithms are seen: GA = .71, SAaj = .68, BF = .65, and SAbp = .60. Although the use of SDM had the smallest effect size among the factors examined (ηp2=.53), it still had a meaningful impact on reliability, consistently leading to lower values, due to the constraints it imposes on item pairing. Specifically, taking GA as an example, reliability was consistently lower when incorporating the desirability constraint, decreasing on average by .01 points compared to the same condition without considering SD, and reaching a decrease of up to .04 points in cases where there is a strong relationship between the scale parameter and SDj. The same pattern was observed for the other procedures. Notable interactions were; algorithm with length (ηp2=.31), algorithm with D-H (ηp2=.54) and SDM with D-H (ηp2=.59). These interactions further prove that these factors are relevant in assembling FCQs, as the algorithm proves to be more important when the correlation between aj and SDj is higher and whether you form heteropolar blocks or not, and that this same factor interacts with SDM. In all cases, the ordering of the methods described above is preserved.

As for RMSEθ^θ, Figure 3 shows relevant differences across algorithms that align with the results found in ρθ^θ2. Generally, GA has lesser error, as it is closer to zero, and SA blueprint is the furthest method from zero. RMSE is lower for θ values near zero and increases at the extremes, as well as longer questionnaires reduce error, and greater differences among the algorithms are observed in the condition where raj+−SDj+=.80 in the 40-block questionnaire.

Regarding the ipsativity indicator (BiasΦ^), several patterns emerge from the absolute values in Table 2. As seen in Table S3 in the Supplementary Material (see Escudero et al., 2026), the ANOVA results follow the same pattern as the ρθ^θ2 results previously described. First, mixed banks with 25% heteropolar blocks produce negligible bias across all algorithms (BiasΦ^ ranges from .00 to .01). In contrast, bias increases as the correlation between the scale parameter, aj, and SDj grows and when only positively keyed items are used. For the most demanding condition (i.e., raj+−SDj+=.80, 0% heteropolar blocks, SDM, and Length 40), the bias reaches -.12 for GA, -.17 for SA with aj optimization, -.25 for BF, and -.32 for SA blueprint. Across all conditions, GA consistently shows the lowest ipsativity levels, followed by SA with aj optimization, BF, and SA blueprint. This ordering is also reflected in the grand means: -.04 (GA), -.07 (SAaj), -.10 (BF), and -.15 (SAbp). These differences also have a large effect size (ηp2=.82). Thus, while all methods show increasing bias, as the conditions are more demanding, GA systematically yields the least distorted correlation matrices.

In conclusion, we recommend using the GA for assembling FCQs, as it consistently produces high-quality solutions by accounting for key psychometric properties. Its advantages are particularly evident in challenging scenarios such as short questionnaires, high correlation between aj and SDj, and the need to match items on SD. Although SDM may slightly reduce reliability due to its restrictive nature, it remains essential for minimizing response bias. As the forced-choice format continues to gain popularity over traditional SS formats, the use of optimization algorithms becomes increasingly important. These methods enable fast and reliable questionnaire assembly while accommodating additional constraints such as SD control and the inclusion of heteropolar blocks, making them an essential tool for advancing non-cognitive assessment.