Methodology

A Weibull-Link Response Model for Measuring Unipolar-Skewed Constructs With Continuous Responses

2026-08-31T11:49:02+00:00

In certain noncognitive applications attempts are made to measure unipolar traits expected to have positively skewed distributions in the target populations, two conditions that clash head-on with the bipolarity and normality assumptions on which most standard IRT models are based. Unipolar-skewed models are an alternative to be used in these conditions, and we propose here a model intended for double-bounded continuous response items in which: (a) the link function or item response function is a cumulative Weibull curve, and (b) the initial latent distribution is lognormal. The model is simple, has desirable psychometric properties, and, conceptually, provides a plausible account of the response functioning. We propose a full development which has been also implemented in a free R program. Moreover, an empirical study is included, and its results show that the model behaves appropriately when used with real data, delivering the anticipated accuracy and external validity outcomes.

Nuances of Information Criteria for Bayesian Psychometric Models

2026-08-31T11:49:02+00:00

It is common practice to compare Bayesian psychometric models via information criteria such as DIC and WAIC. Especially because these criteria can be automatically computed by MCMC software, it is easy to ignore the intricacies related to their computation. This often leads researchers to use noisy criteria that may lead to suboptimal analysis decisions. In this paper, we first review different forms of Bayesian information criteria that could be computed for psychometric models. We then consider best practices, highlighting computational pitfalls that can occur even when one is attempting to follow best practices. Finally, we provide recommendations for the metrics’ practical uses. The paper is intended to clarify conflicting recommendations from the literature and to raise awareness about ways that information criteria can behave unexpectedly.

Moving From a Sketch to a Painting: Toward an Informative Mobility Effect

2026-08-31T11:49:03+00:00

This article develops an identification strategy for estimating the mobility effect in linear models by enriching the mobility term with theoretically grounded proxy measures that do not linearly covary with class origin and destination. It introduces three substantively informative metrics of social mobility: permeability, defined as the difference in frequency or probability mass between mobility positions and their corresponding nonmobile diagonals; atypicality, captured by the deviation between individuals’ predicted probabilities of occupying a given mobility position and those associated with the diagonal; and social distance, measured as the covariate-based Gower distance between mobility positions and their diagonal counterparts. By incorporating these proxies, the proposed framework enables simultaneous estimation of the net effects of origin, destination, and mobility, while relaxing restrictive functional constraints in conventional specifications. This reformulation effectively addresses previously underexplored research scenarios, including reference-group shifts, counterfactual decomposition, sociodemographic matching, and continuous measures of socioeconomic status. Two empirical examples are provided to demonstrate the analytical leverage and interpretive gains of the proposed approach.

Revisiting Equivalent Structural Models Through the Mediation Models’ Markov Equivalence Class

2026-08-31T11:49:03+00:00

Rational inquiry in psychology aims not just to describe phenomena but also to explain them. Researchers continue to use the classic trivariate mediation model as a key tool in this pursuit. The problem of equivalent structural models is revisited as a Markov equivalence class (MEC) (Andersson et al., 1997), illustrated using the trivariate mediation model. We highlight how the MEC complicates causal interpretation of the classic mediation model. To illustrate this, we examine two recent articles from a widely read psychology journal that apply the classic trivariate mediation model, one demonstrating strong causal control, the other weak causal control. We also introduce an R Shiny application that helps users understand this linkage and explore how causal assumptions shape the number of equivalent models in the MEC of a mediation model.

A Simulation-Based Comparison of Minimization, Rerandomization, and Anticlustering for Creating Experimental Conditions

2026-06-30T08:26:46+00:00

Anticlustering has been used as a novel method to assign subjects to conditions in experiments. Anticlustering can be applied when covariate measurements are available at the beginning of an experiment and minimizes differences in covariates between conditions. In a simulation study implementing a two-group between-subjects design, we compared anticlustering with established methods for minimizing covariate imbalance: rerandomization and minimization. Anticlustering most strongly reduced covariate imbalance, followed by rerandomization and minimization. Lower covariate imbalance increased the precision of the effect size estimate. The average statistical power of the unadjusted analysis (independent t-test) was not improved when using covariate-based assignment as compared to random assignment. However, with random assignment, the statistical power of the unadjusted analysis depended on observed covariate imbalance; with covariate-based assignment, the statistical power of the unadjusted analysis was less affected by covariate imbalance because imbalance was minimized. Statistical adjustment via regression was most important to maximize statistical power.

Beyond Scalar Invariance: Evaluating the Validity of Person-Level Score Comparisons

2026-06-30T08:26:45+00:00

Scalar invariance is widely regarded as essential for comparing test-score means across groups. However, it is less clear when test scores can be meaningfully compared at the individual level — specifically, whether individuals from different groups who share the same observed score have the same expected value of the latent trait. I show that scalar invariance alone is insufficient for meaningful person-level comparisons based on sum scores. In addition to scalar invariance, person comparison invariance requires equality of latent variable means and omega coefficients across groups. Nevertheless, non-invariance effects can be relatively small if the omega coefficients are high and similar in magnitude across groups. I relate person comparison invariance to predictive invariance and provide R code to test person comparison invariance and to visualise the effects of non-invariance.

How to Use Independent Validation in Python

2026-06-30T08:26:46+00:00

To statistically test whether two groups or models differ, classifier accuracy is compared. However, common accuracy estimates like cross-validation have unknown distributions, making them unsuitable for statistical inference. Alternatives like permutation tests or train-test splits are computationally expensive and limited to frequentist tests against chance. Independent Validation (IV) is a more flexible alternative providing a known estimate distribution. This enables both conventional hypothesis testing and Bayesian analysis of classifier performance. Although Python is most widely used for machine learning, a Python implementation of IV has been lacking so far. This article introduces such an implementation; beyond the core IV algorithm, the package allows to: (1) plot accuracy against training set size, (2) estimate the posterior distribution of the asymptotic accuracy, and (3) query the posterior for statistics and credible intervals. This makes it easy to apply IV when comparing accuracy posteriors across classes, datasets, or classifiers on the same data.

A Comparison of Optimization Algorithms for Forced-Choice Questionnaire Assembly

2026-06-30T08:26:46+00:00

Forced-choice questionnaires (FCQs) are increasingly favored over traditional Likert-type formats due to their reduced susceptibility to faking and social desirability (SD). Their construction typically involves pairing items from existing single-stimulus banks. This study compares four methods for assembling FCQs: a genetic algorithm (GA), two simulated annealing (SA) strategies (blueprint-based and scale-parameter-optimized), and brute-force (BF) random search. These methods are evaluated via simulation and an empirical example, focusing on trait score recovery. The effects of questionnaire length and SD matching on recovery are also examined. Three item banks varying in the aj-SDj relationship and inclusion of heteropolar blocks were used to assess performance across pairing scenarios. GA consistently produced the most reliable scores, followed by SA with aj optimization. All examined factors significantly affected reliability. GA is recommended for FCQ assembly, especially with short questionnaires, no heteropolar blocks, and high aj-SDj correlation.

Analyzing Group Differences and Measurement Fairness in Process Data: A Sequential Response Model With Covariates

2026-03-27T05:15:39+00:00

This article introduces the sequential response model with covariates (SRM-C) for analyzing process data, with emphasis on three key capabilities: detecting potential measurement bias in response processes, evaluating group differences in ability distributions and improving parameter estimation precision. The SRM-C combines measurement and structural components, with the measurement component modeling response sequences conditional on abilities and covariates, and the structural component characterizing group-specific ability distributions. Sparsity assumptions implemented through horseshoe prior distributions address identification issues within the Bayesian framework. Monte Carlo simulations demonstrated robust parameter recovery and effective differential item functioning (DIF) detection. An empirical analysis of PISA problem-solving data illustrated the model’s utility in distinguishing ability differences from potential measurement bias. The SRM-C offers a comprehensive framework for understanding group differences in process data while ensuring measurement fairness.

Controlling for Time-Varying Confounding in the Longitudinal Fixed-Effects Model: A Latent Variable Approach

2026-04-02T06:37:53+00:00

Fixed-effects regression models are commonly used in longitudinal studies as a means to estimate causal effects while controlling for unobserved time-invariant confounders. However, unobserved time-varying confounding remains potentially problematic, and identifying and measuring such confounders can be resource-intensive and costly. We propose the Time-Varying Confounding Structural Equation Model (TVC-SEM), a simple longitudinal model that builds on previous “common factor” models and which can serve as a robustness check for the assumption of no unobserved time-varying confounding in the fixed-effects approach. We posit a model with a latent autoregressive variable Zit, which represents the combined influence of both time-invariant and time-varying unobservables, and which is linked to the independent and dependent variables over time. Through Monte Carlo simulations and analyses of data from the Early Childhood Longitudinal Studies Kindergarten cohort (ECLS-K) and the Rural Substance Abuse and Violence Project (RSVP), we show that, under most conditions, TVC-SEM provides less biased estimates than several variants of the traditional fixed-effects model. Our proposed approach offers applied researchers a practical check for gauging the extent to which the fixed-effects assumption of no time-varying confounding may produce bias in the estimation of causal effects.