Stochastic Constrained Test Assembly for AI-Enabled Assessment Systems

Abstract

Test assembly, the process of constructing a complete test form from an item pool subject to blueprint constraints, has traditionally been treated as a static optimization problem. In AI-enabled assessment environments, however, item pools evolve continuously as newly generated items enter with uncertain psychometric parameters, and delivery is on demand. These conditions make test assembly a sequential decision-making problem under uncertainty: which form should be deployed now, given current but incomplete knowledge of item quality, to simultaneously maximize measurement precision, satisfy content-blueprint constraints, maintain pool sustainability, and accelerate calibration of uncertain new items? This paper proposes the Stochastic Constrained Hybrid (SCH) framework as a principled answer to this question. SCH recasts form-level assembly as a multi-armed bandit (MAB) problem with Fisher information as the reward, extending recent item-level approaches in computerized adaptive testing (CAT) to the form-level setting. A simulation study comparing six test assembly methods is also presented. The main contribution of this paper is a framework for incorporating items with uncertain parameters into the automatic test assembly process for linear test forms.