A Psychometric and Practical Comparison of Standard Moodle-Based and STACK-Based Step-by-Step Tests in University Calculus

Abstract

This paper compares two formats of online assessment in university Calculus: a standard Moodle-based step-by-step test and a STACK-based step-by-step test. Both tests assess integration by parts and divide the solution into consecutive response fields, but they differ in their validation mechanisms. The standard test relies on predefined scoring patterns, whereas the STACK-based test uses symbolic validation rules. The comparison is based on final manually verified scoring matrices from two student cohorts and follows a Classical Test Theory framework, including score distributions, reliability estimates, response-field-level indicators, and correlation-based measures. The results show high overall performance and ceiling effects in both formats. However, the STACK-based test required fewer manual corrections, showed higher internal consistency, and produced a more coherent relationship between solution steps and the total score.