E-TRIALS: Empowering Data-Driven Decisions to Enhance Computer-Based Learning Platforms

Abstract

Computer-based learning platforms (CBLPs) have become a common medium in schools, transforming how students learn and interact with educational content. However, researchers still lack adequate tools to address the diverse set of challenges that students face in these environments. In this paper, we introduce \textbf{Ed-Tech Research Infrastructure to Advance Learning Sciences (E-TRIALS)}, a free tool developed by ASSISTments to help researchers conduct randomized controlled trials in the realm of learning sciences. We describe its features, the types of experiments it supports, and how it can address critical research questions. We showcase E-TRIALS' capabilities through two real-world interventions. Finally, we evaluate the efficacy of interventions using three average treatment effect (ATE) estimators. Student's t-test, regression, and Leave-One-Out Potential outcomes (LOOP). The results demonstrate that the unbiased LOOP estimator can achieve greater precision by adjusting for baseline covariates compared to the Student's t test. Our work demonstrates the potential of E-TRIALS to advance research and contribute to the development of more effective, inclusive, and adaptive CBLP. The code used for this work is available at https://osf.io/xp6ch/.