Methodologies for Improving the Quality of AI Tutoring in K-12 Education
Tushar Udeshi, Anna Khazenzon, Kabir Khan, Nick Breen, RJ Corwin, Chris DiGiano, Kodi Weatherholtz, Marek Zaluski
Abstract
Many AI tutors leverage large language models (LLMs) today. Given that LLMs are opaque black boxes, robust evaluation and live experimentation to measure the impact of every change are essential. We pioneered AI-powered tutoring for K-12 with the launch of Khanmigo (Khan Academy, 2023). We describe the metrics we use to measure AI tutoring quality and student engagement as well as various experiments we have run. We highlight the changes that have moved our metrics, including models, prompting, personalization and agents.
Create a lesson
Related papers
Prepared Or Unprepared? Evaluating Healthcare Workforce Readiness for Clinical Adoption of Artificial Intelligence in Nigeria
Abbas M. Rabiu, Abdulrazaq A. Zubair, Um-mulkhairi Ibrahim et al.
Could Underwater Data Centers Pose a Risk to AI Treaty Verification?
James Teague, Ashmita Rajmohan, Yannick Muehlhaeuser
Control-Theoretic Content Moderation
Benedetta Tessa, Serena Tardelli, Marco Avvenuti et al.
"If I Had to Buy Just ONE: Galaxy S26 Ultra": Auditing AI-Generated Product Recommendations
Lucas G. Uberti-Bona Marin, Thales Bertaglia, Giovanni Astante et al.
Understanding AI Provider Recommendations in Local Service Markets
Hazem Ibrahim, Yasir Zaki
Toward a Time-Aware Assessment Framework for the Carbon Cost of AI-Enabled Decarbonization
Chenrui Xu, Burcu Akinci, Christopher McComb