Design and analysis of experiments linking on-line drilling methods to improvements in knowledge

Abstract

An on-line drilling system, the tutor-web, has been developed and used for teaching mathematics and statistics. The system was used in a basic course in calculus including 182 students. The students were requested to answer quiz questions in the tutor-web and therefore monitored continuously during the semester. Data available are grades on a status exam conducted in the beginning of the course, a final grade and data gathered in the tutor-web system. A classification of the students is proposed using the data gathered in the system; a Good student should be able to solve a problem quickly and get it right, the "diligent" hard-working Learner may take longer to get the right answer, a guessing (Poor) student will not take long to get the wrong answer and the remaining (Unclassified) apparent non-learning students take long to get the wrong answer, resulting in a simple classification GLUP. The (Poor) students were found to show the least improvement, defined as the change in grade from the status to the final exams, while the Learners were found to improve the most. The results are used to demonstrate how further experiments are needed and can be designed as well as to indicate how a system needs to be further developed to accommodate such experiments.