Visualizing changes in student responses using consistency plots

Abstract

Traditional methods of reporting changes in student responses have focused on class-wide averages. Such models hide information about the switches in responses by individual students over the course of a semester. We extend unpublished work by Steven Kanim on "escalator diagrams" which show changes in student responses from correct to incorrect (and vice versa) while representing pre- and post-instruction results on questions. Our extension consists of "consistency plots" in which we represent three forms of data: method of solution and correctness of solution both before and after instruction. Our data are from an intermediate mechanics class, and come from (nearly) identical midterm and final examination questions.