SAVE: Situated Assessment Using Virtual Environments of Science Inquiry and Content

We are investigating the use of immersive virtual environments (IVEs) for assessment that can more authentically reflect scientific complexity. Our team has developed an IVE, to assess middle school children’s understanding of both science content and process. Participants complete the modules by interacting with characters and objects in the IVE, collecting and analyzing clues, and using their existing understanding to draw inferences about embedded problems. Students can solve the problems in multiple ways, many of which are equally correct while others uncover misconceptions. We hypothesized that this type of contextualized assessment will yield new insights into student understanding.

In this session, we will discuss what we are learning about student understanding through their choices and actions in the assessment modules, including the methods we use to analyze those actions. Analysis is complicated by the fact that the embedded problems are open-ended, resulting in a large data set reflecting the many and varied actions students can take while completing the assessments. Our analysis has been evolving and becoming more sophisticated as we make sense of our data.

Currently, we are investigating whether patterns of behavior can be ‘chunked’ to indicate student understanding. This ongoing analysis has as an ultimate goal the creation of visualizations that would allow a teacher to review quickly a student’s overall performance. This addresses one of the key problems in the current push to have teachers use data driven decision-making (DDDM) to drive their instruction: the lack of time and understanding of the tools needed. It has become recognized that for teachers to use DDDM successfully, interpretation and visualization tools are necessary. Using discriminative pattern mining, we can identify levels of understanding in students with 67% accuracy. We are now focusing on improving our accuracy by deepening our analysis of the students for whom we were inaccurate.