Qualitative simulations as instruments for learning: An evaluation study

Qualitative Reasoning (QR) is a research area within Artificial Intelligence that focuses on means to articulate and communicate conceptual knowledge such as system structure, causality, the start and end of processes, the assumptions and conditions under which facts are true, qualitative distinct behaviours, etc. [3]. Cognitive science research has shown that when learners develop a causal model of a system's behaviour, they are better able to apply their knowledge to new situations [6]. QR models are a way to develop such causal models, because they explicitly capture the fundamental knowledge and elements of a system, while suppressing irrelevant details. In the last few years, tools have been developed that take a diagrammatic approach to having learners interact with QR software, such as Betty's Brain [1], Vmodel [4], and Garp3 [2]. Diagrammatic representations help reduce working memory load (cognitive offloading), allowing students to work through more complex problems. Such representations also help them in presenting their ideas to others for discussion and collaboration [5]. Particularly, the Garp3 workbench (http://www.garp3.org) allows users to investigate the logical consequences of their common sense ideas and use expert knowledge to improve their own understanding of phenomena. But how useful is the Garp3 software for education? We are particularly interested in the 'naturalness' of the workbench, i.e. the extent to which it blends in as a cognitive tool to empower the minds of learners. Does the vocabulary and diagrammatic visualization of the workbench communicate knowledge in such way that the subject matter is understandable to learners? And does the overhead of operating the many features of the software not prevent learners from actually learning the subject matter? This paper presents the results of a study to investigate the use of Garp3. As subject matter a model previously developed by experts about the United Nations' seventh Millennium Development Goal (MDG7) was used (http://www.un.org/millenniumgoals/). This model includes three of the indicators selected for monitoring this goal (25, 26 and 30). The results of the experiment turned out to be encouraging. Students in the treatment group could easily operate the software, that is: open the models, run them, and inspect the simulations. In fact, they did not ask for any help in doing so and they did not report interaction issues in the evaluation forms. The results support the idea that the interface of the Garp3 software communicates in a way that is intuitive (naturalness principle). Furthermore, the treatment was effective in creating a significant learning effect, as observed in comparisons between pre- and post-tests.