Going Beyond the Problem Given: How Human Tutors Use Post-Solution Discussions to Support Transfer

Issue title: "Caring for the Learner" in honour of John Self

Article type: Research Article

Authors: Katz, Sandra | Allbritton, David | Connelly, John

Affiliations: Learning Research and Development Center, University of Pittsburgh, 3939 O'Hara Street, Pittsburgh, PA 15260 USA. {katz, Connelly}@pitt.edu | Department of Psychology, DePaul University, 2219 N. Kenmore Ave.,Chicago, IL 60614 USA. [email protected]

Abstract: Two studies investigated the role and effectiveness of post-solution, reflective dialogues in physics tutorials. The first study investigated the instructional roles of post-solution discussions, their relationship to problem-solving discussions, and features that predict learning. Seven tutors individually guided 15 students as they worked on problems in the Andes physics tutoring system. Tutors adapted the post-solution discussions to students? ability levels and their performance on the current problem. Qualitative analysis of the transcripts revealed several roles of the post-solution dialogues – most prominently, explaining conceptual knowledge and integrating this knowledge with strategic, problem-solving knowledge. The number of post-solution discussions students had with their tutor, the number of discussions that abstracted from the current problem, and the number of tutor-initiated discussions predicted transfer, as measured by pre-test to post-test gain score on problems similar to those solved in Andes. Several tutorial strategies that are distributed between problem solving and post-solution reflection were identified. A framework for describing distributed plans for reflection is proposed based on these analyses. The second study investigated whether reflection questions such as those asked by the tutors in the first study lead to better conceptual understanding and problem-solving ability, as measured by overall gain scores and gain scores on conceptual and quantitative questions. It also examined whether human tutor-provided feedback on students?responses – with its often multi-exchange, dialectic character – is more effective than a single, canned explanation. Forty-six students solved problems in Andes in one of three conditions: with no reflection questions after problem solving, with reflection questions discussed with human tutors, or with the same reflection questions followed by canned feedback (without a human tutor). Students learned more with reflection questions and feedback than without, but the canned feedback and human tutored conditions did not differ significantly. Hence, overall, these studies support the practice of implementing post-solution reflective activities in intelligent tutoring systems, but call into question the need for natural-language processing techniques to support these activities.

Journal: International Journal of Artificial Intelligence in Education, vol. 13, no. 1, pp. 79-116, 2003