Teachers often struggle with how to help learners interpret and argue from visual representations of scientific phenomena (e.g., data tables, graphs, diagrams). Each kind of representation has distinct affordances for how it helps organize information, and norms for how it should be used in particular contexts. Our project, Representational Reasoning Assistant: Building AI Models to Help Middle School Students Interpret Science Diagrams, aims to support teachers as they provide in the moment help to students who are using and interpreting representations. We will develop, implement, and test a novel GenerativeAI (GenAI) based tool, the Representational Reasoning Assistant or RRA, for helping learners to interpret representations in ways that align with and support classroom teachers’ learning goals. This project will be innovative in that it both narrows the interpretation based on teacher input, aims to offer feedback in a pedagogically supportive manner, and gives feedback based on a visual representation (rather than text). The proposed work aims to explore how generative AI models that understand the visual representations that support science learning can be leveraged to help students learn key science content, particularly how to make and support claims about scientific phenomena using visual representations.
Representational Reasoning Assistant: Building AI Models to Help Middle School Students Interpret Science Diagrams
Cindy Hmelo-Silver
Distinguished Professor; Associate Dean for Research and Development
Anne Ottenbreit-Leftwich
Professor; Associate Vice President, Learning Technologies
Luddy Professor of Computer Science; Director of Center for Machine Learning
Kelli Paul
Research Scientist
Ali Magzari
Graduate Student
Jie Sheng
Graduate Student
