A Conceptual Architecture for Automatically Generating Multimodal Data Storytelling Interfaces for Supporting Team Learning

The different components of the architecture are described in this page. Also, the code used for the research team as a proof of concept of the architecture is presented all along in the different repos.

A conceptual view of the process

We use the aproximation presented in:

• Roberto Martinez-Maldonado, Vanessa Echeverria, Gloria Fernandez Nieto, and Simon Buckingham Shum. 2020. From Data to Insights: A Layered Storytelling Approach for Multimodal Learning Analytics. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ‘20). Association for Computing Machinery, New York, NY, USA, 1–15. DOI:https://doi.org/10.1145/3313831.3376148

A conceptual approach is introduced for providing meaning to multimodal learning data, collected from physical spaces (e.g. classroom). Such approach indicated that by using aspects such the learning design (e.g. critical actions that are required from students) or the teachers’ pedagogical intentions (e.g. timeliness of students in reacting to certain actions) it is possible to interrogate multimodal data to provice visual/narrative evidence of teamwork in f2f settings. Additionally, this approach focused on the communication of insights as multimodal learning analytic interfaces incoporating storytelling principles (e.g. explaining the purpose of a visual, highlight the more relevant aspect of data) to support teachers/students interpretations. The approach is visually described as follows:

Architecture to automatically generate MMLA data stories

Based on the previous approach the research team conceptualise and implemented an architecture that automatically generates such MMLA interfaces, which are also enhaced with the use of data storytelling principles. See the scope of the architecture in the following diagram: