Multimodal Learning Analytics Using Hierarchical Models for Analyzing Team Performance

This paper presents a comprehensive hierarchical model for teamwork by extending the well-known Affective, Behavioral, and Cognitive (ABC) approach for analyzing team per- formance. We develop a framework for interpreting individual and collective team processes, which allows us to create mappings from collections of actions to individual and team perfor- mance measures. The performance analysis algorithms use rich multimodal video, speech, and eye tracking data to monitor user activities in the teamwork scenario. We present an initial case study of soldiers training in small groups in a mixed reality training environment. Using data from the training scenarios, we demonstrate the use of multimodal learning analytics methods to analyze video data, infer soldiers’ individual and collective actions as a scenario evolves, and establish a proof-of-concept for our analysis methods by comparing against expert judgment. We conclude that our hierarchical ABC model combined with MMLA presents an effective approach to analyzing and evaluating team performance in training applications.

Vatral, C., et al. "Multimodal learning analytics using hierarchical models for analyzing team performance" Proceedings of the 15th International Conference on Computer-Supported Collaborative Learning. (2022): pp. 403-406. International Society of the Learning Sciences.
Download Paper