Research Project

Socially Aware AI for Teamwork Enhancement

Assessing embodied interactions in physical environments over time can reveal valuable insights into team dynamics. SWATE explores the creation of a Socially-Aware AI agent that enhances team training by providing real-time insights and actionable feedback from 3D articulated bodies and audio signals, as descriptors of teamwork dynamics.

Reference: 2023.18417.ICDT
Funding: Fundação para a Ciência e a Tecnologia
Institutions: INESC-ID · GAIPS
Duration: 2025 – 2028

Overview

Objectives

SWATE investigates the significant challenges in unpacking complex human behaviour in teamwork processes: how low-level data traces from commodity sensors can inform high-level constructs of teamwork, serving as evidence of the team's and each member's capabilities, a step towards technology that improves teamwork efficiency and efficacy. We outline the following scientific and technological challenges:

Identify markers of effective teamwork

Using continuous and unobtrusive capture from cameras placed in high-fidelity environments, SWATE identifies key indicators of effective teamwork that forecast team and individual outcomes, offering trainers and trainees insight into performance and the team's adaptive coordination. Combining video with speech signals lets us decipher high-level social activities and their underlying intentions from low-level data traces, including the temporal aspect of teamwork data, which is often overlooked.
Develop an AI with knowledge of team dynamics

SWATE develops new algorithms that conceptualize team performance in terms of interpersonal and cognitive processes: the former covering situation monitoring, communication, and assistance during task execution; the latter concerned with how a team builds a shared mental model. This is the foundation of the agent's mind: a mechanism that judges what the team knows, what it needs to do, and how it performs against the core components of teamwork.
Enhance team training with actionable insight

SWATE designs a collaborative interaction between the AI and the team trainer to augment the trainer's perception of social processes and coordinative adaptations, furnishing trainers with practical tools for designing complex training interventions, richer in-context feedback for trainees, and better-informed debrief sessions.
Validate the tool in clinical simulation training

In partnership with Hospital da Luz Learning Health, SWATE investigates the application of the developed tool in healthcare teams, using state-of-the-art clinical simulation facilities. These simulations offer a controlled yet realistic setting to practice everything from routine procedures to high-stress emergencies without risk to patients, prioritizing the fidelity needed for an authentic learning experience. The goal is to prepare healthcare professionals not only technically but for the psychological demands of practice, ultimately improving patient outcomes through better-prepared, more resilient teams.