Feeling the Learning: Designing Affect-Aware AI Systems Through Multimodal Analytics and the Future of Emotionally Intelligent Systems
Learners do not simply think through complex educational experiences; they feel them. Yet, the social and emotional dimensions of learning remain among the most underutilized signals in AI-driven educational systems. Drawing on two decades of research examining cognitive, affective, metacognitive, and motivational (CAMM) processes, this keynote spans work across intelligent tutoring systems, serious games, and immersive learning environments. It then synthesizes key empirical findings that demonstrate how multimodal data streams, such as facial expression recognition, eye tracking, physiological sensors, and interaction logs, can detect, model, and respond to learners' affective states in real time. Rather than treating affect as a nuisance variable, this presentation positions emotion as central to AIED design. Findings from MetaTutor and similar systems reveal how co-occurring emotional states, like confusion-to-frustration transitions, interact with metacognitive accuracy and learning outcomes. These insights inform UX decisions, adaptive scaffolding strategies, and interface transparency. Finally, the keynote addresses critical challenges and presents a forward-looking vision for emotionally intelligent, human-centered learning systems.
Biography
Dr. Azevedo is a Pegasus Professor in the School of Modeling, Simulation, and Training at the University of Central Florida. He is also an affiliated faculty member in the Departments of Computer Science and Internal Medicine at the University of Central Florida and the lead scientist for the Learning Sciences Faculty Cluster Initiative. He received his PhD in Educational Psychology from McGill University and completed his postdoctoral training in Cognitive Psychology at Carnegie Mellon University. His main research area includes examining the role of cognitive, metacognitive, affective, and motivational self-regulatory processes during learning, reasoning, and problem solving with intelligent learning technologies such as intelligent tutoring systems, hypermedia, multimedia, simulations, serious games, immersive virtual learning environments, human digital twins, and simulated learners). He has published over 300 peer-reviewed papers, chapters, and refereed conference proceedings in the areas of educational, learning, cognitive, and computational sciences. He is a fellow of the American Psychological Association, the American Educational Research Association, and the recipient of the prestigious Early Faculty Career Award from the National Science Foundation. He was recently inducted into the Academy of Science, Engineering and Medicine of Florida.
