The 4th Workshop on Nonverbal Cues for Human-Robot Cooperative Intelligence

banner_1
banner_2

About the Workshop

This workshop is dedicated to discussing computational methods for sensing and recognition of nonverbal cues and internal states in the wild to realize cooperative intelligence between humans and intelligent systems. We gather researchers from different expertise, yet having the common goal, motivation, and resolve to explore and tackle this delicate issue considering the practicality of industrial applications. We are calling for papers to discuss novel methods to realize human-robot cooperative intelligence by sensing and understanding humans’ behavior, internal states, and to generate empathetic interactions.

  • Human internal state inference, e.g., cognitive, emotional, intention models.
  • Recognition of nonverbal cues, e.g., gaze and attention, body language, para-language.
  • Multi-modal sensing fusion for scene perception.
  • Nonverbal behavior generation for robots/agents, e.g., gaze salience, gesture.
  • Synchronization of nonverbal and verbal behavior
  • Learning algorithms, e.g., cross-embodiment and cross-context learning, imitation learning.
  • Generative and adversarial algorithms to enhance human-robot interaction, e.g., LLMs, diffusion models, VLMs.
  • Empathetic interaction between humans and intelligent systems.
  • Robust sensing of facial and body key points.
  • Modeling of social dynamics, e.g., group harmony, engagement, and cohesion.
  • Personalization and trust modeling based on multimodal nonverbal cues.
  • Real-world applications of cooperative intelligence.

Keywords: "Human: Face, gaze, body, pose, gesture, movement, attention, cognitivestate, emotion state, intention, empathy, Environment: Object"

Secondary subject: "Human-Robot cooperative intelligence", "Nonverbal cues recognition from audiovisual", "Human internal state inference from multi-modality", "Vision applications and systems", "Human-Object interaction and scene understanding"

Sponsors

Organizers

News updates

Mar 24th Workshop webpage was launched.

Call for Papers

Submission Guidelines

We invite authors to submit unpublished papers (2-4 pages excluding references) to our workshop, to be presented at a workshop session upon acceptance. Submissions will undergo a peer-review process by the workshop's program committee and accepted papers will be invited to present their works at the workshop (see presentation format).

We are pleased to announce that award will be given to the best paper accepted by this workshop.

Important Dates

We are reviewing papers on a rolling basis as they are accepted. The submission deadline for the first batch of papers to be reviewed is August 15.

Submission Instructions

Please use the IEEE conferences paper format to write your manuscript. Please submit your paper electronically through the workshop's EasyChair submission system.

Presentation Format

Accepted papers should be presented in three-way presentation approach to foster active participation

Publication Format

Authors are recommended to archive their papers and inform workshop organizers once this procedure is completed. Accepted papers which have been archived will be hosted on the workshop webpage.
As with the previous IROS2024 workshop, extensions of the papers presented at this ICRA2025 workshop will be invited to submit to a special issue journal to-be-announced at a later date.

Program

We plan a half-day event for 4 hours, including talks by two invited speakers. For participants who could not attend in person, we will disseminate the papers and pre-recorded videos on our workshop page, which also consists of a comment section for Q&A.

Invited Speakers

We intend to have speakers from different ethnic backgrounds, countries, and career stages. Specifically, we confirmed the attendance of two speakers.

no image

Invited Talk I

Learning Quantifiable Cues in Social Human–Robot Interaction and Neuro-Symbolic Learning for HRI

Chung Hyuk Park, George Washington University, USA
Link to website: https://www.chunghyukpark.com/about-me.html/
Abstract

In this talk, I will discuss our efforts to identify and learn quantifiable, explainable cues from social interactions and behaviors. We believe this work has important implications for developing intelligent and beneficial human–robot interactions. I will also share our ongoing approaches to studying neurological signals in human–robot interaction and learning.

Biography

Dr. Chung Hyuk Park is a Professor in the Department of Biomedical Engineering, School of Engineering and Applied Science at the George Washington University. As the director of the Assistive Robotics and Tele-Medicine (ART-Med) Lab, his current research interests are: 1) Multimodal human-robot interaction (HRI) and robotic assistance for individuals with disabilities or special needs, 2) Robotic learning and humanized intelligence, 3) Tele-medical robotic assistance and AI-based reasoning for medical perception and decision-making, 4) Trustworthiness in Embodied AI, and 5) Physical AI and Neuro-Sybolic Learning. He is a recipient of an NSF CAREER grant and carried out a National Robotics Initiative (NRI) NIH R01 project as the lead-PI. His work was introduced in diverse media and events such as Voice of America, Robot Trends, USA Today, Arirang TV, and TEDx Pearl Street. He received his PhD degree in Electrical and Computer Engineering from the Georgia Institute of Technology (2012) and MS degree in Electrical Engineering and Computer Science and BS degree in Electrical and Computer Engineering from Seoul National University (2002, 2000, respectively).

no image

Invited Talk II

Building Adaptive Spatial Computing Systems

David Lindlbauer, Carnegie Mellon University
Link to website: https://hcii.cmu.edu/people/david-lindlbauer/
Abstract

Spatial computing systems such as Extended Reality (XR) will transform the way we interact with digital information, and promise a rich set of applications, ranging from productivity and architecture to interaction with smart devices and robotics. Current approaches to spatial computing, however, are static, and users need to manually adjust factors such as what digital tools they need and when, every time they change task or environment. This is distracting and leads to information overload. To overcome these challenges, we aim to understand and predict how users perceive and interact with digital information, and use this information in context-aware systems that automatically adapt when, where and how to present virtual elements. In this talk, I will present computational and multimodal approaches that leverage contextual knowledge about users, the environment and system capabilities to create advanced interactive experiences that blend the digital and physical world.

Biography

David Lindlbauer is an Assistant Professor at the Human-Computer Interaction Institute at Carnegie Mellon University where he leads the Augmented Perception Lab and co-directs the CMU Extended Reality Technology Center. His research focuses on creating and studying enabling technologies and computational approaches for adaptive user interfaces including Extended Reality (XR), to increase the usability of advanced interactive systems, with applications in casual interaction, productivity, health and robotics. He received his PhD from TU Berlin, advised by Prof. Marc Alexa, and was a postdoctoral researcher at ETH Zurich. He has published more than 50 scientific papers at premier venues in Human-Computer Interaction such as ACM CHI and ACM UIST and was awarded the ETH Postdoctoral Fellowship in 2018. He was awarded the ETH Postdoctoral Fellowship in 2018 and NSF CAREER Awards in 2026. His work has attracted media attention in outlets such as MIT Technology Review, Fast Company Design, and Shiropen Japan.

no image

Invited Talk III

Beyond Words: Towards Context-Aware and Adaptive Human–Robot Interaction

Fethiye Irmak Doğan, University of Cambridge
Link to website: https://www.irmakdogan.com/
Abstract

Effective human–robot interaction goes beyond understanding spoken natural language, particularly when robots need to adapt to individual users. People communicate their intentions, preferences, and expectations through a combination of verbal and nonverbal behaviour, while social and environmental context shapes how these signals should be interpreted and what responses are appropriate. The talk will present research on how robots can use these signals to better understand users, reason under uncertainty, and adapt their behaviour accordingly. Specifically, it will examine how ambiguity and conflicting cues can be handled, how human feedback and explanations can support adaptation, and how robots can generate socially appropriate behaviour for different people and situations. Finally, the talk will highlight open challenges in context-aware and adaptive human–robot interaction, including accounting for individual differences without amplifying bias and maintaining interpretable and robust robot decision-making.

Biography

Fethiye Irmak Dogan is a Postdoctoral Research Associate in the Affective Intelligence and Robotics Laboratory (AFAR) at the University of Cambridge, UK. She received her PhD in Computer Science, with a specialisation in Robotics, Perception and Learning, from KTH Royal Institute of Technology, Sweden, in 2023. During her doctoral research, she was a visiting scholar at the Georgia Institute of Technology, USA, from November 2021 to April 2022. Her research focuses on developing autonomous and explainable robots for human environments, with an emphasis on human–robot interaction, machine learning, deep learning, and explainability. Her work has received several recognitions, including the KROS Interdisciplinary Research Award in Social HRI at IEEE RO-MAN 2025, the Seal of Excellence from the European Commission, and she was a finalist for the Early Career Contribution Award in the Outstanding Women in Robotics & Automation Paper Awards at IEEE ICRA 2025. She currently serves as an Associate Editor for IEEE Robotics and Automation Letters (RA-L).

Flash talks

09:32 09:37
Enabling Effective Human-Robot Collaboration through Efficient On-Board Vision-Language Models Francesco La Carpia, Mauro Martini, Simone Angarano, Marcello Chiaberge
09:37 09:42
An offline approach to fNIRS-guided reinforcement learning for robot behavior Julia Santaniello, Madelaine Brower, Benson Jiang, Donatello Sassaroli, Chenyuan Zhang, Robert Jacob, Jivko Sinapov
09:42 09:47
A Scenario-Knowledge-Driven Pipeline for Just-in-Time Assistance Zhiyuan Li, Tatsunori Hara, and Jun Ota
09:47 09:52
RobotSTAR: Next-Scale Autoregressive Sign Language Motion Translation for Humanoid Robots Yujia Zeng, Chensheng Peng, Yuxin Chen, and Masayoshi Tomizuka
09:52 09:57
Social Intuition vs. Machine Reasoning: Anticipating Human-Robot Interaction from multiple modalities Raphael Lorenzo-Louis, Bertrand Luvison, Serena Ivaldi
09:57 10:02
Ask Before It Tells: Benchmark-to-Robot Body-Cue Transfer for a Question-First Bedside Robot Dongsik Yoon
10:02 10:07
Level-Aware AI Coaching for Human Skill Development Wei Wang, Enlin Gu, Antonio Loquercio, Haimin Hu, Rahul Mangharam
10:07 10:12
Toward Robust Assistive Robot Navigation with EEG Motor-Imagery Mohamed A. Hanafy, Ashutosh Prakash, Saykhun Yusufjonov, Madan M. Rayguru, Dan O. Popa
10:12 10:17
HINT-Blimp: Human INTent Inference from Multimodal Cues for Robotic Blimps Subhadeep Koley, Benjamin Greenberg, Yifei Simon Shao, Nadia Figueroa, David Saldana
10:17 10:22
A Probabilistic Shared Control Framework for SE(3) Grasping Pinhao Song, Zhongxi Li, Ze Fu, Federico Ulloa Rios, Renaud Detry

Motivation and Background

Humans can perceive social cues and the interaction context of another human to infer the internal states including cognitive and emotional states, empathy, and intention. This unique ability to infer internal states leads to effective social interaction between humans desirable in many intelligent systems such as collaborative and social robots, and humanmachine interaction systems. However, it is challenging for machines to perceive human states under noisy real-world settings, which are usually measured by noninvasive sensors. Recent works investigating the potential solutions for the estimation of human states under controlled conditions using facial features with the off-the-shelf camera by leveraging deep learning methods. This workshop aims to bring interdisciplinary researchers across computer vision, artificial intelligence, robotics, and human-computer interaction together to share current research achievements and discuss future research directions for human behavior and state understanding, and their potential application, especially in the wild environment. Specifically, we are interested in cognition-aware computing by integrating environment contexts and multi-modal nonverbal social cues not limited to gaze interaction, body language and para language. More importantly, we extend multi-modal human behavior research to infer the internal states of humans. This is a challenging problem yet important to realize effective interaction between humans and intelligent systems.

It is desirable for intelligent systems like robots, virtual agents, human-machine interfaces to collaborate and interact seamlessly with humans in the era of Industry 5.0, where intelligent systems must work alongside humans to perform a variety of tasks anywhere at home, factories, offices, transit, etc. The underlying technologies to achieve efficient and intelligent collaboration between humans and ubiquitous intelligent systems can be realized by cooperative intelligence, spanning interdisciplinary studies between robotics, AI, human-robot and -computer interaction, computer vision, cognitive science, etc.

One of the main considerations to achieve cooperative intelligence between humans and intelligent systems is to enable everyone and everything to know each other well, like how humans can trust or infer the implicit internal states like intention, emotion, and cognitive states of each other. The importance of empathy to facilitate human-robot interaction has been highlighted in previous studies . However, it is difficult for intelligent systems to estimate the internal states of humans because they are dependent on the complex social dynamics and environment contexts. This requires intelligent systems to be capable of sensing the multi-modal inputs, reasoning the underlying abstract knowledge, and generating the corresponding responses to collaborate and interact with humans.

There are many studies on estimating internal states of humans through measurements of wearables and non-invasive sensors, but it would be difficult to implement these solutions in the wild because of the additional sensors to be worn by humans. One promising solution is to use audiovisual data like nonverbal behavior cues consisting of gaze interaction, facial expression, body language and paralanguage to infer the internal states of humans. Researchers in cognitive and social psychology have long advocated that these nonverbal behaviors are subconsciously generated by humans and reflect the internal states of humans under different contexts. Some salient examples are the studies on emotion recognition using facial and body language in controlled environment. It remains an open question for intelligent systems to sense and recognize nonverbal cues and reason the rich underlying internal states of humans in the wild and noisy environments.

Organizers

Jouh Yeong Chew

Honda Research Institute Japan

jouhyeong.chew@jp.honda-ri.com
Andreas Bulling

University of Stuttgart, Germany

andreas.bulling@vis.uni-stuttgart.de
Iolanda Leite

KTH Royal Institute of Technology

iolanda@kth.se
Daisuke Kurabayashi

Tokyo Institute of Technology

kurabayashi.d.aa@m.titech.ac.jp
Eiichi Yoshida

Tokyo University of Science

eiichi.yoshida@rs.tus.ac.jp
Sarah Gillet

KTH Royal Institute of Technology

sgillet@kth.se
Siyu Tang

ETH Zürich

siyu.tang@inf.ethz.ch
Xucong Zhang

TU Delft

xucong.zhang@tudelft.nl