Narrative Review of Emotional Expression Support in XR: Psychophysiology of Speech-to-Text Interfaces

Abstract

This narrative review examines recent advancements, limitations, and research gaps in integrating emotional expression into speech-to-text (STT) interfaces within extended reality (XR) environments. Drawing from 37 peer-reviewed studies published between 2020 and 2024, we synthesized literature across multiple domains, including affective computing, psychophysiology, captioning innovation, and immersive human-computer interaction. Thematic categories include communication enhancement technologies for Deaf and Hard of Hearing (DHH) users, emotive captioning strategies, visual and affective augmentation in AR/VR, speech emotion recognition, and the development of empathic systems. Despite the growing accessibility of real-time STT tools, such systems largely fail to convey affective nuance, limiting the richness of communication for DHH users and other caption consumers. This review highlights emerging approaches such as animated captions, emojilization, color-coded overlays, and avatar-based emotion visualization, but finds a persistent gap in real-time emotion-aware captioning within immersive XR contexts. We identify key research opportunities at the intersection of accessibility, XR, and emotional expression, and propose future directions for the development of affect-responsive, user-centered captioning interfaces.