Multilingual and Multimodal Abuse Detection
Rini Sharon, Heet Shah, Debdoot Mukherjee, Vikram Gupta
Abstract
The presence of abusive content on social media platforms is undesirable as it severely impedes healthy and safe social media interactions. While automatic abuse detection has been widely explored in textual domain, audio abuse detection still remains unexplored. In this paper, we attempt abuse detection in conversational audio from a multimodal perspective in a multilingual social media setting. Our key hypothesis is that along with the modelling of audio, incorporating discriminative information from other modalities can be highly beneficial for this task. Our proposed method, MADA, explicitly focuses on two modalities other than the audio itself, namely, the underlying emotions expressed in the abusive audio and the semantic information encapsulated in the corresponding textual form. Observations prove that MADA demonstrates gains over audio-only approaches on the ADIMA dataset. We test the proposed approach on 10 different languages and observe consistent gains in the range 0.6%-5.2% by leveraging multiple modalities. We also perform extensive ablation experiments for studying the contributions of every modality and observe the best results while leveraging all the modalities together. Additionally, we perform experiments to empirically confirm that there is a strong correlation between underlying emotions and abusive behaviour.
Create a lesson
Related papers
VoxMel TTS Technical Report
Wenhao Guan, Jiaxin Ye, Haitao Li et al.
Clinician Assessment of Laryngeal Disease with Auditory AI before Laryngoscopy: A Paired Reader Study
Siyuan Hou, Shuting Yang, Zengrui Jin et al.
ProTTS: End-to-End Prominence-aware Natural and Controllable Text-to-Speech via Automatic Word Stress Discovery
Sai Harshitha Aluru, Chiranjeevi Yarra
Rethinking Post-Training for Expressive Zero-Shot Text-to-Speech
Jingbin Hu, Xinfa Zhu, Hao Wang et al.
Leveraging Dual-Process Theory for Multimodal Multiparty Turn-Taking Prediction
Rutherford A. Patamia, Ming Liu, Wei Luo et al.
Pixel-KWS: Rendered-Glyph Representations for Orthographically Robust Open-Vocabulary Keyword Spotting
Adarsh Arigala, Ramesh Gundluru, Srinivasan Umesh et al.