Do Language Models Know Their Slang? Queer Slang Understanding in User-Generated Content
Arianna Denitto, Beatrice Savoldi
Abstract
Despite its cultural relevance and diffusion, queer slang remains underrepresented in Natural Language Processing research. Towards addressing this gap, we introduce Slang-Q, a manually curated dataset of naturally user-generated English sentences paired with queer slang terms and reference definitions, built upon a newly constructed taxonomy of 118 queer terms. We use this resource to conduct a first exploratory evaluation of language models on their ability to understand and define queer slang under varying prompting conditions. Slang-Q is intended as a basis for studying how current models handle sensitive, community-specific language and whether they can provide accurate and reliable information about such forms of identity and linguistic expression.
Create a lesson
Related papers
Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations
Leon Bergen, Usha Bhalla, Andrew Lee et al.
Reporting Practice Matters: The Impact of Reference Choice on Chest X-ray Report Evaluation
Daniel P. Jeong, Charles Q. Li, Hossein Hosseiny et al.
Long-Lived Characters, Local Inference: Incremental Memory Maintenance for Game NPCs
Zimu Xu
Beyond Outcomes: Dual-View Relational Learning for Efficient Agent Benchmarking
Xinshuai Guo, Junjie Wu, Dolly Deng et al.
How Much is a Human Right Worth? ECtHR-NPD: A Benchmark for Predicting Non-Pecuniary Damage Awards
Yanyi Pu, Damian A. Gonzalez-Salzberg, Zheng Yuan et al.
Structured Claim-Level Discourse Representations for Dense Health Narratives
Farnoushsadat Nilizadeh, Elham Pourabbas Vafa, Shirin Nilizadeh et al.