Assessing System Agreement and Instance Difficulty in the Lexical Sample Tasks of Senseval-2
Ted Pedersen
Abstract
This paper presents a comparative evaluation among the systems that participated in the Spanish and English lexical sample tasks of Senseval-2. The focus is on pairwise comparisons among systems to assess the degree to which they agree, and on measuring the difficulty of the test instances included in these tasks.
Create a lesson
Related papers
RSI-Anything: Unlocking Recursive Self-Improvement with World Models
Taoyong Cui, Pinxian Zeng, Wanghan Xu et al.
When VLMs Destroy What They Know: Spatial Representations Fall Below Lexical Priors
Shubh Garg, Chen Shani
Structural Automata for Neural Semantic Parsing
Xiao Zhang
Task Context Shapes Component Roles in Language Models
Sovesh Mohapatra, Dani S Bassett
Can Language Models Learn When to Abstain? Policy Learning and Calibrated Answer Selection
Juan Terven, Diana-Margarita Córdova-Esparza, Ruben Gomez et al.
Error Rate is Not Enough: Costs of Augmentation in Low-Resource ASR
Katsumi Ibaraki, David Chiang