The Basque task: did systems perform in the upperbound?
Eneko Agirre, Elena Garcia, Mikel Lersundi, David Martinez, Eli Pociello
Abstract
In this paper we describe the Senseval 2 Basque lexical-sample task. The task comprised 40 words (15 nouns, 15 verbs and 10 adjectives) selected from Euskal Hiztegia, the main Basque dictionary. Most examples were taken from the Egunkaria newspaper. The method used to hand-tag the examples produced low inter-tagger agreement (75%) before arbitration. The four competing systems attained results well above the most frequent baseline and the best system scored 75% precision at 100% coverage. The paper includes an analysis of the tagging procedure used, as well as the performance of the competing systems. In particular, we argue that inter-tagger agreement is not a real upperbound for the Basque WSD task.
Create a lesson
Related papers
ToolSpace: Scaling Tool-Centric Environment Synthesis for Long-Horizon Agents
Guanting Dong, Zhichao Sun, Xinyu Yang et al.
Whose Effect Is It? How Objectives Inflate Learned Subspace Interventions
Toygar Tanyel, Serkan Ayvaz
Who is the Pronoun Pro? Comparing LLMs and Humans at Number-Disambiguation of 'they'
Carlos Hartmann
Shuffled OPSD: When Semantic Consistency Hurts On-Policy Self-Distillation?
Qingyuan Zheng, Zhihao Dou
HERIT: Enabling Global Access to Hanja-to-English Historical Translation by Mitigating Temporal Bias
Dayeon Jeong, Kyeongpil Kang
Residual Attribute Leakage Is an Interface: Auditing Privacy Evaluation After Text De-identification
Tuan Thanh Duong, Joongmin Shin, Jeongho Yoon et al.