Cross-lingual keyword assignment
Ralf Steinberger
Abstract
This paper presents a language-independent approach to controlled vocabulary keyword assignment using the EUROVOC thesaurus. Due to the multilingual nature of EUROVOC, the keywords for a document written in one language can be displayed in all eleven official European Union languages. The mapping of documents written in different languages to the same multilingual thesaurus furthermore allows cross-language document comparison. The assignment of the controlled vocabulary thesaurus descriptors is achieved by applying a statistical method that uses a collection of manually indexed documents to identify, for each thesaurus descriptor, a large number of lemmas that are statistically associated to the descriptor. These associated words are then used during the assignment procedure to identify a ranked list of those EUROVOC terms that are most likely to be good keywords for a given document. The paper also describes the challenges of this task and discusses the achieved results of the fully functional prototype.
Create a lesson
Related papers
CritICL: Inference-Time Weak-to-Strong Generalization from Small Language Model Failure Modes
Yufan Wu, Yinghui He, Zhengyi Hu et al.
TTPO: Test-Time Policy Optimization
Aozhe Wang, Zhengxi Lu, Jianze Wang et al.
Stochastic Estimation of Transduced Language Models
Vésteinn Snæbjarnarson, Samuel Kiegeland, Manuel de Prada Corral et al.
Boosting LLM Exploration via Weak-Model Guidance in RLVR
Xingyu Shen, Huishuai Zhang, Peng Li et al.
Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms
Siye Wu, Kai Yang, Yuchen Cai et al.
How Language Models Organize and Structure Moral Knowledge
Orion Reblitz-Richardson