TnT - A Statistical Part-of-Speech Tagger
Thorsten Brants
Abstract
Trigrams'n'Tags (TnT) is an efficient statistical part-of-speech tagger. Contrary to claims found elsewhere in the literature, we argue that a tagger based on Markov models performs at least as well as other current approaches, including the Maximum Entropy framework. A recent comparison has even shown that TnT performs significantly better for the tested corpora. We describe the basic model of TnT, the techniques used for smoothing and for handling unknown words. Furthermore, we present evaluations on two corpora.
Create a lesson
Related papers
Latent Compaction with Test-Time Training
Yingfa Chen, Xingyu Shen, Chaojun Xiao et al.
Manacá-1B-Instruct-2: Reproducible Safety Alignment for a Native Brazilian-Portuguese LLM. Negative DPO Results, a Controlled Composition Scan, and a Silent Failure Mode of the LLM Judge
Bruno Leonardo Santos Menezes, Carlos Leonardo Souza Cardoso, Fabio André Machado Porto
PerSo: Investigating Facets of LLMs' Social Intelligence in Real-World Persuasion Scenarios
Ali Zamani, Soroush Esfahanian, Erfan Shahabi et al.
PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesis
Qiushi Sun, Jingyang Gong, Yang Liu et al.
VASPO: Self-Distillation with Privileged Value Estimation
Fei Wu, Zhenrong Zhang, Jianshu Zhang et al.
ReaDy: Recurrent States as Dynamic Context for Speculative Drafting in Hybrid Linear Attention Large Language Models
Bohan Sun, Zunhai Su, Xialie Zhuang et al.