Combining semantic and syntactic structure for language modeling
Rens Bod
Abstract
Structured language models for speech recognition have been shown to remedy the weaknesses of n-gram models. All current structured language models are, however, limited in that they do not take into account dependencies between non-headwords. We show that non-headword dependencies contribute to significantly improved word error rate, and that a data-oriented parsing model trained on semantically and syntactically annotated data can exploit these dependencies. This paper also contains the first DOP model trained by means of a maximum likelihood reestimation procedure, which solves some of the theoretical shortcomings of previous DOP models.
Create a lesson
Related papers
SpecCompact: Accelerating LLM Agent Context Compaction with Slot-Parallel Semantic Speculative Verification
Yulu An, Yifei Feng, Xiangyang Yin
Balancing hierarchical representation and quantization: A performance trade-off in text embeddings
Kai Ponel, Marco Polignano, Erasmo Purificato et al.
Persona Perspectives: Adopting Viewpoints Reorganizes Representations in LLMs
Austin C. Kozlowski, Avital Mintz, Madison Lee
Before You Continue: Evaluating Time-Sensitive Safety Awareness in Conversational Assistants
Peidi Dong, Yunze Xiao, Yu-Lin Tsai et al.
LooM: Loop-Invariant Memory for Causal Encoder-Decoder Language Models
Lele Yang, Muxi Diao, Zhonghao Yan et al.
SAUCE: Summary Analysis Using Conversation Entailment
Man-Ling Sung, Hemanth Kandula, Jeff Ma et al.