Experiments on predictability of word in context and information rate in natural language
Dmitrii Manin
Abstract
Based on data from a large-scale experiment with human subjects, we conclude that the logarithm of probability to guess a word in context (unpredictability) depends linearly on the word length. This result holds both for poetry and prose, even though with prose, the subjects don't know the length of the omitted word. We hypothesize that this effect reflects a tendency of natural language to have an even information rate.
Create a lesson
Related papers
Deterministic Identification over Additive Gaussian Channels
Jonathan E. W. Huffmann, Holger Boche
Knowledge Distillation Driven Semantic NOMA with GAN Refinement for 6G Robotic Vehicle Networks
Qifei Wang, Zhen Gao, Li Qiao et al.
Real-Time Reconstruction of Markov Sources over MPR Channels
Pansee S. Elessawy, Nikolaos Pappas
Minimum Rate For Partially Observable Linear System with Side Information: LQG Plant and Gaussian-Markov Source
Sijie Li, Hyeji Kim
Spectral Approximation and Ergodic-Capacity Convergence of HMIMO Channels under Spatial-Wavenumber Domain Mismatch
Hangsong Yan, Hong Yang, Shu Sun
Sharp Minimax Regret for Infinite-Memory Logistic Prediction
Vaneet Aggarwal