Redundancy Estimates for Word-Based Encoding of Sequences Produced by a Bernoulli Source

Abstract

The efficiency of a code is estimated by its redundancy $R$, while the complexity of a code is estimated by its average delay $\bar N$. In this work we construct word-based codes, for which $R \lesssim \bar N^{-5/3}$. Therefore, word-based codes can attain the same redundancy as block-codes while being much less complex. We also consider uniform on the output codes, the benefit of which is the lack of a running synchronization error. For such codes $\bar N^{-1} \lesssim R \lesssim \bar N^{-1}$, except for a case when all input symbols are equiprobable, when $R \leqslant \bar N^{-2}$ for infinitely many $\bar N$.