On difference in word frequencies in a symmetric Bernoulli process

Abstract

In a symmetric Bernoulli process, all binary strings, or ``words'' of the same length have the same long term frequency. However, between two such words, one may have a ``frequency advantage'' in the sense that in any long enough segment of the Bernoulli process, the probability that the word occurs more times than the other word is greater than the probability the other way around. To characterize the frequency advantage in the long run, the asymptotics of the difference between the two probabilities as the length of the segment of the Bernoulli process tends to infinity is derived.