The exact power law and Pascal pyramid

Abstract

Let $\omega_0, \omega_1,\ldots, \omega_n$ be a full set of outcomes (letters, symbols) and let positive $p_i$, $i=0,\ldots,n$, be their probabilities ($\sum_{i=0}^n p_i=1$). Let us treat $\omega_0$ as a stop symbol; it can occur in sequences of symbols (we call them words) only once, at the very end. The probability of a word is defined as the product of probabilities of its letters. We consider the list of all possible words sorted in the non-increasing order of their probabilities. Let $p(r)$ be the probability of the $r$th word in this list. We prove that if at least one of ratios $\log p_i/\log p_j$, $i,j\in\{ 1,\ldots,n\}$, is irrational, then the limit $\lim_{r\to\infty} p(r)/r^{1/\gamma}$ exists and differs from zero; here $\gamma$ is the root of the equation $\sum_{i=1}^n p_i^\gamma=1$. Some weaker results were established earlier. We are first to write an explicit formula for this limit constant at the power function; it can be expressed (rather easily) in terms of the entropy of the distribution~$(p_1^\gamma,\ldots,p_n^\gamma)$.