In data compression, a universal code for integers is a prefix code that maps the positive integers onto binary codewords, with the additional property that whatever the true probability distribution on integers, as long as the distribution is monotonic ≥ p, the expected lengths of the codewords are within a constant factor of the expected lengths that the optimal code for that probability distribution would have assigned. A universal code is asymptotically optimal if the ratio between actual and optimal expected lengths is bounded by a function of the information entropy of the code that, in addition to being bounded, approaches 1 as entropy approaches infinity. In general, most prefix codes for integers assign longer codewords to larger integers. Such a code can be used to efficiently communicate a message drawn from a set of possible messages, by simply ordering the set of messages by decreasing probability and then sending the index of the intended message. Universal codes are generally not used for precisely known probability distributions, and no universal code is known to be optimal for any distribution used in practice. A universal code should not be confused with universal source coding, in which the data compression method need not be a fixed prefix code and the ratio between actual and optimal expected lengths must approach one. However, note that an asymptotically optimal universal code can be used on independent identically-distributed sources, by using increasingly large blocks, as a method of universal source coding.
Universal and non-universal codes
These are some universal codes for integers; an asterisk indicates a code that can be trivially restated in lexicographical order, while a double dagger indicates a code that is asymptotically optimal:
Byte coding where a special bit pattern is used to mark the end of the code — for example, if an integer is encoded as a sequence of nibbles representing digits in base 15 instead of the more natural base 16, then the highest nibble value can be used to indicate the end of the integer.
Golomb coding, which has Rice coding and unary coding as special cases.
Their nonuniversality can be observed by noticing that, if any of these are used to code the Gauss–Kuzmin distribution or the Zeta distribution with parameter s=2, expected codeword length is infinite. For example, using unary coding on the Zeta distribution yields an expected length of On the other hand, using the universal Elias gamma coding for the Gauss–Kuzmin distribution results in an expected codeword length near entropy .
Relationship to practical compression
and arithmetic coding give at least as good, and often better compression than any universal code. However, universal codes are useful when Huffman coding cannot be used — for example, when one does not know the exact probability of each message, but only knows the rankings of their probabilities. Universal codes are also useful when Huffman codes are inconvenient. For example, when the transmitter but not the receiver knows the probabilities of the messages, Huffman coding requires an overhead of transmitting those probabilities to the receiver. Using a universal code does not have that overhead. Each universal code, like each other self-delimiting binary code, has its own "implied probability distribution" given by p=2−l where l is the length of the ith codeword and p is the corresponding symbol's probability. If the actual message probabilities are q and Kullback–Leibler divergenceDKL is minimized by the code with l, then the optimal Huffman code for that set of messages will be equivalent to that code. Likewise, how close a code is to optimal can be measured by this divergence. Since universal codes are simpler and faster to encode and decode than Huffman codes, the universal code would be preferable in cases where DKL is sufficiently small. For any geometric distribution, a Golomb code is optimal. With universal codes, the implicit distribution is approximately a power law such as . For the Fibonacci code, the implicit distribution is approximately, with where is the golden ratio. For the ternary comma code, the implicit distribution is a power law with. These distributions thus have near-optimal codes with their respective power laws.
