4 Contextual Adaptive coding

Adaptive coding can be made contextual by utilizing the fact that only three properties are needed for a time-invariant Markov process to completely be defined:

1. set of all possible states
2. transition probabilities between the states
3. starting probability distribution

In the implementation of a contextual adaptive encoder. The set of all possible states is the expected alphabet size. The transition probabilities can be approximated to a uniform distribution as discussed in the previous section or or another prior as discussed later.

4.1 Advantages and challenges of longer block lengths

When working with estimated probabilities, there are special considerations when using longer block lengths, in addition to those considered in 2.2. Firstly, the longer the block length, the less frequently this block will be observed in other parts of the source and the longer the parsing is required to generate significant probability for the blocks. The longer the context, the more important it is to actually having a good estimator for the first few symbols. This makes \(\delta\) more important. Secondly, as discussed in 3.3, a large block length fails to capture local distributions and the estimated probabilities may be underfitting for the block.