7 Conclusion

Shannon noted in 1951 that a good encoder could be the speaker of the language instead of a statistical model [9]. With the advent of natural language processing models such as Generative Pre-trained Transformer 3 (GPT3), it becomes increasingly probable to implement something that Shannon imagined to predict the next text as well as GPT3 producese them. Yet, even if the black-box models would take over the world, the understanding of the fundamental coding algorithms and potential modification to make them practical and efficient should still be in the toolbox of every information engineer.