Explore the evolution of natural language processing and language models in this comprehensive conference talk from Data Science Conference Europe 2023. Journey through the developmental progression from Recurrent Neural Networks (RNNs) to the revolutionary Attention Mechanisms, culminating in the groundbreaking Transformer architecture that dominates modern NLP. Examine the intriguing question of minimal model requirements for coherent English language generation, while gaining deep insights into the historical milestones that shaped today's language modeling landscape. Learn from speaker Jovan Bubonja's expertise as he breaks down complex concepts and demonstrates how these architectural advances have transformed our approach to natural language processing.