What is Multimodal RAG, and Can We Build a Village with It?

This 40-minute conference talk by Alexander Chatzizacharias explores the concept of Multimodal Retrieval-Augmented Generation (RAG) and its creative application in building interactive virtual environments. Discover how RAG technology combines textual, visual, and audio data to enhance AI prompts, creating more dynamic and responsive non-player characters (NPCs) in gaming environments. Follow along as Alexander demonstrates how to implement RAG workflows that enable NPCs to interact with each other, pursue goals, and move dynamically within virtual spaces. Learn the practical aspects of setting up both basic and advanced RAG systems using tools like Langchain4j to deliver factual, up-to-date responses. Suitable for both experienced backend engineers and curious developers, this engaging Devoxx presentation blends educational content with playful demonstrations to illuminate the potential of multimodal RAG technology.