Scalable and Observable RAG for Question-Answering in a Box

This lightning talk demonstrates how to build a self-hosted Retrieval-Augmented Generation (RAG) system for natural language querying across diverse enterprise data sources. Learn how to leverage Large Language Models (LLMs) on a Kubernetes cluster to extract valuable insights from public documentation, internal wikis, and ticketing systems. Discover techniques for implementing auto-scaling and observability at both application and infrastructure levels, enabling a systematic approach from pilot deployments to production. The 13-minute presentation shows how GenAI application developers can create cost-effective question-answering solutions that scale with demand while maintaining visibility into system performance. Presented by Selvi Kadirvel from Elotl at a CNCF event, this talk provides a practical framework for implementing "Question-Answering-in-a-Box" powered by cloud-native technologies.