Observability Without Oversharing - Privacy-Conscious Telemetry for LLMs

Learn to implement robust observability for Large Language Models while maintaining user privacy through this hands-on technical workshop presented by Joaquin Rodriguez and Amin Espinoza de los Monteros from Microsoft at the Linux Foundation's Open Source Summit. Master the essential balance between monitoring LLM performance, usage, and costs while safeguarding sensitive data using open-source tools including OpenTelemetry, OpenLIT, Prometheus, and Grafana. Engage in guided practical exercises designed to develop ethical AI monitoring skills, focusing on privacy-conscious telemetry implementation for production LLM systems. Gain hands-on experience with open-source solutions that address privacy challenges in LLM deployments, making this workshop ideal for developers, data scientists, and AI practitioners committed to responsible AI observability practices.