Secure by Design: Strategies for LLM Adoption in Cloud-Native Environments

Explore strategies for secure LLM adoption in cloud-native environments through this informative conference talk. Delve into the common journey of software development companies integrating AI technology, addressing challenges faced by development and platform teams. Learn about the evolution from initial resistance to external LLM APIs to building solutions with non-public data and open-source models. Examine critical security questions surrounding AI/LLM applications and discover practical approaches to data privacy in Retrieval-Augmented Generation architectures, AI-agent architectures, and general security hardening. Gain insights from real-world practices, including the development of a threat model for AI-based systems aligned with the OWASP Top 10 for LLM Applications. Understand the implementation of both offensive and defensive approaches, featuring the integration of tools like garak (LLM Vulnerability Scanner) and NVIDIA NeMo Guardrails into cloud-native stacks.