Beyond AI Accuracy - Building Trustworthy and Responsible AI Applications Through Mosaic AI Framework

Learn to build trustworthy and responsible AI applications by moving beyond generic accuracy metrics to create custom evaluation frameworks tailored to your business needs in this 37-minute conference talk. Discover why standard LLM metrics fall short of meeting real-world business requirements and explore advanced strategies for developing bespoke AI evaluation systems. Master best practices for LLM evaluation strategies, including when to implement LLM judges versus statistical metrics, and gain insights into autonomous AI evaluation suites for complex multi-agent systems. Follow along with a live demonstration using Databricks' Mosaic AI Framework to build custom AI metrics specifically designed for your GenAI applications. Explore techniques for generating ground truth data at scale and implementing effective production monitoring strategies. Benefit from real-world case studies and actionable insights drawn from extensive customer experience to establish robust AI evaluation frameworks that ensure your AI solutions are both powerful and aligned with your organization's specific needs and business objectives.