Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Learn to build an intelligent model routing AI agent that optimizes both cost and user value across different large language models in this 37-minute conference talk from Databricks. Discover how to balance cost reduction with maximizing use case value by considering factors like latency, multi-modality, API costs, user needs, and prompt complexity. Explore cost-effective model training techniques using AI gateway logs, user feedback, prompt analysis, and model features to create sophisticated routing systems. Master various model routing strategies and their deployment in Mosaic AI, including re-training processes and evaluation through A/B testing within end-to-end Databricks workflows. Dive deep into technical implementation details covering training data collection, feature engineering, prompt formatting, custom loss functions, architectural modifications, and solutions for cold-start problems. Understand advanced techniques for query embedding generation and clustering through VectorDB, plus reinforcement learning policy-based exploration methods for continuous improvement of routing decisions.
