Finetuning - 500M AI Agents in Production with 2 Engineers

Learn how Method Financial successfully deployed and scaled AI financial agents handling hundreds of thousands of daily interactions with just two engineers in this conference talk from the AI Engineer Summit 2025. Discover the battle-tested techniques and hard-won lessons from 18+ months of running 500 million AI agents in production, including critical strategies for fine-tuning models, implementing real-time guardrails and sanity checks, building effective observability platforms, and deploying intelligent caching systems. Gain insights into the specific challenges and solutions for maintaining operational effectiveness at scale, with practical advice from Mustafa Ali, founding engineer at Method Financial who leads robust real-time data integration systems with major financial institutions, and Kyle Corbitt, co-founder and CEO of OpenPipe who has fine-tuned thousands of customer models serving tens of millions of daily inference requests. Understand the technical architecture and operational considerations necessary for deploying AI agents in high-stakes financial environments where reliability and accuracy are paramount.