AI Engineering 201 - Inference

Explore advanced AI inference concepts in this comprehensive workshop designed for AI engineers seeking to deepen their understanding of running machine learning models in production. Learn the fundamentals of inference workloads and compare proprietary versus open-source models, examining their respective advantages and trade-offs in real-world applications. Discover various execution environments including end-user devices and network-based solutions, while mastering the principles of serving inference at scale. Dive into inference-as-a-service providers and understand how to leverage cloud infrastructure and serverless GPU solutions for optimal performance. Examine rack-and-stack approaches for inference deployment and master GPU arithmetic calculations essential for resource planning. Explore specialized hardware options including TPUs and other custom silicon designed for inference optimization. Conclude by learning containerization strategies for inference services, enabling scalable and maintainable AI deployments across different environments.