AI Inferencing Sizing Considerations on Nutanix Enterprise AI

In this 28-minute Tech Field Day presentation, learn practical AI inferencing sizing guidance based on real-world experience from Jesse Gonzales, Staff Solution Architect at Nutanix. Discover how to properly size AI infrastructure for inferencing workloads by understanding model requirements, GPU device types, and inference engine roles. Follow detailed explanations of CPU and memory requirements for Kubernetes worker nodes based on inference engine selection. Gain insights into managing administrative overhead and ensuring high availability when deploying LLM endpoints within Kubernetes clusters. Explore Nutanix Enterprise AI's pre-validated models with specific resource recommendations for production environments. Understand the industry shift from proof-of-concept to centralized systems for sharing large models, while accounting for planned maintenance and pod migration capacity. Master the sizing process from model selection through GPU identification to CPU and memory calculation. Consider FinOps and cost management aspects, including upcoming metrics integration for request counts, latency, and token-based consumption. Examine deployment and licensing options for on-premises, bare metal, and cloud scenarios based on existing infrastructure. Appreciate Nutanix's flexible approach supporting various infrastructure choices, virtualization options, and Kubernetes distributions to streamline AI deployment and management. Recorded live in Santa Clara, California on April 24, 2025, as part of AI Infrastructure Field Day.