This course was developed by members of AWS Technical Field Communities (TFC), an AWS community of technical experts. The content is intended to complement our standard training curriculum and augment your AWS learning journey. We are aware some courses have accessibility limitations and are working to address. If you require accommodation, please contact AWS Training and Certification Customer Support.
In this course, learners will understand the cold start problem that occurs when deploying large language models for inference on Amazon SageMaker HyperPod -- where downloading container images and model weights can delay serving traffic by 30 minutes or more. They will learn how model caching works by pre-loading model weights and container images onto local NVMe storage on cluster nodes, enabling pods to start serving in seconds instead of tens of minutes. Learners will also gain practical knowledge on how to enable model caching and review benchmark results demonstrating its performance benefits.
