LLM Inference Courses and Certifications

LLM Inference Optimization

1 rating at Udacity

Master advanced techniques to optimize large language model inference performance, including quantization, speculative decoding, and model parallelism for faster deployment.

• Udacity
• 11 hours
• On-Demand
• Paid Course

Google TechTalks

Streaming Attention Approximation via Discrepancy Theory

Discover BalanceKV, an algorithm using geometric properties to compress key-value cache in LLMs, reducing memory requirements while maintaining attention quality through discrepancy theory.

• YouTube
• 33 minutes
• On-Demand
• Free Video

Open Compute Project

Accelerating LLM Serving with Prompt Cache Offloading via CXL

Discover how CXL-attached memory integration boosts LLM inference performance by 38.5% through efficient prompt cache offloading, reducing latency and infrastructure costs significantly.

• YouTube
• 14 minutes
• On-Demand
• Free Video

Nvidia

Dynamo KVBM - Managing Memory at Scale

Discover how Dynamo's KV Block Manager optimizes memory management for high-throughput LLM deployments, covering architecture, runtime integration, and practical implementation strategies.

• YouTube
• 29 minutes
• On-Demand
• Free Video

Anyscale

Inside NVIDIA Dynamo - Faster, Scalable AI Deployment

Discover how NVIDIA Dynamo revolutionizes large-scale LLM inference with smart scheduling, hierarchical memory management, and low-latency KV-cache transfer for maximum efficiency.

• YouTube
• 34 minutes
• On-Demand
• Free Video

MLOps.community

Exploring the Latency, Throughput, and Cost Space for LLM Inference

Explore LLM inference stacks, optimizing model selection, hardware, and code for cost-effective performance. Gain insights on open-source models and future developments in AI serving technology.

• YouTube
• 30 minutes
• On-Demand
• Free Video

AI Engineer

Mastering LLM Inference Optimization - From Theory to Cost Effective Deployment

Dive into LLM inference optimization covering GPU selection, KV Cache management, quantization, parallelism techniques, and cost-effective deployment strategies for production-grade systems.

• YouTube
• 34 minutes
• On-Demand
• Free Video

Databricks

Accelerating LLM Inference with vLLM

Explore vLLM, an open-source engine for LLM inference, offering state-of-the-art performance. Learn about paged attention, continuous batching, and new innovations like Speculative Decoding and Prefix Caching.

• YouTube
• 36 minutes
• On-Demand
• Free Video

Anyscale

Optimizing LLM Inference with AWS Trainium, Ray, vLLM, and Anyscale

Explore cost-effective LLM inference scaling using AWS accelerators, Ray, vLLM, and Anyscale. Learn to build a complete stack on EKS and leverage advanced cluster management for enterprise-grade GenAI workloads.

• YouTube
• 46 minutes
• On-Demand
• Free Video

Arm Software Developers

Demos on Vulkan SDK, LLM Inference, and Advancing Mobile Graphics

Explore Vulkan SDK, LLM inference on Android, and mobile graphics advancements. Gain insights from experts and discover new learning paths for software development on Arm platforms.

• YouTube
• 54 minutes
• On-Demand
• Free Video

USENIX

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve

Efficient LLM inference scheduler Sarathi-Serve improves throughput and latency in serving large language models, achieving significant performance gains across various models and hardware configurations.

• YouTube
• 16 minutes
• On-Demand
• Free Video

The Machine Learning Engineer

LLMOps: Accelerate LLM Inference in GPU Using TensorRT-LLM

Acelere la inferencia de LLM en GPU utilizando TensorRT-LLM para mejorar el rendimiento y la eficiencia en aplicaciones de aprendizaje automático.

• YouTube
• 17 minutes
• On-Demand
• Free Video

Anyscale

Accelerated LLM Inference with Anyscale - Ray Summit 2024

Dive into Anyscale's latest LLM enterprise features and open-source contributions. Explore advancements in vLLM, including FP8 support and speculative decoding, doubling throughput and latency efficiency for improved inference performance.

• YouTube
• 30 minutes
• On-Demand
• Free Video

Devoxx

Practical LLM Inference in Modern Java

Implement efficient local LLM inference in Java, leveraging modern features for CPU-based performance without specialized hardware. Explore flexible frameworks adaptable to multiple architectures.

• YouTube
• 51 minutes
• On-Demand
• Free Video

DevConf

PagedAttention - Revolutionizing LLM Inference with Efficient Memory Management

Discover how PagedAttention transforms LLM inference by solving memory bottlenecks through virtual memory paging, achieving 30× higher throughput and 96% cache efficiency in vLLM framework.

• YouTube
• 28 minutes
• On-Demand
• Free Video