The New PyTorch Architecture for TensorRT-LLM

Explore a 52-minute talk from Nvidia that introduces the new PyTorch-based architecture for TensorRT-LLM, designed to enhance user experience and developer velocity for large language model (LLM) deployments. Learn how this architecture makes it easier to build custom models, integrate new kernels, and extend runtime functionality while delivering state-of-the-art performance on NVIDIA GPUs. Through concrete examples, discover the flexibility of this PyTorch-based architecture and how it enables quick customizations while maintaining optimal performance for LLM deployments on the NVIDIA platform.