Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve

USENIX via YouTube Direct link

OSDI '24 - Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve

1 of 1

1 of 1

OSDI '24 - Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve

Class Central Classrooms beta

YouTube videos curated by Class Central.

Classroom Contents

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve

Automatically move to the next video in the Classroom when playback concludes
  1. 1 OSDI '24 - Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free
Someone learning on their laptop while sitting on the floor.