Completed
11:07 Infrastructure for Batch Inference
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Serving Voice AI at $1/hr - Open-source, LoRAs, Latency, Load Balancing
Automatically move to the next video in the Classroom when playback concludes
- 1 00:00 Introduction to Gabber and Real-Time AI
- 2 02:15 Gabber's Mission for Consumer AI
- 3 04:17 The Orpheus Voice Model
- 4 05:43 Challenges in Voice Cloning
- 5 07:44 Latency Management and "Head of Line Silence"
- 6 11:07 Infrastructure for Batch Inference
- 7 11:36 Leveraging vLLM and Dynamic Quantization
- 8 13:21 Load Balancing with a Consistent Hash Ring
- 9 14:17 System Architecture Overview
- 10 15:07 Conclusion and Open Source Shout-outs
