Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Professional Quality Voice Cloning - Open Source vs ElevenLabs

Trelis Research via YouTube

Overview

Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Learn to create professional-quality voice cloning systems by comparing open-source solutions with commercial platforms like ElevenLabs in this comprehensive 59-minute tutorial. Master the complete workflow of fine-tuning text-to-speech models using Unsloth, starting with understanding token-based speech-to-text model architectures and progressing through practical implementation. Discover data preparation techniques including audio cleaning, chunking, and transcription processes, then explore dataset creation and management using HuggingFace Hub. Practice loading and configuring models like Sesame CSM-1B with LoRA adapters, understanding training hyperparameters, and running inference on fine-tuned models. Compare different approaches by working with Orpheus by Canopy Labs, examining its unique data loading requirements and evaluating output quality. Analyze the differences between open-source voice cloning capabilities and commercial solutions through hands-on examples and quality comparisons. Gain insights into model evaluation techniques using tensorboard logs and prepare for advanced deployment scenarios with vLLM serving capabilities.

Syllabus

0:00 Fine-tuning Text-to-Speech Models with Unsloth
0:53 Video Overview
1:47 Video Resources
2:26 Voice Quality Examples: ElevenLabs vs Open Source
4:52 The recipe for professional quality voice cloning
6:23 How do token based speech to text models?
14:08 Data Preparation and Training Overview
16:02 Data preparation, cleaning and chunking for voice cloning
24:05 Audio transcription from uploaded audio
25:42 Dataset chunking and pushing to HuggingFace Hub
29:49 Loading Sesame CSM-1B and LoRA adapters full fine-tuning also possible! And in the repo
34:36 Dataset loading and creating and eval split
37:42 Training Hyperparameters
40:08 Running inference on the fine-tuned model, and evaluating
43:57 LoRA fine-tuning of Orpheus by Canopy Labs - Data loading and is very different!
50:27 Running inference and Listening to the quality with Orpheus
53:15 Professional Voice Cloning with Eleven Labs
56:18 Examining tensorboard logs from the Sesame LoRA fine-tuning
57:27 Upcoming video on serving Orpheus with vLLM
58:10 Conclusion

Taught by

Trelis Research

Reviews

Start your review of Professional Quality Voice Cloning - Open Source vs ElevenLabs

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.