Trelis Research Courses

Full Fine-Tuning vs LoRA and QLoRA - Comparison and Best Practices

Explore full fine-tuning vs (Q)LoRA techniques, comparing VRAM usage, training time, and quality. Learn parameter selection, tips for QLoRA, and step-by-step guidance for TinyLlama and Mistral 7B fine-tuning.

• YouTube
• 53 minutes
• Self-Paced
• Free Video

Multi-GPU Fine-tuning with DDP and FSDP

Master multi-GPU fine-tuning techniques using DDP and FSDP. Explore VRAM optimization, distributed training strategies, and practical implementation with code examples and demos.

• YouTube
• 1 hour 8 minutes
• Self-Paced
• Free Video

Combined Preference and Supervised Fine-Tuning with ORPO

Explore combined preference and supervised fine-tuning with ORPO. Learn about fine-tuning methods, loss functions, and performance improvements. Includes practical demos and evaluations.

• YouTube
• 31 minutes
• Self-Paced
• Free Video

Deploying Serverless Inference Endpoints - Setup and Cost Comparison

Deploy serverless inference endpoints efficiently. Learn setup, use cases, and cost comparisons for API-based machine learning model deployment.

• YouTube
• 20 minutes
• Self-Paced
• Free Video

Improved Retrieval Augmented Generation with ALL-SORT - Advanced Techniques

Explore advanced retrieval augmented generation techniques using ALL-SORT for improved LLM-assisted information retrieval, with practical demonstrations and performance evaluations.

• YouTube
• 57 minutes
• Self-Paced
• Free Video

Fine-tuning LLMs - Every Step Explained for Memorization Tasks

Explore LLM fine-tuning techniques, from dataset generation to hyperparameter optimization. Learn to overcome challenges like the reversal curse and optimize model performance for specific tasks.

• YouTube
• 47 minutes
• Self-Paced
• Free Video

Mistral Large vs GPT-4: Practical Benchmarking and LLM Evaluation

Explore practical benchmarking of Mistral Large vs GPT4. Compare performance, run custom evaluations, and gain insights into LLM capabilities for informed decision-making.

• YouTube
• 28 minutes
• Self-Paced
• Free Video

Fine-tuning Optimizations - DoRA, NEFT, LoRA+, and Unsloth

Explore advanced fine-tuning techniques like DoRA, NEFT, LoRA+, and Unsloth to optimize language models. Learn implementation, comparisons, and practical applications through code examples and demonstrations.

• YouTube
• 33 minutes
• Self-Paced
• Free Video

Fine-tuning Multi-modal LLaVA Vision and Language Models

Explore advanced techniques for fine-tuning multi-modal vision and language models like LLaVA and IDEFICS. Learn about architectures, data creation, training, and evaluation methods.

• YouTube
• 51 minutes
• Self-Paced
• Free Video

Understanding Mamba and State Space Models

Explore Mamba and state space models, comparing them to transformers. Learn basic equations, run inference in Colab, and understand training processes and future directions for this compression-focused technology.

• YouTube
• 28 minutes
• Self-Paced
• Free Video

Data Extraction with Large Language Models

Master data extraction techniques using large language models for efficient and accurate information retrieval from various sources. Enhance your skills in JSON and YAML extraction, and compare model performances.

• YouTube
• 56 minutes
• Self-Paced
• Free Video

Fine-Tuning Whisper for Speech Transcription

Explore advanced Whisper fine-tuning techniques for improved speech transcription. Learn to transcribe YouTube videos, understand model mechanics, and evaluate performance using LoRA.

• YouTube
• 49 minutes
• Self-Paced
• Free Video

Pushing Models and Adapters to HuggingFace

Learn to push, merge, and manage various LLM model formats on HuggingFace. Covers setup, fine-tuning, and best practices for training and inference. Includes practical demonstrations and resource recommendations.

• YouTube
• 45 minutes
• Self-Paced
• Free Video

The Best Tiny Language Models - Performance, Fine-tuning, and Function-calling

Explore the benefits, performance, and fine-tuning of small language models. Learn about function-calling with quantized models and discover the best tiny LLMs for various applications.

• YouTube
• 1 hour 2 minutes
• Self-Paced
• Free Video

Speculative Decoding: Techniques for Faster LLM Inference

Explore advanced speculative decoding techniques for faster LLM inference, including naive, n-gram, lookahead, and assisted methods. Compare performance and learn optimization tips.

• YouTube
• 38 minutes
• Self-Paced
• Free Video