Overview

Google, IBM & Meta Certificates – 40% Off

One Coursera Plus subscription covers most Professional Certificates on Coursera.

Moving a large language model (LLM) application from prototype to production requires measurable quality, the right model for each task, and reliable operations at scale. This course covers how to evaluate, optimize, and scale LLM systems. Through follow-along demos, you will improve retrieval with re-ranking and hybrid search, score RAG pipelines with RAGAS, fine-tune open models with LoRA and QLoRA on Hugging Face, and deploy a containerized GenAI API with FastAPI and Docker. By the end of this course, you will be able to: • Evaluate RAG pipelines using ranking metrics and RAGAS faithfulness scores. • Improve retrieval with re-ranking, HyDE, hybrid search, and advanced retrievers. • Select models by cost, latency, and quality, and fine-tune them with LoRA. • Build multimodal features with vision-language and speech models. • Deploy GenAI services with FastAPI, Docker, caching, and provider fallbacks. • Operate LLM systems with guardrails, red-teaming, tracing, and drift detection. This course is designed for AI engineers, ML engineers, backend developers, and MLOps practitioners responsible for shipping LLM features. You should be comfortable building LLM applications with Python, APIs, and RAG before you begin. Enroll now to evaluate, fine-tune, and scale LLM systems with confidence.

Syllabus

Advanced Retrieval Techniques and RAG Evaluation
- Analyze the limitations of basic RAG by examining re-ranking, query transformations, hybrid search, and advanced retrievers. Develop foundational skills by implementing cross-encoder re-ranking, query rewriting, HyDE, hybrid search, and parent-document and self-querying retrievers. Build practical capabilities by measuring retrieval quality and generation faithfulness with ranking metrics and RAGAS, and by improving weak RAG pipelines.
Model Selection and Parameter-Efficient Fine-Tuning
- Evaluate when to use prompting, RAG, or fine-tuning by examining model selection trade-offs in cost, latency, and quality. Develop foundational skills by benchmarking candidate models and preparing instruction-tuning datasets. Build practical capabilities by fine-tuning models with LoRA, QLoRA, and adapters on Hugging Face and comparing quality before and after fine-tuning.
Multimodal Generative AI Systems
- Analyze how multimodal LLMs process and combine text, image, and audio inputs. Develop foundational skills by generating image captions and answering questions over images with vision-language models. Build practical capabilities by integrating speech-to-text and text-to-speech into generative AI applications.
Reliability, Serving, and Deployment
- Analyze reliability patterns for GenAI services by examining caching, retries, provider fallbacks, and observability. Develop foundational skills by building a GenAI API with FastAPI and containerizing it with Docker. Build practical capabilities by deploying containerized GenAI applications to the cloud.
LLMOps and Production Safety
- Evaluate LLMOps practices by examining how versioning, experiment tracking, guardrails, and red-teaming keep GenAI systems safe in production. Develop foundational skills by versioning prompts and outputs and tracking experiments. Build practical capabilities by implementing guardrails and red-teaming applications for toxicity and hallucination.
Observability, Monitoring, and the Specialization Capstone
- Evaluate the production readiness of GenAI systems by examining tracing, monitoring, and drift detection. Develop foundational skills by tracing and monitoring deployed applications and detecting model and data drift. Build practical capabilities by designing a deployed, guarded, and monitored end-to-end GenAI application that integrates retrieval, fine-tuning, and LLMOps practices.

Taught by

Edureka

Reviews

Start your review of Evaluating and Scaling LLMs

Taught by

Never Stop Learning.