Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Linux Foundation

Hardware, Software, Performance and Costs for Llama-2 70b and Mixtral 8x7b LLM Inference with Low Concurrency

Linux Foundation via YouTube

Start learning Write review
OpenLearning Ad

Future-Proof Your Career: AI Manager Masterclass

Learn More → Corporate Finance Institute Ad

Build the Finance Skills That Lead to Promotions, Not Just Certificates

Learn More →

Overview

Google, IBM & Meta Certificates – 40% Off
One Coursera Plus subscription covers most Professional Certificates on Coursera.
Unlock All Certificates
This course examines the hardware, software, performance, and costs involved in running unquantized Llama-2 70B and Mixtral 8x7B inference at low concurrency. It compares on-premises and public cloud configurations with closed APIs and shows how to configure servers and reproduce benchmarks.

Syllabus

HW, SW, Performance and Costs for Llama-2 70b and Mixtral 8x7b LLM Inference with Low...- Ivan Baldo

Taught by

Linux Foundation

Tags

Reviews

Start your review of Hardware, Software, Performance and Costs for Llama-2 70b and Mixtral 8x7b LLM Inference with Low Concurrency

Start learning

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free
Someone learning on their laptop while sitting on the floor.