Is the Nvidia Tesla V100 Still Good for AI - Inspur DGX V100 vs RTX 5090
Overview
Coursera Spring Sale
40% Off Coursera Plus Annual!
Grab it
Explore the viability of eight-year-old Nvidia Tesla V100 GPUs for modern AI workloads in this comprehensive 25-minute video analysis. Examine the performance of eight Tesla V100s housed in an Inspur DGX V100 system against contemporary hardware like the RTX 5090 through rigorous testing with 70B and 120B large language models. Learn about the system specifications including dual Intel Xeon 8260 processors, 256GB DDR4-3200 memory, and the complete hardware configuration. Follow detailed benchmark comparisons using MLPerf testing with Llama 3.1 8B and Phi 4 Reasoning 14B models, plus practical performance evaluation through LM Studio with Llama 3.3 70B and GPT-OSS 120B models. Discover whether older enterprise AI hardware can provide a cost-effective alternative to expensive modern solutions for businesses looking to implement AI workflows without the prohibitive costs of current-generation equipment. Gain insights into power consumption, cooling requirements, and real-world performance metrics that will help determine if Tesla V100s still have viable applications in today's AI landscape.
Syllabus
0:00 - Intro
2:11 - Sponsor - Verda.com
3:29 - Speeds and Feeds
6:10 - Llama 3.1 8B MLPerf
8:16 - Phi 4 Reasoning 14B MLPerf
9:06 - LM Studio Llama 3.3 70B + GPT-OSS 120B
14:56 - As Always, I'm Mikey
20:15 - Wrapping Up
22:57 - Deschutes Fresh Squeezed NA
Taught by
Craft Computing