Benchmarking Courses and Certifications

Benchmarking LLMs with QA

Master LLM evaluation through QA benchmarking, fuzzy matching, ROUGE scoring, and semantic similarity. Compare GPT models using zero-shot to few-shot prompting while exploring perplexity metrics.

• CodeSignal
• 1 hour
• On-Demand
• Free Certificate

Benchmarking LLMs on Text Generation

Explore benchmarking techniques for LLMs in text generation tasks, comparing GPT models through prompting strategies, ROUGE metrics, and semantic similarity evaluation methods.

• CodeSignal
• 1 hour
• On-Demand
• Free Certificate

Behavioral Benchmarking of LLMs

Explore LLM evaluation beyond accuracy through hands-on experiments with token efficiency, temperature effects, output consistency, and hallucination detection using API testing.

• CodeSignal
• 1 hour
• On-Demand
• Free Certificate

Trelis Research

Mistral Large vs GPT-4: Practical Benchmarking and LLM Evaluation

Explore practical benchmarking of Mistral Large vs GPT4. Compare performance, run custom evaluations, and gain insights into LLM capabilities for informed decision-making.

• YouTube
• 28 minutes
• On-Demand
• Free Video

ACM SIGPLAN

ValBench: Benchmarking Exact Value Analysis

Explore ValBench, a comprehensive benchmark suite for evaluating value analysis tools in Java and Android. Learn about its test cases, evaluation framework, and insights into current challenges in program analysis.

• YouTube
• 18 minutes
• On-Demand
• Free Video

SNIAVideo

Benchmarking Storage with AI Workloads - Performance Analysis and Optimization

Explore how storage systems impact AI/ML workload performance through comprehensive benchmarking analysis, focusing on data ingestion, training patterns, and inference workload optimization.

• YouTube
• 46 minutes
• On-Demand
• Free Video

LinaroOrg

Attempting to Measure the Graviton - Benchmarking Performance on AWS Graviton3 and Graviton4 Instances

Dive into the analysis of benchmark variations on AWS Graviton instances, exploring how shared infrastructure affects SPEC CPU 2017 test reproducibility and performance consistency.

• YouTube
• 27 minutes
• On-Demand
• Free Video

Chris Hay

Benchmarking LLMs: Claude 4, Gemini 2.5, and Mixture of Expert Models

Explore comprehensive benchmarking of leading AI models including Claude 4, Gemini 2.5, GPT-4, and Mistral, revealing performance insights and identifying which use Mixture of Experts architecture.

• YouTube
• 38 minutes
• On-Demand
• Free Video

nanohubtechtalks

Benchmarking Universal Machine Learning Force Fields with CHIPS-FF

Explore CHIPS-FF, an open-source platform for benchmarking universal machine learning force fields in semiconductor applications and complex material properties.

• YouTube
• 1 hour 5 minutes
• On-Demand
• Free Video

Code Sync

Benchee - 9 Years of Benchmarking on the BEAM

Explore 9 years of Benchee development, BEAM performance insights, open source lessons, architectural decisions, and benchmarking best practices in the Elixir/Erlang ecosystem.

• YouTube
• 56 minutes
• On-Demand
• Free Video

MLOps.community

Benchmarking Intelligence - ARC Prize and AI Evaluation Strategies

Explore AI benchmarking strategies, from human-easy puzzles that challenge models to measuring intelligence through learning speed and the philosophical quest toward AGI.

• YouTube
• 49 minutes
• On-Demand
• Free Video

Open Compute Project

Benchmarking is Hard - A Hermetically Sealed Container for CMS Workload Benchmarking

Discover how to deploy standardized, hermetically sealed containers for fair CMS workload benchmarking across diverse hardware platforms with configurable parameters and comparable results.

• YouTube
• 12 minutes
• On-Demand
• Free Video

GERAD Research Center

Benchmarking Algorithms on Large Test Sets

Discover advanced benchmarking techniques for optimization algorithms, including convergence plots, performance profiles, and extensions to constrained, multi-objective, and surrogate-based optimization.

• YouTube
• 40 minutes
• On-Demand
• Free Video

Presto Foundation

Diving into and Benchmarking Presto C++ - Hands-on Virtual Lab

Dive into Presto C++ architecture, deployment, and performance benchmarking using pbench tool while comparing it with Presto Java in this practical hands-on workshop.

• YouTube
• 1 hour 17 minutes
• On-Demand
• Free Video

DevConf

Benchmarking LLMs: Metrics, Challenges, and Best Practices for Evaluation

Explore benchmarking metrics for LLMs, covering evaluation challenges, best practices, and ethical considerations for enterprise adoption, with insights on qualitative and quantitative assessment techniques.

• YouTube
• 35 minutes
• On-Demand
• Free Video