USENIX Courses

Harvesting Memory-bound CPU Stall Cycles in Software with MSH

Innovative software system for efficiently harvesting memory-bound CPU stall cycles, offering configurable latency and high concurrency to improve datacenter workload performance.

• YouTube
• 16 minutes
• On-Demand
• Free Video

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve

Efficient LLM inference scheduler Sarathi-Serve improves throughput and latency in serving large language models, achieving significant performance gains across various models and hardware configurations.

• YouTube
• 16 minutes
• On-Demand
• Free Video

IntOS: Persistent Embedded Operating System for Multi-threaded Intermittent Computing

Innovative embedded OS for battery-less devices, enabling efficient multi-threaded computing with crash consistency and simplified programming through novel replay-and-bypass approach.

• YouTube
• 18 minutes
• On-Demand
• Free Video

Anvil: Verifying Liveness of Cluster Management Controllers

Innovative framework for verifying liveness in cluster management controllers, ensuring continuous reconciliation to desired states despite failures and asynchrony in cloud environments.

• YouTube
• 17 minutes
• On-Demand
• Free Video

Chop Chop: Byzantine Atomic Broadcast to the Network Limit

Innovative Byzantine Atomic Broadcast system achieving high throughput and low latency through novel batching technique. Demonstrates superior performance in geo-distributed deployments for various applications.

• YouTube
• 12 minutes
• On-Demand
• Free Video

μSlope - High Compression and Fast Search on Semi-Structured Logs

Innovative system for compressing and searching semi-structured log data, offering superior compression ratios and faster search speeds compared to existing solutions for managing large-scale log datasets.

• YouTube
• 16 minutes
• On-Demand
• Free Video

SquirrelFS - Using the Rust Compiler to Check File-System Crash Consistency

SquirrelFS: A novel approach to building crash-safe file systems using Rust's typestate pattern for compile-time enforcement, offering performance comparable to state-of-the-art alternatives.

• YouTube
• 16 minutes
• On-Demand
• Free Video

ServiceLab: Preventing Tiny Performance Regressions at Hyperscale through Pre-Production Testing

Explore Meta's ServiceLab platform for detecting minute performance regressions in large-scale systems, addressing challenges in noisy cloud environments through statistical analysis and machine factor studies.

• YouTube
• 16 minutes
• On-Demand
• Free Video

nnScaler: Constraint-Guided Parallelization Plan Generation for Deep Learning Training

Innovative framework for efficient deep learning training parallelization, using primitives and constraints to generate optimized execution plans, outperforming existing solutions for popular DNN models.

• YouTube
• 14 minutes
• On-Demand
• Free Video

Validating the eBPF Verifier via State Embedding

Innovative technique for validating eBPF verifier correctness using state embedding, uncovering critical Linux kernel security bugs and improving overall system reliability.

• YouTube
• 15 minutes
• On-Demand
• Free Video

ChameleonAPI: Automatic and Efficient Customization of Neural Networks for ML Applications

Automatic customization of neural networks for ML applications, reducing incorrect decisions by 43% compared to commercial APIs without changing application code.

• YouTube
• 16 minutes
• On-Demand
• Free Video

DistServe: Disaggregating Prefill and Decoding for Goodput-Optimized Large Language Model Serving

Innovative approach to optimize large language model serving by separating prefill and decoding phases, improving performance and meeting latency requirements for various applications.

• YouTube
• 15 minutes
• On-Demand
• Free Video

Automatically Reasoning About How Systems Code Uses the CPU Cache

Technique for automatic analysis of CPU cache usage in systems code. Enables developers to identify performance issues and security vulnerabilities without extensive benchmarking.

• YouTube
• 15 minutes
• On-Demand
• Free Video

Caravan - Practical Online Learning of In-Network ML Models with Labeling Agents

Practical online learning system for in-network ML models, addressing automatic labeling and efficient performance monitoring. Improves model adaptability to dynamic network environments.

• YouTube
• 18 minutes
• On-Demand
• Free Video

USHER: Holistic Interference Avoidance for Resource Optimized ML Inference

Optimizes ML inference systems by maximizing GPU resource utilization and minimizing interference, resulting in higher goodput and cost-efficiency for deep learning model deployment.

• YouTube
• 15 minutes
• On-Demand
• Free Video