Controlling Memory Footprint of Stateful Streaming Graph Processing

Explore cutting-edge techniques for controlling memory usage in stateful streaming graph processing systems in this conference talk from USENIX ATC '21. Dive into the challenges of analyzing dynamic graphs and learn about innovative memory-efficient stateful iterative models that significantly reduce memory footprint while maintaining performance. Discover the Selective Stateful Iterative Model and the Minimal Stateful Iterative Model, understanding their implementation strategies and benefits. Examine experimental results demonstrating how these models enable processing of larger graphs that traditional approaches struggle with. Gain insights into the future of efficient streaming graph analysis and its applications in handling ever-growing datasets.

Syllabus

Intro
Graph Analytics
Streaming Graph Processing
Stateful Iterative Processing Model
Streaming Graph Systems: GraphBolt & DZIG
Memory-Efficient Stateful Iterative Models
Selective Stateful Iterative Model: Challenges
Selectively Tracking Intermediate State
Selective Incremental Processing
Distributive Update Property • Computation distributed into sub-computations on subsets of inputs
Minimal Stateful Iterative Model
Experimental Setup
Other Experiments
Conclusion