The Queue Continuum - Applied Queuing Theory

An overview of how queueing theory informs application scaling, including the differences between concurrency and parallelism and when to use more threads or processes. Examples use Sidekiq and Puma with different concurrency settings in input/output scenarios.

Syllabus

Intro
Transporters Capacity Planning
Basic Terminology Queueing Things
Little's Law
USE Method - Intel Engineer Brendan Gregg
Saturation Causes Service Delays
Vertical Scale Bigger is Better
Vertical scaling makes horizontal scaling more expensive Scaling Quantum
Parallelism is Concurrency Concurrency is not exactly the same na parallelism
Benefits of Threaded Concurrency Sharing Available Resources
Queueing for the GVL Threads Lock On to the Ruby VM
How much concurrency? But you don't have to take my word for it
Amdahl's Law
Job Queueing: Sidekiq Distributed Processing of a Centralized Queue
Latency Based Queueing Latency Bed Scaling
Request Queueing: Puma
Heroku Intelligent Routing Random Routing
Customer Satisfaction
Waiting too long causes frustration
Finding Balance
All Good Things...