A Deep Dive and Comparison of Python Drivers for Cassandra and Scylla

Explore a comprehensive conference talk that delves into Python drivers for Cassandra and Scylla databases. Learn about cluster architecture, replication factors, and virtual nodes for improved data distribution. Discover the differences between naive and token-aware clients, and understand how routing keys and partition hashing work. Gain insights into Scylla's shard-aware drivers, including the implementation of shard-awareness in the Python driver. Examine the challenges of connecting to every core and how they were addressed. Investigate performance optimizations, such as moving shard ID calculations to Cython. Understand the core logic behind scylla-driver's shard-awareness and the importance of expectation checks in ensuring proper functionality.

Syllabus

Intro
A cluster is a collection of nodes
Replication Factor provides higher data availability
Virtual Nodes better partition distribution between nodes
Scylla's Virtual Nodes are split into shards bound to cores!
Rows are located on nodes by hashing their partition key
Naive clients route queries to any node (coordinator)
Token Aware clients route queries to the right node(s)!
TokenAware Policy Statement + routing key = node(s)
Scylla shard aware drivers: Python was missing!
Implementing shard-awareness for scylla-driver
scylla-driver shard-awareness detection
scylla-driver connections to shards/cores
The Connection to every core problem
scylla-driver enhanced connections to shards/cores
scylla-driver routing key token to core calculation
Performance concern move shard id calculation to Cython
At the heart of scylla-driver's shard-awareness logic
scylla-driver expectations checks