Securing Data Applications at Pinterest With Finer Grained Access Control on Kubernetes

Learn how Pinterest implemented Finer Grained Access Control (FGAC) in their Kubernetes-based data processing platform called Moka to secure massive-scale data operations. Discover the architecture and design decisions behind securing a platform that runs nearly 90,000 jobs on 20,000 nodes while processing 200PB of data daily for machine learning models, user insights, and data lakes. Explore how FGAC integrates Kubernetes and AWS features including namespaces, sidecars, service accounts, RBAC, STS, EKS, and IRSA to authenticate with internal services through servicemesh, mTLS, and IAM proxy for creating a secure multi-tenant environment. Understand the implementation details for supporting Spark, Ray, and Flink workloads while meeting evolving regulatory requirements and maintaining least-privileged access principles. Examine the trade-offs and design decisions that resulted in improved data isolation, better scalability, enhanced resource utilization, and a simpler overall approach compared to Pinterest's previous Hadoop/Kerberos-based solution, along with insights into their current migration status from the legacy system.