Building a High-Performance Binary Serialization Format with In-Place Modification

Learn to design and implement a high-performance binary serialization format with in-place modification capabilities in this 55-minute conference talk from CppCon 2025. Explore the landscape of existing binary serialization formats including protobufs, Thrift, Avro, MessagePack, flatbuffers, BSON, and CBOR, understanding their differences, performance characteristics, and trade-offs between codegen vs non-codegen approaches, schema-based vs schemaless designs, and zero-copy implementations. Discover the development process of a custom schema-less data format that supports arrays and maps while enabling efficient in-place mutations without re-encoding entire buffers, essentially creating a compact binary equivalent of JSON. Master the implementation of binary serialization using tags and values, arrays through offset tables and indirection, SIMD optimization for fast offset table searches, and complex data structures in contiguous buffers for map implementations. Understand techniques for minimizing copies during in-place updates and gain insights from comprehensive benchmarking against other serialization formats and standard library containers. Develop an ergonomic C++ interface that allows users to interact with encoded data similarly to standard vectors and maps, while exploring the fundamental trade-offs inherent in high-performance serialization design. Gain valuable knowledge applicable to data structures and high-performance computing beyond serialization, presented by Hamish Morrison, a software engineer at Bloomberg specializing in high-performance market data processing systems.