Reordering 59 Million NYT Publishing Assets Using Go and BadgerDB

Explore the process of reordering 59 million New York Times publishing assets using Go and BadgerDB in this GopherCon 2020 talk. Discover how to chronologically and topologically organize every version of NYT articles since 1851, along with related assets like images and tags. Learn about encoding assets, inserting them into databases, resolving missing references, and removing duplicates. Gain insights into Go development philosophy, BadgerDB performance optimization, and practical techniques such as string enumeration, timestamp handling, and error management in goroutines.

Syllabus

Intro
My Background
T Publishing Assets
Encode Assets and Insert Into Database
Find and Resolve Missing References
Find and Remove Duplicates
T Development Philosophy
Fast Performance (v 1.6.1)
I'm New to Go
T Badger Wrapper
T Batch DB Wrapper
TEncoding Strings via Enumeration
T Marshal - Read
T Timestamps as Keys
Refresher: Endianness
Endianness - Sorting
T Unique Timestamp
TUnique Timestamp Cache
T Panic Error Handling - Goroutines