Why I Wrote My Own Rsync - Michael Stapelberg - Media.CCC.De

Discover the motivations and process behind creating a custom rsync implementation in Go during this conference talk. Learn about the inner workings of rsync, including its history, protocol, and the famous "rsync algorithm." Explore the speaker's journey in replacing C software with Go programs for enhanced performance, security, and learning. Gain insights into router7, a home internet router project, and gokrazy, a Go appliance platform for Raspberry Pi. Examine resource usage visualizations, project status updates, and real-world use cases for the custom rsync implementation. Understand the approach to tackling open-ended projects and the benefits of reimagining established tools in modern programming languages.

Syllabus

Intro
rsync: a fast, versatile, remote file-copying tool
rsync: history
compact file list entry on the wire
how does rsync work? (protocol version 27)
hash search ("the rsync algorithm")
visualization: resource usage
visualization resource usage sync receiver
Why did I write my own rsync?
fiber7 hardware upgrade
custom-built PC with 25 Gbit/s network card
router7 is built on gokrazy
gokrazy case study: scan drive
gokrazy case study: scandrive
router7: architecture
router7 server merge
prototyping: how I ran tridge rsyncd on gokrazy
better rsync on gokrazy
gokrazy/rsync: project status: functionality
gokrazy/rsync: project status: performance
Use case: distri Linux
Use case: scan drive backup: gokrazy packer
Use case: scan drive backup: flags and config
Use case: scan2drive backup: flags and config
Use case: scan drive backup: SSH + rsync
how does one approach such open-ended projects?
Thank you for your attention!