Handling Text from Around the World in Go

Explore the intricacies of handling text from around the world in Go with Marcel van Lohuizen's GopherCon 2016 talk. Dive into Go's native UTF-8 support, string models, and normal forms. Learn about sequential text processing, character iteration, and the Transformer interface. Discover language identification techniques, language tags, and matching algorithms. Examine examples of language matching in Go, custom locale-specific data, and string comparison methods. Investigate the x/text/search and x/text/collate packages for advanced text operations. Gain insights into secure string comparisons, translation insertion, and best practices for internationalizing Go code.

Syllabus

Intro
All Native Go
Go and UTF-8
String Model
Normal Forms
No Random Access
Sequential nature of text
Iterate over Characters
Transformer Interface
Using Transformers
Package cases
Chaining Transforms
Language Identification
Language Tags
Matching is Non-Trivial • Swiss German speakers usually understand German gsw=de
Language Matching in Go
Example language matching
Custom locale-specific data var matcher = language. NewMatcher |language. Tag
Comparing strings
x/text/search Example SEARCH
x/text/collate Example
Secure comparison
Translation Insertion
Translating Text
Insert Translations in Code
Conclusion