Text Processing in APL - Data-Parallel and Idiomatic Parsing Techniques

Explore text processing capabilities in APL through this 25-minute conference talk from Dyalog '22. Dive into parsing techniques, architectural considerations, and handling text-oriented applications. Learn about structured models for parsing tasks, limitations of regular expressions, and recursive parsing expression grammars. Examine performance issues with recursive parsers and discover data-parallel, idiomatic parsing approaches. Follow along with practical examples including balanced parentheses parsing and variable binding. Gain insights into the new co-dfns parser and its advantages. Conclude with valuable notes on achieving flexibility, maintainability, and performance in APL text processing applications.

Syllabus

APL for processing text data
Most text parsing tasks result in a structured model such as a tree
Regular expressions regex are convenient but limited
Parsing expression grammars are recursive
Issues with a recursive parser
Performance of recursive parsers
How to do data-parallel, idiomatic parsing
Balanced parentheses example
High level overview of the new co-dfns parser
Link variables to their bindings example
Parsing strands example
Notes on flexibility, maintainability and performance