Structure-Sensitive Dependency Learning in Recurrent Neural Networks - 2017

Explore the capabilities of recurrent neural networks (RNNs) in learning structure-sensitive dependencies from natural language corpora, focusing on English subject-verb number agreement. Delve into Tal Linzen's research examining LSTMs' ability to predict verb number in various sentence types, analyzing their internal representations and comparing their performance to human agreement attraction errors. Discover how the networks approximate syntactic structure in common sentences but struggle with complex constructions, highlighting the need for stronger inductive biases. Learn about the potential of multi-task learning to address these limitations and gain insights into using linguistic and psycholinguistic methods to evaluate "black-box" neural network models. This hour-long lecture, delivered by Assistant Professor Tal Linzen from Johns Hopkins University, offers valuable perspectives on the intersection of cognitive science, linguistics, and artificial intelligence in natural language processing.