Deep Learning and NLP with Spark

Explore deep learning and natural language processing with Apache Spark in this Scala Days conference talk. Dive into the implementation of a Spark-ready Long Short-Term Memory (LSTM) neural network, a complex deep learning model used for challenging NLP tasks like automatic summarization, machine translation, and question answering. Learn about machine learning fundamentals, the intricacies of deep learning, and the challenges of scalability. Discover how to leverage Spark Notebook and Spark Streaming for interactive, real-time visualizations. Follow along as the speaker demonstrates practical examples, including word prediction, auto-generated Shakespeare text, and sentiment classification. Gain insights into data parallelization, back propagation, and the integration of deep learning frameworks like DL4J with Spark. Understand the structure of recurrent neural networks and their applications in NLP. Get hands-on experience with importing models, creating data structures, and performing training and prediction tasks within the JVM environment.