Applied Natural Language Processing in Engineering Part 1

Northeastern University via Coursera

Go to class Write review

Details

Go to class

Provider

Coursera
Pricing

Paid Course
Languages

English
Certificate

Paid Certificate Available
Duration & workload

1 day 3 hours 45 minutes
Sessions

On-Demand
Level

Intermediate
Subtitles

English

Found in

Overview

Coursera Flash Sale

40% Off Coursera Plus for 3 Months!

Grab it

Welcome to this course on applied natural language processing in engineering. This course is designed to provide you with an in-depth understanding of NLP, a pivotal area of artificial intelligence that empowers computers to comprehend, interpret, and generate human language. Throughout this course, you will explore a wide range of topics, from fundamental NLP tasks like text classification and Named Entity Recognition (NER) to advanced techniques in neural machine translation and optimization methods critical for machine learning. We will delve into the complexities of teaching language to machines, addressing challenges like ambiguity, grammar, and cultural nuances. By the end of this part 1 course, you will have a foundational understanding of how modern NLP systems work - particularly those involving machine learning and deep learning. These topics will equip you to build, analyze and improve NLP systems across many applications.

Syllabus

Introduction to Speech, Language, and Natural Language Processing

This module provides an in-depth exploration of Natural Language Processing (NLP), a crucial area of artificial intelligence that enables computers to understand, interpret, and generate human language. By combining computational linguistics with machine learning, NLP is applied in various technologies, from chatbots and sentiment analysis to machine translation and speech recognition. The module introduces fundamental NLP tasks such as text classification, Named Entity Recognition (NER), and neural machine translation, showcasing how these applications shape real-world interactions with AI. Additionally, it highlights the complexities of teaching language to machines, including handling ambiguity, grammar, and cultural nuances. Through the course, you will gain hands-on experience and knowledge about key techniques like word representation and distributional semantics, preparing them to solve language-related challenges in modern AI systems.

Gradient Descent & Optimization Techniques

This module focuses on optimization techniques critical for machine learning, particularly in natural language processing (NLP) tasks. It introduces Gradient Descent (GD), a fundamental algorithm used to minimize cost functions by iteratively adjusting model parameters. You’ll explore variants like Stochastic Gradient Descent (SGD) and Mini-Batch Gradient Descent to learn more about their efficiency in handling large datasets. Advanced methods such as Momentum and Adam are covered to give you insight on how to enhance convergence speed by smoothing updates and adapting learning rates. The module also covers second-order techniques like Newton’s Method and Quasi-Newton methods (e.g., BFGS), which leverage curvature information for more direct optimization, although they come with higher computational costs. Overall, this module emphasizes balancing efficiency, accuracy, and computational feasibility in optimizing machine learning models.

Neural Networks & Cost Functions

This module explores Named Entity Recognition (NER), a core task in Natural Language Processing (NLP) that identifies and classifies entities like people, locations, and organizations in text. We’ll begin by examining how logistic regression can be used to model NER as a binary classification problem, though this approach faces limitations with complexity and context capture. We’ll then transition to more advanced techniques, such as neural networks, which excel at handling the complex patterns and large-scale data that traditional models struggle with. Neural networks' ability to learn hierarchical features makes them ideal for NER tasks, as they can capture contextual information more effectively than simpler models. Throughout the module, we compare these methods and highlight how deep learning approaches such as Recurrent Neural Networks (RNNs) and transformers like BERT improve NER accuracy and scalability.

Embeddings, GloVe, Evaluation Techniques

The Word2Vec and GloVe models are popular word embedding techniques in Natural Language Processing (NLP), each offering unique advantages. Word2Vec, developed by Google, operates via two key models: Continuous Bag of Words (CBOW) and Skip-gram, focusing on predicting a word based on its context or vice versa (Word2Vec). The GloVe model, on the other hand, created by Stanford, combines count-based and predictive approaches by leveraging word co-occurrence matrices to learn word vectors (GloVe). Both models represent words in a high-dimensional vector space and capture semantic relationships. Word2Vec focuses on local contexts, learning efficiently from large datasets, while GloVe emphasizes global word co-occurrence patterns across the entire corpus, revealing deeper word associations. These embeddings enable tasks like analogy-solving, semantic similarity, and other linguistic computations, making them central to modern NLP applications.

Evaluation Techniques

This module delves into the evaluation techniques for Natural Language Processing (NLP) models, focusing on both intrinsic and extrinsic evaluation methods. Intrinsic evaluation assesses the model's performance based on internal criteria, such as word embedding quality, parsing accuracy, and language model perplexity. In contrast, extrinsic evaluation measures the model's effectiveness in real-world applications, including tasks like machine translation, sentiment analysis, and named entity recognition. You’ll also learn more about key differences between these evaluation types, and the importance of context and application in determining a model's utility. Additionally, you’ll review specific metrics like cross-entropy loss, perplexity, BLEU, and ROUGE scores, providing a comprehensive understanding of how to evaluate and improve NLP models.

Topic Modeling

This module explores various techniques for topic modeling in natural language processing (NLP), focusing on methods like Latent Semantic Analysis (LSA), Non-Negative Matrix Factorization (NMF), and Latent Dirichlet Allocation (LDA). It begins with an introduction to matrix factorization and the importance of transforming textual data into numerical representations. You’ll delve into the mechanics of LSA and NMF, paying attention to their use of TF-IDF and Singular Value Decomposition (SVD) to uncover latent semantic structures. Additionally, you’ll review LDA's probabilistic approach to topic modeling, explaining its reliance on Dirichlet distributions and Bayesian inference to identify hidden topics within a corpus. Through detailed examples and mathematical explanations, the module provides a comprehensive understanding of how these techniques can be applied to extract meaningful topics from large text datasets.

Dependency Parsing

This module delves into the essential techniques of syntactic and semantic parsing in natural language processing (NLP). You’ll begin with an exploration of linguistic structures, focusing on phrase structure and dependency structure, which are fundamental for understanding sentence syntax. Then you’ll review various parsing methods, including transition-based and graph-based dependency parsing, highlighting their respective advantages and challenges. Additionally, you’ll touch on neural transition-based parsing, which leverages neural networks for improved accuracy and efficiency. Finally, the module touches on semantic parsing, emphasizing its role in mapping sentences to formal representations of meaning, crucial for applications like dialogue systems and information extraction.