Overview

Train more powerful models with a single GPU, learn how hardware can speed up model training and the key considerations when training models on a GPU.

Train more powerful models with a single GPU. In this course, you will learn how hardware can speed up model training and the key considerations when training models on a GPU. First, you will learn how to estimate the number of computations and the amount of computer memory required to train large neural networks. You will then discover techniques for reducing the computing and memory requirements when training a model. Techniques which you will apply for fine-tuning a Gemma model with 4 billion parameters. Finally, you will consider the potential environmental impacts of machine learning, with a focus on where questions of energy, water, and e-waste intersect with justice and equity.

Syllabus

Introduction

In this module, you will learn about specialized hardware for training neural network models, called Graphical Processing Units (GPUs). You will explore the tradeoff between model efficiency, that is, how fast a model can be trained and make predictions, and performance, that is, how well a model can solve a task. You will see that models with more parameters generally work better but can are also be slower and require more computer memory. You will also map the stakeholders affected by the potential environmental impacts of AI such as energy use, water consumption, and e-waste in order to see how different groups experience both risks and potential benefits. This exercise will help you understand why environmental justice in AI requires considering diverse perspectives, from local communities to developers, policymakers, and future generations.

Compute

In this module, you will discover which computations are performed when you train a model on a GPU and which computations are performed when you perform inference. You will learn how computers represent numbers and how changing the number representation affects computations and computer memory requirements. You will explore techniques for reducing the computational effort without any or very little reduction in model performance. You will also explore your own role as a developer by mapping your carbon footprint, the positive impact you can make through choices that reduce energy use and resource consumption in the AI pipeline. This reflection will help you see how everyday technical decisions, like model size or training location, connect directly to broader goals of sustainability and environmental justice.

GPU memory

In this module, you will explore the details around memory when training models and performing inference on a GPU. You will learn how to estimate how much computer memory you need for training a specific model. You will then experiment with and apply methods for decreasing memory requirements, such as representing numbers as bfloat16.

Other considerations

In this module, you will gain an overview of more advanced techniques for reducing memory requirements. You will learn about gradient accumulation and how this can be used as an alternative to larger batch sizes, and you will apply this technique for fine-tuning a model with 4 billion parameters.

Challenge

In this module, you will reflect on how AI’s energy demands intersect with questions of access, equity, and environmental justice in Africa, weighing both potential benefits and risks. In the challenge activity, you will build on this reflection by designing a sustainability plan for your own LLM project, ensuring it aligns with principles of fairness, accountability, and energy justice.

Continue your journey

In this module, you will have the opportunity to consult additional resources and further reading to investigate the topics you have covered in more detail. Finally, you will consider your next steps and how you can build on what you have learned in the course.