Efficient Programming

Overview

This course is targeted to scientists, engineers, scholars, or anyone seeking to solve problems efficiently in high-performance computing environments or in the cloud. Students completing this course will have a basic understanding of how to find bottlenecks in their programs as well as how to address those bottlenecks. The course will provide a high-level introduction to modern compute node architectures of high-performance and cloud computing instances. This course can be taken for academic credit as part of CU Boulder’s Master of Science in Data Science (MS-DS) degree offered on the Coursera platform. The MS-DS is an interdisciplinary degree that brings together faculty from CU Boulder’s departments of Applied Mathematics, Computer Science, Information Science, and others. With performance-based admissions and no application process, the MS-DS is ideal for individuals with a broad range of undergraduate education and/or professional experience in computer science, information science, mathematics, and statistics. Learn more about the MS-DS program at https://www.coursera.org/degrees/master-of-science-data-science-boulder.

Syllabus

Finding Performance Bottlenecks

In this module, we cover an approach to analyze and optimize program performance, such as profiling, using optimized libraries, and compiler options for increasing efficiency.

Simple Optimization Techniques

In this module, we examine simple techniques that help with program performance. We are looking at scalar and loop optimization methods that can have a large impact on a program’s floating-point performance.

Computer Architecture and Vectorization

In this module, we introduce the basic architecture of modern computers focusing on how the architecture influences program performance. We are looking at processor level data parallelism and how optimized code for parallelism has a much increased floating-point performance.

Computer Architecture

Memory performance is generally the main performance bottleneck since the speed of the main memory has not kept up with the capabilities of processors to process floating-point numbers. We introduce how layers of fast memory, called cache memory, can speed up computations and provide an example of how to optimize algorithms for better memory performance.

Parallel and High Throughput Computing

This module will provide an introduction to parallel and high throughput computing. It will also demonstrate slurm job arrays, where there are mechanisms for working with many similar jobs quickly and easily. Finally, this module will look at running many jobs concurrently with GNU Parallel.