Stanford CS149 - Parallel Computing 2023

Stanford University via YouTube

Start learning Write review

Details

Start learning

Provider

YouTube
Pricing

Free Video
Languages

English
Duration & workload

1 day 19 minutes
Sessions

On-Demand
Level

Advanced

Found in

Explore the fundamental principles and engineering trade-offs of modern parallel computing systems through this comprehensive Stanford University course taught by Kayvon Fatahalian and Kunle Olukotun. Master both parallel hardware and software design concepts essential for understanding ubiquitous parallel processing found in smartphones, multi-core CPUs, GPUs, supercomputers, and large-scale websites. Begin with foundational concepts including why parallelism and efficiency matter, then dive deep into modern multi-core processor architecture and ISPC programming abstractions. Learn parallel programming basics, performance optimization techniques focusing on work distribution, scheduling, locality, communication, and contention management. Develop expertise in GPU architecture and CUDA programming while mastering data-parallel thinking and distributed data-parallel computing using Spark. Gain practical knowledge of efficiently evaluating deep neural networks on GPUs and understand critical concepts like cache coherence, memory consistency, and fine-grained synchronization including lock-free programming. Explore advanced topics such as domain-specific programming languages, transactional memory systems, and hardware specialization for parallel computing. Access hands-on programming experience with modern parallel programming techniques necessary to effectively utilize parallel machines, while building a deep understanding of key machine performance characteristics that inform good parallel program design.

Syllabus

Stanford CS149 I Parallel Computing I 2023 I Lecture 1 - Why Parallelism? Why Efficiency?
Stanford CS149 I Parallel Computing I 2023 I Lecture 2 - A Modern Multi-Core Processor
Stanford CS149 I 2023 I Lecture 3 - Multi-core Arch Part II + ISPC Programming Abstractions
Stanford CS149 I Parallel Computing I 2023 I Lecture 4 - Parallel Programming Basics
Stanford CS149 I 2023 I Lecture 5 - Performance Optimization I: Work Distribution and Scheduling
Stanford CS149 I Lecture 6 - Performance Optimization II: Locality, Communication, and Contention
Stanford CS149 I Parallel Computing I 2023 I Lecture 7 - GPU architecture and CUDA Programming
Stanford CS149 I Parallel Computing I 2023 I Lecture 8 - Data-Parallel Thinking
Stanford CS149 I 2023 I Lecture 9 - Distributed Data-Parallel Computing Using Spark
Stanford CS149 I Parallel Computing I 2023 I Lecture 10 - Efficiently Evaluating DNNs on GPUs
Stanford CS149 I Parallel Computing I 2023 I Lecture 11 - Cache Coherence
Stanford CS149 I Parallel Computing I 2023 I Lecture 12 - Memory Consistency
Stanford CS149 I 2023 I Lecture 13 - Fine-Grained Synchronization and Lock-Free Programming
Stanford CS149 I Parallel Computing I 2023 I Lecture 14 - Midterm Review
Stanford CS149 I Parallel Computing I 2023 I Lecture 15 - Domain Specific Programming Languages
Stanford CS149 I Parallel Computing I 2023 I Lecture 16 - Transactional Memory 1
Stanford CS149 I Parallel Computing I 2023 I Lecture 17 - Transactional Memory 2
Stanford CS149 I Parallel Computing I 2023 I Lecture 18 - Hardware Specialization
Stanford CS149 I Parallel Computing I 2023 I Lecture 19 - Accessing Memory + Course Wrap Up

Taught by

Stanford Online

Reviews

5.0 rating, based on 3 Class Central reviews

Start your review of Stanford CS149 - Parallel Computing 2023

ARJUN VASHISHTHA 23BCE11644

Very Nice course,I recently completed the course and was thoroughly impressed by the depth and structure of the material. The instructor was incredibly knowledgeable, breaking down complex topics into digestible, engaging modules that made learning both fun and practical.
What I appreciated most was the balance between theory and real-world application, allowing me to immediately apply new techniques to my daily work. The course materials were well-organized, and the platform was easy to navigate. This has been a worthwhile investment for my professional development, and I highly recommend it to anyone looking to improve their skills in this field.
Shriyam Rastogi

CS149 isn't just about learning syntax; it’s a fundamental shift in how you think about problem-solving. In most classes, you care about $O(n)$ vs $O(n^2)$. Here, you care about why your $O(n)$ algorithm is crawling because of a cache miss or false sharing.Workload: High. Expect to spend significant hours in the "basement" (or your remote equivalent) debugging race conditions that only appear once every 1,000 runs.Prerequisites: You must be comfortable with C++. If you’re still shaky on pointers or memory management, this course will be a trial by fire.
Suchit Chavan

I completed the Stanford CS149 Parallel Computing course and found it very useful for understanding how programs can run faster using parallel execution. The course explains important ideas like parallelism, threads, synchronization, and performance…

I completed the Stanford CS149 Parallel Computing course and found it very useful for understanding how programs can run faster using parallel execution. The course explains important ideas like parallelism, threads, synchronization, and performance improvement in a clear and step-by-step way. The instructors use practical examples that help in understanding how parallel code works in real systems. Even though the topics are advanced, the explanations are simple and easy to follow if you have basic knowledge of programming. This course also helped me think differently about writing efficient code and using system resources properly. Overall, it is a good course for students who want to learn parallel computing and improve their technical skills.