Stanford CS149 - Parallel Computing 2023

Stanford University via YouTube

Start learning Write review

Explore the fundamental principles and engineering trade-offs of modern parallel computing systems through this comprehensive Stanford University course taught by Kayvon Fatahalian and Kunle Olukotun. Master both parallel hardware and software design concepts essential for understanding ubiquitous parallel processing found in smartphones, multi-core CPUs, GPUs, supercomputers, and large-scale websites. Begin with foundational concepts including why parallelism and efficiency matter, then dive deep into modern multi-core processor architecture and ISPC programming abstractions. Learn parallel programming basics, performance optimization techniques focusing on work distribution, scheduling, locality, communication, and contention management. Develop expertise in GPU architecture and CUDA programming while mastering data-parallel thinking and distributed data-parallel computing using Spark. Gain practical knowledge of efficiently evaluating deep neural networks on GPUs and understand critical concepts like cache coherence, memory consistency, and fine-grained synchronization including lock-free programming. Explore advanced topics such as domain-specific programming languages, transactional memory systems, and hardware specialization for parallel computing. Access hands-on programming experience with modern parallel programming techniques necessary to effectively utilize parallel machines, while building a deep understanding of key machine performance characteristics that inform good parallel program design.

Syllabus

Stanford CS149 I Parallel Computing I 2023 I Lecture 1 - Why Parallelism? Why Efficiency?
Stanford CS149 I Parallel Computing I 2023 I Lecture 2 - A Modern Multi-Core Processor
Stanford CS149 I 2023 I Lecture 3 - Multi-core Arch Part II + ISPC Programming Abstractions
Stanford CS149 I Parallel Computing I 2023 I Lecture 4 - Parallel Programming Basics
Stanford CS149 I 2023 I Lecture 5 - Performance Optimization I: Work Distribution and Scheduling
Stanford CS149 I Lecture 6 - Performance Optimization II: Locality, Communication, and Contention
Stanford CS149 I Parallel Computing I 2023 I Lecture 7 - GPU architecture and CUDA Programming
Stanford CS149 I Parallel Computing I 2023 I Lecture 8 - Data-Parallel Thinking
Stanford CS149 I 2023 I Lecture 9 - Distributed Data-Parallel Computing Using Spark
Stanford CS149 I Parallel Computing I 2023 I Lecture 10 - Efficiently Evaluating DNNs on GPUs
Stanford CS149 I Parallel Computing I 2023 I Lecture 11 - Cache Coherence
Stanford CS149 I Parallel Computing I 2023 I Lecture 12 - Memory Consistency
Stanford CS149 I 2023 I Lecture 13 - Fine-Grained Synchronization and Lock-Free Programming
Stanford CS149 I Parallel Computing I 2023 I Lecture 14 - Midterm Review
Stanford CS149 I Parallel Computing I 2023 I Lecture 15 - Domain Specific Programming Languages
Stanford CS149 I Parallel Computing I 2023 I Lecture 16 - Transactional Memory 1
Stanford CS149 I Parallel Computing I 2023 I Lecture 17 - Transactional Memory 2
Stanford CS149 I Parallel Computing I 2023 I Lecture 18 - Hardware Specialization
Stanford CS149 I Parallel Computing I 2023 I Lecture 19 - Accessing Memory + Course Wrap Up

Taught by

Stanford Online

Reviews

4.8 rating, based on 17 Class Central reviews

Start your review of Stanford CS149 - Parallel Computing 2023

VIJENDHAR M 717824F159

Stanford CS149: Parallel Computing (2023) by Stanford University is an outstanding advanced-level course that provides a deep understanding of parallel programming and modern computer architecture. Taught by Professors Kayvon Fatahalian and Kunle Ol…

Stanford CS149: Parallel Computing (2023) by Stanford University is an outstanding advanced-level course that provides a deep understanding of parallel programming and modern computer architecture. Taught by Professors Kayvon Fatahalian and Kunle Olukotun, the course covers essential topics such as multicore processors, SIMD, GPU architecture, CUDA programming, cache coherence, memory consistency, synchronization, and performance optimization. The lectures are well-structured, combining theoretical concepts with practical examples and challenging programming assignments that help learners build real-world parallel computing skills. Although the course is demanding and best suited for students with a background in C/C++ and computer architecture, it is widely regarded as one of the best free resources for learning high-performance computing, GPU programming, and efficient software design. Overall, it is highly recommended for computer science students, systems programmers, and AI engineers who want to understand how modern hardware executes parallel programs and how to write high-performance code.
PRABAKARAN P

The course provided a well-structured and engaging learning experience that helped build a strong understanding of the subject. The content was organized logically, starting from basic concepts and gradually moving toward more advanced topics, makin…

The course provided a well-structured and engaging learning experience that helped build a strong understanding of the subject. The content was organized logically, starting from basic concepts and gradually moving toward more advanced topics, making it easy to follow even for beginners. The explanations were clear and supported by practical examples, which made complex ideas easier to grasp. Assignments and quizzes were useful in reinforcing the concepts and encouraged consistent learning. The instructor demonstrated good knowledge and explained topics in a simple and effective manner. One of the best aspects of the course was its focus on real-world applications, which made the learning more relevant and interesting. Overall, this course is highly recommended for anyone looking to strengthen their knowledge and gain practical insights in this field
MIRACLIN GIFTY S 717824F131

The Parallel Distributed Computing course provided a strong understanding of parallel algorithms, distributed systems, and high-performance computing concepts. The course effectively explained topics such as process synchronization, message passing,…

The Parallel Distributed Computing course provided a strong understanding of parallel algorithms, distributed systems, and high-performance computing concepts. The course effectively explained topics such as process synchronization, message passing, load balancing, and fault tolerance through practical examples and programming assignments. Hands-on experience with parallel programming tools improved problem-solving and coding skills. The instructor presented complex concepts clearly, making them easier to understand and apply. Overall, the course enhanced my knowledge of modern computing architectures and prepared me to design efficient, scalable applications for real-world problems. It is highly valuable for students interested in advanced computing and system design.
Mansi Kapse - 23BCE10627

The course provided a well-structured and engaging learning experience that helped build a strong understanding of the subject. The content was organized logically, starting from basic concepts and gradually moving toward more advanced topics, makin…

The course provided a well-structured and engaging learning experience that helped build a strong understanding of the subject. The content was organized logically, starting from basic concepts and gradually moving toward more advanced topics, making it easy to follow even for beginners. The explanations were clear and supported by practical examples, which made complex ideas easier to grasp. Assignments and quizzes were useful in reinforcing the concepts and encouraged consistent learning. The instructor demonstrated good knowledge and explained topics in a simple and effective manner. One of the best aspects of the course was its focus on real-world applications, which made the learning more relevant and interesting. Overall, this course is highly recommended for anyone looking to strengthen their knowledge and gain practical insights in this field.
KHAVEENAA G S 717824F230

Good .It is useful for my studies .I will use it for my further study purpous also. It is useful to inprove my skills .
DEEPIKA B 717824F214

Good! I understood all the concepts clearly. The videos were clear and understandable. This course helped me to explore new concepts and new technology .
ROHITH R

This is very useful to me and the teaching method also easy for learning both beginners and experienced ones .
Saksham Kohli 23bce10558

Stanford CS149 is an excellent course that provides a strong foundation in parallel computing and performance optimization. It covers key topics such as SIMD, multithreading, GPU computing, and memory hierarchy, helping learners understand how moder…

Stanford CS149 is an excellent course that provides a strong foundation in parallel computing and performance optimization. It covers key topics such as SIMD, multithreading, GPU computing, and memory hierarchy, helping learners understand how modern hardware executes programs efficiently. The lectures are clear and well-structured, making complex concepts easier to grasp.

The assignments are challenging but highly practical, allowing students to implement parallel algorithms and optimize performance. However, beginners may find the course demanding, especially without prior knowledge of C/C++ or computer architecture.

Overall, CS149 is a highly valuable course for anyone interested in systems programming, high-performance computing, or AI. It enhances problem-solving skills and develops a deeper understanding of efficient computation.
SANOFAR NISHA S 717824F247

Well-structured course with clear guidance, practical tasks, and engaging lessons throughout my learning
Munish Upadhyay 23bce10925

This course on Parallel and Distributed Computing provides a strong foundation in both theoretical concepts and practical implementation. It clearly explains key topics such as concurrency, synchronization, parallel architectures, and distributed sy…

This course on Parallel and Distributed Computing provides a strong foundation in both theoretical concepts and practical implementation. It clearly explains key topics such as concurrency, synchronization, parallel architectures, and distributed system design. The inclusion of real-world examples and hands-on exercises makes complex ideas easier to understand. I particularly appreciated the focus on performance optimization and scalability, which are crucial in modern computing. However, some advanced topics could have been explored in greater depth. Overall, it is a well-structured and valuable course for anyone looking to build skills in high-performance and distributed systems, especially for careers in cloud computing and large-scale application development.
Yoshita Upadhyay 23BCE11822

This course offered a clear and engaging introduction to parallel and distributed computing concepts. Complex ideas were broken down into simple, understandable sections, making it easier to grasp core principles. The structured approach helped buil…

This course offered a clear and engaging introduction to parallel and distributed computing concepts. Complex ideas were broken down into simple, understandable sections, making it easier to grasp core principles. The structured approach helped build confidence while exploring topics like concurrency, scalability, and performance. Practical insights and examples made the learning experience more meaningful and applicable. It also encouraged critical thinking about real-world computing challenges. Overall, the course was well-organized, informative, and highly beneficial for strengthening foundational knowledge. It has significantly improved my understanding and sparked a deeper interest in advanced computing systems and their applications in solving modern problems.
Suchit Chavan

I completed the Stanford CS149 Parallel Computing course and found it very useful for understanding how programs can run faster using parallel execution. The course explains important ideas like parallelism, threads, synchronization, and performance…

I completed the Stanford CS149 Parallel Computing course and found it very useful for understanding how programs can run faster using parallel execution. The course explains important ideas like parallelism, threads, synchronization, and performance improvement in a clear and step-by-step way. The instructors use practical examples that help in understanding how parallel code works in real systems. Even though the topics are advanced, the explanations are simple and easy to follow if you have basic knowledge of programming. This course also helped me think differently about writing efficient code and using system resources properly. Overall, it is a good course for students who want to learn parallel computing and improve their technical skills.
Akhil Astitva

I am extremely happy to get this course done without any issue.
I got to learn each and every topic about The Parallel And Distributive Computing very deeply.They teached us in a very effictive way to get the concepts in a blink of an eye.
The script of the subject seems very easy to understand and anyone new to this course will surely gonna love the way the teacher teaches.
ARJUN VASHISHTHA 23BCE11644

Very Nice course,I recently completed the course and was thoroughly impressed by the depth and structure of the material. The instructor was incredibly knowledgeable, breaking down complex topics into digestible, engaging modules that made learning both fun and practical.
What I appreciated most was the balance between theory and real-world application, allowing me to immediately apply new techniques to my daily work. The course materials were well-organized, and the platform was easy to navigate. This has been a worthwhile investment for my professional development, and I highly recommend it to anyone looking to improve their skills in this field.
Shriyam Rastogi

CS149 isn't just about learning syntax; it’s a fundamental shift in how you think about problem-solving. In most classes, you care about $O(n)$ vs $O(n^2)$. Here, you care about why your $O(n)$ algorithm is crawling because of a cache miss or false sharing.Workload: High. Expect to spend significant hours in the "basement" (or your remote equivalent) debugging race conditions that only appear once every 1,000 runs.Prerequisites: You must be comfortable with C++. If you’re still shaky on pointers or memory management, this course will be a trial by fire.
Kalpaang Sen

It is good course to learn Parallel Computing. It provides students with video tutorials and help them understand the basic concept of Parallel and Distributed Computing
Lakshya Tatoo 23bce11403

Knowledge fruitful experience that was very interesting and learned asking experience.
It helped me understand more about the parallel computing