Stanford CS149 - Parallel Computing 2023

Stanford University via YouTube

Start learning Write review

Details

Start learning

Provider

YouTube
Pricing

Free Video
Languages

English
Effort

1 day 19 minutes
Sessions

Self-Paced
Level

Advanced

Found in

Explore the fundamental principles and engineering trade-offs of modern parallel computing systems through this comprehensive Stanford University course taught by Kayvon Fatahalian and Kunle Olukotun. Master both parallel hardware and software design concepts essential for understanding ubiquitous parallel processing found in smartphones, multi-core CPUs, GPUs, supercomputers, and large-scale websites. Begin with foundational concepts including why parallelism and efficiency matter, then dive deep into modern multi-core processor architecture and ISPC programming abstractions. Learn parallel programming basics, performance optimization techniques focusing on work distribution, scheduling, locality, communication, and contention management. Develop expertise in GPU architecture and CUDA programming while mastering data-parallel thinking and distributed data-parallel computing using Spark. Gain practical knowledge of efficiently evaluating deep neural networks on GPUs and understand critical concepts like cache coherence, memory consistency, and fine-grained synchronization including lock-free programming. Explore advanced topics such as domain-specific programming languages, transactional memory systems, and hardware specialization for parallel computing. Access hands-on programming experience with modern parallel programming techniques necessary to effectively utilize parallel machines, while building a deep understanding of key machine performance characteristics that inform good parallel program design.

Syllabus

Stanford CS149 I Parallel Computing I 2023 I Lecture 1 - Why Parallelism? Why Efficiency?
Stanford CS149 I Parallel Computing I 2023 I Lecture 2 - A Modern Multi-Core Processor
Stanford CS149 I 2023 I Lecture 3 - Multi-core Arch Part II + ISPC Programming Abstractions
Stanford CS149 I Parallel Computing I 2023 I Lecture 4 - Parallel Programming Basics
Stanford CS149 I 2023 I Lecture 5 - Performance Optimization I: Work Distribution and Scheduling
Stanford CS149 I Lecture 6 - Performance Optimization II: Locality, Communication, and Contention
Stanford CS149 I Parallel Computing I 2023 I Lecture 7 - GPU architecture and CUDA Programming
Stanford CS149 I Parallel Computing I 2023 I Lecture 8 - Data-Parallel Thinking
Stanford CS149 I 2023 I Lecture 9 - Distributed Data-Parallel Computing Using Spark
Stanford CS149 I Parallel Computing I 2023 I Lecture 10 - Efficiently Evaluating DNNs on GPUs
Stanford CS149 I Parallel Computing I 2023 I Lecture 11 - Cache Coherence
Stanford CS149 I Parallel Computing I 2023 I Lecture 12 - Memory Consistency
Stanford CS149 I 2023 I Lecture 13 - Fine-Grained Synchronization and Lock-Free Programming
Stanford CS149 I Parallel Computing I 2023 I Lecture 14 - Midterm Review
Stanford CS149 I Parallel Computing I 2023 I Lecture 15 - Domain Specific Programming Languages
Stanford CS149 I Parallel Computing I 2023 I Lecture 16 - Transactional Memory 1
Stanford CS149 I Parallel Computing I 2023 I Lecture 17 - Transactional Memory 2
Stanford CS149 I Parallel Computing I 2023 I Lecture 18 - Hardware Specialization
Stanford CS149 I Parallel Computing I 2023 I Lecture 19 - Accessing Memory + Course Wrap Up

Taught by

Stanford Online

Reviews

4.7 rating, based on 24 Class Central reviews

Start your review of Stanford CS149 - Parallel Computing 2023

Jonard Doctolero

The lecture content and visual explanations of parallel concepts are top-tier, making complex topics like multi-threading, cache coherency, and GPU architecture genuinely intuitive to grasp. However, the viewing experience on YouTube comes with nota…

The lecture content and visual explanations of parallel concepts are top-tier, making complex topics like multi-threading, cache coherency, and GPU architecture genuinely intuitive to grasp. However, the viewing experience on YouTube comes with notable friction. Because these videos are raw, unedited classroom recordings, the microphone picks up a frustrating amount of ambient room noise, echo, and background chatter, which makes it tiring to listen to and hard to follow along consistently. More importantly, student questions from the hall are rarely repeated on mic, leaving online viewers in the dark about what prompted certain discussions. To make matters worse, several past lectures intermittently become unavailable or get taken down by the department without clear re-uploads, which completely disrupts continuity and learning momentum for self-paced viewers trying to finish the entire course sequentially. It is a fantastic educational resource, but you definitely need a good pair of noise-isolating headphones and the official course slide decks open on the side to fill in the gaps.
VIJENDHAR M 717824F159

Stanford CS149: Parallel Computing (2023) by Stanford University is an outstanding advanced-level course that provides a deep understanding of parallel programming and modern computer architecture. Taught by Professors Kayvon Fatahalian and Kunle Ol…

Stanford CS149: Parallel Computing (2023) by Stanford University is an outstanding advanced-level course that provides a deep understanding of parallel programming and modern computer architecture. Taught by Professors Kayvon Fatahalian and Kunle Olukotun, the course covers essential topics such as multicore processors, SIMD, GPU architecture, CUDA programming, cache coherence, memory consistency, synchronization, and performance optimization. The lectures are well-structured, combining theoretical concepts with practical examples and challenging programming assignments that help learners build real-world parallel computing skills. Although the course is demanding and best suited for students with a background in C/C++ and computer architecture, it is widely regarded as one of the best free resources for learning high-performance computing, GPU programming, and efficient software design. Overall, it is highly recommended for computer science students, systems programmers, and AI engineers who want to understand how modern hardware executes parallel programs and how to write high-performance code.
PRABAKARAN P

The course provided a well-structured and engaging learning experience that helped build a strong understanding of the subject. The content was organized logically, starting from basic concepts and gradually moving toward more advanced topics, makin…

The course provided a well-structured and engaging learning experience that helped build a strong understanding of the subject. The content was organized logically, starting from basic concepts and gradually moving toward more advanced topics, making it easy to follow even for beginners. The explanations were clear and supported by practical examples, which made complex ideas easier to grasp. Assignments and quizzes were useful in reinforcing the concepts and encouraged consistent learning. The instructor demonstrated good knowledge and explained topics in a simple and effective manner. One of the best aspects of the course was its focus on real-world applications, which made the learning more relevant and interesting. Overall, this course is highly recommended for anyone looking to strengthen their knowledge and gain practical insights in this field
Paladugu Tejaswini

This course is very informative and helpful for understanding the basics of parallel computing. The lectures explain concepts clearly with practical examples and make complex topics easier to understand. It is a good learning resource for students interested in computer science and parallel computing.
MIRACLIN GIFTY S 717824F131

The Parallel Distributed Computing course provided a strong understanding of parallel algorithms, distributed systems, and high-performance computing concepts. The course effectively explained topics such as process synchronization, message passing,…

The Parallel Distributed Computing course provided a strong understanding of parallel algorithms, distributed systems, and high-performance computing concepts. The course effectively explained topics such as process synchronization, message passing, load balancing, and fault tolerance through practical examples and programming assignments. Hands-on experience with parallel programming tools improved problem-solving and coding skills. The instructor presented complex concepts clearly, making them easier to understand and apply. Overall, the course enhanced my knowledge of modern computing architectures and prepared me to design efficient, scalable applications for real-world problems. It is highly valuable for students interested in advanced computing and system design.
Leo John

Masterly, rigorous lectures demystifying GPU hardware, cache coherence, multithreading, and parallel software design.
Jessie Lei Reynancia

With the help of class central, I review and somewhat understand how parallel computing works and its terms. It also help me to review for my upcoming exam in parallel computing.
Chiarra Marie D. Rivera

I learned enough how parallel computing. It was quite confusing at first so i have to keep rewatching but i understand it now. The videos are really helpful.
Josh Ashley Untalan

helped me review and understand parallel computing a bit better, especially the common terms and how it works in general.
SHRIYA SUBRAMANI

This is a well-structured and informative course on parallel computing. The lectures explain key concepts clearly with practical examples. It is suitable for students who want to understand parallel programming and improve their problem-solving skills. Highly recommended.
Mansi Kapse - 23BCE10627

The course provided a well-structured and engaging learning experience that helped build a strong understanding of the subject. The content was organized logically, starting from basic concepts and gradually moving toward more advanced topics, makin…

The course provided a well-structured and engaging learning experience that helped build a strong understanding of the subject. The content was organized logically, starting from basic concepts and gradually moving toward more advanced topics, making it easy to follow even for beginners. The explanations were clear and supported by practical examples, which made complex ideas easier to grasp. Assignments and quizzes were useful in reinforcing the concepts and encouraged consistent learning. The instructor demonstrated good knowledge and explained topics in a simple and effective manner. One of the best aspects of the course was its focus on real-world applications, which made the learning more relevant and interesting. Overall, this course is highly recommended for anyone looking to strengthen their knowledge and gain practical insights in this field.
Saksham Kohli 23bce10558

Stanford CS149 is an excellent course that provides a strong foundation in parallel computing and performance optimization. It covers key topics such as SIMD, multithreading, GPU computing, and memory hierarchy, helping learners understand how moder…

Stanford CS149 is an excellent course that provides a strong foundation in parallel computing and performance optimization. It covers key topics such as SIMD, multithreading, GPU computing, and memory hierarchy, helping learners understand how modern hardware executes programs efficiently. The lectures are clear and well-structured, making complex concepts easier to grasp.

The assignments are challenging but highly practical, allowing students to implement parallel algorithms and optimize performance. However, beginners may find the course demanding, especially without prior knowledge of C/C++ or computer architecture.

Overall, CS149 is a highly valuable course for anyone interested in systems programming, high-performance computing, or AI. It enhances problem-solving skills and develops a deeper understanding of efficient computation.
KHAVEENAA G S 717824F230

Good .It is useful for my studies .I will use it for my further study purpous also. It is useful to inprove my skills .
DEEPIKA B 717824F214

Good! I understood all the concepts clearly. The videos were clear and understandable. This course helped me to explore new concepts and new technology .
Munish Upadhyay 23bce10925

This course on Parallel and Distributed Computing provides a strong foundation in both theoretical concepts and practical implementation. It clearly explains key topics such as concurrency, synchronization, parallel architectures, and distributed sy…

This course on Parallel and Distributed Computing provides a strong foundation in both theoretical concepts and practical implementation. It clearly explains key topics such as concurrency, synchronization, parallel architectures, and distributed system design. The inclusion of real-world examples and hands-on exercises makes complex ideas easier to understand. I particularly appreciated the focus on performance optimization and scalability, which are crucial in modern computing. However, some advanced topics could have been explored in greater depth. Overall, it is a well-structured and valuable course for anyone looking to build skills in high-performance and distributed systems, especially for careers in cloud computing and large-scale application development.
Yoshita Upadhyay 23BCE11822

This course offered a clear and engaging introduction to parallel and distributed computing concepts. Complex ideas were broken down into simple, understandable sections, making it easier to grasp core principles. The structured approach helped buil…

This course offered a clear and engaging introduction to parallel and distributed computing concepts. Complex ideas were broken down into simple, understandable sections, making it easier to grasp core principles. The structured approach helped build confidence while exploring topics like concurrency, scalability, and performance. Practical insights and examples made the learning experience more meaningful and applicable. It also encouraged critical thinking about real-world computing challenges. Overall, the course was well-organized, informative, and highly beneficial for strengthening foundational knowledge. It has significantly improved my understanding and sparked a deeper interest in advanced computing systems and their applications in solving modern problems.
ROHITH R

This is very useful to me and the teaching method also easy for learning both beginners and experienced ones .
SANOFAR NISHA S 717824F247

Well-structured course with clear guidance, practical tasks, and engaging lessons throughout my learning
Akhil Astitva

I am extremely happy to get this course done without any issue.
I got to learn each and every topic about The Parallel And Distributive Computing very deeply.They teached us in a very effictive way to get the concepts in a blink of an eye.
The script of the subject seems very easy to understand and anyone new to this course will surely gonna love the way the teacher teaches.
Suchit Chavan

I completed the Stanford CS149 Parallel Computing course and found it very useful for understanding how programs can run faster using parallel execution. The course explains important ideas like parallelism, threads, synchronization, and performance…

I completed the Stanford CS149 Parallel Computing course and found it very useful for understanding how programs can run faster using parallel execution. The course explains important ideas like parallelism, threads, synchronization, and performance improvement in a clear and step-by-step way. The instructors use practical examples that help in understanding how parallel code works in real systems. Even though the topics are advanced, the explanations are simple and easy to follow if you have basic knowledge of programming. This course also helped me think differently about writing efficient code and using system resources properly. Overall, it is a good course for students who want to learn parallel computing and improve their technical skills.