Complete Guide to Parallel and Concurrent Programming with C++

Introduction

• Learn parallel programming basics
• What you should know
• Installing Cygwin and Visual Studio Code

1. Parallel Computing Hardware

• Sequential vs. parallel computing
• Parallel computing architectures
• Shared versus distributed memory

2. Threads and Processes

• Thread versus process
• Thread versus process: C++ demo
• Concurrent versus parallel execution
• Execution scheduling
• Execution scheduling: C++ demo
• Thread life cycle
• Thread life cycle: C++ demo
• Detached thread
• Detached thread: C++ demo

3. Mutual Exclusion

• Data race
• Data race: C++ demo
• Mutual exclusion
• Mutual exclusion: C++ demo
• Atomic objects: C++ demo

4. Locks

• Recursive mutex
• Recursive mutex: C++ demo
• Try lock
• Try lock: C++ demo
• Shared mutex
• Shared mutex: C++ demo

5. Liveness

• Deadlock
• Deadlock: C++ demo
• Abandoned lock
• Abandoned lock: C++ demo
• Starvation
• Starvation: C++ demo
• Livelock
• Livelock: C++ demo

6. Synchronization

• Condition variable
• Condition variable: C++ demo
• Producer-consumer
• Producer-consumer: C++ demo
• Semaphore
• Semaphore: C++ demo

7. Barriers

• Race condition
• Race condition: C++ demo
• Barrier
• Barrier: C++ demo
• Latch: C++ demo

8. Asynchronous Tasks

• Computational graph
• Thread pool
• Thread pool: C++ demo
• Future
• Future: C++ demo
• Divide and conquer
• Divide and conquer: C++ demo

9. Evaluating Parallel Performance

• Speedup, latency, and throughput
• Amdahl's law
• Measure speedup
• Measure speedup: C++ demo

10. Designing Parallel Programs

• Partitioning
• Communication
• Agglomeration
• Mapping

11. Challenge Problems

• Welcome to the challenges!
• Challenge: Matrix multiply
• Solution: Matrix multiply
• Challenge: Merge sort
• Solution: Merge sort
• Challenge: Download images
• Solution: Download images

Conclusion

• Next steps