Credit points: 3.
Grading Policy: Homework 30%, Competitive Project 70%.
Shared-Memory Parallelism: CPUs, GPUs and in-between
Shared-Memory Parallelism: CPUs, GPUs and in-between
Lecturer: Dr. Gal Oren
Assistant: Yehonatan Fridman
This course is primarily designed for under and graduate students to learn the basics and the advances of shared memory programming with OpenMP.
This course will introduce you to the multiple forms of parallelism found in modern multi and many core architecture processors and teach you the programming frameworks for handling this parallelism in applications. You will get access to a cluster of modern multi (CPUs) and many (GPUs) core processors for experiments with graded programming exercises.
You will learn how to handle data parallelism with vector instructions, task parallelism in shared memory with threads, vector programming and SIMD extensions, and offloading to accelerators. This knowledge will help you to accelerate computational applications by orders of magnitude, all the while keeping your code portable and future-proof.
This course can apply to various HPC and datacenter workloads and framework including artificial intelligence (AI). We will use oneAPI - an open standard for a unified application programming interface intended to be used across different compute processors and accelerator (coprocessor) architectures, and Intel DevCloud, which offers complimentary access to a wide range of Intel architectures. We will also review other heterogeneous APIs' new avenues.
Prerequisites:
- Programming in C/C++ or Fortran - MUST (we will also use Python!)
- Programming in the Linux environment and Linux shell proficiency (navigation, file copying, editing files in text-based editors, compilation).
- OS profound understanding.
Tentative Course Outline:
- Introduction to OpenMP
- Creating Threads
- Synchronization
- Parallel Loops
- Data Environment
- Memory Model
- Irregular Parallelism and Tasks
- Memory Access / NUMA
- Thread Affinity
- SIMD -- Vectorization
- Heterogeneous Architectures
- OneAPI base and HPC toolkits
Main References:
This is a restricted list of various interesting and useful books that will be touched during the course. You need to consult them occasionally.
- Van der Pas, Ruud, Eric Stotzer, and Christian Terboven. Using OpenMP -- The Next Step: Affinity, Accelerators, Tasking, and SIMD. MIT press, 2017.
- Mattson, Timothy G., Yun Helen He, and Alice E. Koniges. The OpenMP Common Core: Making OpenMP Simple Again. MIT Press, 2019.