This course provides students with an understanding of the concepts and techniques of parallel computing. This includes an understanding of the interaction between hardware and software, the power and limitations of parallelism and its benefits and challenges. Examples are used throughout the course to illustrate concepts and issues in parallel computing.
- Introduction and Motivation. Motivation for Parallelism: parallel computing, speed up, Moore's law, grand challenge problems, trends.
- Parallel Computing Systems. Flynn's taxonomy, shared memory architectures, distributed memory architectures, clusters, networks of workstations, heterogeneous architectures.
- Performance Analysis. Performance Measures: speed up, efficiency, cost. Amdahl's law, Gustafson's law. Parallel Computational Models: PRAM, BSP, Cluster cost model.
- Shared Memory Parallelism. Threads. OpenMP Compiler Directives. Partitioning Techniques. Examples: Mandelbrot set, Monte Carlo methods, N-body problem, Barnes Hut algorithm.
- Distributed Memory Parallelism. Message Passing Libraries: processes, point-to-point and collective communication. MPI message passing routines. Data Partitioning. Examples: bucket sort, numerical integration. Divide-and-Conquer. Examples: merge sort, adaptive quadrature. Pipelining: Type 1, 2 and 3 pipelines. Examples: sum of sequence, insertion sort, prime number generation, back substitution.
- Synchronous Computations. Data Parallel Programming, Global and Local Synchronization. Examples: solving linear equations, cellular automata.
- Scheduling and Load Balancing. Scheduling. Static Load Balancing. Dynamic Load Balancing. Example: Moore's Algorithm. Performance Tools.
- Applications. Sorting Algorithms. Examples: rank sort, compare and exchange, bubble sort, quicksort, bitonic mergesort. Numerical Algorithms. Examples: matrix algorithms, solving linear equations, Gaussian elimination; Jacobi iteration.
Prerequisite graph
Parallel Computing
Unlocks
Available indexes
| Mon | Tue | Wed | Thu | Fri | |
|---|---|---|---|---|---|
| 1030 | COMMON LEC (CS4) 1030-1130 Mon LT11 | COMMON LEC (CS4) 1030-1130 Thu LT11 | |||
| 1100 | |||||
| 1130 | 10443 TUT (CS4) 1130-1230 Mon LT11 | ||||
| 1200 | |||||
| 1230 | |||||
| 1300 | |||||
| 1330 | |||||
| 1400 | |||||
| 1430 | 10443 LAB (FCS1) 1430-1630 Wed SWLAB1 Wk4,6,8,10,12 | ||||
| 1500 | |||||
| 1530 | |||||
| 1600 |
Other offerings
Other relevant mods
Planning guides
View all articlesBDE · 17 min read
Complete NTU BDE List: Every BDE Category and Module (AY2026/2027 Semester 2)
A complete NTU BDE list by category: languages, business, computing, humanities, arts, science, engineering, and more. Updated for AY2026/2027 Semester 2. Browse all 780+ BDE modules on NTUMods.
STARS · 9 min read
How to Bid for a BDE at NTU (STARS Registration Guide)
Step-by-step: how NTU STARS bidding works, what to do in Round 1 vs Round 2, and how to still get your BDE during add/drop if your first choice fills up.
STARS · 9 min read
NTU STARS Guide 2026: Registration Rounds, Bidding & Planner Tips
NTU STARS planner guide — how bidding rounds work, how to prepare a shortlist before registration opens, and what to do when a module fills up. Includes a free STARS planner tool.