ECS 158: Programming on Parallel Architectures

Summary of Course Content

1. Programming Paradigms
   1. Shared-memory
   2. Message-passing
   3. Bulletin-board (tuple space)
   4. Distributed shared memory (DSM)
   5. Software packages: MPI, PVM, Linda, p4, Adsmith, pthreads
2. Overview of Parallel Architectures
   1. Shared-memory and message-passing hardware
   2. Interconnect structures, e.g. bus, crossbar,Omega-net, mesh
   3. Cache coherency
3. Interprocess Synchronization
   1. Review of operating systems material: Processes, timesharing, OS-based interprocess synchronization
   2. Synchronization hardware, e.g. test-and-set and fetch-and-add instructions
   3. Locks, barriers
4. Latency Problems
   1. Memory/network delays
   2. Latency hiding, e.g. using nonblocking messages
5. Collective Communication
   1. Broadcast, scatter/gather, all-scatter/all-gather
6. Overview of Parallel Applications
   1. Sorting, database
   2. Graph algorithms
   3. Scientific computing Computer Usage: Extensive programming assignments on our CSIF networked machines. The shared-memory work is done using Adsmith and the MulSim simulator, and the message-passing component using MPI. Engineering Design Statement: Students in the course write sophisticated, systems-level programs according to their own design. Many of the programs involve analysis of tradeoffs between efficient use of the hardware and ease of programming.

Illustrative Reading
N. Matloff, Programming on Parallel Machines, 2012. Open-source textbook, available at http://heather.cs.ucdavis.edu/parprocbook

Potential Course Overlap
Comparison of this course with other existing courses does not indicate any significant overlap.