ECS 158: Programming on Parallel Architectures

ECS 158
Programming on Parallel Architectures
Effective Term
2018 Winter Quarter
Learning Activities
Lecture: 3 hours
Discussion: 1 hour
Techniques for software development using the shared-memory and message-passing paradigms, on parallel architectures and networks of workstations. Locks, barriers, and other techniques for synchronization. Introduction to parallel algorithms. GE Prior to Fall 2011: SciEng. GE: SE.
ECS 150; ECS 154B recommended.
Enrollment Restrictions
Pass One open to Computer Science and Computer Science Engineering Majors only.

Summary of Course Content

  1. Programming Paradigms
    1. Shared-memory
    2. Message-passing
    3. Bulletin-board (tuple space)
    4. Distributed shared memory (DSM)
    5. Software packages: MPI, PVM, Linda, p4, Adsmith, pthreads
  2. Overview of Parallel Architectures
    1. Shared-memory and message-passing hardware
    2. Interconnect structures, e.g. bus, crossbar,Omega-net, mesh
    3. Cache coherency
  3. Interprocess Synchronization
    1. Review of operating systems material: Processes, timesharing, OS-based interprocess synchronization
    2. Synchronization hardware, e.g. test-and-set and fetch-and-add instructions
    3. Locks, barriers
  4. Latency Problems
    1. Memory/network delays
    2. Latency hiding, e.g. using nonblocking messages
  5. Collective Communication
    1. Broadcast, scatter/gather, all-scatter/all-gather
  6. Overview of Parallel Applications
    1. Sorting, database
    2. Graph algorithms
    3. Scientific computing Computer Usage: Extensive programming assignments on our CSIF networked machines. The shared-memory work is done using Adsmith and the MulSim simulator, and the message-passing component using MPI. Engineering Design Statement: Students in the course write sophisticated, systems-level programs according to their own design. Many of the programs involve analysis of tradeoffs between efficient use of the hardware and ease of programming.

Illustrative Reading
N. Matloff, Programming on Parallel Machines, 2012. Open-source textbook, available at

Potential Course Overlap
Comparison of this course with other existing courses does not indicate any significant overlap.

Course Category