Cornell insignia

Cornell University
Computing and Information Science 4205:
Effective Use of High Performance Computing

Steve Lantz, Instructor for Spring 2009

Course Location and Time:
Upson 205, MWF 10:10–11:00

8 weeks, Jan. 19–Mar. 13

2 Credit Hours, student option
Successful/Unsuccessful or Letter Grade

Proficiency in C, C++,
Fortran, or Fortran 90/95

Recommended Text:
Parallel Programming in C with MPI and OpenMP, by
Michael J. Quinn, on reserve at the Engineering Library
(preview available on Google Book Search)

Office Hours:
533 Rhodes Hall, by appointment
Send email to

Grading and Policies:  follow this link

A hands-on introduction to high-performance computing (HPC) for graduate students or advanced undergraduate students who will use HPC as a tool in their work. Various HPC architectural platforms are described with a focus on computational clusters. Students learn how to identify and exploit parallelism in algorithms and legacy applications; how to measure parallel speedup and efficiency; and how to diagnose bottlenecks affecting performance. Parallel programming with MPI, OpenMP, and task-farming techniques (for Web services and grid computing) is covered in depth. Examples and assignments are taken from typical application areas such as matrix and Monte Carlo computations. The goal of the class is for students to gain practical HPC experience for use in their specific fields.

Course Outline and Lecture Notes:
The outline is tentative and may be modified—students will be given a voice in what is ultimately presented.

(Note: Lectures may contain concepts and examples from the textbook and other sources listed below)

Week 1: Introduction to High Performance Computing
Lecture Notes ( pptpdf )

Week 2: Building and Running Parallel Applications, Pt. 1;
Parallel Application Design
Lecture Notes ( pptpdf )

Week 3: Building and Running Parallel Applications, Pt. 2;
Distributed Memory Programming Using Basic MPI
Lecture Notes ( pptpdf )

Week 4: Distributed Memory Programming Using Advanced MPI
Lecture Notes ( pptpdf )

Weeks 5 and 6: In-Class “Guerrilla” Development of MPI Examples
Shared Memory Programming with Basic and Advanced OpenMP
Lecture Notes ( pptpdf )

Weeks 7 and 8: Parallel Performance
Lecture Notes ( pptpdf )

Sample Codes:

Week 2:
helloworld.c, heat2d_ser.c, heatmovie.gif

Week 3:
api_examples.c, roundtrip.c
Remove .txt from the filename to use any of the following shell scripts:,,

Week 4:
pi.c, gather.c, gather2.c, alltoall.c, tp1.c, bagboy.c, pp1.c

Week 5:
montepi.c,, mw_skel.c

Week 6:
omp_pi.c, omp_firstpriv.c, omp_lastpriv.c, omp_single.c

Week 7:
Assignment #3 solutions: bagboys.c, bagboys_better.c
Assignment #4 solutions: naive.c, omp_naive.c, sieve.c, omp_sieve.c

Week 8:
omp_split.c, omp_quinn1.c, omp_quinn2.c, omp_quinn3.c

Quizzes, Assignments, and Project Milestones:

QUIZ #1 ANNOUNCED for Wed. 1/28 based on Week 1 Lecture Notes!

Download the gzipped tar file from Tutorial 7 at
and build the code within it on any Linux computer by following the steps delineated in 7.2–7.6. Then: PROJECT MILESTONE #1 DUE 2/6
Propose a problem in parallel computing that you would like to solve as an outcome of this course.
It should involve the following elements: ASSIGNMENT #2 DUE 2/11
Build the roundtrip.c example using mpicc: QUIZ #2 ANNOUNCED for Mon. 2/16 based on Week 2 Lecture Notes!

Rewrite the bagboy.c code to create a grocery store you’d much rather shop in! PROJECT MILESTONE #2 DUE 2/20
Present a parallel program design that will solve the problem you described in your proposal:
Write serial and parallel versions of a code to find prime numbers using the “Sieve of Eratosthenes”: PROJECT MILESTONE #3 DUE 3/6 - no late penalty if handed in by 3/13
Write a program based on the parallel algorithm design you created for your problem:
QUIZ #3 ANNOUNCED for Mon. 3/9 based on MPI and OpenMP (functional knowledge, not syntax details)

ASSIGNMENT #5 - REVISED - WAS DUE IN CLASS 3/11, now may be submitted by 3/18, no late penalty
Run tests with the improved “Sieve of Eratosthenes” codes that were used as examples in class:
Please fill out the course survey

PROJECT MILESTONE #4 DUE 3/13 - no late penalty if handed in by 3/23
Improve the parallel program you put together for your problem and demonstrate some results from it:

Quick Reference Links:

MPI Documents specifying the standard MPI interface
C99 Draft Standard, plain text format
Archive of this course’s website from previous years

Online Resources:

  1. Chandra, Rohit, et al. Parallel Programming in OpenMP. Morgan Kaufmann Publishers, 2001. Books24x7.
  2. Foster, Ian. Designing and Building Parallel Programs (Online). Addison-Wesley, 1995. Argonne/CRPC.
  3. Petersen, Wesley, and Peter Arbenz. Introduction to Parallel Computing: A Practical Guide with
    Examples in C
    . Oxford University Press, 2004. Books24x7.
  4. Silva, Vladimir. Grid Computing for Developers. Cengage Charles River Media, 2006. Books24x7.
  5. Snir, Marc, et al. MPI: The Complete Reference. MIT, 1995. Netlib.

Reference Texts:

[1] Using MPI: Portable Parallel Programming with the Message-Passing Interface
By: William Gropp, Ewing Lusk, & Anthony Skjellum
October 1994; ISBN 0-262-57104-8

[2] Introduction to Parallel Computing
By: Ted G. Lewis, Hesham El-Rewini
Copyright 1992 - Prentice-Hall Inc.
ISBN: 0-13-498924-4

[3] An Introduction to Parallel Programming
By: K. Mani Chandy, Stephen Taylor
Copyright 1992 - Jones and Bartlett Publishers, Inc.
ISBN: 0-86720-208-4

[4] Designing and Building Parallel Programs, Concepts and Tools for Parallel Software Engineering
By: Ian Foster
Copyright 1995 - Addison-Wesley Publishing Company, Inc.
ISBN: 0-201-57594-9

[5] Introduction to Parallel and Vector Solution of Linear Systems
By: James M. Ortega
Copyright 1988 - Plenum Press
ISBN: 0-306-42862-8

[6] Parallel Sorting Algorithms
By: Selim G. Akl
Copyright 1995 - Academic Press, Inc
ISBN: 0-12-047680-0

[7] Portable Programs for Parallel Processors
By: James Patterson, Terrence Disz, Ralph Butler, Rick Stevens, Ross Overbeek, Barnett Glickfeld, Ewing Lusk and James Boyle
Copyright 1987 Holt, Rinehart and Winston, Inc.
ISBN: 0-03-014404-3

Questions? Comments? Email me,
-- The background color for this page is RGB #caccac (what else?) --
Last updated 3/11/09