CS 475 / 575 -- Parallel Programming

On-campus Edition

Spring Quarter 2017


Lectures Projects Handouts Grades VHR

This page was last updated: June 14, 2017


Here are Some Background Resources for You

Over the last couple of months, various faculty have been putting together mini-tutorials on topics that we (apparently) expect you to magically know, without us ever actually showing you. Here they are. We hope they help.

Class Videos in MP4 Format

Go here to play or download the class video lecture files.

What We Will Be Doing This Quarter

The goals of this course are to leave you "career-ready" (i.e., both work-ready and research-ready) for tasks that require desktop parallelism, both on a CPU and on a GPU.

CS 475/575 topics include:


This course will use C/C++ for its programming. Already being comfortable with function calls, arrays, for loops, structures, arrays of structures, structures of arrays, pointers, stacks, queues, trees, and linked lists is important. It is strongly suggested that you not use this class as an opportunity to learn C for the first time.

Many of the assignments can be done on Linux systems. It would be good if you already know how to use Linux and know at least one Linux-based editor (vim is good).

Beyond that, understanding the analysis of algorithms and Big-O notation is highly recommended.

Some knowledge about computer architecture (e.g., cores, cache) would be a plus, but not critical.

Learning Outcomes

On completion of the course, students will have demonstrated the ability to:

  1. Explain the clockspeed limitations of computing using physics and Moore's Law
  2. Explain the limitations of parallel computing using Amdahl's Law
  3. Demonstrate "parallel thinking" in program design
  4. Explain the difference between TLP, DLP, and SIMD
  5. Demonstrate the ability to program parallel algorithms in TLP, DLP, and SIMD.
  6. Characterize what types of problems are best able to be parallelized
  7. Characterize different parallel programming patterns and what types of problems they best address
  8. Chacacterize how cache issues affect parallel performance
  9. Demonstrate the proper use of synchronization to avoid race conditions and deadlock
  10. Characterize the benefits of using a CPU versus using a GPU for parallel programming
  11. Characterize the benefits of using a GPU versus using a CPU for parallel programming

In addition, those taking this course as CS 575 will also have deminstrated the ability to:

  1. Read a parallel-programming-related research paper and write a 5-page analysis paper of it. (I will make some of these available for you, or you can propose your own. It has to be a real research paper, though.)


The class is being taught by Professor Mike Bailey.

Office: Kelley 2117
Phone: 541-737-2542
E-mail: mjb@cs.oregonstate.edu

Prof. Bailey Office Hours:

Mondays 2:00 - 4:00 Kelley 2117
Tuesdays 12:00 - 2:00 Kelley 2117
Fridays 10:30 - 12:30 Kelley 2117
    or, anytime my office door is open
    or, by appointment -- send email

Note that anytime my office door is open, it becomes an implied Office Hour time. However, this also means that when my office door is not open, I have made a deliberate choice not to make this an impled Office Hour time. Please respect this.


The TAs for this course are:

Alrik Firl, firla@oregonstate.edu,
Mondays8:00 - 10:00 AM
Tuesdays8:00 AM - Noon
Wednesdays3:00 - 5:00
Thursdays8:00 AM - 1:00 PM
The plan is to hold these hours in the Computer Graphics Education Lab, Batcheller Hall room 244.


Zehuan Chen, chenzeh@oregonstate.edu.
Mondays2:00 - 6:00
Tuesdays2:00 - 6:00
Thursdays2:00 - 6:00
Fridays4:00 - 7:00
The plan is to hold these hours in the Computer Graphics Education Lab, Batcheller Hall room 244.

The Virtual Hand Raise (VHR)

I recognize that it sometimes takes a certain amount of courage to ask a question in class. But, the worst thing of all is to not ask! So, this class also uses something called the Virtual Hand Raise. Click here to get into it. It will allow you to send me a question or comment, completely anonymously. I will answer questions submitted this way by email to the class or in class.


There is no required textbook for this course. We will use notes that will be made available on the class web site.

There is a free Parallel Programming book available. Click here to get to it. I'm not sure how much it helps for this class, but it is certainly interesting to look through.

Lecture Schedule

To see an Academic Year calendar, click here.

Class lecture time is: Monday, Wednesday, and Friday, 1:00 - 1:50. Unless otherwise specified, all classes will be in Dearborn 118.

  Date Topics
1 April 3 Introduction. Syllabus. What this course is ... and isn't.
Project notes: timing, graphing. (This is really important.)
2 April 5 Simple OpenMP. Project #0.
Parallel programming background information. The three things we care about Parallel Programming for. Von Neumann architecture.
3 April 7 Multithreading
OpenMP: fork-join model, pragmas, what it does for you, what it doesn't do for you,
OpenMP: parallelizing for-loops
OpenMP: variable sharing
4 April 10 OpenMP: Reduction. Not doing anything special vs. critical vs. atomic vs. reduction.
Trapezoid integration.
Project #1.
5 April 12 Moore's Law. What holds, what doesn't.
Multicore. Hyperthreading.
Timing. Speedup. Amdahl's Law. Parallel efficiency. Gustafson's observation.
6 April 14 OpenMP: dynamic vs. static thread assignment.
Project #2.
Sections, tasks.
7 April 17 OpenMP: bubble sort (red-black), tree traversal using tasks.
8 April 19 Caches. Architecture. Hits. Misses.
9 April 21 Caches. False sharing.
Project #3.
10 April 24 Designing parallel programs.
11 April 26 Designing parallel programs.
12 April 28 Functional Decomposition
Project #4
13 May 1 Project #4.
Test #1 review
Vectorization: what it is, how to use it.
14 May 3 More vectorization
Xeon Phi
Project #5
15 May 5 Test #1
16 May 8 Go over the test answers.
GPU 101.
17 May 10 GPUs -- what they are good at.
GPUs -- what they are not good at.
18 May 12 Prof. Bailey out of town -- No Class Today
Remember that Sunday is Mother's Day!
19 May 15 OpenCL: What is it? Diagram. Mapping onto GPU architecture.
OpenCL library. Querying configurations.
20 May 17 OpenCl Events
21 May 19 Engineering Expo -- No Class Today
22 May 21 OpenCL Reduction.
Project #6.
23 May 24 OpenCL / OpenGL Interoperability
24 May 26 More OpenCL / OpenGL Interoperability
Project #7.
25 May 29 Memorial Day -- OSU Holiday -- No Class Today
26 May 31 Looking at OpenCL Assembly Language.
OpenGL Compute Shaders
27 June 2 Prof. Bailey out of the office -- No Class Today.
28 June 5 Guest Speaker: Chris Schultz, NVIDIA: "GPU Architectures" Confirmed.
29 June 7 The Message Passing Interface (MPI)
30 June 9 Class Evaluations.
Compute Shaders.
Test #2 review.
More Information.
T2 June 15 Test #2 Thursday, June 15, 2:00 - 3:30 PM.
(You can confirm this for yourself by going here.
G June 16, 3:30 EECS Graduation Celebration: La Sells Stewart Center
G June 17 OSU Graduation: meet at MU Quad


Project # Points Title Due Date
0 30 Simple OpenMP Experiment April 10
1 100 OpenMP: Numeric integration April 20
2 100 OpenMP: Static vs Dynamic and Small vs. Large Chunksize May 1 (=April 31)
3 70 False sharing. May 9
4 100 Functional Decomposition May 16
5 60 Vectorized Array Multiplication and Reduction using SSE May 23
6 120 OpenCL Array Multiplication and Reduction June 2
7A 120 OpenCL/OpenGL Particle System June 13 -- No BDs
7B 120 Autocorrelation using OpenMP, SIMD, and OpenCL June 13 -- No BDs
575-only paper 100 Paper analysis June 13 -- No BDs

Project Notes

Refer back to the Project Notes handout.


Grades will be posted through this web page. To protect your privacy, they will be posted by your alias that you give me in Project #1.

Click here to see the current grade posting.

CS 475/575 will be graded on a fill-the-bucket basis. There will be 8 projects, 10 quizzes, and two tests. You get to keep all the points you earn.

In addition, the CS 575 people have an extra 100-point project.
The CS 475 people automatically get 100 points for this project, even though they didn't have to do it.

Your final grade will be based on your overall class point total. Based on an available point total of 1100, grade cutoffs will be no higher than:

Points Grade
1050 A- 
1030 B+
990 B- 
970 C+
930 C- 
910 D+
870 D- 

Project Turn-In Procedures

Bonus Days and Late Assignments

Projects are due at 23:59:59 on the listed due date, with the following exception:

Each of you has been granted five Bonus Days, which are no-questions-asked one-day extensions which may be applied to any project, subject to the following rules:

  1. No more than 2 Bonus Days may be applied to any one project
  2. Bonus Days cannot be applied to tests
  3. Bonus Days cannot be applied such that they extend a project due date past the start of Test #2.

Click here to get a copy of the Bonus Day Submission Form.
Fill this out and turn it in to me within a week of when the project was due. You can turn it in in class, or can drop it off in my office (under my door is fine).

Downloadable Files

None yet.


Don't print these until you are told to do so in the Announcements section.
Sometimes I will put notes out here that are not quite complete, just to show you where we are headed.

  1. Project Notes 1pp, 2pp, 6pp
  2. Simple OpenMP 1pp, 2pp, 6pp
  3. Parallel programming: background Information 1pp, 2pp, 6pp
  4. Moore's Law and Multicore: 1pp, 2pp, 6pp
  5. OpenMP 1pp, 2pp, 6pp
  6. OpenMP 4.0 Quick Reference Card: http://openmp.org/mp-documents/OpenMP-4.0-C.pdf
  7. Bubble Sort Case Study 1pp, 2pp, 6pp
  8. Trapezoid Intergration with OpenMP 1pp, 2pp, 6pp
  9. Speedups and Amdahl's Law 1pp, 2pp, 6pp
  10. rabbit.engr.oregonstate.edu 1pp, 2pp, 6pp
  11. Caching Issues in Multicore Performance 1pp, 2pp, 6pp
  12. Tree Traversal using Tasks 1pp, 2pp, 6pp
  13. Prefetching 1pp, 2pp, 6pp
  14. Parallel Program Design Patterns and Strategies 1pp, 2pp, 6pp
  15. Vector Parallel Programming (SIMD) 1pp, 2pp, 6pp
  16. Xeon Phi 1pp, 2pp, 6pp
  17. GPU 101 1pp, 2pp, 6pp
  18. OpenCL 1pp, 2pp, 6pp
  19. first.cpp C++ source
  20. first.cl OpenCL source
  21. OpenCL 1.1 Quick Reference Card: PDF.
  22. OpenCL 1.2 Quick Reference Card: PDF.
  23. OpenCL 2.0 Quick Reference Card: PDF.
  24. OpenCL Events 1pp, 2pp, 6pp
  25. OpenCL Reduction 1pp, 2pp, 6pp
  26. Looking At OpenCL Assembly language 1pp, 2pp, 6pp
  27. OpenCL / OpenGL Vertex Buffer Interoperability 1pp, 2pp, 6pp
  28. The Message Passing Interface (MPI) 1pp, 2pp, 6pp
  29. Compute Shaders 1pp, 2pp, 6pp
  30. Finding More Information on Parallel Programming PDF

Lecture Recordings

I am simultaneously teaching this class on Ecampus. I will put links to those classes here so that you can relive them if that would help. Warning: We often talk about other things in class, and those are eligible to be on the test, even if they didn't happen in the Ecampus recordings!

None yet.

Class Rules

Students With Disabilities

Accommodations for students with disabilities are determined and approved by Disability Access Services (DAS). If you, as a student, believe you are eligible for accommodations, but have not obtained approval, please contact DAS immediately at 541-737-4098 or at http://ds.oregonstate.edu. DAS notifies students and faculty members of approved academic accommodations and coordinates implementation of those accommodations. While not required, students and faculty members are encouraged to discuss details of the implementation of individual accommodations.

Other Useful Online Parallel Programming Information