CS 475 / 575 -- Parallel Programming

Spring Quarter 2021

http://cs.oregonstate.edu/~mjb/cs575

Resources Page


Projects Handouts Ed Discussion PB Zoom


This page was last updated: May 12, 2021


Coronavirus Information

Your well-being is our #1 concern! I want all of you to have ready-access to the latest information:

Stuff You Should Be Interested In:


What We Will Be Doing

The goals of this course are to leave you "career-ready" (i.e., both work-ready and research-ready) for tasks that require you to speed the execution of programs using parallelism.

CS 475/575 topics include:


Prerequisites and Course Incoming Expectations

This course will use C/C++ for most of its programming. You should be comfortable with the concepts of function calls, arrays, for-loops, structures, arrays of structures, structures of arrays, pointers, stacks, queues, trees, and linked lists. It is strongly suggested that you not use CS 475/575 as an opportunity to learn programming for the first time.

Many of the assignments can be done on Linux systems, which you will hve ready-access to. It would be good if you already know how to use the Linux command line and know at least one Linux-based editor (vim is good).

It will help if you know the Linux commands: ls, mv, cp, mkdir, cd, pwd, rm, echo, gcc/g++, and diversion to a file.

This class does a lot of graphing performance data. You will need access to a program that will let you enter data into a 2D table and graph it (Excel is good, but there are others). You will need to be able to copy-and-paste those tables and graphs into a word processing document, add your own text around them, and then produce a PDF file from it.


Professor

The class is being taught by Professor Mike Bailey.

Office: Kelley 2117 (2nd floor, south side)
Phone: 541-737-2542
Email: mjb@cs.oregonstate.edu

TAs

The TAs for this course are: Braxton Cuneo, Hongyu Nie, Botong Qu, Jian Tang, Mikhail Tokarev, and Mian Xie. We are lucky to have them!

Total Office Hours

  Braxton Cuneo Hongyu Nie Botong Qu Jian Tang Mikhail Tokarev Mian Xie Prof. Bailey
Email @oregonstate.edu cuneob nieh qub tangjian tokarevm xiemia mjb
Zoom-room Zoom Zoom Zoom Zoom Zoom Zoom Zoom
Mondays   1:00-7:00 9:00-1:00 8:00-9:00 , 10:00-12:00 4:00-6:00   2:00-4:00
Tuesdays 11:00-12:00 , 2:00-4:00     8:00-9:00 4:00-6:00 8:00-11:00 , 12:00-4:00 3:00-5:00
Wednesdays 11:00-12:00 9:00-1:00   8:00-12:00 , 2:00-6:00 2:00-6:00    
Thursdays 11:00-12:00 , 2:00-4:00     8:00-9:00 4:00-6:00 8:00-11:00 , 12:00-4:00 12:00-2:00
Fridays   9:00-1:00 1:00-4:00 8:00-9:00 12:00-4:00   5:00-6:00

  8:00 9:00 10:00 11:00 12:00 1:00 2:00 3:00 4:00 5:00 6:00
Mondays JT BQ BQ JT BQ JT BQ HN HN HN HN MT HN MT HN
Tuesdays MX JT MX MX BC MX MX MX BC MX PB BC MT PB MT  
Wednesdays JT HN JT HN JT HN JT BC HN   JT MT JT MT JT MT JT MT  
Thursdays MX JT MX MX BC MX PB MX PB MX BC MX BC MT MT  
Fridays JT HN HN HN HN MT BQ MT BQ MT BQ MT   PB  
PB = Prof. Bailey
BC = Braxton Cuneo
HN = Hongyu Nie
BQ = Botong Qu
JT = Jian Tang
MT = Mikhail Tokarev
MX = Mian Xie
The links point to the Zoom-Rooms.


Live Lectures

Every Monday and Wednesday, starting at 12:00 noon, I will be holding Live Lectures. We will go over some current materials, some upcoming materials, and maybe some extra materials. It will be a time for you to also ask live questions. Attendance at the live session is optional, but the material we cover there is eligible to be on quizzes and tests. These sessions will be recorded (see below) in case you missed them.

Here is my Zoom-room.
If you need a passcode, use ZeldaDog

Week #DayChatVideo
1MondayPDFLV
1WednesdayPDFLV
2MondayPDFLV
2WednesdayPDFLV
3MondayPDFLV
3WednesdayPDFLV
4MondayPDFLV
4WednesdayPDFLV
5MondayPDFLV
5WednesdayPDFLV
6MondayPDFLV
6WednesdayPDFLV
7MondayPDFLV
7WednesdayPDFLV


Notes in place of a textbook

Course material will consist of my notes, web pages, and notes taken in class.

Note Handouts and Recorded Videos

LV = Lecture Video

These notes are in final form for the Spring 2021 version of the class:

WeekTopic
1 Course Introduction 1pp 2pp 4pp 6pp LV
1 Project Notes 1pp 2pp 4pp 6pp LV
1 Windows Powershell Scripting 1pp 2pp 4pp 6pp LV
1 Simple OpenMP 1pp 2pp 4pp 6pp LV
1 Parallel Programming: Background Information 1pp 2pp 4pp 6pp LV-1 LV-2
2 OpenMP 1pp 2pp 4pp 6pp LV-1 LV-2 LV-3
2 OpenMP 4.0 Quick Reference Card: PDF
2 Trapezoid Integration with OpenMP 1pp 2pp 4pp 6pp LV
2 Speedups and Amdahl's Law 1pp 2pp 4pp 6pp LV
2 Moore's Law and Multicore: 1pp 2pp 4pp 6pp LV
2 Hyperthreading and "Almost Amdahl" 1pp 2pp 4pp 6pp LV
3 Tree Traversal using Tasks 1pp 2pp 4pp 6pp LV
3 Caching Issues in Multicore Performance 1pp 2pp 4pp 6pp LV-1 LV-2 LV-3
3 Functional Decomposition 1pp 2pp 4pp 6pp LV
4 Data Decomposition 1pp 2pp 4pp 6pp LV-1 LV-2
4 SIMD Vector Parallel Programming 1pp 2pp 4pp 6pp LV
4 Prefetching 1pp 2pp 4pp 6pp LV
5 GPU 101 1pp 2pp 4pp 6pp LV-1 LV-2 LV-3
5 Test #1 Review HTML
6 CUDA 1pp 2pp 4pp 6pp LV
6 CUDA Quick Reference Card PDF
6 DGX System 1pp 2pp 4pp 6pp LV
6 CUDA Array Multiplication 1pp 2pp 4pp 6pp LV
6 CUDA Matrix Multiplication 1pp 2pp 4pp 6pp LV
7 OpenCL 1pp 2pp 4pp 6pp LV-1 LV-2 LV-3 LV-4
7 OpenCL 2.0 Quick Reference Card: PDF
7 OpenCL Events 1pp 2pp 4pp 6pp LV
8 OpenCL Reduction 1pp 2pp 4pp 6pp LV
8 Looking At OpenCL Assembly language 1pp 2pp 4pp 6pp LV
8 OpenCL / OpenGL Vertex Buffer Interoperability 1pp 2pp 4pp 6pp LV-1 LV-2
9 Message Passing Interface (MPI) 1pp 2pp 4pp 6pp LV-1 LV-2 LV-3
10 Parallelism Jeopardy 1pp 2pp 4pp 6pp LV
10 Finding More Information on Parallel Programming PDF LV
10 Test #2 Review HTML


Downloadable Files

CUDA ArrayMult Visual Studio Solution Folder NewCudaArrayMul2019.zip Un-zip and double-click on the .sln file, then select Build→Clean Solution, then select Build→Build Sample, then select Debug→Start Without Debugging
CUDA ArrayMult .cu program arrayMul.cu  
CUDA Linux Makefile (works on the DGX systems) Makefile  
Just the CUDA part of the array multiplication code JustTheCuda.txt  
Just the CUDA part of the array multiplication + reduction code JustTheReduceCuda.txt  
Sample OpenCL C++ source first.cpp  
Sample OpenCL CL source first.cl  
OpenCL Code to Select the Best (imho...) OpenCL Platform/Device selectopencldevice.cpp


Quizzes

Quiz # Due Date and Time Over?
1 Friday, April 2, 4:00 PM
2 Sunday, April 11, 11:59 PM
3 Sunday, April 18, 11:59 PM
4 Sunday, April 25, 11:59 PM
5 Sunday, May 2, 11:59 PM
6 Sunday, May 9, 11:59 PM
7 Sunday, May 16, 11:59 PM
8 Sunday, May 23, 11:59 PM
9 Sunday, May 30, 11:59 PM
10 Sunday, June 6, 11:59 PM


Projects

Project # Points Title Due Date Lecture Video
0 30 Simple OpenMP Experiment April 5 LV
1 100 OpenMP: Monte Carlo Simulation April 16 LV
2 100 OpenMP: Numeric integration April 27 LV
3 100 Functional Decomposition May 5 LV
4 60 Vectorized Array Multiplication and Reduction using SSE May 12 LV
5 100 CUDA Monte Carlo Simulation May 19 LV
6 100 OpenCL Array Multiplication and Reduction May 30 LV
7A 120 OpenCL/OpenGL Particle System June 8 -- No BDs LV
7B 120 MPI Autocorrelation June 8 -- No BDs LV
575-only paper 100 Paper analysis June 8 -- No BDs LV

Project Turn-In Procedures

Bonus Days and Late Assignments

Projects are due at 23:59:59 on the listed due date, with the following exception:

Each of you has been granted 5 Bonus Days, which are no-questions-asked one-day extensions which may be applied to any project, subject to the following rules:

  1. No more than 2 Bonus Days may be applied to any one project
  2. Bonus Days cannot be applied to tests
  3. Bonus Days cannot be applied such that they extend a project due date past the start of Test #2.

If you turn in a project 3 or more days late, the score is a zero.

If you turn in a project late and don't have enough Bonus Days left to spend on it, the score is a zero.

You don't need to ask me or even tell me that you are using Bonus Days. Just turn your project in late. I have a script that will check your turn-in date and deduct the Bonus Days.


Grading

Your scores will be posted on Canvas

CS 575 will be graded on a fill-the-bucket basis. There will be 8 programming projects, 11 quizzes, and two tests. You get to keep all the points you earn.

Those taking the class as CS 575 will also do a 100-point essay summarizing a research paper. (Those taking the class as CS 475 will get a free 100 points for this assignment.)

The quizzes will be done via Canvas. They will open each Friday at 00:01 (=12:01 AM) and will close Sunday night at 23:59 (=11:59 PM). Canvas is very unforgiving about due times -- don't push it.

... with the exception of Week #1:

Quiz #1 opens Monday, March 29 at 12:01 AM and is due to me by 4:00 PM on Friday, April 2. All you need to do is go to Quiz #1 on Canvas and, in the space provided, tell me why you are taking this class. What do you hope to get out of being here? Be honest -- no sucking up!

Your final grade will be based on your overall class point total. Based on an available point total of 1110, grade cutoffs will be no higher than:

Points Grade
1060
1040 A- 
1020 B+
1000
980 B- 
960 C+
940
920 C- 
900 D+
880
860 D- 

Notice that this grade scale is not 90%-80%-70%-60%. That is because I just do a soft grade on the projects.


Class Rules


Link to the Code of Student Conduct

https://beav.es/codeofconduct

Students With Disabilities

Accommodations for students with disabilities are determined and approved by Disability Access Services (DAS). If you, as a student, believe you are eligible for accommodations but have not obtained approval please contact DAS immediately at 541-737-4098 or at http://ds.oregonstate.edu. DAS notifies students and faculty members of approved academic accommodations and coordinates implementation of those accommodations. While not required, students and faculty members are encouraged to discuss details of the implementation of individual accommodations.

Religious Holidays

Oregon State University strives to respect all religious practices. If you have religious holidays that are in conflict with any of the requirements of this class, please see me immediately so that we can make alternative arrangements.

Life Events

As {John Lennon? Allen Saunders?} has said: "Life is what happens to you while you're busy making other plans". I care about you as a person. When life happens to you, send me an email and come talk with me. I might be able to help, I might not. But I surely can listen. You are not alone.

CAPS

Oregon State University's Counseling and Psychological Services (CAPS) provides OSU students with individual, relationship, and group counseling. I have met some of these people, and they are really good. If you are in any type of emotional difficulty, don't hesitate to contact to them.

Reach Out for Success

University students encounter setbacks from time to time. If you encounter difficulties and need assistance, it's important to reach out. Consider discussing the situation with me or an academic advisor. Learn about resources that assist with wellness and academic success at http://oregonstate.edu/ReachOut. If you are in immediate crisis, please contact the Crisis Text Line by texting OREGON to 741-741 or call the National Suicide Prevention Lifeline at 1-800-273-TALK (8255)

Basic Needs

Any student who has difficulty affording groceries or accessing sufficient food to eat every day, or who lacks a safe and stable place to live, is urged to contact the Human Services Resource Center (HSRC) for support: hsrc@oregonstate.edu, 541-737-3747. The HSRC has a food pantry, a textbook lending program, and other resources to help. Furthermore, if you are comfortable doing so, please come talk with me. I will do everything I can do to help you.

University Housing & Dining Services has been piloting a meal program called Food for Thought. Using the extra food generated by campus dining locations, they create pre-packaged, balanced meal options available at no cost to Oregon State students who meet eligibility criteria determined by the Human Services Resource Center. For others, these packages are available for purchase at a reduced price.


Other Notes You Might Enjoy!

 University   
CS Skills for Simulation and Game Programming
Intro to Computer Graphics
Paraview
Scientific Visualization
Shaders
Vulkan

Grades 2-12 
Blender
Processing
Scratch
SketchUp
TinkerCad


Other Useful Parallel Programming Information