CS 475 / 575 -- Parallel Programming

Spring Quarter 2022

http://cs.oregonstate.edu/~mjb/cs575

Resources Page


Projects Handouts Ed Discussion PB Zoom


This page was last updated: June 3, 2022


Coronavirus Information

Your well-being is our #1 concern! I want all of you to have ready-access to the latest information:

Stuff You Should Be Interested In:


What We Will Be Doing

The goals of this course are to leave you "career-ready" (i.e., both work-ready and research-ready) for tasks that require you to speed the execution of programs using parallelism.

CS 475/575 topics include:


Prerequisites and Course Incoming Expectations

This course will use C/C++ for most of its programming. You should be comfortable with the concepts of function calls, arrays, for-loops, structures, arrays of structures, structures of arrays, pointers, stacks, queues, trees, and linked lists. It is strongly suggested that you not use CS 475/575 as an opportunity to learn programming for the first time.

Many of the assignments can be done on Linux systems, which you will have ready-access to. It would be good if you already know how to use the Linux command line and know at least one Linux-based editor (vim is good).

It will help if you know the Linux commands: ls, mv, cp, mkdir, cd, pwd, rm, echo, gcc/g++, and diversion to a file.

This class does a lot of graphing performance data. You will need access to a program that will let you enter data into a 2D table and graph it (Excel is good, but there are others). You will need to be able to copy-and-paste those tables and graphs into a word processing document, add your own text around them, and then produce a PDF file from it.


Professor

The class is being taught by Professor Mike Bailey.

Office: Kelley 2117 (2nd floor, south side)
Phone: 541-737-2542
Email: mjb@cs.oregonstate.edu

TAs

The TAs for this course are: Julie Kuang, Hongyu Nie, Anton Nikitin, Jian Tang, Mikhail Tokarev, Shengxuan Wang, and Yichuan Yin. We are lucky to have them!

Total Office Hours

All times are PDT, Pacific Daylight Time. (UTC offset: UTC -7)

You don't need to have specific questions to Zoom into Office Hours -- lurking is definitely encouraged!

  Julie Kuang Hongyu Nie Anton Nikitin Jian Tang Mikhail Tokarev Shenqxuan Wang Yichuan Yin Prof. Bailey
Email @oregonstate.edu kuangqi nieh nikitina tangjian tokarevm wangshe yinyic mjb
Zoom-room Zoom Zoom Zoom Zoom Zoom Zoom Zoom Zoom
Mondays   2:00-6:00 11:00-1:00   8:00-11:00 8:00-11:00,
1:00-4:00
  CS 12:00-2:00,
LL 2:30-4:00
Tuesdays 8:00-11:00,
3:00-6:00
2:00-6:00   8:00-12:00   12:00-3:00   12:00-2:00
Wednesdays     2:00-6:00 8:00-4:00 3:00-6:00 8:00-11:00   CS 12:00-2:00,
LL 2:30-4:00
Thursdays 8:00-11:00,
3:00-6:00
  11:00   8:00-11:00   12:00-6:00 1:00-3:00
Fridays   2:00-6:00 9:00-2:00   9:00-11:00   12:00-6:00 4:00-6:00
Sundays               5:00-6:00

  8:00 9:00 10:00 11:00 12:00 1:00 2:00 3:00 4:00 5:00
Mondays MT, SW MT, SW MT, SW AN CS, AN CS, SW LL, HN, SW LL, HN, SW HN HN
Tuesdays JK, JT JK, JT JK, JT JT PB, SW PB, SW HN, SW HN, JK HN, JK HN, JK
Wednesdays JT, SW JT, SW JT, SW JT CS, JT CS, JT AN, JT, LL AN, JT, LL, MT AN, MT AN, MT
Thursdays JK, MT JK, MT JK, MT AN YY PB, YY PB, YY JK, YY JK, YY JK, YY
Fridays MT AN, MT AN, MT AN AN, YY AN, YY HN, YY HN, YY HN, PB, YY HN, PB, YY
Sundays                   PB

CS = on-campus class time
LL = Zoom Live Lecture time (it actually starts at 2:30, not 2:00)
The links on the initials point to the Zoom-Rooms.

AN = Anton Nikitin
HN = Hongyu Nie
JK = Julie Kuang
JT = Jian Tang
MT = Mikhail Tokarev
PB = Prof. Bailey
SW = Shengxuan Wang
YY = Yichuan Yin


Live Lectures

Every Monday and Wednesday, starting at 2:30, I will be holding Live Lectures. We will go over some current materials, some upcoming materials, and maybe some extra materials. Most of it will be a repeat of what was done in the on-campus class earlier in the day. But, if you couldn't attend the on-campus class (or even if you could), this will be a time for you to hear about the material and to ask live questions.

Attendance at the Live Lectures is optional, but the material we cover there could be on quizzes and tests. These sessions will be recorded (see below) in case you missed them.

Here is my Zoom-room.

Week #DayChatVideo
1MondayPDFLV
1WednesdayPDFLV
2MondayPDFLV
2WednesdayPDFLV
3MondayPDFLV
3WednesdayPDFLV
4MondayPDFLV
4WednesdayPDFLV
5MondayPDFLV
5WednesdayNo Live Lecture
6MondayNo Live Lecture
6WednesdayPDFLV
7MondayPDFLV
7WednesdayPDFLV
8MondayPDFLV
8WednesdayPDFLV
9MondayPDFLV
9WednesdayPDFLV
10MondayNo Live Lecture
10WednesdayPDFLV


Notes in place of a textbook

Course material will consist of my notes and web pages.
All required course materials for this class will cost you $0.00 (i.e., free).

On-campus Section of the Class

Our on-campus class time is: Monday and Wednesday, 12:00 - 1:50. All of our classes will be in the Milam Auditorium (room 026).

Online Section of the Class

The lectures for the Ecampus version of the class will be asynchronous, that is, you are free to review the notes and watch the videos whenever it suits you. But, be careful not to fall behind! The programming projects, and their due dates, will assume that you have been keeping up with the material.

I will also be conducting Live Lectures so that you have a chance to ask questions during the presentation of the material.

Both Sections of the Class

Everyone learns in different ways. As such, feel free to attend/use resources that have been setup for the other course modality.
Ecampus students, if you are in the area, feel free to attend the live class.
On-campus students, feel free to watch the videos.
Everyone should be studying the notes!

Note Handouts and Recorded Videos

To see an Academic Year calendar, click here.
LV = Lecture Video

WeekTopic
1 Class Resources Page HTML       LV
1 Course Introduction 1pp 2pp 4pp 6pp LV
1 Project Notes 1pp 2pp 4pp 6pp LV
1 Pivot Tables 1pp 2pp 4pp 6pp LV
1 Windows Powershell Scripting 1pp 2pp 4pp 6pp LV
1 Simple OpenMP 1pp 2pp 4pp 6pp LV
1 Parallel Programming: Background Information 1pp 2pp 4pp 6pp LV-1 LV-2
2 OpenMP 1pp 2pp 4pp 6pp LV-1 LV-2 LV-3
2 OpenMP 4.0 Quick Reference Card PDF
2 Trapezoid Integration with OpenMP 1pp 2pp 4pp 6pp LV
2 Speedups and Amdahl's Law 1pp 2pp 4pp 6pp LV
2 Moore's Law and Multicore 1pp 2pp 4pp 6pp LV
2 Hyperthreading and "Almost Amdahl" 1pp 2pp 4pp 6pp LV
3 Caching Issues in Multicore Performance 1pp 2pp 4pp 6pp LV-1 LV-2 LV-3
3 OpenMP Tasks 1pp 2pp 4pp 6pp LV
3 Functional Decomposition 1pp 2pp 4pp 6pp LV
4 Data Decomposition 1pp 2pp 4pp 6pp LV-1 LV-2
4 SIMD Vector Parallel Programming 1pp 2pp 4pp 6pp LV
4 Prefetching 1pp 2pp 4pp 6pp LV
5 A Tale of Two Assembly Codes 1pp 2pp 4pp 6pp
5 GPU 101 1pp 2pp 4pp 6pp LV-1 LV-2
5 Test #1 Review HTML
6 CUDA 1pp 2pp 4pp 6pp LV
6 CUDA Quick Reference Card PDF
6 DGX System 1pp 2pp 4pp 6pp LV
6 CUDA Array Multiplication 1pp 2pp 4pp 6pp LV
6 CUDA Matrix Multiplication 1pp 2pp 4pp 6pp LV
7 Transition CUDA ↔ OpenCL 1pp 2pp 4pp 6pp
7 OpenCL 1pp 2pp 4pp 6pp LV-1 LV-2 LV-3 LV-4
7 OpenCL 2.0 Quick Reference Card: PDF
7 OpenCL Events 1pp 2pp 4pp 6pp LV
8 OpenCL Reduction 1pp 2pp 4pp 6pp LV
8 Looking At OpenCL Assembly language 1pp 2pp 4pp 6pp LV
8 OpenCL / OpenGL Vertex Buffer Interoperability 1pp 2pp 4pp 6pp LV-1 LV-2
9 Compute:Communicate Ratio 1pp 2pp 4pp 6pp LV
9 Message Passing Interface (MPI) 1pp 2pp 4pp 6pp LV-1 LV-2 LV-3
10 Parallelism Jeopardy 1pp 2pp 4pp 6pp LV
10 Finding More Information PDF LV
10 Test #2 Review HTML


Downloadable Files

CUDA ArrayMult Visual Studio Solution Folder NewCudaArrayMul2019.zip Un-zip and double-click on the .sln file, then select Build→Clean Solution, then select Build→Build Sample, then select Debug→Start Without Debugging
CUDA ArrayMult .cu program arrayMul.cu  
CUDA Linux Makefile (works on the DGX systems, but not on flip) Makefile  
Just the CUDA part of the array multiplication code JustTheCuda.txt  
Just the CUDA part of the array multiplication + reduction code JustTheReduceCuda.txt  
Sample OpenCL C++ source first.cpp  
Sample OpenCL CL source first.cl.txt  
OpenCL Code to Select the Best (imho...) OpenCL Platform/Device selectopencldevice.cpp


Projects

Project # Points Title Due Date Lecture Video
0 30 Simple OpenMP Experiment April 4 LV
1 100 OpenMP: Monte Carlo Simulation April 15 LV
2 100 OpenMP: Numeric integration April 26 LV
3 100 Functional Decomposition May 4 LV
4 60 Vectorized Array Multiplication and Reduction using SSE May 11 LV
5 100 CUDA: Monte Carlo Simulation May 18 LV
6 100 OpenCL Array Multiplication and Reduction May 29 LV
7A 120 OpenCL/OpenGL Particle System June 7 -- No BDs LV
7B 120 MPI Autocorrelation June 7 -- No BDs LV
575-only paper 100 Paper Analysis Project June 7 -- No BDs LV

Do 7A or 7B -- You cannot get credit for both

Project Turn-In Procedures

Bonus Days and Late Assignments

Projects are due at 23:59:59 on the listed due date, with the following exception:

Each of you has been granted 5 Bonus Days, which are no-questions-asked one-day extensions which may be applied to any project, subject to the following rules:

  1. No more than 2 Bonus Days may be applied to any one project
  2. Bonus Days cannot be applied to quizzes
  3. Bonus Days cannot be applied to tests
  4. Bonus Days cannot be applied to Project #7 or the 575 Paper Project

If you turn in a project three or more days late, the score is a zero.

If you turn in a project late and don't have enough Bonus Days left to spend on it, the score is a zero.

You don't need to ask me or even tell me that you are using Bonus Days. Just turn your project in late. I have a script that will check your turn-in date and deduct the proper number of Bonus Days.

It is your job to track your Bonus Day usage. But, if you lose track, feel free to email me and ask me to look it up.


Grading

Your scores will be posted on Canvas

CS 575 will be graded on a fill-the-bucket basis. There will be 8 programming projects, 10 quizzes, and two tests. You get to keep all the points you earn.

Those taking the class as CS 575 will also do a 100-point essay summarizing a research paper. (Those taking the class as CS 475 will get a free 100 points for this assignment.)

All programming projects will be turned in on Teach, http://teach.engr.oregonstate.edu .

The quizzes will be done on Canvas. They will open each Friday at 00:01 (=12:01 AM) and will close Sunday night at 23:59 (=11:59 PM). Canvas is very unforgiving about due times -- don't push it.

Your final grade will be based on your overall class point total. Based on an available point total of 1110, grade cutoffs will be no higher than:

Points Grade
1060
1040 A- 
1020 B+
1000
980 B- 
960 C+
940
920 C- 
900 D+
880
860 D- 

Notice that this grade scale is not 90%-80%-70%-60%. That is because I do such a soft grade on the projects.


Class Rules


Other Notes You Might Enjoy!

 University   
CS Skills for Simulation and Game Programming
Intro to Computer Graphics
Paraview
Scientific Visualization
Shaders
Vulkan

Grades 2-12 
Blender
Processing
Scratch
SketchUp
TinkerCad


Other Useful Parallel Programming Information

  • vi Quick Reference Card

  • Association of Computing Machinery http://www.acm.org

  • C++ Style Guide used by Google: https://google.github.io/styleguide/cppguide.html

  • Interesting collection of information about the history of the CPU: CPU-DB.pdf


    Student Resources:

    Academic Calendar

    All students are subject to the registration and refund deadlines as stated in the Academic Calendar: https://registrar.oregonstate.edu/osu-academic-calendar

    Students With Disabilities

    Accommodations for students with disabilities are determined and approved by Disability Access Services (DAS). If you, as a student, believe you are eligible for accommodations but have not obtained approval, please contact DAS immediately at 541-737-4098 or at http://ds.oregonstate.edu. DAS notifies students and faculty members of approved academic accommodations and coordinates implementation of those accommodations. While not required, students and faculty members are encouraged to discuss details of the implementation of individual accommodations.

    Link to the Code of Student Conduct

    https://beav.es/codeofconduct

    Student Bill of Rights

    OSU has twelve established student rights. They include due process in all university disciplinary processes, an equal opportunity to learn, and grading in accordance with the course syllabus. See: https://asosu.oregonstate.edu/advocacy/rights

    Reach Out for Success

    University students encounter setbacks from time to time. If you encounter difficulties and need assistance, it's important to reach out. Consider discussing the situation with me or an academic advisor. Learn about resources that assist with wellness and academic success at http://oregonstate.edu/ReachOut. If you are in immediate crisis, please contact the Crisis Text Line by texting OREGON to 741-741 or call the National Suicide Prevention Lifeline at 1-800-273-TALK (8255).

    Religious Holidays

    Oregon State University strives to respect all religious practices. If you have religious holidays that are in conflict with any of the requirements of this class, please let me know immediately so that we can make alternative arrangements.

    Life Events

    As {John Lennon? Allen Saunders?} has said: "Life is what happens to you while you're busy making other plans". I care about you as a person. When life happens to you, send me an email and let's talk. Don't be afraid to do this. I might be able to help, I might not. But I surely can listen. You are not alone.

    Counseling and Psychological Services

    Oregon State University's Counseling and Psychological Services (CAPS) provides OSU students with individual, relationship, and group counseling. I have met some of these people, and they are really good. If you are in any type of emotional difficulty, don't hesitate to contact them. If it makes it easier for you, I will walk over with you.

    Basic Needs

    Any student who has difficulty affording groceries or accessing sufficient food to eat every day, or who lacks a safe and stable place to live, is urged to contact the Human Services Resource Center (HSRC) for support: hsrc@oregonstate.edu, 541-737-3747. The HSRC has a food pantry, a textbook lending program, and other resources to help. Furthermore, if you are comfortable doing so, please come talk with me. I will do everything I can do to help you.

    University Housing & Dining Services has been piloting programs collectively called Food Assistance Program. Using the extra food generated by campus dining locations, they create pre-packaged, balanced meal options available at no cost to Oregon State students who meet eligibility criteria determined by the Human Services Resource Center. For others, these packages are available for purchase at a reduced price. Find out more about it here.