CS 569/589 Winter 2014
Empirical Lab Studies (Quantitative) of Programming

(To understand humans' use of languages, tools, environments, and practices of software development)

Instructor: Dr. Burnett
Office: KEC 3051
E-Mail: burnett@eecs.oregonstate.edu

Dr. Burnett's Office Hours are listed on my home page

Course Description

The course is about empirical methods in understanding humans' use of languages, environments, etc., in software development. There are many possibilities about empirical methods, and we can't cover them all. Thus, this course will focus on a method that is very useful for certain kinds of questions about the HCI (human aspects) of software development, but is not well understood in computer science: how to scientifically conduct and analyze (statistically oriented) laboratory studies with human participants.

This course will cover how you go about designing, preparing for, running, analyzing, and writing-for-publication quantitative (statistical) lab experiments of programming situations involving human participants. This is an end-to-end coverage of the entire process, and will put you in a position to conduct lab studies of your own with human participants.

Note that this is not a statistics course, although we will cover a couple of basic stats. If what you really want is a statistics-for-experiments course, I recommend the Stats 511 and/or Stats 515 courses, which are excellent, and are specifically targeted to non-stats grad students.

Course objectives

You can think of this as a "research methods" course, focusing on the research method of doing this type of empirical work. The goals are that by the end of this course, you will be able to:
  1. Choose when a (statistically oriented) lab study is the right choice of empirical work.
  2. Design, conduct, and gather data in such lab studies...
  3. ... according to accepted ethical principles of dealing with human subjects.
  4. Analyze data in lab studies using quantitative (statistical) methods.
  5. Report quantitative lab study empirical work in research publications.

How to sign up

This course is cross-listed under both CS 569 (Special Topics in Software Engineering) and as 589 (Special Topics in Programming Languages). Register for 4 hours of either one (but not both). CS 589 counts toward the Languages category of requirements, and CS 569 counts toward the Software Engineering category, so you should choose whichever one helps you the most to fulfill your degree requirements.


Contrary to what the catalog or scheduling system might say, the only prerequisite for this course is grad standing in Computer Science.

How the course will be conducted, method of instruction

You'll actually do a lab study of some programming language, tool, or practice, with a team member. You and your teammate will choose the language/tool/practice you want to study:

I'm anticipating little or no programming in this course. There will be some lectures by me; about half of the classes will be like this. The other half of the class will be more studio/discussion style, with teams discussing and critiquing each others' work, and based on team presentations in which the class jointly provides feedback on some aspect of a team's case study. In short, it will be highly interactive.


We will also have selected readings from other sources, but you don't have to buy those.

Components of your grade

There will be one midterm, one final exam, assignments (most of which will contribute toward your team project), and a final presentation of your study. In addition, every student will need to participate as human subjects in 3 (tentative number) of the studies your classmates are conducting.

Note: This class does not have a TA, and so not all assignments can be graded formally. You will not know in advance which assignments will be graded formally and which will not.

I have high expectations, and expect performance worthy of graduate students in computer science. Thus, in this class, "A" does not mean "adequate" or "nothing wrong" -- it means "excellent". For an A, you should expect to dig deep and get the most you can out of the class.


Tentative schedule

Subject to change! Please check back every week for updates.

Things I may work in if time permits:

Handy link to internal page.

Margaret M. Burnett
Date of last update: Feb. 25, 2014