CS 589/569 Winter 2014
Project Resources

(To understand humans' use of languages, environments, and practices of software development)

Project ideas

Standard reporting of empirical studies

Derived from APA guidelines.

Research questions

Usually these are included in the paper's Introduction section. If they aren't there (or if only an overview question is there), put them in full detail here.

Method

This section must be very detailed and clear. It tells the reader that someone else can repeat the experiment just by reading your method section. The method section generally consists of three subsections: participants, apparatus (or materials), and procedure. A fourth, optional, subsection is design.

Participants

The age, sex, and any other relevant demographic data are presented here. State how many subjects participated, how they were selected, and how they were assigned to groups.

Apparatus or Materials

A description of the apparatus used is given here. In the case of standard laboratory equipment, rather than describing the entire apparatus, the company name and model and/or serial number is sufficient. If this is not possible, the equipment should be described in detail.

If materials (such as a questionnaire) were used, either cite your source (if published materials were used) or provide a copy in the appendix of your paper if you devised the instrument yourself. You should describe the instrument in your materials section. For example,

A 50-item six-point Likert-type questionnaire was devised by the experimenter to measure attitudes toward authority figures. Half of the questions were worded such that....The highest (positive) score that could be attained on the measure was 300; the lowest (negative) score was 50. Thus, higher scores reflected more positive attitudes toward authority figures.

Procedure

This section describes what the experimenter did and how it was done. (And why you made any choices that aren't obvious.) It is a detailed description of the events that the experimenter went through from the beginning until the end of the study. Such things as experimental and control group assignment to conditions, order or manner of experimental treatment presentation, and a summary of the instructions to the participants are presented here. Include a statement about your research design and the operational definitions of your variables. (If your design is complex, a separate section can be designated for this information.)

Data

This section is often very short and combined with Procedure, but for this class we want it in detail, and separate. This will enable us to think through your experiment plan in advance.

Results

This section is where you present your data and analyses. The experimenter gives a description and not an explanation of the findings of the experiment. In order to fulfill this requirement, the results section should include descriptive statistics (rather than the raw data) and statistical tests if used.

Regarding your statistics, be very clear about what kind of test you ran on what data. So, nclude degrees of freedom used, obtained values of inferential statistics performed, probability level, and direction of effect. Italicize letters used as statistical symbols, such as "N", "F", "t", "SD", and "p." Make reference to any figures and tables used, for example, "(see Table 1)."

The reference to the table or figure should be close to the relevant material in the text. Never use a figure or table without referring to it in the text.

Tables are often used when presenting descriptive statistics such as means, standard deviations and correlations. Pictures, graphs, and drawings are referred to as figures. Tables/figures should be used as supplements, not to do the entire job of communication. (See the APA manual for detailed guidelines for Tables and for Figures.)

Generally, one reports descriptive statistics, inferential statistics, and states in words what was found.

Discussion

In this section, you state your conclusions on the basis of your analyses. The conclusions should be related to the questions raised in your introduction section. How is this study, and these results, relevant to the field? You should open the discussion section with a statement of support or nonsupport for your original hypothesis. You may want to point out differences or similarities between other points of view and your own. You may remark on certain shortcomings of the study, but avoid dwelling on flaws. In general, this section allows you relatively free rein to examine, interpret, and qualify your results.

Because a separate "Threats to Validity" section is expected in SE and PL empirical papers, we will have one here too. Make it a subsection of your discussion section.

Other resources for your project


Margaret M. Burnett
Date of last update: Feb. 10, 2014