End-User Software Engineering

Overview

The objective of this project is to improve the reliability of software produced by end-user programming languages in general, and by spreadsheet languages in particular. Our approach is to address software engineering issues as an integrated whole in ways that incrementally interact with each other and with the user. For example, the system will notice the user's reactions to sample values she tests, and will then suggest general principles about the spreadsheet, encouraging the user to refine or modify the suggestions. This collaboration between the system and user will incrementally generate formal specifications as the spreadsheet evolves. These specifications can in turn be fed back to enhance reliability -- by automatically suggesting appropriate test values, by helping locate faults, and by ensuring continuing consistency with the specifications.

The research involves three facets: developing an interactive mechanism for user-system collaboration, developing algorithms for the system's part of the collaboration, and conducting experiments to evaluate effectiveness. This is the first research attempting to bring fundamental software engineering principles to bear on end-user programming. Since the use of end-user-written programs and spreadsheets is widespread and their lack of reliability is pervasive, improved reliability will impact a potentially huge number of business and personal computer users.

This project's inspirations and beginnings came from our work on the WYSIWYT methodology for testing.

Our work on end-user software engineering eventually led to the founding of the EUSES Consortium in 2003. See that web page for the most recent results of our work and that of our collaborators.

We welcome the opportunity to collaborate with other organizations on this work. If you are interested in pursuing this, please contact one of the Principal Investigators.

Principal Investigators

• Margaret Burnett, Oregon State University (burnett@cs.orst.edu)
• Gregg Rothermel, Oregon State University (grother@cs.orst.edu)
• Curtis Cook, Oregon State University (cook@cs.orst.edu)

Outside Consultant

• Alan Blackwell, Cambridge University, U.K. (Alan.Blackwell@cl.cam.ac.uk)

Graduate and Undergraduate Student Researchers (Past and Present)

• Miguel Arredondo-Castro (M.S. student)
• Laura Beckwith (Ph.D. student)
• Darren Brown (Ph.D. student)
• Sudheer Chekka (M.S. student)
• Eugene Rogan Creswick (Ph.D. student)
• Marc Fisher (Ph.D. student)
• Dalai Jin (M.S. student)
• Vijay Krishna (M.S. student)
• Omkar Pendse (M.S. student)
• Bing Ren (M.S. student)
• Prashant Shah (M.S. student)
• Jay Summet (M.S. student)
• Christine Wallace (M.S. student)

• Joshua Cantrell (undergraduate research student)
• Ledah Casburn (undergraduate research student)
• Mike Durham (undergraduate research student)
• Orion Granatir (undergraduate research student)
• Daniel Keller (undergraduate research student)
• Andrew Ko (undergraduate research student)
• John LeHoullier (undergraduate research student)
• Martin Main (undergraduate research student)
• Aaron Wilson (undergraduate research student)

Some of Our Papers Related to this Project

• See the EUSES Consortium for papers about end-user software engineering 2003 and later. That site contains a large collection of resources in this area, by us, by our collaborators. and by other researchers.
• "End-User Software Visualizations for Fault Localization", J. Ruthruff, E. Creswick, M. Burnett, C. Cook, S. Prabhakararao, M. Fisher II, M. Main, ACM Symposium on Software Visualization, San Diego, CA, June 2003.
• "End-User Software Engineering with Assertions in the Spreadsheet Paradigm," Margaret Burnett, Curtis Cook, Omkar Pendse, Gregg Rothermel, Jay Summet, and Christine Wallace, International Conference on Software Engineering, Portland, OR, May 2003.
• "Harnessing Curiosity to Increase Correctness in End-User Programming", Aaron Wilson, Margaret Burnett, Laura Beckwith, Orion Granatir, Ledah Casburn, Curtis Cook, Mike Durham, and Gregg Rothermel, ACM Conference on Human Factors in Computing Systems, Ft. Lauderdale, FL, April 5-10 2003.
• "Test Reuse in the Spreadsheet Paradigm", Marc Fisher II, Dalai Jin, Gregg Rothermel, Margaret Burnett, IEEE International Symposium on Software Reliability Engineering, Nov. 2002. (PDF format.)
• "Improving the Design of Visual Programming Language Experiments Using Cognitive Walkthroughs", Andrew Ko, Margaret Burnett, Thomas Green, Karen Rothermel, and Curtis Cook, Journal of Visual Languages and Computing 13(5), Oct. 2002, pp. 517-544. (Postscript format).
• "Reasoning about Many-to-Many Requirement Relationships in Spreadsheets," Laura Beckwith, Margaret Burnett, and Curtis Cook, IEEE Symposia on Human-Centric Computing Languages and Environments, Arlington, VA, Sept. 2002 PDF
• "Assertions in End-User Software Engineering: A Think-Aloud Study" (Tech Note), Christine Wallace, Curtis Cook, Jay Summet, and Margaret Burnett, IEEE Symposium on Human-Centric Languages, Sept. 2002.
• "Applying Attention Investment to End-User Programming" (Tech Note), Alan Blackwell and Margaret Burnett, IEEE Symposium on Human-Centric Languages, Sept. 2002.
• "End-User Programming of Time as an 'Ordinary' Dimension in Grid-Oriented Visual Programming Languages", Margaret Burnett, Nanyu Cao, Miguel Arredondo-Castro, John Atwood, Journal of Visual Languages and Computing 13(4) pp. 421-447, Aug. 2002.
• "Testing Homogeneous Spreadsheet Grids with the 'What You See Is What You Test' Methodology", M. Burnett, A. Sheretov, B. Ren, G. Rothermel, IEEE Trans. Software Engineering, June 2002, 576-594. PDF
• "Automated Test Case Generation for Spreadsheets," Marc Fisher, Mingming Cao, Gregg Rothermel, Curtis R. Cook, Margaret M. Burnett Int'l. Conf. on Software Engineering, Orlando, Florida, 141-151, May 19-25, 2002. PDF
• "Incorporating Incremental Validation and Impact Analysis into Spreadsheet Maintenance: An Empirical Study," Vijay Krishna, Curtis Cook, Daniel Keller, Joshua Cantrell, Chris Wallace, Margaret Burnett, and Gregg Rothermel, IEEE International Conference on Software Maintenance, Nov. 2001, Florence, Italy, pp. 72-81.
• "Visually Testing Recursive Programs in Spreadsheet Languages," Margaret Burnett, Bing Ren, Andrew Ko, Curtis Cook, and Gregg Rothermel, IEEE Symposium on Human-Centric Languages: Visual/Multimedia Approaches to Programming and Software Engineering (HCC'01), Sept. 01, Stresa, Italy, pp. 288-295.
• "What Visual Programming Research Contributes to Universal Access" (Invited Paper), M. Burnett, HCI International: Conference on Universal Access in Human-Computer Interaction, New Orleans, Aug. 5-10, 2001.
• "Software Engineering for Visual Programming Languages," Margaret Burnett, Handbook of Software Engineering and Knowledge Engineering, Vol. 2, World Scientific Publishing Company, June 2001, 77-92. PDF
• "Forms/3: A First-Order Visual Language to Explore the Boundaries of the Spreadsheet Paradigm," M. Burnett, J. Atwood, R. Djang, H. Gottfried, J. Reichwein, S. Yang, Journal of Functional Programming 11(2), March 2001, 155-206. Postscript pre-print
• "A Methodology for Testing Spreadsheets," G. Rothermel, M. Burnett, L. Li, C. DuPuis, and A. Sheretov, ACM Trans. Software Engineering and Methodology 10(1), Jan. 2001, 110-147. PDF
• "Time in Grid-Oriented VPLs: Just Another Dimension?" Margaret Burnett, Nanyu Cao, and John Atwood, IEEE Symposium on Visual Languages, Seattle, WA, September 2000, 137-144.
• "Scaling Up a 'What You See Is What You Test' Methodology to Testing Spreadsheet Grids", Margaret M. Burnett, Andrei Sheretov, Gregg Rothermel, 1999 IEEE Symposium on Visual Languages, Tokyo, Japan, Sept. 1999. Postscript
• "Slicing Spreadsheets: An Integrated Methodology for Spreadsheet Testing and Debugging," James Reichwein, Gregg Rothermel, Margaret Burnett, Conference on Domain Specific Languages, Austin, Texas, October 1999. PDF or Postscript
• "What You See Is What You Test: A Methodology for Testing Form-Based Visual Programs," Gregg Rothermel, Lixin Li, Christopher DuPuis, Margaret Burnett, 1998 International Conference on Software Engineering, April 19-25, 1998, Kyoto, Japan, 198-207. Postscript

Applicability to Other Visual Programming Languages and Environments

The widespread use of spreadsheet-like VPLs makes it imperative that mechanisms to improve reliability be supported in a manner suited to the users of these languages as soon as possible. However, doing so is also expected to yield new insights into ways to facilitate in other kinds of end-user programming environments and ultimately, could even impact interactive visual programming languages and environments used by professional programmers.

Empirical Results

Our empirical results are listed here.

Research Infrastructure

Date of last update: Jul. 23, 2005