End-User Software Engineering
(Research Activities until 2003. See the EUSES Consortium Page for 2003-on.)
The objective of this project is to improve
the reliability of software produced by
end-user programming languages in general, and by spreadsheet
languages in particular. Our approach is to address software
engineering issues as an integrated whole in ways that incrementally
interact with each other and with the user. For example, the system
will notice the user's reactions to sample values she tests, and will
then suggest general principles about the spreadsheet, encouraging the
user to refine or modify the suggestions. This collaboration between
the system and user will incrementally generate formal specifications
as the spreadsheet evolves. These specifications can in turn be fed
back to enhance reliability -- by automatically suggesting appropriate
test values, by helping locate faults, and by ensuring continuing
consistency with the specifications.
The research involves three
facets: developing an interactive mechanism for user-system
collaboration, developing algorithms for the system's part of the
collaboration, and conducting experiments to evaluate effectiveness.
This is the first research attempting to bring fundamental software
engineering principles to bear on end-user programming. Since the use
of end-user-written programs and spreadsheets is widespread and
their lack of reliability is pervasive, improved reliability will
impact a potentially huge number of business and personal computer
This project's inspirations and beginnings came from our work on the
Our work on end-user software engineering eventually led to
the founding of the EUSES
Consortium in 2003. See that web page for the most recent results
of our work and that of our collaborators.
We welcome the opportunity to collaborate with other organizations on
this work. If you are interested in pursuing this, please contact one of the Principal Investigators.
- Alan Blackwell, Cambridge University, U.K. (Alan.Blackwell@cl.cam.ac.uk)
Graduate and Undergraduate Student Researchers (Past and Present)
- Miguel Arredondo-Castro (M.S. student)
- Laura Beckwith (Ph.D. student)
- Darren Brown (Ph.D. student)
- Sudheer Chekka (M.S. student)
- Eugene Rogan Creswick (Ph.D. student)
- Marc Fisher (Ph.D. student)
- Dalai Jin (M.S. student)
- Vijay Krishna (M.S. student)
- Omkar Pendse (M.S. student)
- Bing Ren (M.S. student)
- Prashant Shah (M.S. student)
- Jay Summet (M.S. student)
- Christine Wallace (M.S. student)
- Joshua Cantrell (undergraduate research student)
- Ledah Casburn (undergraduate research student)
- Mike Durham (undergraduate research student)
- Orion Granatir (undergraduate research student)
- Daniel Keller (undergraduate research student)
- Andrew Ko (undergraduate research student)
- John LeHoullier (undergraduate research student)
- Martin Main (undergraduate research student)
- Aaron Wilson (undergraduate research student)
Some of Our Papers Related to this Project
- See the EUSES
Consortium for papers about end-user software engineering 2003
and later. That site contains a large collection of resources in this
area, by us, by our collaborators. and by other researchers.
- "End-User Software Visualizations for Fault Localization", J. Ruthruff, E. Creswick, M. Burnett, C. Cook, S. Prabhakararao, M. Fisher II, M. Main,
ACM Symposium on Software Visualization, San Diego, CA, June 2003.
- "End-User Software Engineering with Assertions in the Spreadsheet Paradigm,"
Margaret Burnett, Curtis Cook, Omkar Pendse, Gregg Rothermel, Jay Summet, and Christine Wallace,
International Conference on Software Engineering,
Portland, OR, May 2003.
- "Harnessing Curiosity to Increase Correctness in End-User Programming", Aaron Wilson, Margaret Burnett, Laura Beckwith, Orion Granatir, Ledah Casburn, Curtis Cook, Mike Durham, and Gregg Rothermel, ACM Conference on Human Factors in Computing Systems, Ft. Lauderdale, FL, April 5-10 2003.
"Test Reuse in the Spreadsheet Paradigm", Marc Fisher II, Dalai Jin,
Gregg Rothermel, Margaret Burnett, IEEE International Symposium on
Software Reliability Engineering, Nov. 2002.
- "Improving the Design of Visual Programming Language Experiments Using
Cognitive Walkthroughs", Andrew Ko, Margaret Burnett, Thomas Green,
Karen Rothermel, and Curtis Cook, Journal of Visual Languages and
Computing 13(5), Oct. 2002, pp. 517-544. (Postscript format).
"Reasoning about Many-to-Many Requirement Relationships in
Spreadsheets," Laura Beckwith, Margaret Burnett, and Curtis Cook,
IEEE Symposia on Human-Centric Computing Languages and Environments,
Arlington, VA, Sept. 2002
"Assertions in End-User Software Engineering: A Think-Aloud Study"
(Tech Note), Christine Wallace, Curtis Cook, Jay Summet, and Margaret
Burnett, IEEE Symposium on Human-Centric Languages,
"Applying Attention Investment to End-User Programming" (Tech Note),
Alan Blackwell and Margaret Burnett, IEEE Symposium on Human-Centric
Languages, Sept. 2002.
- "End-User Programming of Time as an 'Ordinary' Dimension in Grid-Oriented
Visual Programming Languages", Margaret Burnett, Nanyu Cao, Miguel
Arredondo-Castro, John Atwood, Journal of Visual Languages and
Computing 13(4) pp. 421-447, Aug. 2002.
- "Testing Homogeneous Spreadsheet Grids with the 'What You See Is What
You Test' Methodology", M. Burnett, A. Sheretov, B. Ren, G. Rothermel,
IEEE Trans. Software Engineering, June 2002, 576-594.
"Automated Test Case Generation for Spreadsheets," Marc Fisher,
Mingming Cao, Gregg Rothermel, Curtis R. Cook, Margaret M. Burnett
Int'l. Conf. on Software Engineering, Orlando, Florida,
141-151, May 19-25, 2002.
- "Incorporating Incremental Validation and Impact Analysis into
Spreadsheet Maintenance: An Empirical Study," Vijay Krishna, Curtis
Cook, Daniel Keller, Joshua Cantrell, Chris Wallace, Margaret Burnett,
and Gregg Rothermel, IEEE International Conference on Software
Maintenance, Nov. 2001, Florence, Italy, pp. 72-81.
- "Visually Testing Recursive Programs in Spreadsheet Languages,"
Margaret Burnett, Bing Ren, Andrew Ko, Curtis Cook, and Gregg
Rothermel, IEEE Symposium on Human-Centric Languages:
Visual/Multimedia Approaches to Programming and Software Engineering
(HCC'01), Sept. 01, Stresa, Italy, pp. 288-295.
- "What Visual Programming Research Contributes to Universal Access"
(Invited Paper), M. Burnett, HCI International: Conference on
Universal Access in Human-Computer Interaction, New Orleans,
Aug. 5-10, 2001.
"Software Engineering for Visual Programming Languages," Margaret Burnett, Handbook of Software Engineering and Knowledge Engineering, Vol. 2, World Scientific Publishing Company, June 2001, 77-92.
"Forms/3: A First-Order Visual Language to Explore the Boundaries of
the Spreadsheet Paradigm," M. Burnett, J. Atwood, R. Djang, H. Gottfried,
J. Reichwein, S. Yang, Journal of Functional Programming 11(2), March 2001,
- "A Methodology for Testing Spreadsheets," G. Rothermel, M. Burnett,
L. Li, C. DuPuis, and A. Sheretov, ACM Trans. Software Engineering
and Methodology 10(1), Jan. 2001, 110-147.
- "Time in Grid-Oriented VPLs: Just Another Dimension?" Margaret
Burnett, Nanyu Cao, and John Atwood, IEEE Symposium on Visual
Languages, Seattle, WA, September 2000, 137-144.
"Scaling Up a 'What You See Is What You Test' Methodology to Testing
Spreadsheet Grids", Margaret M. Burnett, Andrei Sheretov, Gregg
Rothermel, 1999 IEEE Symposium on Visual Languages, Tokyo, Japan,
"Slicing Spreadsheets: An Integrated Methodology for Spreadsheet
Testing and Debugging," James Reichwein, Gregg Rothermel, Margaret
Burnett, Conference on Domain Specific Languages, Austin, Texas,
PDF or Postscript
- "What You See Is What You Test: A Methodology for Testing Form-Based Visual Programs," Gregg Rothermel, Lixin Li,
Christopher DuPuis, Margaret Burnett, 1998 International Conference
on Software Engineering, April 19-25, 1998, Kyoto, Japan, 198-207.
Applicability to Other Visual Programming Languages and Environments
The widespread use of spreadsheet-like VPLs makes it imperative
that mechanisms to improve reliability be supported in a manner suited
to the users of these languages as soon as possible. However, doing so
is also expected to yield new insights into ways to facilitate
in other kinds of end-user programming environments and ultimately, could even
interactive visual programming languages and environments used by professional
Our empirical results are listed here.
Prototype spreadsheet-like visual programming language -- Forms/3
Date of last update: Jul. 23, 2005