Objective
To contribute to the computer science community via research in artificial intelligence.
Specifically, I plan to finish my PhD and go on to a career in research, continuing work in reinforcement learning.
I am currently seeking a post-doc position or an industry research position.
Education
B.S. in Computer Science.
Montana State University, Bozeman
PhD in Computer Science
Oregon State University
Expected graduation date: Fall 2009
Computer related skills
Programming languages: C/C++, BASIC, Lisp, Assembly, Perl, SQL
Platforms: Linux, Windows, UNIX
Web Skills and languages: HTML, Java, CGI
Relevant coursework
Software Engineering, Programming Language Design
Theory of Computation, Operating Systems, Compilers
Computer Architecture, Computer Networks
Computer Graphics, Image Processing, Databases
Artificial Intelligence, Reinforcement Learning, Cybernetics
Algorithms and Data Structures, Bayesian Networks, Graph Theory
Work History
Webmaster, Jet Propulsion Laboratories, May 1999-December 1999.
Primary designer of a web page for engineers, served across the JPL intranet.
Programmer, Dyonjet Research, 2001
Primary programmer of numerous in-house utilities. Designed GUI software for release to the public.
Research Assistant, Oregon State University, 2002-Present
Current work includes research on scaling reinforcement learning, particularly in multiagent domains.
Research Contributions
Developed several techniques to mitigate the "three curses of dimensionality" (explosions in state, action, and stochastic
branching factor) of reinforcement learning problems,
including a new kind of function approximation that generalizes both tables and linear functions -- "Tabular Linear Functions" --
and the creation of a new average-reward model-based value iteration algorithm based on afterstates, "ASH-learning".
Developed a new technique -- "Assignment-based Decomposition" -- for decomposing states and actions in multi-agent,
multi-task domains that
greatly mitigates the three curses of dimensionality by dividing the action-selection step of a reinforcement learning
algorithm into two stages: an upper assignment level and a lower task performance level.
Further developed and expanded upon assignment-based decomposition by showing how to integrate coordination
graphs into the lower task performance level, how to use it together with transfer learning to enable multi-agent domains to scale
to large numbers of agents, and how to use search techniques to quickly assign agents to appropriate tasks.
Publications
Proper, S., Tadepalli, P., Tang, H., Logendran, R.,
A Reinforcement Learning Approach for Product Delivery by Multiple Vehicles,
for IIE/IERC 2003: Institute of Industrial Engineers/Industrial Engineering Research Conference.
Proper, S., Tadepalli, P.,
Scaling Average-reward Reinforcement Learning for Product Delivery,
for AAAI Real Life Reinforcement Learning Fall Symposium 2004.
Proper, S., Tadepalli, P.,
Scaling Model-Based Average-reward Reinforcement Learning for Product Delivery,
in ECML 2006: Proceedings of the 17th European Conference on Machine Learning, p 735-742.
Proper, S., Tadepalli, P.,
Solving Multiagent Assignment Markov Decision Processes,
in AAMAS 2009: Proceedings of the 8th International Joint Conference on Autonomous Agents and Multiagent Systems., p 681-688
Proper, S., Tadepalli, P.,
Transfer Learning via Relational Templates,
in ILP 2009: Proceedings of the 19th International Joint Conference on Inductive Logic Programming. (to appear)