The overall goal of this project is to learn to perform multiple sequential decision tasks by actively interacting with an in-situ expert and transferring this knowledge across different tasks. The tasks might involve strategic thinking as in playing a real-time strategy game or efficiently searching for targets in a space using limited resources, or more simpler control tasks such as balancing a bicycle or a cart-pole. We are exploring several subproblems of this challenging research topic including Bayesian transfer learning, interactive reinforcement learning, and active imitation learning.

In Bayesian transfer learning, the task knowledge is organized hierarchically into different classes, where related tasks fall under the same class. The number and the definition of classes is variable and learned from experience using the framework of hierarchical Dirichlet processes. The classes correspond to similar Markov Decision processes in model-based reinforcement learning and or similar role-based policies in moodel-free learning.

In interactive reinforcement learning, the goal is to accelerate reinforcement learning by having an expert critique the trajectories generated by the learner and offer advice. The learner combines the self-practice sessions with critique sessions which makes it possible to converge more quickly than either practice or critique by themselves.

In active imitation learning, we are exploring a number of approaches where the learning agent can actively ask queries to quickly learn to imitate the expert. In one approach, we ask state-based queries about what action to take in a given state. In a second approach we ask queries about preferences between multiple trajectories generated by different policies. More recently we are developing approaches that combine priors over utilities with expert advice on actions.