Publications

Sort by: [year] [type] [author]

Deterministic, Locally-Quadratic Value-Function Approximations over Belief Space

Tom Erez and William D. Smart.
In "Proceedings of the NIPS 2009 Workshop on Probabilistic Approaches for Robotics and Control", Whistler, BC, 2009.

We present a deterministic algorithm for optimal control in stochastic domains, where both process noise and observation noise may be state- and action-dependent. Our algorithm uses a quadratic, time-dependent approximation of the value function around a sequence of improving nominal trajectories, and exhibits rapid convergence (like Newton's method) near the optimum. Despite being deterministic and quadratic, out approach can adapt the resulting behavior to stochastic features in the system's dynamics. This is achieved by extending the state space to represent a finite-dimensional subset of probability distributions over states, thereby fitting a value function over belief space, rather than configuration space.

@inproceedings{nips09,
  author = {Erez, Tom and Smart, William D.},
  title = {Deterministic, Locally-Quadratic Value-Function Approximations over Belief Space},
  booktitle = {Proceedings of the NIPS 2009 Workshop on Probabilistic Approaches for Robotics and Control},
  address = {Whistler, BC},
  year = {2009}
}