Publications

Sort by: [year] [type] [author]

Shared Autonomy Perception and Manipulation of Physical Device Controls

Matthew Rueben and William D. Smart.
In "Proceedings of the 12th International Symposium on Visual Computing (ISVC)", volume 9475, pages 830-841, Las Vegas, NV, 2015.

As robots begin to enter our homes and workplaces, they will have to deal with the devices and appliances that are already there. These devices are invariably designed with human perception and manipulation abilities in mind. Unfortunately, this often makes them hard for robots to interact with autonomously. Control elements, such as buttons, switches, and knobs, are often hard to identify with current sensors. Even when they are found, it is often not clear how they should be manipulated to achieve a specific goal without extensive background knowledge of the specific device and task.

In this paper, we describe a shared-autonomy approach to the identification and operation of these device controls. A human operator provides assistance with perception and high-level planning, while the robot takes care of the low-level actions that depend on closed-loop sensor feedback.

We demonstrate our approach by controlling a consumer electronics device, despite not being able to autonomously sense some of its control elements, and give the results of some initial evaluations suggesting that the shared autonomy interface is both easier to use and more efficient than a direct teleoperation interface.

Paper: [PDF]

@inproceedings{isvc2015,
  author = {Rueben, Matthew and Smart, William D.},
  title = {Shared Autonomy Perception and Manipulation of Physical Device Controls},
  booktitle = {Proceedings of the 12th International Symposium on Visual Computing ({ISVC})},
  series = {Lecture Notes in Computer Science},
  volume = {9475},
  pages = {830--841},
  address = {Las Vegas, NV},
  year = {2015}
}