RoboHow

Web-enabled and Experience-based Cognitive Robots that Learn Complex Everyday Manipulation Tasks

The vision of Robohow is to develop cognitive service robots that can autonomously perform a large number of complex everyday manipulation tasks in real-world settings. Our goal is to realize a programming framework that enables programmers to semi-automatically expand autonomous service robot applications with little effort. Mechanisms that enable robots to acquire skills with ease are crucial for enabling them to function autonomously in complex contexts while performing a wide range of complex tasks. These tasks cannot be pre-programmed because the set of potential tasks is so large that the tasks cannot be known in advance.

The Robohow framework represents control programs as concurrent, perception-controlled manipulation plans. Websites, visual instructions, and haptic demonstration are used as primary sources of information. These heterogeneous pieces of information will be integrated and combined together through an interface layer that provides an abstract machine for programming high-level robot manipulation plans. The interpreter for this abstract machine includes novel mechanisms for optimization and constraint-based movement specification as well as percept-guided manipulation.

The framework is designed to cope with many challenges of working in knowledge-intensive, open-world settings. For example, imagine a robot that needs to know how to make a pancake. Existing resources that are widely available for performing human-scale activities, such as wikihow.com, typically consist of vague, incomplete and ambiguous instructions. An instruction such as “stir the batter until the texture is smooth” does not specify what the batter needs to be stirred with, what exact trajectory one has to follow, and with what force or speed. Instead, the robot must be able to decide how to perform appropriate actions on appropriate objects are for each task.

The Intelligent Autonomous Systems group at University Bremen is responsible for coordinating this collaborative project. We create knowledge representation methods to integrate all knowledge sources (web, vision, simulation, teaching), develop plan-based control methods to execute actions using a constraint-based controller, and investigate plan transformations.

We believe in taking personal robotics to the next level through active and free knowledge sharing. The methodology and software for (self)improvement of robots under “open world” conditions developed by Robohow will be made available to the community. We will also refactor and extend existing open source robotics software to support this and make results easily accessible.

For more information and news, visit robohow

Partners:

This project has received funding from the European Union Seventh Framework Programme FP7/2007-2013