The SUTURO VaB team video showing our results and scientific expertise.
Welcome to the SUTURO VaB team! It is a joint team between the Institute for Artificial Intellience at the University of Bremen(IAI) and the TU Vienna(TUW).
By joining forces, our team for RoboCup@Home combines state-of-the-art robotic perception (TUW) with state-of-the-art robot control architectures (IAI).
Both parties in the SUTURO VaB team are strongly interested in autonomous mobile manipulation on robot platforms.
The robot control architecture developed by the IAI is also integrated within the CRC 1320 Everyday Activity Science and Engineering (EASE 🔗).
EASE is an interdisciplinary research center at the University of Bremen that investigates everyday activity science & engineering.
Its core purpose is to advance the understanding of how human-scale manipulation tasks can be mastered by robotic agents.
Within EASE, general software frameworks for robotic agents are created and further developed.
These frameworks (CRAM, KnowRob, Giskard, and RoboKudo) can also be well applied to the RoboCup@Home context, since there is a substantial overlap in robot capabilities required to pass the RoboCup@Home challenges with what capabilities robots within EASE provide.
SUTURO VaB participants will benefit from expertise acquired by EASE researchers through multiple years of experience in working with autonomous robots performing everyday activities.
TUW delivers a comprehensive suite of perception modules that provide semantic information, such as object identities, locations and grasp configurations.
The focus of the research of these perception approaches is the target domain of mobile robot manipulation.
We are looking forward to successfully employing and extending our approaches based on the exiciting challenges in the RoboCup@Home domain.
Reactive placing using the force-torque-sensor.
Failure detection and recovery. This gif is accelerated to approximately 200% speed.
Opening doors and detecting if the gripper slips off.
==== Motivation and Goal====
We are participating in the 27th edition of the RoboCup@Home which will take place in Eindhoven in 2024.
RoboCup@Home wants to provide a stage for service robots and the problems these robots can tackle, like interacting with humans and helping them in their everyday-life. In the competition, the participants have to absolve different challenges in two stages. We chose to go for the following challenges:
Stage 1
Carry my Luggage
GPSR
Receptionist
Serve Breakfast
Storing Groceries
Stage 2
Clean the Table
EGPSR
Restaurant
The RoboCup@Home presents a series of increasingly challenging tasks designed to advance the development of autonomous domestic service robots. These tasks, set within a home-like environment, require robots to demonstrate a variety of skills that are crucial for real-world applications.
Common key challenges include object recognition and manipulation, where robots must accurately identify and interact with various household items. This demands sophisticated vision systems capable of processing complex visual information in dynamic settings. Navigation and obstacle avoidance are also critical, necessitating advanced perception-action loops for smooth and safe movement within a household space. These robots must also possess a robust knowledge base to understand and execute complex, underspecified commands.
RoboCup@Home tasks are very broad in nature and require the robot to act competently in areas like human robot interaction, spatial reasoning, natural language understanding or object manipulation.
Effective planning, reasoning and decision-making algorithms are essential for such robots to autonomously perform tasks like assisting humans, cleaning, or fetching objects.
Solving such challenges requires a complex architecture of specialized software frameworks developed for robotic applications. Our system is split into integrated subsystems of Perception, Manipulation, Knowledge, Natural Language Processing and Planning. The methodology of these subsystems is explained below.
Our team has participated in previous RoboCup@Home events (German Open 2019, World 2021, World 2023).
In 2023, we have placed 2. in the Carry my Luggage task in the DSPL league.
====Team====
We are happy to present our whole team:
Team Lead
{{:robocupfiles:suturo23-24_team_lead.jpg?nolink&600 |}}
Vanessa Hassouna hassouna[at]uni-bremen[.]de
Team Co-Lead
{{:robocupfiles:suturo23-24_team_co-lead.jpg?nolink&600 |}}
Knowledge: To fulfill complex tasks a robot needs knowledge and memory of its environment. While the robot acts in its world, it recognizes objects and manipulaties them through pick-and-place tasks. With the use of KnowRob, a belief state provides episodic memory of the robot's experience, recording the robot's memory of each cognitive activity. Via ontologies objects can be classified and put into context, which enables logical reasoning over the environment and intelligent decision making.
Manipulation:
For manipulation tasks in the environment, we use the open source motion planning framework Giskard.
It uses constraint and optimization based task space control to generate trajectories for the whole body of mobile manipulators.
Giskard offers interfaces to plan and execute motion goals and to modify its world model.
A selection of predefined basic motion goals can be arbitrarily combined to describe a motion.
If an environment model is present, such goals can also be defined on the environment, e.g., to open a door.
Simulation of the HSR while opening a human sized door. This gif is accelerated to 200% speed. This results with a real-time factor of approximately 0.46 to an effective speed of 92%.
Planning:
Planning is responsible for the high-level control and failure handling of the autonomous robot system by utilizing generic cognitive strategies. It combines the frameworks of perception, knowledge and manipulation within high-level plans written within the CRAM (Cognitive Robot Abstract Machine) system. CRAM enables the implementation of various recovery strategies for failures, as well as a lightweight simulation tool for prospection, allowing to simulate the potential outcome of the current plans and their respective parameters before executing it on the real robot, hereby increasing the success of the performed action by discarding faulty parameters in advance. CRAM also allows the high-level plans to be written generically in a way, so that he plans are robot-platform independent. The plan execution results can furthermore be recorded in order to be reasoned about in the future, adapting and increasing the success of upcoming plan performance.
Perception:
The perception framework has the task to process the visual data received by the robot's camera sensors and establish the communication between the high-level and visual perception. RoboKudo is an open source robotic perception framework based on the principles of unstructured information management.
The framework allows for the creation of perception systems that employ an ensembles of experts approach and treat perception as a question-answering problem.
Based on the queries issued to the system a perception plan is created consisting of a list of experts to be executed.
The perception experts generate object hypotheses, annotate these hypotheses and test and rank them in order to come up with the best possible interpretation of the data and generate the answer to the query. The methods provided by TUW are integrated as experts into the RoboKudo perception framework allowing the reasoning about the perception results and the communication with the high-level planning CRAM to close the perception-action loop.
{{ :robocupfiles:objects_perception01.png?nolink&200 |}}
NLP:
The NLP (Natural Language Processing) team focuses on transforming spoken language
into text using tools like whisper and rasa. Audio is captured by the robot,
filtering keywords essential for the Knowledge and Planning teams.
However, achieving accurate text recognition under diverse conditions,
including different accents or loud ambient noise, can be challenging.
The central goal is about extracting semantics from the given input, recognizing elements
such as locations, persons, and objects. NLP faces the challenge of understanding user
intent, a crucial aspect for precise task execution, ensuring effective human-robot
communication in dynamic environments.
Gayane Kazhoyan, Simon Stelter, Franklin Kenghagho Kenfack, Sebastian Koralewski, and Michael Beetz.
The robot household marathon experiment. In Proc. of IEEE ICRA, pages 9382-9388, 2021.
Welcome to the SUTURO VaB team! It is a joint team between the Institute for Artificial Intellience at the University of Bremen(IAI) and the TU Vienna(TUW).
By joining forces, our team for RoboCup@Home combines state-of-the-art robotic perception (TUW) with state-of-the-art robot control architectures (IAI).
Both parties in the SUTURO VaB team are strongly interested in autonomous mobile manipulation on robot platforms.
The robot control architecture developed by the IAI is also integrated within the CRC 1320 Everyday Activity Science and Engineering (EASE 🔗).
EASE is an interdisciplinary research center at the University of Bremen that investigates everyday activity science & engineering.
Its core purpose is to advance the understanding of how human-scale manipulation tasks can be mastered by robotic agents.
Within EASE, general software frameworks for robotic agents are created and further developed.
These frameworks (CRAM, KnowRob, Giskard, and RoboKudo) can also be well applied to the RoboCup@Home context, since there is a substantial overlap in robot capabilities required to pass the RoboCup@Home challenges with what capabilities robots within EASE provide.
SUTURO VaB participants will benefit from expertise acquired by EASE researchers through multiple years of experience in working with autonomous robots performing everyday activities.
TUW delivers a comprehensive suite of perception modules that provide semantic information, such as object identities, locations and grasp configurations.
The focus of the research of these perception approaches is the target domain of mobile robot manipulation.
We are looking forward to successfully employing and extending our approaches based on the exiciting challenges in the RoboCup@Home domain.
