Google Summer of Code 2019

In the following we shortly present the open source frameworks that are currently participating for this year's Google Summer of Code. This can be useful if you would like to propose your own topic.

Organization page.

For the proposed topics see section further below.

For Q/A check out our gitter page.

pracmln is a toolbox for statistical relational learning (SRL) and reasoning and as such also includes tools for standard probabilistic graphical models. pracmln is an implementation of Markov logic networks (MLN) that supports efficient learning and inference in relational domains. Its learning and inference engines are entirely written in the Python programming language. Markov logic networks generalize both first-order logic and probabilistic graphical models and have proven successful in many real-world applications such as natural-language understanding and robot perception. This makes MLNs one of the most general and most powerful representation formalisms for uncertain knowledge.

pracmln was designed with the particular needs of technical systems in mind. Our methods are geared towards practical applicability and can easily be integrated into other applications. The availability of graphical tools makes both learning or inference sessions a user-friendly experience. Scripting support enables automation, and a browser-based web application is intended for teaching and getting familiar with MLNs.

pracmln is an open-source project hosted on GitHub. It has its own project page (http://www.pracmln.org) that provides extensive documentation and tutorials that facilitate getting started with MLNs. It is provided as a pip package in the Python package index (PyPI).

RobCoG (Robot Commonsense Games) is a framework consisting of various open source games and plugins (https://github.com/robcog-iai) written in the Unreal Engine with the intention of collecting and equipping robots with commonsense and naive physics knowledge. Various Game prototypes are created where users are asked to execute kitchen related tasks. During gameplay developed game plugins automatically collecting symbolic and sub-symbolic data. The automatically annotated data is then stored in the web-based knowledge service openEASE. This allows robots to access it and reason about it.

The games are split into two categories: (1) VR/Full Body Tracking with physics based interactions, where data as close as possible to reality is collected. The users are immersed in a virtual environment and are asked to perform tasks using natural movements. (2) FPS style, web-based games, where the users interact with objects using a keyboard and mouse. This allows for easy crowdsourcing capabilities since these games could be run from a browser (http://open-ease.org/robcogweb). The data will be less precise for more low level learning, however still valuable at a more higher level (e.g. positioning of objects, the order of executing actions etc.).

KnowRob is a knowledge processing system that combines knowledge representation and reasoning methods with techniques for acquiring knowledge from different sources and for grounding the knowledge in a physical system. It provides robots with knowledge to be used in their tasks, for example action descriptions, object models, environment maps, and models of the robot's hardware and capabilities. The knowledge base is complemented with reasoning methods and techniques for grounding abstract, high-level information about actions and objects in the perceived sensor data.

KnowRob became the main knowledge base in the ROS ecosystem and is actively being used in different academic and industrial research labs around the world. Several European research projects use the system for a wide range of applications, from understanding instructions from the Web (RoboHow), describing multi-robot search-and-rescue tasks (SHERPA), assisting elderly people in their homes (SRS) to industrial assembly tasks (SMErobotics).

KnowRob is an open-source project hosted at GitHub that also provides extensive documentation on its website – from getting-started guides to tutorials for advanced topics in robot knowledge representation.

OpenEASE is a generic knowledge database for collecting and analyzing experiment data. Its foundation is the KnowRob knowledge processing system and ROS, enhanced by reasoning mechanisms and a web interface developed for inspecting comprehensive experiment logs. These logs can be recorded for example from complex CRAM plan executions, virtual reality experiments, or human tracking systems. OpenEASE offers interfaces for both, human researchers that want to visually inspect what has happened during a robot experiment, and robots that want to reason about previous task executions in order to improve their behavior.

The OpenEASE web interface as well as further information and publication material can be accessed through its publicly available website. It is meant to make complex experiment data available to research fields adjacent to robotics, and to foster an intuition about robot experience data.

In the following, we list our proposals for the Google Summer of Code topics that contribute to the aforementioned open-source projects.

Main Objective: The current implementation of pracmln is entirely written in pure Python. The main objective of this project is to port the computationally heavy parts of the learning and inference algorithms to Cython, an extension to Python that allows static compilation of Python modules to C libraries. Cython allows to add static type declarations to Python, which can significantly speed up execution (up to a factor of 1000 compared to regular Python code).

As most of the Python libraries for machine learning and scientific computing (e.g. scikit-learn, numpy) make use of static compilation, it is expected that the practical applicability of Markov logic networks will substanially be pushed to more demanding real-world scenarios.

Task Difficulty: There is alreay a high-quality code base, however, solving this task properly requires understanding and experience in gradient-based optimization, machine learning and logics.

Requirements: Good programming skills in the Python programming language (CPython/Cython), experience in Artificial Intelligence and Machine Learning (ideally SRL technques and logic). Knowledge about C/C++ will be very helpful.

Expected Results: The core components of pracmln, i.e. the learning and inference modules are expected to be ported to the Cython language exploiting the means of implementational optimization provided by Cython.

Remarks: If you have questions about this project in advance, about your application, qualification or ways to get started, please post your question in the pracmln Gitter chat.

Main Objective: Integrating PhysX (3.4/4.0) Articulations in Unreal Engine.

Task Difficulty: Medium, as both frameworks have a large user base for finding examples. The main difficulty lays in the correct integration with UE4.

Requirements: Good programming skills in C++. Good knowledge of the Unreal Engine and PhysX API. Experience with physics engines and robotic simulators is a plus.

Expected Results: To have a seamless integration of the PhysX articulation models in Unreal. Thus allowing precise simulations of robots.

Assignment: In order to be considered for this topic you will need to solve a short programming assignment described here.

Contact / Q&A: Gitter

Since ROS has cross platform support, it would be of a great advantage to wrap it as a module in the Unreal Engine framework. This would greatly improve communication between our RobCoG modules and the ROS world. As a further step the module should be extended to work under Linux as well. This can be done using the unreal build system (UBT).

Task Difficulty: The task is to be placed in the medium difficulty level, as it requires programming skills of various frameworks (ROS, Linux, Unreal Engine).

Requirements: Good programming skills in C++. Good knowledge of the Unreal Engine API. Experience with ROS, C++ library linkage in Unreal Engine.

Expected Results We expect to have an integrated communication level with ROS and Unreal Engine on Windows and Linux side.

Assignement: In order to be considered for this topic you need to solve a short programming assignement described here.

Contact / Q&A: Gitter

For this topic we would like to extend the modules from RobCoG with intuitive Unreal Engine Editor Panels. This would allow easier and faster manipulation/visualization of various parameters.

Task Difficulty: The task is to be placed in the easy difficulty level, as it only requires familiarity with the SLATE framework from Unreal Engine.

Requirements: Good programming skills in C++. Good knowledge of the Unreal Engine API. Experience with the SLATE framework.

Expected Results We expect to have intuitive Unreal Engine UI Panels for editing, visualizing various RobCoG plugins data and features.

Assignment: In order to be considered for this topic you need to solve a short programming assignment described here.

Contact / Q&A: Gitter

For this topic we would like to create a minimal version of openEASE in Unreal Engine. A user should be able to connect to a KnowRob instance from the Unreal Engine Editor and perform various queries. For example to verify if the items from the Unreal Engine world are present in the ontology of the robot. It should be able to upload new data directly from the editor.

Task Difficulty: The task is to be placed in the medium difficulty level, as it required knowledge of various frameworks/libraries (Unreal Engine, openEASE, KnowRob SLATE)

Requirements: Good programming skills in C++. Good knowledge of the Unreal Engine API. Experience with the SLATE framework.

Expected Results We expect to have a minimal version of openEASE in the Unreal Engine editor.

Assignment: In order to be considered for this topic you need to solve a short programming assignment described here.

Contact / Q&A: Gitter