Cyber Valley Research Fund

Support for basic research in AI and robotics

Learning of physics-based models for visio-tactile object perception and manipulation

cyvy Research Project

Today’s autonomous systems still lack behind the versatile capabilities of humans in scene perception and object manipulation tasks. This project addresses the research question of how robots can learn to perceive and manipulate objects from visual and tactile feedback in a self-supervised way. Jörg Stückler and his team will develop methods that will allow robots to learn models of their interaction with objects from camera images and tactile measurements. The scientists will investigate the use of learned models for perception and control in several robotic object manipulation tasks.

MachineData: Machine

cyvy Research Project

When it comes to machine learning, it is all about the data. Machines that are fed large data sets learn to independently discover regularities in the data – just as the brain draws conclusions from observing the environment over and over again. The availability of large data sets has played a major role in recent breakthroughs in machine learning. In recent years, data sets have enabled or improved language processing, speech recognition, or computer vision, to name just a few examples. Some researchers even argue that data is more important than new algorithms. 

However, this is different for machines and engineering systems such as autonomous robots or self-driving cars. In the domain of physical systems, data sets have not yet led to comparable breakthroughs. 


Sebastian Trimpe and his “Intelligent Control Systems” research group are focusing on the question of whether rich and high-quality data sets collected on machines (for example, data collected on an engine) can be similarly beneficial for learning in the context of physical machines. The team will collect diverse data sets on a variety of physical machines in collaboration with Cyber Valley industry partners and empirically investigate whether such data is useful for a learning task on a different machine. 

In addition, they will address conceptual and fundamental research questions such as: When is data sufficiently informative for a dynamic learning problem? What data can be meaningfully transferred from one machine to another? And how can available data sets be used for modeling and control on new machines? 

The developed machine dynamics data sets and benchmarks, as well as all scientific results are planned to be made publicly available to the research community.

Self-supervised learning of mobility affordances for vision-based navigation

cyvy Research Project

If the mobile service robots of the future, among them delivery robots and self-driving cars, were able to learn independently from their environments, they would also be capable of adapting to changes in their surroundings and thus move about more efficiently. In turn, this would eliminate the need for engineers to tune robots manually to their environments.


This research project addresses the question of how mobile robots can learn their driving capabilities in their environment (i.e. mobility affordances) in a self-supervised way. Jörg Stückler and his team will develop methods for learning motion models that will allow mobile robots to predict the effects of their actions. The scientists will develop a vision-based navigation approach that use learned models for motion planning. They will then evaluate this approach for the autonomous navigation of a mobile robot. 

Relaxing restrictive interdependence assumptions in networks

cyvy Research Project

In various fields of research, such as the social sciences, biology, and computer science, network models are often applied to help describe complex systems with many individual elements interacting. In recent years, these models have often been used to draw new conclusions from observed data. The availability of large amounts of data has promoted this development.


Generative models are a popular network model. Here, latent variables are introduced which integrate the scientific findings in this field of knowledge (the "domain knowledge") and capture complex interactions. However, interactions among individuals are usually so complex that they are often approximated as independent. Conditioning upon these variables, the network edges are assumed to be independent and the distribution of probabilities within the network can be simplified. The disadvantage of these models is that in some real-world scenarios, the interactions within the network are not well captured. This means that the model's mathematical description does not correspond well to what is observed in real data. The coupling between variables, which are  too limited, are the main problem here. In comparison, network ensemble models do not use such latent variables, but rather network-specific variables (e.g. degree of distribution or clustering coefficient). However, these models also suffer from various problems that limit their practical application.


This project will combine certain features of the generative model and the network ensemble model with methods from statistical physics. The aim is to develop better principle-based models. In addition, the project aims to ensure that these models can be efficiently applied to concrete problems (e.g. repeatability or the simultaneous occurrence of different forms of relationships between two nodes). 

A scalable machine leaning approach to improving human decision making

cyvy Research Project

As technological development accelerates, millions of low-skilled workers are destined to lose their jobs to automation. To mitigate the resulting societal problems, this project aims to develop a scientific and technological foundation for rapidly and inexpensively teaching people the skills they will need to stay or become employable in the workplace of the future, which will be increasingly cognitively demanding.

Building on computational models of human learning and decision-making, Falk Lieder’s group proposes a general and scalable approach that leverages machine learning and artificial intelligence to teach workers the strategies they will need to meet the self-management challenges of the knowledge economy.


The researchers will test this approach by developing a series of intelligent tutors that develop and teach optimal decision strategies for increasingly realistic scenarios. They will illustrate the potential of this approach by developing and evaluating a simulation-based intelligent tutor that teaches high-level employees, freelancers, entrepreneurs, and academics far-sighted strategies for planning their projects, prioritizing their tasks, and managing themselves more effectively.

ACTrain: A personalised companion for enhancing executive functions based on adaptive meta-cognitive feedback

cyvy Research Project

Many students struggle to stay focused long enough to learn effectively, and social media has exacerbated the problem. Constant distractions at work have led to losses in productivity, which cost the economy billions. These serious issues not only have a negative implications for the lives of individual people, but also for society as a whole.


In this project, Dr. Falk Lieder and Jun.-Prof. Dr. Maria Wirzberger will address the challenge of staying focused in the face of distractions by developing a brain training app called ACTrain together with their project team. ACTrain will be a personal assistant with a name and a customized appearance that will train people to stay focused on a task and effectively resume it after getting distracted. Unlike conventional brain training apps, ACTrain will allow people to train while they are working or studying, thereby turning their daily lives into a gym for the mind. ACTrain can thus be used in many different contexts, including education and the workplace.


The heart of ACTrain is an intelligent feedback mechanism based on computational models of how attention control skills are learned. Based on these models, the application gives people feedback when they get distracted. The feedback communicates the benefits of regaining focus for their productivity and success. In both online courses and the workplace, this software could improve the lives of millions of students and working professionals.

Mechanisms of representation transfer

cyvy Research Project

This project focuses on investigating new ways of transferring characteristics of the human visual system to artificial neural networks, with the aim of making them more robust against changes in image features. These can include, for example, changes in image style that do not alter the image's content. At present, no learning algorithm is capable of robustly generalize what it has learned to other untrained image features. Artificial neural networks quickly make mistakes when the image changes even slightly, for instance when noise is added or style changes are made. Humans have no problems recognizing the content of an image in such instances. Even if most of us grow up under the influence of a certain environment with specific visual characteristics (such as the Black Forest), our visual system easily generalizes to completely different environments (such as a desert environment or a painting).


Previous work has shown that deep artificial neural networks use very different image features for decision making than our visual system. For example, while we usually categorize objects by their shape, these networks rely mainly on local patterns in the images. It is still very difficult to incorporate the image features humans use to perceive into artificial systems, as we simply know too little about the exact properties of biological systems.


This is why we want to develop mechanisms that can transfer robust features directly from measurements of brain activity to artificial systems. Under controlled conditions, we will first investigate the mechanisms with which these features can be transferred between networks. In the final phase of the project, we will use publicly available measurements of neural activity from the visual system to test which of the neural properties can be transferred to artificial networks using the methods we have developed.



Soft-sensing interfaces with multifunctional smart materials

cyvy Research Project

The Cyber Valley “Locomotion in Biorobotic and Somatic Systems” research group investigates the biomechanics of locomotion and underlying morphological adaptations, as evolved by nature. The researchers then apply their biological findings to develop life-like robots and functional materials that are similar to how they occur in nature. Their research is at the interface of engineering and biology – a relatively new and promising field. 

Dr. Ardian Jusufi and Hritwick Banerjee envision developing a flexible, stretchable, and biocompatible external sensor made from multi-functional smart materials that could one day be applied in healthcare, both for humans and in non-invasive veterinary care. The sheet-like sensor would adhere externally to the human or animal exterior as smoothly as a second layer of skin, and would stay in place no matter how a person or animal moves. The sensor could then detect a person’s health, sense blood pressure and other biometric values, or whether a person had an irregular heart beat that could indicate adverse health events such as heart attacks. In addition to a broad of range of biomedical applications, the soft and flexible sensor could also be built into smart clothes, wearable electronics, or soft robotics, to name just a few examples. They could also be used to improve human-machine interaction. For instance, self-driving cars could be equipped with such sensors. If a person touched the sensor while sitting in the vehicle, it could detect an imminent medical emergency and send a signal to the autopilot, which would immediately drive the car to the nearest hospital. 

There are substantial technological challenges on the path to developing soft interfaces of this kind, which would have to potentially gather a broad range of healthcare information while being wrapped around an arm or leg like silk. The fundamental features of such a sensor must be significantly improved to enable performance. This is why fundamental basic research is required to explore flexibility, sensitivity, repeatability, linearity, durability, and stimuli-responsive material, for instance. 

Top sum up: the key aims of the scientists’ research project are as follows: 

  • manufacturing pressure-sensitive tactile sensing which is strain invariant, and improving the interface between highly stretchable and biocompatible conducting materials which provide excellent adhesion
  • developing a sensing sleeve with a multi-stimuli response embedded into a single hybrid platform that could actively conform to the device or body without compromising efficacy, and
  • exploring innovative automobile, entertainment industry applications for cutting- edge soft sensors, including integration with mobile soft robots, rehabilitative systems, and possibly collision-aware surgical robotics

The Cyber-Physical Twin of Human Organs

cyvy Research Project

Airline pilots train many hundreds of hours in flight simulators before they take to the skies. In contrast, surgeons have very limited access to simulators, and those that are available do not offer sufficiently realistic conditions. Medical instruments used for robotic and minimally-invasive surgery are often tested on grapes or cans of meat and thus do not accurately reflect reality.


Dr. Tian Qiu, leader of the Cyber Valley “Biomedical Microsystems” research group, has set out to improve the situation. His research focuses on developing very realistic organ phantoms that optimize surgical training procedures and make them quantitatively measurable. Not only are these phantoms authentic physical replicas, they also have a cyber component. In other words. Qiu’s research program proposes to develop a “cyber-physical twin” of human organs.


Each 3D-printed organ twin is made of soft materials very similar to real organs in terms of anatomy and tissue properties. The cyber aspect is that the model can sense what it experiences and that this data is collected. Such data would be impossible to record if, for instance, a medical procedure were trained on a real human organ. With the data generated by a cyber-physical organ twin, the outcome of a surgery training session can now be clearly visualized, which is not even possible in a real surgical situation. The performance of a medical student who is training to become a surgeon could thus be evaluated automatically, and the feedback can be provided immediately after the training session to improve the training experience.


Such smart cyber-physical organ twins will one day transform surgical training. Tian Qiu and his team believe that they could gradually substitute medical training on human bodies and reduce animal experiments. The organ replicas not only offer the opportunity to develop and test new medical instruments, but also to develop better safety products such as helmets and airbags, for example when body part replicas are used in crash tests. Vital data on how they are affected in an accident can be collected and analyzed. 

Thumb ticker allgower1
(University of Stuttgart)
Thumb ticker pi  053
(University of Tübingen)
Thumb ticker ports 160922 1261headcrop2
(Max Planck Institute for Intelligent Systems)
Thumb ticker kjkuchenbecker1
(Max Planck Institute for Intelligent Systems)
Thumb ticker bernhard sch%c3%b6lkopf  germany
(Max Planck Institute for Intelligent Systems)
Thumb ticker bildschirmfoto 2019 03 15 um 14.12.22
(Max Planck Institute for Intelligent Systems)
no image
(Porsche AG)
Thumb ticker petergehler copy
no image
(ZF Friedrichshafen AG)
Thumb ticker bosch research thomas kropf cr p 0578
(Robert Bosch GmbH)
no image
(BMW Group)
no image