Vision

Algorithmic advances in machine learning are key to realising the next breakthroughs in artificial intelligence. Equally important, however, is to integrate these algorithmic advances into fully functional intelligent and autonomously (inter)acting systems and to also consider societal and ethical aspects in their development. This requires to bridge the gap between computer science and the engineering sciences, such as electrical and mechanical engineering, as well as the social sciences, and to address a range of new research challenges at the intersection of these fields. Our unit has a strong expertise in multiple research areas that are closely aligned with the various algorithmic efforts within the ELLIS network. As one of only few places in Germany, Stuttgart is also well-known for its long-standing and broad expertise in the engineering and social sciences. Our vision for our ELLIS unit is therefore to leverage the existing expertise in algorithms, to bridge it with expertise locally available in the engineering and social sciences, and to contribute this unique interdisciplinary mix to ELLIS.

Core research areas

Interactive Intelligent Systems

AI and human-computer interaction (HCI) have long coalesced in the area of interactive intelligent systems, and six members of our team have extensive experience in this area. Bulling is widely know for his work on fundamental computational methods to sense and model everyday non-verbal human behavior and cognition as well as on leveraging these methods for intelligent interaction. Bürkner has recently started working on an interactive AI-assisted Bayesian workflow that can be used to help people with analyzing data in a principled Bayesian way. Kuchenbecker has worked on automatic evaluation of surgical skills from haptic measurements using machine learning to rate task performance as well as real-time computer vision and haptic perception for a responsive hugging robot. Schulte im Walde’s research focuses on cognitive plausibility and interpretability of ML models, as well as multimodal semantic abstraction modes in human communication. Staab has worked on intelligent human-computer mediation, i.e., helping users with diverse needs by mapping their multi-modal utterances to features of a graphical user interface. Finally, Vu’s research focuses on extracting information from human speech and on conversational agents that collaborate with users on tasks that neither humans nor machines can solve alone. [Image from Karessli 2017]

Natural and Programming Language Processing

This research area combines two fields that use machine learning to represent and reason about languages: natural language processing (NLP) and programming language processing (PLP). Several members have a strong background in NLP. Schulte im Walde and Vu are experts in distributional representations, linguistic interpretations of structured textual human language, and machine learning for NLP. Staab has been working on text clustering, learning ontologies and constructing knowledge graphs. Bulling explores methods to bridge between cognitive models of reading and deep and recurrent neural networks. Others pioneer methods to reason about programming languages using machine learning, e.g. to train a model on code examples to make predictions about previously unseen code. Pradel has contributed the first fully machine-learned bug detector, the first learning-based type prediction technique deployed in industry, and a popular learning-based program repair tool. Other members (Bulling, Staab) have started collaborations in this field. While natural and programming languages differ they also share many commonalities to be exploited by joint work in our unit. In particular, we plan to work on "mixed tasks" that require to combine NLP and PLP models, e.g., to reason about code comments, identifier names, or technical documentation, and on processing natural language input provided by programmers, e.g., requirements specifications.

Learning Theory

Many modern ML methods have a variety of hyper-parameters whose influence on the overall learning process is often still not well understood. In addition, predictions often tend to be unstable and easily influenced by small perturbations in the data or the model itself. Finally, if massive amounts of data are available, computationally cheaper yet even less understood approximations of the original algorithms may be required. These and other short-comings call for a better theoretical understanding of ML methods. For example, aspects such as robustness and uncertainty quantification can be approached either using tools from e.g. robust statistics and the theory of loss functions or from a Bayesian probabilistic perspective. Our unit has expertise in both schools of thought: Steinwart works on the theory of machine learning, e.g. on kernel methods, cluster analysis, robustness, and on the role of loss functions from a frequentist perspective. Bürkner works on methods for Bayesian machine learning including multilevel modeling, Gaussian processes, deep simulation-based inference, convergence diagnostics, and cross-validation. We plan to collaborate not only on the natural overlap of kernel methods and Gaussian processes but also on the better understanding of deep neural networks, as the latter share aspects of both kernel methods and Gaussian processes. At the same time, investigating the interpretability of neural network architectures provides a strong link to our NLP members regarding their application to linguistic ML models. [Image from García-Durán 2018]

Robot Learning

Our unit is in the process of establishing a new research area on robot learning by combining the region’s outstanding strength in hardware engineering with modern AI methods to create intelligent robotic systems that can learn to interact in complex environments. The intricate coupling between hardware and software and the challenge of accurately modeling physical interactions make hand-programming of robots intractable; machine learning is thus the method of choice for such systems. The University of Stuttgart, the Max Planck Society, and the German state in which Stuttgart is located are making substantial investments in robot learning research in Stuttgart. Together with Kuchenbecker, two new faculty members will form the core of our Robot Learning research area. Kuchenbecker uses learning to endow autonomous robots with robust touch sensing for object perception and physical human-robot interaction. [Image from Sun et al. 2022]