Versatile Imitation Learning and Reinforcement Learning with Motion Primitives in Robotics

In this talk, I will present our recent activities on imitation learning and reinforcement learning in robotics. I will give an overview of motion primitives (MP) in robotics, which have been used very successfully for smooth motion generation, and discuss their pros and cons in comparison to standard action-based policies. Here, I will also introduce a novel MP representation, called Probabilistic Dynamic Movement Primitives (ProDMPs), that can be easily embedded in a neural network architecture while guaranteeing smooth motion generation. In terms of imitation learning, I will present novel augmented reality-based user interfaces that can be used for low-effort data generation. Using this interface, we collected versatile benchmark datasets for evaluating recent imitation learning algorithms. Here, diffusion-based policies show convincing performance. Toward this end, we extend the ProDMP framework with diffusion-based policies to represent versatile behavior while guaranteeing smoothness. Regarding reinforcement learning (RL), I will show the beneficial exploration properties of MPs if we explore in the parameter space of MPs instead of in the action space of the robot. Toward this end, I will introduce MP-based RL algorithms that abstract the RL problem as a black-box optimization problem where the algorithm needs to choose the MP parameters instead of low-level control actions. We introduce a novel RL algorithm that is based on differentiable trust region projection layers and is well suited for RL problems with such high-dimensional action spaces and extend this algorithm for replanning and learning versatile behavior. In the end, I will give an outlook on how learning from human feedback approaches can make reinforcement learning more applicable to practitioners without the requirement of defining complex reward functions.