SCENE: Reasoning about Traffic Scenes using Heterogeneous Graph Neural Networks

Thomas Monninger, Julian Schmidt, Jan Rupprecht, David Raba, Julian Jordan, Daniel Frank, Steffen Staab, Klaus Dietmayer

IEEE Robotics and Automation Letters, , pp. 1–8, 2023.

Abstract

Understanding traffic scenes requires considering heterogeneous information about dynamic agents and the static infrastructure. In this work we propose SCENE, a methodology to encode diverse traffic scenes in heterogeneous graphs and to reason about these graphs using a heterogeneous Graph Neural Network encoder and task-specific decoders. The heterogeneous graphs, whose structures are defined by an ontology, consist of different nodes with type-specific node features and different relations with type-specific edge features. In order to exploit all the information given by these graphs, we propose to use cascaded layers of graph convolution. The result is an encoding of the scene. Task-specific decoders can be applied to predict desired attributes of the scene. Extensive evaluation on two diverse binary node classification tasks show the main strength of this methodology: despite being generic, it even manages to outperform task-specific baselines. The further application of our methodology to the task of node classification in various knowledge graphs shows its transferability to other domains.

Links

doi: 10.1109/LRA.2023.3234771

BibTeX

@article{monninger23_ral, title = {SCENE: Reasoning about Traffic Scenes using Heterogeneous Graph Neural Networks}, author = {Monninger, Thomas and Schmidt, Julian and Rupprecht, Jan and Raba, David and Jordan, Julian and Frank, Daniel and Staab, Steffen and Dietmayer, Klaus}, year = {2023}, journal = {IEEE Robotics and Automation Letters}, pages = {1--8}, doi = {10.1109/LRA.2023.3234771} }