Papers accepted at NeurIPS

We are delighted to announce that 4 papers written by members of our unit have been accepted to NeurIPS. Congratulations to the authors!

1. Ordered Subgraph Aggregation Networks

   Chendi Qian, Gaurav Rattan, Floris Geerts, Christopher Morris, Mathias Niepert

   Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS), pp. , 2022.

   Abstract BibTeX Project

   Numerous subgraph-enhanced graph neural networks (GNNs) have emerged recently, provably boosting the expressive power of standard (message-passing) GNNs. However, there is a limited understanding of how these approaches relate to each other and to the Weisfeiler–Leman hierarchy. Moreover, current approaches either use all subgraphs of a given size, sample them uniformly at random, or use hand-crafted heuristics instead of learning to select subgraphs in a data-driven manner. Here, we offer a unified way to study such architectures by introducing a theoretical framework and extending the known expressivity results of subgraph-enhanced GNNs. Concretely, we show that increasing subgraph size always increases the expressive power and develop a better understanding of their limitations by relating them to the established k-𝖶𝖫 hierarchy. In addition, we explore different approaches for learning to sample subgraphs using recent methods for backpropagating through complex discrete probability distributions. Empirically, we study the predictive performance of different subgraph-enhanced GNNs, showing that our data-driven architectures increase prediction accuracy on standard benchmark datasets compared to non-data-driven subgraph-enhanced graph neural networks while reducing computation time.

   @inproceedings{qian22_neurips, title = {Ordered Subgraph Aggregation Networks}, author = {Qian, Chendi and Rattan, Gaurav and Geerts, Floris and Morris, Christopher and Niepert, Mathias}, year = {2022}, booktitle = {Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS)}, pages = {} }

2. PDEBENCH: An Extensive Benchmark for Scientific Machine Learning

   Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Dan MacKinlay, Francesco Alesiani, Dirk Pflüger, Mathias Niepert

   Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS), pp. , 2022.

   Abstract Links BibTeX Project

   Machine learning-based modeling of physical systems has gained increasing interest in recent years. Despite recent progress, there is still a lack of such benchmarks for scientific ML with sufficient volume and variety that are easy to use but still challenging and representative for a wide range of problems. In this paper, we introduce PDEBench, a benchmark suite of time-dependent simulation tasks based on Partial Differential Equations (PDEs). PDEBench comprises both code and data to benchmark the performance of novel machine learning models against both classical numerical simulations and machine learning baselines. Our proposed set of benchmark problems contributes in particular the following unique features: (1) A much wider range of PDEs than existing approaches, ranging from relatively common examples to more realistic and difficult ones; (2) much larger ready-to-use datasets than state-of-the-art, comprising multiple simulation-runs across varying initial or boundary conditions and model parameters; (3) and it provides easily extensible source codes with user-friendly APIs for data generation and baseline results with advanced machine learning models (FNO, U-Net, PINN, Gradient-based inverse method). PDEBench allows researchers to extend the dataset freely for their own purposes using a standardized API, and to compare the performance of their new models. Finally, we propose new metrics to help to understand and evaluate a given ML model in the context of scientific ML. With those metrics we identified tasks which the present ML methods cannot provide acceptable accuracy, and propose them as future challenge-task for the community.

   Code: https://github.com/pdebench/PDEBench

   Preprint: https://openreview.net/forum?id=dh_MkX0QfrK

   @inproceedings{takamoto22_neurips, title = {PDEBENCH: An Extensive Benchmark for Scientific Machine Learning}, author = {Takamoto, Makoto and Praditia, Timothy and Leiteritz, Raphael and MacKinlay, Dan and Alesiani, Francesco and Pflüger, Dirk and Niepert, Mathias}, year = {2022}, booktitle = {Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS)}, pages = {}, preprint = {https://openreview.net/forum?id=dh_MkX0QfrK} }

3. Pseudo-Riemannian Graph Convolutional Networks

   Bo Xiong, Shichao Zhu, Nico Potyka, Shirui Pan, Chuan Zhou, Steffen Staab

   Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS), pp. , 2022.

   Abstract BibTeX Project

   Graph Convolutional Networks (GCNs) are typically studied through the lens of Euclidean geometry. Non-Euclidean Riemannian manifolds provide specific inductive biases for embedding hierarchical or spherical data, but cannot align well with data of mixed topologies. We consider a larger class of semi-Riemannian manifolds with indefinite metric that generalize hyperboloid and sphere as well as their submanifolds. We develop new geodesic tools that allow for extending neural network operations into geodesically disconnected semi-Riemannian manifolds. As a consequence, we derive a principled Semi-Riemannian GCN that first models data in semi-Riemannian manifolds of constant nonzero curvature in the context of graph neural networks. Our method provides a geometric inductive bias that is sufficiently flexible to model mixed heterogeneous topologies like hierarchical graphs with cycles. Empirical results demonstrate that our method outperforms Riemannian counterparts when embedding graphs of complex topologies.

   @inproceedings{xiong22_neurips, title = {Pseudo-Riemannian Graph Convolutional Networks}, author = {Xiong, Bo and Zhu, Shichao and Potyka, Nico and Pan, Shirui and Zhou, Chuan and Staab, Steffen}, year = {2022}, booktitle = {Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS)}, pages = {} }

4. Hyperbolic Embedding Inference for Structured Multi-Label Prediction

   Bo Xiong, M. Cochez, Mojtaba Nayyeri, Steffen Staab

   Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS), pp. , 2022.

   Abstract BibTeX Project

   @inproceedings{xiong22_neurips_2, title = {Hyperbolic Embedding Inference for Structured Multi-Label Prediction}, author = {Xiong, Bo and Cochez, M. and Nayyeri, Mojtaba and Staab, Steffen}, year = {2022}, booktitle = {Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS)}, pages = {} }