PDEBENCH: An Extensive Benchmark for Scientific Machine Learning

Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Dan MacKinlay, Francesco Alesiani, Dirk Pflüger, Mathias Niepert

Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS), pp. , 2022.

---

Abstract

Machine learning-based modeling of physical systems has gained increasing interest in recent years. Despite recent progress, there is still a lack of such benchmarks for scientific ML with sufficient volume and variety that are easy to use but still challenging and representative for a wide range of problems. In this paper, we introduce PDEBench, a benchmark suite of time-dependent simulation tasks based on Partial Differential Equations (PDEs). PDEBench comprises both code and data to benchmark the performance of novel machine learning models against both classical numerical simulations and machine learning baselines. Our proposed set of benchmark problems contributes in particular the following unique features: (1) A much wider range of PDEs than existing approaches, ranging from relatively common examples to more realistic and difficult ones; (2) much larger ready-to-use datasets than state-of-the-art, comprising multiple simulation-runs across varying initial or boundary conditions and model parameters; (3) and it provides easily extensible source codes with user-friendly APIs for data generation and baseline results with advanced machine learning models (FNO, U-Net, PINN, Gradient-based inverse method). PDEBench allows researchers to extend the dataset freely for their own purposes using a standardized API, and to compare the performance of their new models. Finally, we propose new metrics to help to understand and evaluate a given ML model in the context of scientific ML. With those metrics we identified tasks which the present ML methods cannot provide acceptable accuracy, and propose them as future challenge-task for the community.

---

Links

---

BibTeX