The aim of the talk is to discuss probabilistic solvers for ordinary and partial differential equations (ODEs/PDEs) and their implementation using Bayesian filters and smoothers. Probabilistic numerical solving of ODEs can be formulated as Gaussian process (GP) regression, where the observations are derivatives of the vector field (i.e., the right-hand side) of the ODE. When the GP has a state-space representation, the problem can be reduced to a non-linear Bayesian filtering and smoothing problem. In particular, the iterated extended Kalman smoother (IEKS) can be used to compute maximum a posteriori (MAP) estimate of the solution along with its uncertainty. We also discuss the extension of the IEKS solution to PDEs of non-linear Cauchy type. In these models, the PDE can be approximated in the spatial direction using finite differences or basis function expansions, which then reduces the PDE to an ODE, where the IEKS solution can be applied.