This repository is a curated collection of resources on the intersection of differential equations and deep learning, targeting researchers and practitioners in scientific machine learning. It provides a comprehensive overview of Neural Ordinary Differential Equations (Neural ODEs), Neural Stochastic Differential Equations (Neural SDEs), Neural Controlled Differential Equations (Neural CDEs), Neural Operators, and their applications in areas like model discovery, control, and generative modeling.

How It Works

The core of this collection revolves around representing deep learning models as continuous-time dynamical systems, often by parameterizing the derivative of a hidden state with a neural network. This approach bridges deep learning with dynamical systems theory, enabling models to learn continuous transformations and potentially offer better generalization and interpretability. Key techniques include using ODE solvers for forward and backward passes, leveraging Hamiltonian and Lagrangian mechanics for physics-informed learning, and employing Fourier transforms for efficient operator learning.

Quick Start & Requirements

This is a collection of research papers and software libraries, not a single executable project. To get started with the underlying technologies:

• torchdyn: pip install torchdyn
• torchdiffeq: pip install torchdiffeq
• torchsde: pip install torchsde
• torchcde: pip install torchcde
• Julia DiffEqFlux: Requires Julia and the DiffEq ecosystem.

Dependencies generally include Python (3.7+), PyTorch, and potentially Julia with specific packages. GPU support is highly recommended for performance.

Highlighted Details

• Extensive coverage of Neural ODEs, including theoretical insights, training acceleration techniques, and applications in control.
• Detailed sections on Neural Operators, Fourier Neural Operators, and their use in solving Partial Differential Equations (PDEs).
• Resources on Neural SDEs and Neural CDEs for handling stochasticity and irregularly sampled time series.
• Categorization of works by topic, aiding navigation through the diverse landscape of differential equations in ML.

Maintenance & Community

The repository is maintained by Zymrael and welcomes contributions via Issues or Pull Requests. It serves as a community-driven effort to catalog advancements in this rapidly evolving field.

Licensing & Compatibility

The repository itself is a collection of links and does not have a specific license. The linked papers and software libraries have their own licenses, which vary. Users must consult the individual licenses of the software libraries (e.g., PyTorch, Julia DiffEq) for compatibility and usage terms.

Limitations & Caveats

This is a curated list of resources, not a unified framework. Understanding and implementing these concepts requires a strong background in both deep learning and differential equations. Some libraries are under active development, and performance or stability may vary.