pykan  by KindXiaoming

Kolmogorov-Arnold Networks (KANs) as a drop-in replacement for MLPs

Created 1 year ago
15,920 stars

Top 3.0% on SourcePulse

GitHubView on GitHub
Project Summary

Kolmogorov-Arnold Networks (KANs) offer an alternative to Multi-Layer Perceptrons (MLPs) by placing activation functions on edges rather than nodes, grounded in the Kolmogorov-Arnold representation theorem. This architectural shift aims to improve model accuracy and interpretability, particularly for scientific computing tasks. The project provides a Python library for implementing and experimenting with KANs.

How It Works

KANs leverage the Kolmogorov-Arnold representation theorem, which states that any continuous function of several variables can be represented as a finite sum of continuous functions of one variable. Unlike MLPs, where activation functions are applied at nodes, KANs apply learnable activation functions to the edges connecting neurons. This allows KANs to potentially represent complex functions more efficiently and with greater interpretability than traditional MLPs.

Quick Start & Requirements

  • Installation: pip install pykan or pip install -e . after cloning the repository.
  • Prerequisites: Python 3.9.7+, PyTorch 2.2.2, NumPy, SciPy, Matplotlib, Scikit-learn, Pandas, Seaborn, PyYAML.
  • Resources: Examples are runnable on a single CPU in under 10 minutes; paper examples take under a day. PDE training can take hours to days on a CPU. GPUs are recommended for larger-scale tasks.
  • Documentation: https://github.com/KindXiaoming/pykan
  • Tutorials: https://github.com/KindXiaoming/pykan/tree/main/tutorials

Highlighted Details

  • KANs are designed for tasks requiring high accuracy and interpretability.
  • Includes model.plot() for visualization and model.prune() for interpretability-focused refinement.
  • Offers advice on hyperparameter tuning, emphasizing starting simple and gradually increasing complexity.
  • The library is primarily geared towards scientific computing and research, not necessarily out-of-the-box ML applications.

Maintenance & Community

The project is maintained by KindXiaoming. The author notes that the code was developed with small-scale math and physics examples in mind and may lack efficiency or reusability optimizations. Further development for efficiency is not a primary focus, with external projects like efficientkan and fouierkan suggested.

Licensing & Compatibility

The repository does not explicitly state a license in the provided README. Users should verify licensing for commercial or closed-source use.

Limitations & Caveats

The author acknowledges that KANs may not be a simple plug-in for general machine learning tasks and require significant hyperparameter tuning. Efficiency and reusability are noted as areas for improvement, and the current implementation is not optimized for large-scale ML. The applicability to LLMs is uncertain due to differing definitions of accuracy and interpretability.

Health Check
Last Commit

8 months ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
0
Star History
80 stars in the last 30 days

Explore Similar Projects

Starred by Maxime Labonne Maxime Labonne(Head of Post-Training at Liquid AI), Lewis Tunstall Lewis Tunstall(Research Engineer at Hugging Face), and
1 more.

ML-Notebooks by dair-ai

0.0%
3k
ML notebooks for education/research
Created 3 years ago
Updated 1 year ago
Feedback? Help us improve.