This repository provides a comprehensive collection of Convolutional Kolmogorov-Arnold Networks (CKANs) for researchers and practitioners exploring novel neural network architectures. It offers implementations of various CKAN variants, model architectures like ResNet and DenseNet adapted with CKANs, and training scripts for benchmarking on standard datasets, aiming to advance the state-of-the-art in deep learning.

How It Works

The core innovation lies in replacing traditional convolutional kernels (weight matrices) with learnable univariate functions (phi functions), inspired by the Kolmogorov-Arnold representation theorem. This approach allows for more flexible and potentially more efficient feature extraction by learning non-linear transformations directly within the convolution operation, rather than relying solely on subsequent activation functions.

Quick Start & Requirements

• Install: pip install -r requirements.txt
• Prerequisites: Python 3.9+, CUDA Toolkit, cuDNN.
• Usage:
  • MNIST/CIFAR benchmarks: python mnist_conv.py
  • Accelerate-based training: accelerate launch <script_name>.py (e.g., cifar.py, tiny_imagenet.py)
• Links:
  • Project Repo: https://github.com/IvanDrokin/torch-conv-kan
  • Paper: https://arxiv.org/abs/2407.01092

Highlighted Details

• Implements KANConv, KALNConv, FastKANConv, KACNConv, KAGNConv, WavKANConv, KAJNConv, and ReLU KAN Conv layers.
• Offers Bottleneck Convolutional KAN layers to reduce parameter count.
• Includes ResNet-like (ResKANets), DenseNet-like (DenseKANets), VGG-like (VGGKANs), and U-Net-like (UKANet, U2KANet) model architectures.
• Achieves competitive results on MNIST and Tiny ImageNet, with preliminary benchmarks on CIFAR datasets and ImageNet1k.
• Provides pretrained checkpoints for VGG-like models on ImageNet1k.

Maintenance & Community

• Project status: Under Development with frequent updates.
• Based on several other KAN-related repositories (TorchKAN, FastKAN, etc.).
• Contributions are welcome; issues can be raised for problems.

Licensing & Compatibility

• The repository does not explicitly state a license in the README. It is based on other projects which may have their own licenses. Users should verify licensing for commercial or closed-source use.

Limitations & Caveats

The README notes that results are preliminary and model architectures are not exhaustively explored. ChebyKAN-based convolutions exhibit stability issues. Performance on CIFAR-10/100 is reported to be significantly lower than classical convolutions, indicating a need for further architectural research for these datasets.