Summary

SplitFM is an open-source framework for parameter-efficient fine-tuning (SplitLoRA) and inference (SplitInfer) of foundation models. It addresses edge deployment challenges by enabling resource-constrained, data-sensitive environments through techniques like Federated Learning, Split Learning, and cloud offloading, thereby enhancing privacy and efficiency.

How It Works

SplitLoRA combines Federated Learning (FL) for data privacy with Split Learning (SL) for computational offloading, based on the LoRA technique. This allows fine-tuning by training minimal parameters, reducing load and enhancing privacy. SplitInfer enables large foundation model inference on edge devices by leveraging cloud resources. It partitions models, performing inference without transmitting sensitive data to external servers, thus preserving privacy and enabling deployment on low-resource devices.

Quick Start & Requirements

• Installation: pip install loralib. Dependencies via pip install -r requirement.txt. Docker image nvcr.io/nvidia/pytorch:20.03-py3 available.
• Prerequisites:
  • SplitLoRA: Ubuntu 18.04, Python 3.7.16, PyTorch 1.7.1+cu110, Transformers 3.3.1. Supports nn.Linear, nn.Embedding, nn.Conv2d. Requires original pre-trained checkpoints.
  • SplitInfer: Ubuntu 18.04, Python 3.8.20, PyTorch 2.