FauxPilot provides an open-source, locally-hosted alternative to GitHub Copilot, targeting developers who want to run AI-powered code completion without relying on external cloud services. It enables self-hosting of code generation models for enhanced privacy and control.

How It Works

FauxPilot leverages Salesforce's CodeGen models, running them within NVIDIA's Triton Inference Server utilizing the FasterTransformer backend. This combination is chosen for optimized inference performance on NVIDIA GPUs, allowing efficient execution of large language models for code generation.

Quick Start & Requirements

• Install: Requires Docker and docker compose (>= 1.28).
• Prerequisites: An NVIDIA GPU with Compute Capability >= 6.0 and sufficient VRAM. nvidia-docker, curl, and zstd are also needed.
• Setup: Run the setup.sh script to select and download a model, which is then converted for FasterTransformer. Detailed setup guides are available on the wiki.
• Links: How to set-up a FauxPilot server, How to set-up a client

Highlighted Details

• Supports splitting models across multiple GPUs for users with less VRAM per card.
• Offers multiple client connection methods, including OpenAI API compatibility and Copilot plugins.
• Utilizes FasterTransformer for potentially improved inference speed.

Maintenance & Community

• Minimal information on formal support or warranty, with questions directed to a discussion forum.
• Links to community resources are available on the wiki.

Licensing & Compatibility

• The README does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project explicitly states "lmao" regarding support and warranty, indicating a community-driven, best-effort development model. Formal documentation and support structures appear minimal.