LLamaSharp provides a C#/.NET library for efficient local execution of Large Language Models (LLMs) like LLaMA and LLaVA. It targets .NET developers seeking to integrate LLM capabilities into their applications, offering RAG support and higher-level APIs for ease of use.

How It Works

LLamaSharp is built upon the llama.cpp project, leveraging its optimized C++ backend for efficient inference on both CPU and GPU (CUDA, Metal, Vulkan). This approach allows .NET applications to benefit from the performance optimizations of llama.cpp without requiring direct C++ development. The library abstracts the complexities of model loading and inference, providing a managed interface for .NET developers.

Quick Start & Requirements

• Install via NuGet: PM> Install-Package LLamaSharp
• Install a backend package (e.g., LLamaSharp.Backend.Cuda12, LLamaSharp.Backend.Cpu).
• Models must be in GGUF format.
• Requires .NET 6.0+ for RAG support.
• Official Documentation: https://www.llamasharp.ai/docs/
• Console Demo: https://www.llamasharp.ai/docs/console-demo

Highlighted Details

• Cross-platform support for Windows, Linux, and macOS.
• GPU acceleration via CUDA, Metal, and Vulkan.
• Integrations with popular frameworks like Semantic Kernel, Kernel Memory, Langchain, and BotSharp.
• Numerous examples available, including Unity, Blazor, and ASP.NET integrations.
• Supports RAG (Retrieval Augmented Generation) via LLamaSharp.kernel-memory.

Maintenance & Community

• Active development with regular releases.
• Community chat available on Discord.
• Detailed contributor wall of fame and version mapping to llama.cpp commits.

Licensing & Compatibility

• MIT License.
• Permissive license suitable for commercial use and integration into closed-source applications.

Limitations & Caveats

The project requires careful matching of backend packages to system configurations (e.g., CUDA version). Compatibility between LLamaSharp versions and specific llama.cpp commits is crucial, as noted in the version table, and using mismatched commits can lead to crashes.