TinyAgent enables Small Language Models (SLMs) to perform complex reasoning and function calling securely and privately at the edge. It targets users seeking on-device AI assistance for tasks like email management, scheduling, and file operations, offering a privacy-preserving alternative to cloud-based LLMs.

How It Works

TinyAgent leverages LLMCompiler and a technique called ToolRAG to enhance SLM capabilities. ToolRAG intelligently retrieves relevant tools and in-context examples for specific queries, minimizing prompt space and reducing hallucinations. This approach allows even smaller models to achieve high accuracy in function calling and task execution.

Quick Start & Requirements

• Install: Desktop app available. Programmatic use via Python.
• Prerequisites: macOS required for desktop app. OpenAI API key or local model setup (LMStudio, llama.cpp server) for LLM providers. Whisper.cpp requires a local server setup for voice commands.
• Resources: Fine-tuned models TinyAgent-1.1B and TinyAgent-7B are provided.
• Links: Blog Post, Paper, Hugging Face

Highlighted Details

• Supports 16 functions for interacting with macOS applications (Mail, Contacts, Calendar, Maps, Zoom, etc.).
• Offers PDF summarization and file reading capabilities.
• Integrates with OpenAI Whisper API or local whisper.cpp for voice commands.
• Fine-tuned models (TinyAgent-1.1B, TinyAgent-7B) claim to outperform GPT-4-turbo on specific benchmarks when combined with ToolRAG.

Maintenance & Community

• Project associated with EMNLP 2024.
• No explicit community links (Discord/Slack) mentioned in the README.

Licensing & Compatibility

• License details not explicitly stated in the README. The provided models on Hugging Face are typically under permissive licenses like Apache 2.0, but this should be verified.
• Designed for edge deployment, implying compatibility with local, private environments.

Limitations & Caveats

• Adding custom tools is currently limited to GPT models; open-source models and ToolRAG are fine-tuned only on the original toolset.
• The desktop application is macOS-specific.