Summary

SmallCode enables coding tasks using small, locally-run LLMs (7B-20B parameters) on consumer hardware. This terminal-native agent compensates for smaller models' limitations—context, tool-calling—via an intelligent, privacy-focused architecture, facilitating local development without cloud reliance.

How It Works

SmallCode's architecture optimizes for small LLMs. It manages context via summarization/budgeting and uses a forgiving parser for imperfect tool calls. Complex tasks decompose into TODO-file managed steps, with editing primarily using safer search-and-replace patches. Key innovations include MarrowScript for generating robust AI cognition logic (caching, validation, budgets) from concise declarations, and BoneScript, compiling .bone files into complete Node.js/TypeScript backends to reduce tool call overhead and boost reliability.

Quick Start & Requirements

• Install: npm install -g smallcode or npx smallcode.
• Prerequisites: Node.js 18+ (LTS recommended for SQLite binaries) and a local LLM server (e.g., LM Studio, Ollama, OpenAI-compatible endpoint).
• Optional Dependencies: better-sqlite3 for code graph features requires native compilation tools (Python 3, make, C++ compiler) if prebuilt binaries are unavailable; SmallCode falls back to JSON memory if build fails.
• Configuration: Set SMALLCODE_MODEL and SMALLCODE_BASE_URL in .env. Optional cloud API keys for model escalation.

Highlighted Details

• MarrowScript Cognition Layer: Declarative AI logic compiles to robust TypeScript with caching, retries, validation, and budget enforcement.
• BoneScript Integration: Compiles .bone files into full Node.js/TypeScript backends, reducing tool calls from 8-15 to 1-2 for improved reliability.
• Model Escalation: Opt-in fallback to cloud models (Claude, OpenAI, DeepSeek) on local failure, session-limited.
• Context Budget Engine: Manages context via token capping, mid-turn eviction, and semantic compression.
• Forgiving Tool Call Parser: Parses and auto-repairs tool calls from JSON, YAML, XML, or plain text.
• Patch-First Editing & TODO Planning: Uses search-and-replace editing and TODO-driven decomposition for small model reliability.
• Web Browsing: Integrated Playwright (optional, requires SMALLCODE_WEB_BROWSE=true) for web search/fetching with anti-detection.

Maintenance & Community

No specific details regarding maintainers, community channels, or roadmap were found in the provided README.

Licensing & Compatibility

• License: MIT.
• Compatibility: Permissive MIT license allows broad compatibility, including commercial use and closed-source linking.

Limitations & Caveats

• Code graph features require build tools if better-sqlite3 prebuilt binaries are unavailable.
• Web browsing is optional and requires explicit enablement.
• Cloud model escalation is opt-in and incurs costs.