Neuronpedia is an open-source platform for interpreting neural network features, targeting researchers and developers. It provides tools for exploring activations, steering models, and automatically generating explanations, enabling deeper understanding of AI behavior.

How It Works

Neuronpedia employs a microservices architecture, separating the webapp, database, inference, and auto-interpretation services. This modular design allows for independent development and extensibility, enabling users to swap components or run services individually. It leverages OpenAPI schemas for typed communication between services and generates clients for seamless integration.

Quick Start & Requirements

• Instant Deploy: Deploy a custom instance via Vercel (requires a free Vercel account).
• Local Demo:
  • Install Docker.
  • Run make init-env, make webapp-demo-build, make webapp-demo-run.
  • Access at localhost:3000. Connects to public demo data (GPT-2 small, Gemma-2-2b) and inference servers.
• Local Development: Requires Node.js. Run make install-nodejs, make webapp-localhost-install, make webapp-localhost-dev.
• Inference Server: Requires Poetry. Build with make inference-localhost-build-gpu (CUDA) or make inference-localhost-build (no CUDA). Run with make inference-localhost-dev-gpu or make inference-localhost-dev, specifying MODEL_SOURCESET.
• Auto-Interp Server: Requires Poetry. Build with make autointerp-localhost-build-gpu (CUDA) or make autointerp-localhost-build (no CUDA). Run with make autointerp-localhost-dev-gpu or make autointerp-localhost-dev.
• Hardware: At least 16GB RAM recommended. CUDA is required for the embedding scorer in the auto-interp server.

Highlighted Details

• Supports multiple models including GPT-2, Gemma, and DeepSeek.
• Integrates with SAELens and SAEDashboard for custom data generation and visualization.
• Features a search explanation capability requiring an OpenAI API key for semantic similarity.
• Monorepo structure with clear separation of concerns for webapp, inference, and auto-interp services.

Maintenance & Community

• Community support via Slack (#neuronpedia).
• Contact: johnny@neuronpedia.org.
• Contributions welcome via CONTRIBUTING.md.

Licensing & Compatibility

• License not explicitly stated in the provided README text.

Limitations & Caveats

• The local demo database is read-only; a local database setup is required for writing data.
• The auto-interp server's embedding scorer requires CUDA and does not support Mac MPS or CPU.
• Some sections like "high volume autointerp explanations" and "generate your own dashboards/data" are marked as "under construction."
• Data import into the local admin panel is finicky and lacks resume functionality.