Summary

AutoFigure-edit addresses the challenge of creating publication-ready scientific illustrations directly from method text. It transforms textual descriptions into fully editable SVG figures, offering researchers and technical users a powerful tool for generating and refining complex diagrams with style transfer capabilities. The primary benefit is a streamlined, flexible workflow for producing high-quality, customizable scientific visuals.

How It Works

The AutoFigure-edit pipeline converts method text into editable SVGs through a four-stage process. First, a text-to-image LLM generates a raster draft. Next, SAM3 segmentation identifies and delineates icons and text regions, outputting masks and bounding boxes. Third, backgrounds are removed from detected icons using RMBG-2.0, and an LLM constructs a placeholder SVG template. Finally, cropped icons are integrated into the template, optionally refined by an LLM optimizer, to produce a final, editable SVG. This approach enables lossless modification of all figure components and allows for style transfer by mimicking reference images.

Quick Start & Requirements

Installation involves cloning the repository, installing core dependencies via pip install -r requirements.txt, and separately installing SAM3 (git clone https://github.com/facebookresearch/sam3.git && cd sam3 && pip install -e .). Execution can be via CLI (python autofigure2.py ...) or a local web interface (python server.py). Key prerequisites include Python 3.10+ (SAM3 targets 3.12+), PyTorch 2.7+, and CUDA 12.6+ for GPU builds. Hugging Face authentication is required for SAM3 checkpoints, and API keys are necessary for LLM providers (OpenRouter, Bianxie, fal.ai, Roboflow). Links to the SAM3 repository (https://github.com/facebookresearch/sam3) and Hugging Face (https://huggingface.co/facebook/sam3) are provided.

Highlighted Details

• Direct generation of scientific figures from method text.
• Robust icon detection and segmentation using SAM3.
• Production of fully editable, code-based SVG outputs.
• Integrated browser-based SVG-Edit canvas for interactive refinement.
• Style transfer capability to match user-provided reference image aesthetics.
• Comprehensive artifact outputs, including intermediate icons and SVG templates.

Maintenance & Community

Community support is primarily facilitated through a WeChat discussion group. Contact information for joining or inquiries is provided, including WeChat IDs nauhcutnil and email tuchuan@mail.hfut.edu.cn. No specific details on core maintainers, sponsorships, or a public roadmap are present in the README.

Licensing & Compatibility

The project is released under the permissive MIT License. This license allows for broad usage, including commercial applications and integration into closed-source projects, with minimal restrictions beyond attribution.

Limitations & Caveats

A significant adoption barrier is the external dependency on SAM3, which requires a separate, complex installation process with specific Python, PyTorch, and CUDA version requirements, along with Hugging Face authentication. Furthermore, the system relies on external LLM APIs, necessitating API keys and potentially incurring usage costs. Potential version conflicts between AutoFigure-edit's Python 3.10+ target and SAM3's 3.12+ target should be managed.