Flame is an open-source multimodal AI system designed to translate UI design mockups into high-quality React code. It addresses the limitations of existing models in generating static, non-modular code for front-end development by providing a comprehensive framework for data synthesis, model training, and evaluation, targeting front-end developers and AI researchers.

How It Works

Flame integrates the Siglip Vision encoder with the Deepseek coder model, enhanced by a novel automated data synthesis pipeline. This pipeline generates diverse, high-fidelity datasets by extracting, rendering, and annotating front-end code snippets, supporting both single and multi-image inputs for improved visual reasoning. The framework employs a three-stage training strategy, including vision encoder pretraining, image layout interpretation, and full instruction-tuning for image-to-code generation.

Quick Start & Requirements

• Install via conda env create -f environment.yml and npm install.
• Requires Python 3.x, Node.js, and environment.yml dependencies.
• Data preparation involves cloning repositories, extracting components, rendering code to images, and generating instructions.
• See Installation and Usage for detailed steps.

Highlighted Details

• Comprehensive data preparation pipeline with three synthesis methods: Evolution-Based, Waterfall-Model-Based, and Additive Development.
• End-to-end training pipeline with a three-stage strategy.
• Evaluation pipeline includes the Flame-React-Eval dataset and automated testing for functional correctness and visual fidelity.
• Supports multi-image inputs for iterative UI refinement.

Maintenance & Community

• Open to community contributions for dataset, model, and evaluation pipeline improvements.
• See Contributing for contribution guidelines.

Licensing & Compatibility

• Released under the Apache 2.0 License.
• Permissive license suitable for commercial use and integration with closed-source projects.

Limitations & Caveats

The framework is currently optimized for React-based development, though the methodology is extensible to other frameworks like Vue and Angular. The README does not specify hardware requirements for training or inference, which may be substantial given the multimodal nature of the model.