This project provides an AI-driven local automation assistant that uses natural language to control computer operations, targeting users seeking to automate repetitive tasks without coding. It aims to redefine user-computer interaction by leveraging large language models for task planning and execution.

How It Works

autoMate functions as an AI+RPA tool, utilizing OmniParser for understanding requirements and planning tasks. It simulates human vision to comprehend screen content and makes autonomous decisions based on task objectives. The system supports local deployment, ensuring data security and privacy.

Quick Start & Requirements

• Installation: Recommended via conda with Python 3.12.

  git clone https://github.com/yuruotong1/autoMate.git
  cd autoMate
  conda create -n "automate" python==3.12
  conda activate automate
  python install.py
  python main.py
  

• Prerequisites: NVIDIA GPU recommended for optimal performance due to OCR usage. Minimum 4GB VRAM suggested. CUDA version compatibility with PyTorch is crucial.
• Access: Web UI available at http://localhost:7888/.
• Models: Supports OpenAI models (GPT-4o, etc.) and Yeka.

Highlighted Details

• No-code automation via natural language descriptions.
• Full interface control across any visual application.
• Simplified installation, with support for Chinese environments.
• Local deployment for data privacy.

Maintenance & Community

The project is in early stages with rapid iteration. Technical discussions and development notes are shared via a "Knowledge Planet" (QR code provided in README).

Licensing & Compatibility

The README does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

Performance may be slow without an NVIDIA GPU due to heavy OCR reliance. The project is in rapid development, implying potential for breaking changes. Support is currently limited to specific OpenAI and Yeka models due to multimodal and structured output requirements.