Taxy AI is an open-source browser extension that leverages GPT-4 to automate repetitive web tasks. It's designed for technical users and researchers who want to build or test browser automation workflows, offering a local execution environment without sending data to external servers.

How It Works

Taxy operates via an "Action Cycle." It extracts the DOM, simplifies it to interactive elements, and sends this, along with user instructions, to an LLM (GPT-3.5/GPT-4). The LLM can then choose to perform actions like click(id) or setValue(id, text). Taxy executes these actions using the chrome.debugger API, updates the DOM, and repeats the cycle until the task is complete, the user intervenes, or an error occurs. This approach allows for dynamic, instruction-based automation driven by powerful language models.

Quick Start & Requirements

• Install: Clone the repo, run yarn install, then yarn start. Load the generated build folder as an unpacked extension in Chrome (chrome://extensions/).
• Prerequisites: Node.js >= 16, OpenAI API Key.
• Usage: Activate via popup (cmd/ctrl+shift+y) or DevTools panel. Paste API key, navigate to a webpage, and provide instructions.
• Resources: Getting Started with Chrome Extensions

Highlighted Details

• Supports GPT-3.5 and GPT-4.
• Uses Chrome Extension Manifest V3.
• Demonstrates automation for GitHub branch protection, Netflix search, and OpenAI playground essay writing.
• Action history is sent to the LLM for context.

Maintenance & Community

• Currently in "research preview" with potential for workflow failures.
• Open to contributions via PRs for demos.
• Discord server available for community interaction.

Licensing & Compatibility

• The repository does not explicitly state a license.

Limitations & Caveats

The extension is in a research preview state, meaning many workflows may fail or confuse the agent. It is not yet available on the Chrome Web Store, requiring local installation. The maximum action limit per task is 50.