This repository demonstrates an automated UI testing agent leveraging OpenAI's CUA model and the Responses API. It's designed for developers and QA engineers looking to automate frontend testing by translating natural language test cases into browser interactions, with a focus on ease of use and integration.

How It Works

The system comprises three components: a Next.js frontend for test configuration and monitoring, a Node.js cua-server that interfaces with the OpenAI CUA model and Playwright for browser automation, and a sample e-commerce application for testing. The CUA model interprets test cases and directs Playwright to perform actions within a browser instance, automating the testing workflow.

Quick Start & Requirements

• Install: Clone the repo, set OPENAI_API_KEY in .env.development files for each component, run npm install, and npx playwright install.
• Run: Execute npm run dev. The UI is accessible at http://localhost:3000.
• Prerequisites: Node.js, npm, Playwright, and an OpenAI API key.
• Demo Credentials: ADMIN_USERNAME=test_user_name, ADMIN_PASSWORD=test_password (add to sample-test-app/.env.development).

Highlighted Details

• Automates frontend testing using natural language test cases.
• Integrates OpenAI's CUA model with Playwright for browser control.
• Provides a Next.js UI for configuring and observing tests.
• Core agent logic resides in the cua-server for potential integration into other applications.

Maintenance & Community

OpenAI welcomes issues and PRs, though review is not guaranteed. Specific community channels or roadmap details are not provided in the README.

Licensing & Compatibility

Licensed under the MIT License, permitting commercial use and integration with closed-source projects.

Limitations & Caveats

The Computer Use (CUA) model is in preview and should not be trusted in authenticated or high-stakes environments due to potential exploits and errors. The project is intended for test environments only; avoid using real user data.