This project provides a wearable system that enables AI to control human body actions via Electrical Muscle Stimulation (EMS) and relays, driven by voice and first-person vision input. It targets researchers and developers interested in embodied AI, human-computer interaction, and advanced wearable technology, offering a novel approach to AI-driven physical augmentation and learning. The system won the MIT Hard Mode 2026 Learn Track, showcasing its innovative capabilities.

How It Works

The Human Operator system integrates AI-driven action planning with direct physical actuation. The main application (app.py) captures camera frames and voice commands, processing them through an LLM (Anthropic's Claude) to generate an action plan. This plan is translated into a timestamped sequence of relay and EMS commands. A Flask gateway (utils/receiver.py) receives these commands and orchestrates their execution, interfacing with an Arduino-compatible microcontroller running custom firmware (human_operator_ems.ino) to control relays and optional EMS devices. This architecture separates high-level AI decision-making from low-level, time-sensitive hardware control.

Quick Start & Requirements

• Primary install/run command: Clone the repository, set up a Python virtual environment, install dependencies via pip install -r requirements.txt, configure the ANTHROPIC_API_KEY in a .env file, and run ./run_hardware.sh for relay-only mode.
• Prerequisites: Python 3.10+, macOS/Linux shell, Arduino IDE, Anthropic API key, Arduino-compatible relay MCU, relay board, and an optional EMS/stimulator device and camera.
• Links: GitHub Repository

Highlighted Details

• Winner of the MIT Hard Mode 2026 (Learn Track).
• Supports distinct operational modes: Relay-Only (default, stable) and Full Mode (integrating EMS/stimulator).
• Includes a PyQt-based GUI (manual_control_app.py) for detailed hardware calibration and direct stimulation testing.
• Clear system architecture separating AI planning (app.py) from hardware execution (utils/receiver.py and firmware).

Maintenance & Community

The project lists six core team members: Peter He, Valdemar Danry, Daniel Kaijzer, Yutong Wu, Ashley Neall, and Sean Hardesty Lewis. No community channels (e.g., Discord, Slack) or specific roadmap links are provided in the README.

Licensing & Compatibility

The repository's README does not specify a software license. This lack of clear licensing information presents a significant adoption blocker, particularly for commercial use or integration into closed-source projects. The system requires a macOS or Linux environment.

Limitations & Caveats

The project necessitates careful handling due to its control over physical electrical stimulation hardware, requiring appropriate supervision, informed consent, and conservative intensity settings. Users with contraindications to EMS should not use the system. Troubleshooting notes indicate potential issues with serial port conflicts leading to simulated hardware and camera access problems. Full EMS stimulation is intentionally disabled unless explicitly enabled via HARDWARE_MODE=full.