agents  by astronomer

AI agents for streamlined data engineering

Created 1 month ago
277 stars

Top 93.7% on SourcePulse

GitHubView on GitHub
Project Summary

Astronomer Agents provides AI-powered tooling for data engineering workflows, focusing on Apache Airflow and data warehouses. It extends popular AI coding agents with specialized capabilities for Airflow management and data tasks, enabling natural language interaction to streamline development and operations for engineers and power users.

How It Works

The project integrates an MCP server for Airflow REST API interaction and a CLI tool (af) for terminal access. Its core is a suite of "skills" that equip AI coding agents with domain-specific knowledge for data warehousing, DAG development, and lineage tracing. This allows AI to assist with data analysis, schema discovery, DAG authoring, and debugging directly within AI development environments.

Quick Start & Requirements

Installation typically uses npx skills add astronomer/agents --skill '*'. Claude Code users should install the plugin (claude plugin marketplace add astronomer/agents). Prerequisites include compatible AI coding agents (e.g., Claude Code, Cursor) and Airflow 2.x/3.x for the MCP server. uvx is required for MCP components. Configuration involves setting up warehouse connections via ~/.astro/agents/warehouse.yml and environment variables for remote Airflow.

Highlighted Details

  • MCP Server: Full Airflow REST API integration for DAG management, task triggering, logs, and health.
  • Skills Suite: Comprehensive skills for data discovery, analysis, profiling, freshness, lineage, DAG authoring, testing, debugging, deployment, dbt integration, and Airflow migration.
  • Airflow CLI (af): Direct terminal interaction for status checks, DAG listing, and run triggering.
  • Multi-Database Support: Connects to Snowflake, PostgreSQL, BigQuery, and over 25 SQLAlchemy-compatible databases.

Maintenance & Community

Developed by Astronomer, the project welcomes contributions. A roadmap details planned skills for advanced DAG operations and broader data science workflows. Specific community channels are not detailed in the provided README.

Licensing & Compatibility

Licensed under Apache 2.0, ensuring broad compatibility for commercial use and integration with open-source Apache Airflow. It functions with any MCP-compatible client.

Limitations & Caveats

The af CLI collects anonymous usage telemetry by default (opt-out available). Some skills are tailored towards Astronomer's Astro platform, though the tooling is usable with open-source Airflow. Claude Code users are advised to use the dedicated plugin for optimal integration.

Health Check
Last Commit

3 days ago

Responsiveness

Inactive

Pull Requests (30d)
45
Issues (30d)
9
Star History
73 stars in the last 30 days

Explore Similar Projects

Feedback? Help us improve.