data-enrichment  by langchain-ai

Agent for web research and structured data generation

Created 2 years ago
256 stars

Top 99.0% on SourcePulse

GitHubView on GitHub
Project Summary

Summary

This LangGraph Studio template addresses the challenge of generating structured data from open-ended web research. It provides an agent capable of automating information gathering, extraction, and organization into user-defined JSON formats, benefiting users needing to populate databases or spreadsheets efficiently.

How It Works

The core approach utilizes LangGraph to orchestrate an agent workflow. The agent accepts a research topic and a JSON extraction_schema, performs web searches via Tavily, reads relevant website content, extracts key details, structures the findings according to the schema, and validates the output. This modular design allows for flexible integration and customization of research targets and data formats.

Quick Start & Requirements

Setup requires installing LangGraph Studio and configuring a .env file with necessary API keys. Primary dependencies include API access for the Tavily search engine and a chosen LLM provider (Anthropic or OpenAI). Users must obtain and input their respective API keys (e.g., ANTHROPIC_API_KEY, OPENAI_API_KEY). The default LLM is anthropic/claude-3-5-sonnet-20240620, but other compatible models like openai/gpt-4o-mini can be configured. Links to LangGraph Studio and documentation are available.

Highlighted Details

  • Schema-Driven Extraction: Custom JSON schemas define the precise structure of desired output data.
  • Model Flexibility: Supports configuration of various LLM providers and models beyond the Anthropic default.
  • Extensible Tooling: New tools can be added via tools.py, and agent logic extended within graph.py.
  • Development Workflow: Features hot-reloading, state-based debugging, and prompt customization for rapid iteration.
  • Integration: Seamless integration with LangSmith for advanced tracing and collaboration.
  • Deployment: Options for deployment via LangGraph Cloud.

Maintenance & Community

The provided README does not detail specific contributors, sponsorships, or community channels (e.g., Discord, Slack).

Licensing & Compatibility

The README does not specify a software license. This omission requires clarification regarding usage rights, particularly for commercial applications or integration into closed-source projects.

Limitations & Caveats

Documentation for LangGraph is noted as "under construction," suggesting potential for ongoing changes and API evolution. The README does not detail specific limitations regarding platform support, known bugs, or the project's maturity level.

Health Check
Last Commit

3 days ago

Responsiveness

Inactive

Pull Requests (30d)
8
Issues (30d)
0
Star History
1 stars in the last 30 days

Explore Similar Projects

Starred by Elie Bursztein Elie Bursztein(Cybersecurity Lead at Google DeepMind), Michael Chiang Michael Chiang(Cofounder of Ollama), and
2 more.

enrichmcp by featureform

0%
644
ORM for AI agents
Created 1 year ago
Updated 6 months ago
Starred by Chaoyu Yang Chaoyu Yang(Founder of Bento), Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), and
3 more.

DB-GPT by eosphoros-ai

0.0%
20k
AI-native data app development framework with agentic workflow
Created 3 years ago
Updated 5 days ago
Feedback? Help us improve.