Discover and explore top open-source AI tools and projects—updated daily.
tavily-aiAI-powered CLI for web research and data extraction
Top 84.0% on SourcePulse
This repository provides a command-line interface (CLI) toolkit for advanced web research and data extraction. It offers developers and researchers a streamlined way to perform tasks like web searching, content extraction from dynamic pages, site mapping, and comprehensive AI-powered research, all powered by the Tavily API. The primary benefit is automating and simplifying complex data gathering workflows.
How It Works
The project centers around the Tavily CLI (tvly), a tool that exposes various "skills" for interacting with web data. These skills leverage Tavily's AI models to provide LLM-optimized search results, extract clean markdown or text content (even from JavaScript-rendered pages), discover URLs across entire websites, and synthesize information from multiple sources into concise research summaries with citations. The workflow encourages starting with simple search and escalating to more complex research tasks as needed.
Quick Start & Requirements
npx skills add https://github.com/tavily-ai/skillscurl -fsSL https://cli.tavily.com/install.sh | bash or uv tool install tavily-cli or pip install tavily-cli.tvly login --api-key tvly-YOUR_KEY, tvly login (browser OAuth), or by setting the TAVILY_API_KEY environment variable.Highlighted Details
tavily-search: LLM-optimized web search supporting domain filtering, time ranges, and multiple search depths.tavily-extract: Extracts clean markdown/text content from specific URLs, adept at handling JS-rendered pages.tavily-crawl: Crawls websites to extract content from multiple pages, saving results as local markdown files.tavily-map: Efficiently discovers all URLs on a website without content extraction.tavily-research: Provides comprehensive AI-powered research with citations and multi-source synthesis in 30-120 seconds.Maintenance & Community
The provided README does not contain specific details regarding maintainers, community channels (e.g., Discord, Slack), or project roadmaps.
Licensing & Compatibility
The README does not specify a software license. This omission makes it unclear whether the project is open-source, its terms of use, or its compatibility with commercial or closed-source applications.
Limitations & Caveats
Operation is dependent on obtaining and configuring a Tavily API key, which may involve costs or usage limitations. The lack of explicit licensing information presents a significant adoption blocker, particularly for commercial use cases. Documentation for integrating with specific agents mentioned (e.g., Claude Code, Cursor) is not detailed within the README.
1 month ago
Inactive
Intelligent-Internet