union-search-skill  by runningZ1

Unified multi-platform search and content retrieval

Created 2 months ago
450 stars

Top 66.6% on SourcePulse

GitHubView on GitHub
Project Summary

Summary

This project provides a unified, cross-platform content search solution via a single CLI tool. It addresses the fragmentation of online information by aggregating search capabilities across over 30 platforms, including social media, developer communities, and general search engines. The primary benefit is a streamlined, efficient way for users to access and process information from diverse online sources with standardized output and built-in redundancy.

How It Works

The core of Union Search Skill is a Python-based Command Line Interface (CLI) that acts as a universal adapter for numerous search APIs. It employs a modular design with platform-specific adapters, enabling a single command to query multiple services simultaneously. A key advantage is its robust fallback mechanism, automatically switching to alternative channels if a primary search provider fails, ensuring higher success rates and data availability.

Quick Start & Requirements

Installation involves pip install requests python-dotenv lxml. Optional dependencies exist for image search (pyimagedl), Bilibili (curl_cffi), Tavily (tavily-python), and RSS (feedparser). Configuration requires setting up API keys and credentials in a .env file, with specific keys needed for platforms like GitHub, TikHub, SerpAPI, Google, YouTube, Tavily, Metaso, Volcengine, and cookies for Zhihu/Weibo.

Highlighted Details

  • Extensive Platform Support: Aggregates search across 30+ platforms, far exceeding typical single-purpose tools.
  • Unified Interface: A single CLI command (python union_search_cli.py search ...) standardizes interaction.
  • Automatic Fallback: Redundant channel design ensures service continuity if primary APIs fail.
  • Rich Feature Set: Includes dedicated support for image search (18 platforms), social media (Douyin, Xiaohongshu, Bilibili, Twitter), AI-driven search (Tavily, Metaso), and web-to-Markdown conversion.
  • Developer Focus: Strong support for developer-centric platforms like GitHub, Reddit, and Zhihu.

Maintenance & Community

The project outlines a roadmap including upcoming features like multi-media download tools, expanded search channels, and result caching. Contribution guidelines are provided, welcoming Issues and Pull Requests, though explicit community links (Discord/Slack) are absent.

Licensing & Compatibility

The project is released under the permissive MIT License. This license generally allows for commercial use and integration into closed-source projects without significant restrictions.

Limitations & Caveats

The multi-media download functionality is currently under development. The RSS module is experimental and dependent on external third-party services. Some platforms, like Volcengine, may experience occasional parsing issues, and others like Weibo and Zhihu require user-provided cookies for access. Many platforms necessitate API keys, with some free tiers having usage limitations.

Health Check
Last Commit

2 weeks ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
2
Star History
108 stars in the last 30 days

Explore Similar Projects

Starred by Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems") and Simon Willison Simon Willison(Coauthor of Django).

semantra by freedmand

0%
3k
CLI tool for semantic document search
Created 3 years ago
Updated 1 year ago
Starred by John Resig John Resig(Author of jQuery; Chief Software Architect at Khan Academy), Simon Horup Eskildsen Simon Horup Eskildsen(Cofounder of Turbopuffer), and
21 more.

meilisearch by meilisearch

0.3%
57k
Search engine API for integrating AI-powered hybrid search
Created 8 years ago
Updated 2 days ago
Starred by Andrej Karpathy Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n), Assaf Elovic Assaf Elovic(Cofounder of Tavily), and
9 more.

Vane by ItzCrazyKns

0.4%
34k
AI-powered search engine alternative
Created 2 years ago
Updated 1 day ago
Feedback? Help us improve.