trustfall  by obi1kenobi

Query engine for diverse data sources

created 3 years ago
2,738 stars

Top 17.7% on sourcepulse

GitHubView on GitHub
Project Summary

Trustfall is a powerful query engine designed to unify data access across diverse sources like APIs, databases, files, and even AI models. It empowers developers and researchers to query disparate datasets using a single, expressive query language, simplifying complex data integration tasks and enabling novel cross-source analysis.

How It Works

Trustfall employs a GraphQL-like query language that is compiled and executed by a Rust-based engine. Its core innovation lies in its adaptable adapter system, allowing users to define how various data sources are exposed and queried. This approach enables efficient, declarative data fetching and transformation, abstracting away the complexities of individual data source interactions. The engine can be compiled to WebAssembly for client-side execution, as demonstrated in the interactive playground.

Quick Start & Requirements

  • Python Bindings: pip install trustfall
  • Prerequisites: Python 3.7+, Rust toolchain for building from source or contributing.
  • Demo: The project includes a demo showcasing cross-API and file querying, with instructions in the demo-hytradboi directory.
  • Playground: An in-browser WASM playground is available for testing queries against HackerNews and Rust documentation APIs: https://play.predr.ag/hackernews

Highlighted Details

  • Supports querying REST APIs, RSS/Atom feeds, CSV files, and structured data.
  • Powers the cargo-semver-checks tool for Rust package version analysis.
  • Demonstrates client-side execution via WebAssembly for interactive applications.
  • Features a declarative query language with filtering and output directives.

Maintenance & Community

The project is actively maintained by its author, Predrag Gruevski. Community engagement and contributions are encouraged, with Python bindings available.

Licensing & Compatibility

Licensed under the Apache License, Version 2.0. This license permits commercial use and integration into closed-source projects.

Limitations & Caveats

While the project offers Python bindings, a dedicated getting started guide for Python is still forthcoming. The WASM build is primarily for demonstration and client-side embedding.

Health Check
Last commit

5 days ago

Responsiveness

Inactive

Pull Requests (30d)
11
Issues (30d)
1
Star History
63 stars in the last 90 days

Explore Similar Projects

Starred by Dominik Moritz Dominik Moritz(Professor at CMU; ML Researcher at Apple) and Casey Caruso Casey Caruso(Managing Partner of Topology Ventures).

latent-scope by enjalot

0.4%
717
Scientific tool for latent space investigation
created 2 years ago
updated 2 months ago
Starred by Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems).

super-rag by superagent-ai

0.3%
380
RAG pipeline for AI apps
created 1 year ago
updated 1 year ago
Starred by Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems) and Elie Bursztein Elie Bursztein(Cybersecurity Lead at Google DeepMind).

LightRAG by HKUDS

1.0%
19k
RAG framework for fast, simple retrieval-augmented generation
created 10 months ago
updated 18 hours ago
Feedback? Help us improve.