trustfall  by obi1kenobi

Query engine for diverse data sources

Created 4 years ago
2,809 stars

Top 16.8% on SourcePulse

GitHubView on GitHub
Project Summary

Trustfall is a powerful query engine designed to unify data access across diverse sources like APIs, databases, files, and even AI models. It empowers developers and researchers to query disparate datasets using a single, expressive query language, simplifying complex data integration tasks and enabling novel cross-source analysis.

How It Works

Trustfall employs a GraphQL-like query language that is compiled and executed by a Rust-based engine. Its core innovation lies in its adaptable adapter system, allowing users to define how various data sources are exposed and queried. This approach enables efficient, declarative data fetching and transformation, abstracting away the complexities of individual data source interactions. The engine can be compiled to WebAssembly for client-side execution, as demonstrated in the interactive playground.

Quick Start & Requirements

  • Python Bindings: pip install trustfall
  • Prerequisites: Python 3.7+, Rust toolchain for building from source or contributing.
  • Demo: The project includes a demo showcasing cross-API and file querying, with instructions in the demo-hytradboi directory.
  • Playground: An in-browser WASM playground is available for testing queries against HackerNews and Rust documentation APIs: https://play.predr.ag/hackernews

Highlighted Details

  • Supports querying REST APIs, RSS/Atom feeds, CSV files, and structured data.
  • Powers the cargo-semver-checks tool for Rust package version analysis.
  • Demonstrates client-side execution via WebAssembly for interactive applications.
  • Features a declarative query language with filtering and output directives.

Maintenance & Community

The project is actively maintained by its author, Predrag Gruevski. Community engagement and contributions are encouraged, with Python bindings available.

Licensing & Compatibility

Licensed under the Apache License, Version 2.0. This license permits commercial use and integration into closed-source projects.

Limitations & Caveats

While the project offers Python bindings, a dedicated getting started guide for Python is still forthcoming. The WASM build is primarily for demonstration and client-side embedding.

Health Check
Last Commit

6 days ago

Responsiveness

1 day

Pull Requests (30d)
9
Issues (30d)
0
Star History
8 stars in the last 30 days

Explore Similar Projects

Starred by Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), Elie Bursztein Elie Bursztein(Cybersecurity Lead at Google DeepMind), and
12 more.

mindsdb by mindsdb

0.2%
38k
AI query engine for federated data sources
Created 7 years ago
Updated 1 day ago
Starred by Andrej Karpathy Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n), Anton Troynikov Anton Troynikov(Cofounder of Chroma), and
47 more.

llama_index by run-llama

0.3%
46k
Data framework for building LLM-powered agents
Created 3 years ago
Updated 3 days ago
Feedback? Help us improve.