trustfall  by obi1kenobi

Query engine for diverse data sources

Created 3 years ago
2,762 stars

Top 17.2% on SourcePulse

GitHubView on GitHub
Project Summary

Trustfall is a powerful query engine designed to unify data access across diverse sources like APIs, databases, files, and even AI models. It empowers developers and researchers to query disparate datasets using a single, expressive query language, simplifying complex data integration tasks and enabling novel cross-source analysis.

How It Works

Trustfall employs a GraphQL-like query language that is compiled and executed by a Rust-based engine. Its core innovation lies in its adaptable adapter system, allowing users to define how various data sources are exposed and queried. This approach enables efficient, declarative data fetching and transformation, abstracting away the complexities of individual data source interactions. The engine can be compiled to WebAssembly for client-side execution, as demonstrated in the interactive playground.

Quick Start & Requirements

  • Python Bindings: pip install trustfall
  • Prerequisites: Python 3.7+, Rust toolchain for building from source or contributing.
  • Demo: The project includes a demo showcasing cross-API and file querying, with instructions in the demo-hytradboi directory.
  • Playground: An in-browser WASM playground is available for testing queries against HackerNews and Rust documentation APIs: https://play.predr.ag/hackernews

Highlighted Details

  • Supports querying REST APIs, RSS/Atom feeds, CSV files, and structured data.
  • Powers the cargo-semver-checks tool for Rust package version analysis.
  • Demonstrates client-side execution via WebAssembly for interactive applications.
  • Features a declarative query language with filtering and output directives.

Maintenance & Community

The project is actively maintained by its author, Predrag Gruevski. Community engagement and contributions are encouraged, with Python bindings available.

Licensing & Compatibility

Licensed under the Apache License, Version 2.0. This license permits commercial use and integration into closed-source projects.

Limitations & Caveats

While the project offers Python bindings, a dedicated getting started guide for Python is still forthcoming. The WASM build is primarily for demonstration and client-side embedding.

Health Check
Last Commit

2 days ago

Responsiveness

1 day

Pull Requests (30d)
13
Issues (30d)
1
Star History
20 stars in the last 30 days

Explore Similar Projects

Starred by Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), Elie Bursztein Elie Bursztein(Cybersecurity Lead at Google DeepMind), and
12 more.

mindsdb by mindsdb

0.3%
36k
AI query engine for federated data sources
Created 7 years ago
Updated 13 hours ago
Starred by Andrej Karpathy Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n), Anton Troynikov Anton Troynikov(Cofounder of Chroma), and
44 more.

llama_index by run-llama

0.3%
44k
Data framework for building LLM-powered agents
Created 2 years ago
Updated 18 hours ago
Feedback? Help us improve.