Summary

px-kvstore is an in-memory, sharded, LRU key-value store designed for high-performance AI caching and general-purpose data storage. It offers a robust HTTP API, Time-To-Live (TTL) support that survives restarts, atomic disk snapshots, and efficient key scanning. The project specifically targets AI applications by providing deterministic, content-addressed caching for prompts and responses, enabling cost control, reproducibility, and operational visibility.

How It Works

The store employs a sharded architecture, routing keys to N shards using crc32(key) % N. Each shard manages its own LRU list and lock, enhancing concurrency by isolating hot keys. TTL is enforced lazily and persists across restarts via snapshots that store remaining lifetimes, not absolute timestamps. Atomic snapshot writes use a *.tmp file and os.replace to guarantee data integrity. An AI cache feature computes stable, content-addressed keys from request parameters (prompt, model, config) using canonical JSON and SHA-256.

Quick Start & Requirements

• Install: pip install -e .
• Run Server: python server.py
• Prerequisites: Python environment. No specific hardware (GPU/CUDA) or OS requirements are detailed.
• Dependencies: Standard Python packages implied by editable install.
• Links: README serves as primary documentation.

Highlighted Details

• AI Cache: Implements deterministic, content-addressed caching for AI prompts/responses, facilitating cost savings, reproducibility, and ROI tracking via detailed metrics (lookups, hits, misses, stores).
• Sharded LRU & Concurrency: Per-shard LRU lists and locks prevent contention, improving throughput and isolating performance impacts.
• Persistent TTL: TTL values are snapshotted as remaining lifetimes, allowing keys to retain their expiry duration after server restarts.
• Atomic Snapshots: Disk persistence uses atomic file replacement (os.replace) to prevent corruption from partial writes.
• Cursor Scan API: Supports efficient, paginated key retrieval with prefix filtering and start_after options.
• Fault Injection: Built-in capabilities to simulate request latency and snapshot failures via environment variables for testing resilience.

Maintenance & Community

• Author: Chao Chen.
• Community: No explicit community channels (Discord, Slack) or contribution guidelines are provided in the README.
• Roadmap: Features like pluggable eviction policies, improved scan scalability, request tracing, and fault injection have been completed.

Licensing & Compatibility

• License: The repository's license is not specified in the provided README.
• Compatibility: No explicit notes regarding commercial use or linking compatibility are present.

Limitations & Caveats

• Cross-shard Atomicity: Operations involving multiple shards (e.g., mset, mget) are not atomic.
• Lazy TTL Enforcement: TTL checks occur during read/write/scan/snapshot operations, not via a dedicated background process.
• Unspecified License: The absence of a clear license is a significant blocker for adoption decisions, especially for commercial or derivative works.
• Value Encoding: Handles JSON parsing failures by storing raw bytes and decodes JSON values as UTF-8 with replacement characters for invalid sequences.