Queryable is an open-source iOS application enabling offline, natural language photo search within a user's photo library. It targets users concerned about privacy and seeking more powerful search capabilities than native iOS offers, allowing for descriptive queries like "a brown dog sitting on a bench."

How It Works

The app encodes all photos using CLIP's Image Encoder, generating image vectors. For each text query, it computes a text vector using the Text Encoder. Similarity is then calculated between the text vector and all image vectors, returning the top K most similar results. This approach allows for semantic understanding of image content, moving beyond simple keyword matching.

Quick Start & Requirements

• Install: Clone the repository and open the project in Xcode.
• Prerequisites: Download TextEncoder_mobileCLIP_s2.mlmodelc and ImageEncoder_mobileCLIP_s2.mlmodelc from Google Drive and place them in the CoreMLModels/ directory. Requires Xcode.
• Resources: Model files are approximately 100MB each.
• Links: Blog, App Store, Website, Story, 故事

Highlighted Details

• Supports Apple's MobileCLIP model for efficient on-device inference.
• Offers offline operation, ensuring user privacy.
• Includes scripts for exporting Core ML models, enabling custom implementations.
• An Android version (PicQuery) is available.

Maintenance & Community

• Active development with recent updates (Sept 2024) to support MobileCLIP.
• Community contributions are welcomed, with specific issues highlighted for optimization.
• Contact: Discord, Twitter, Reddit

Licensing & Compatibility

• MIT License.
• Permissive license allows for commercial use and integration into closed-source products.

Limitations & Caveats

The author notes they are not a professional iOS engineer, and the Swift code quality may vary. While the core functionality is present, advanced optimizations or native language support might require further development. Early versions had precision errors in image encoding, though community efforts have addressed some issues.