Summary

Jarvis MLX offers an offline, Stark-level productivity suite for MacBooks, integrating state-of-the-art AI models via Apple's MLX framework. It targets users seeking advanced, on-device AI capabilities without cloud dependency, enhancing local workflow efficiency.

How It Works

This project leverages Apple's MLX, an ML framework optimized for Apple Silicon, enabling local execution of advanced AI models. Speech-to-text is powered by OpenAI's Whisper, a compact yet performant model. For large language tasks, it employs Phi 3, achieving 60 tokens/sec on M1 Max, with support for custom finetuned models. Text-to-speech is handled by MeloTTS, a fast, finetunable model.

Quick Start & Requirements

• Installation: Requires a native Python environment for MLX (verify processor: python -c "import platform; print(platform.processor())" must output "arm"). A suggested conda setup is CONDA_SUBDIR=osx-arm64 conda create -n native numpy -c conda-forge. Install dependencies via pip install -r requirements.txt.
• Prerequisites: Apple Silicon Mac, Python 3.x (arm architecture).
• Links: No specific quick-start, docs, or demo links are provided in the README.

Highlighted Details

• Full offline operation on MacBook hardware.
• Optimized performance on Apple Silicon via the MLX framework.
• Integrates SOTA models: Whisper (STT), Phi 3 (LLM, 60 tokens/sec on M1 Max), and MeloTTS (TTS).
• Supports finetuning of both LLM and TTS models for customization.

Maintenance & Community

The project is explicitly a "Work in progress!". Updates are communicated via X (formerly Twitter). Contributions are welcomed through Pull Requests.

Licensing & Compatibility

The provided README does not specify a software license. Consequently, compatibility for commercial use or closed-source linking remains undetermined.

Limitations & Caveats

As a "Work in progress!", the project may be unstable or incomplete. The default text-to-speech voice is female; custom MeloTTS model training is required for changes. Achieving optimal LLM behavior necessitates finetuning.