Discover and explore top open-source AI tools and projects—updated daily.
AEON-7Uncensored, capability-enhanced LLM with advanced quantization
Top 70.7% on SourcePulse
This project provides an "uncensored" and capability-enhanced version of the Qwen3.6-27B language model, specifically optimized for NVIDIA DGX Spark and Blackwell hardware using NVFP4 quantization. It targets researchers and power users seeking to bypass safety alignment restrictions and leverage the model's full potential, offering significant performance improvements and restored reasoning capabilities.
How It Works
The core of the project is a multi-stage "lossless abliteration" process. It begins with repairing an upstream defect in Qwen3.6's GatedDeltaNet (Mamba-style) layers. Subsequently, it employs abliterix v1.4, a multi-objective optimization tool, to surgically remove the model's refusal direction while minimizing capability degradation, validated against a 10-axis spot-check to prevent word-salad outputs. Finally, it applies NVFP4 quantization, a 4-bit floating-point format designed for Blackwell-era hardware, preserving BF16 accuracy with true 4-bit compute throughput. Critical components like SSM layers retain BF16 precision to ensure correctness.
Quick Start & Requirements
vllm-aeon-ultimate-dflash container (ghcr.io/aeon-7/vllm-aeon-ultimate-dflash:qwen36-v4) with docker-compose.spark-xs.yml. Requires Docker, NVIDIA GPUs (DGX Spark/GB10), and CUDA. Setup involves hf auth login, hf download for model weights, and docker compose up -d. First boot takes ~10-12 minutes due to caching.docker-compose.bf16.yml. Requires Docker, NVIDIA GPUs (A100/H100), and CUDA.other-hardware/ directory.Highlighted Details
Maintenance & Community
The project leverages and collaborates with upstream maintainers of abliteration tooling and vLLM. Specific community links like Discord or Slack are not explicitly provided in the README.
Licensing & Compatibility
The model is released under the Apache 2.0 license, inherited from the base Qwen3.6-27B model. This license generally permits commercial use and integration into closed-source projects.
Limitations & Caveats
This is an uncensored model; users are solely responsible for implementing downstream safety layers (input validation, output filtering, moderation) as the model will execute instructions without internal judgment. NVFP4 quantization offers no throughput advantage on pre-Blackwell hardware (e.g., A100/H100) where it dequantizes to BF16. The DDTree v5 research track is experimental and not production-ready.
1 week ago
Inactive
mit-han-lab
casper-hansen
mit-han-lab
Tiiny-AI