Summary

TornadoVM is a Java framework for high-performance computing on heterogeneous hardware, including NVIDIA, AMD, Intel GPUs, and Apple Silicon. It enables Java developers to run applications efficiently on accelerators by compiling Java bytecode directly, abstracting native programming models and integrating vendor libraries.

How It Works

The framework employs Just-In-Time (JIT) compilation, transforming Java bytecode into GPU-specific code (PTX, OpenCL C, SPIR-V, MSL) at runtime. Computations are defined within a TaskGraph, supporting explicit thread control (KernelContext) or automatic mapping (@Parallel). This allows identical Java code to execute across diverse hardware, managing data transfers and device orchestration transparently.

Quick Start & Requirements

Installation is via SDKMAN! (sdk install tornadovm), with backend-specific builds available. Prerequisites include JDK 21+ (or GraalVM), GCC/G++ ≥ 13, and device drivers (e.g., CUDA Toolkit for NVIDIA features). Official binaries, Docker images, and deployment guides are provided.

Highlighted Details

• NVIDIA Ecosystem Integration: Direct Java access to cuBLAS, cuFFT, cuDNN within TaskGraphs, sharing device buffers and CUDA streams.
• Advanced NVIDIA Features: Exposes Tensor Core mma.sync intrinsics (FP16/INT8) and enables CUDA Graph capture for efficient replay.
• Broad Hardware Support: Targets NVIDIA (PTX/CUDA), AMD/Intel/others (OpenCL), Intel (SPIR-V), and Apple Silicon (Metal), allowing task migration.
• Performance Examples: Powers projects like GPULlama3.java (117 tok/s on RTX 5090 for LLM inference) and ESA's Gaia Mission data processing.

Maintenance & Community

Active development is supported by community channels (GitHub Discussions, Slack) and contribution guidelines. The project benefits from academic/industrial collaborations and NVIDIA's Inception Program.

Licensing & Compatibility

Tornado-API is Apache 2.0, allowing commercial use and closed-source integration. Runtime components (Tornado-Runtime, Tornado-Drivers) use GPLv2 with Classpath Exception, similar to OpenJDK, avoiding copyleft obligations on user applications.

Limitations & Caveats

Achieving optimal performance may require understanding underlying GPU architectures and tuning kernel designs. Managing the multi-backend system and its specific configurations introduces a learning curve for developers.