#VoxxedDaysCERN26

In this talk we will present TornadoVM (www.tornadovm.org); a state-of-the-art programming framework that enables native GPU acceleration of Java programs. TornadoVM allows developers to program GPUs from within Java while the system handles compilation, execution, and optimization completely transparently.
TornadoVM is an open-source project (https://github.com/beehive-lab/TornadoVM) and it is being used by the European Space Agency GAIA mission to accelerate its data processing pipelines resulting in significant reduction of execution times. In addition, it offers a rich toolset ecosystem to help developers throughout their journey (coding, debugging, performance optimizations) and it recently added to its ecosystem GPULlama3.java (https://github.com/beehive-lab/GPULlama3.java); the first GPU-accelerated native Java implementation for AI inference on a plethora of LLM models which has been integrated with Lanchain4J and Quarkus.

The target audience is developers who wish to make a first transition into the GPU programming world, or just practitioners who wish to use the high performance Java AI libraries that TornadoVM provides.

The key takeaways are:
1) Java is AI ready since we can harness GPU power natively for LLM inference
2) Programming GPUs from within Java is easier with the mature ecosystem that TornadoVM provides
3) Having a native Java platfrom for AI and general purpose GPU acceleration enhances both productivity and integration

Devoxx

TornadoVM is a JDK plugin developed over 10 years at the University of Manchester that enables Java developers to run code on GPUs transparently. It supports OpenCL, CUDA, and SPIR-V backends, with Apple Metal in progress. The programming model offers two APIs: a simple loop-parallel annotation-based API and a lower-level kernel API for fine-grained GPU control. Data management leverages Project Panama's foreign memory API to avoid JVM heap/GC conflicts. A task graph abstraction handles kernel chaining and data transfer optimization. The IntelliJ plugin 'Tornado Insight' provides static analysis and debugging for GPU-targeted code. A GPU-accelerated Llama 3 implementation in pure Java demonstrates production readiness, with integrations for LangChain4J and Quarkus. TornadoVM is already in production at the European Space Agency's Gaia mission, achieving over 500x speedup on specific algorithms and reducing processing time from one month to five days.

TornadoVM: Supercharge your Java with GPU acceleration - Christos Kotselidis, Mary Xekalaki