A full-stack developer and Angular consultant explains how AI models can be treated as local assets, enabling better performance and privacy.

The New Stack is a publication covering trends and technologies in cloud-native development, DevOps, and software delivery. Developers can learn about containerization, Kubernetes, and cloud computing, as well as explore topics such as microservices architecture, serverless computing, and continuous integration/continuous delivery (CI/CD) pipelines.

The New Stack

Running AI models locally in the browser using ONNX Runtime Web offers significant advantages over cloud-based approaches. Local execution eliminates privacy concerns by keeping sensitive data on-device, enables offline functionality, and provides instant feedback loops. ONNX acts as a universal format for ML models, allowing models trained in PyTorch or TensorFlow to run anywhere via JavaScript. Angular's Signals feature (v16+) provides the performance isolation needed for heavy inference operations. The approach enables mixing local models for low-latency tasks with cloud calls for complex reasoning, while maintaining transparency about data handling.

Why the Frontend Should Run AI Models Locally With ONNX

Lightweight sentence transformers can be run on onnx.
Yet to see inference optimisations with web assembly furthermore. But there is always limitation on how much can be deffered to local assets (file size, network, io).
I can clearly see frontend inferences as secondary / partial features as of now. But for heavy content generation, a user might prefer fast inferencing and wouldn’t care about if its frontend or cloud based (data privacy concerned people would anyway run it from their own cloud/ vpc).
Quite interesting and hopeful for the future.

‌Nice one and now is the LLMs era, and for that purpose we use Ollama.

ONNX doesn’t support tool and function calling. So, it’s nice for plain, simple SLM work. But useless beyond that.