LLM Inference On-Device in React Native: The Practical Aspects
Presented
Speaker
Artur Morys-Magiera explored what it actually means to run LLM inference directly on mobile devices in React Native applications. Instead of treating “AI” as a buzzword, he narrowed the focus to LLMs and examined why teams might move inference on-device: reliability without network dependency, stronger privacy guarantees, and lower latency without cloud queues. He also highlighted real user-facing constraints, including model size, disk usage, and hardware variability across iOS and Android devices.
From there, the talk moved into the practical engineering layer: hardware acceleration (GPU, NPU, CPU), runtime fragmentation, debugging challenges with abstraction layers like OpenCL, and real-world performance issues traced to memory layout differences. Artur compared available libraries, explained their tradeoffs, and showed how a unified API approach can simplify integration while still supporting optimizations such as quantization, compilation-time improvements, and model selection based on device capability.
What you’ll walk away with:
- When running LLMs locally improves reliability, privacy, and latency
- How model size and OS-provisioned models impact real user experience
- Why hardware acceleration and device fragmentation shape performance decisions
- The tradeoffs between TF Lite, ONNX, ExecuTorch, MLC, llama.cpp, and Apple-based solutions
- How quantization, compilation optimizations, and unified APIs reduce integration risk
