Google launches FunctionGemma for on-device tool use

Running AI actions on a phone usually means round-tripping your audio or text to a massive cloud model, waiting for an API payload to come back, and praying your signal doesn't drop.

Google is tackling that latency and privacy tax head-on. The company just released FunctionGemma, a specialized version of its tiny Gemma 3 270M model explicitly tuned for function calling at the edge.

A 270M agent built for local tool use

According to Google, Gemma downloads surged from 100 million to over 300 million in 2025. After dropping the ultra-compact Gemma 3 270M, developers overwhelmingly asked for one feature: native tool execution.

FunctionGemma does not try to be a philosopher. It acts as a bridge between human prompts and software APIs, generating structured function calls and switching right back to plain English to summarize the result.

Out of the box on Google's Mobile Actions evaluation, zero-shot accuracy sits at a modest 58%. But when fine-tuned on task-specific data, reliability jumps all the way to 85%.

Thanks to a 256k vocabulary shared with the Gemma family, it tokenizes structured JSON and multilingual input with fewer tokens, cutting down inference latency. It runs easily on hardware as modest as an NVIDIA Jetson Nano or standard mobile chipsets.

Community developer @xenovacom even built an in-browser physics playground running FunctionGemma locally via Transformers.js. Google also showcased TinyGarden, a mobile game where voice commands like planting and watering crops are parsed into coordinates and engine calls entirely offline.

The tooling ecosystem is already in place. Developers can fine-tune with Hugging Face, Unsloth, Keras, or NeMo, and serve via LiteRT-LM, Ollama, Llama.cpp, MLX, or vLLM.

Why it matters

We think FunctionGemma represents a sharp and necessary pivot in edge AI. Instead of struggling to squeeze a chatty general-purpose LLM onto mobile RAM, Google built a tiny, razor-sharp switchboard.

For developers building smart home gear, mobile apps, or automotive interfaces, this is well worth testing immediately. It gives you a deterministic, private controller that costs virtually zero battery and operates completely offline.

The real power move here is the compound architecture. FunctionGemma can resolve routine tasks like flipping system toggles or creating calendar events locally in milliseconds. When a query requires genuine reasoning, it simply hands off the context to larger cloud models or a local Gemma 3 27B.

The trade-off is clear: with only 270 million parameters, you cannot prompt-engineer your way to reliability. You have to fine-tune it on your specific API schema. But for production edge environments where privacy and speed trump open-ended trivia, a tuned specialist beats an overloaded generalist every single time.