On-Device SLM Decision Tree

A decision tree for FunctionGemma and edge AI use cases — based on real experiments

flowchart TD START([🤔 I have a mobile feature idea
that could use AI]) --> Q1 Q1[Is the user input
natural language?] Q1 -->|No| D1_DESC[Structured inputs — sensors,
GPS, buttons, or toggles — and
rule-based processing don't
need language models] --> D_COMMON[✅ Use deterministic code] Q1 -->|Yes| Q3 Q3[Can the task be solved
with a finite set of
predefined actions?] Q3 -->|No| D3[☁️ Use a cloud LLM
Open-ended generation,
complex reasoning, and
creative tasks exceed
on-device model capacity] Q3 -->|Yes| Q4 Q4[Is failure tolerable?
Can you retry or
fall back gracefully?] Q4 -->|No| D4_DESC[Safety-critical, financial,
or medical decisions need
100% reliability, not 85%] --> D_COMMON Q4 -->|Yes| Q5 Q5[Can all processing
happen on-device?] Q5 -->|Yes| Q6 Q5 -->|No| Q7 Q6[Are you ready
to fine-tune?] Q6 -->|No| D5[⏸️ Wait or reconsider
Base FunctionGemma scores
58% without fine-tuning.
Not reliable enough for
production use] Q6 -->|Yes| Q9[Is the task
fully local?] Q7[Does on-device routing
add real value before
the remote call?] Q7 -->|Yes| Q6 Q7 -->|No| D7[☁️ Use a cloud LLM directly
If you need the network anyway
and routing is simple, skip
the on-device overhead] Q9 -->|Yes| D6[📱 FunctionGemma
fine-tuned for your tools
The sweet spot: bounded API,
local-first, failure-tolerant] Q9 -->|No| D8[📱☁️ FunctionGemma as router
+ cloud backend
Local intent classification,
remote execution via
agent orchestration] style START fill:#1a1a2e,stroke:#e94560,color:#fff,stroke-width:2px style Q1 fill:#16213e,stroke:#0f3460,color:#fff,stroke-width:1px style Q3 fill:#16213e,stroke:#0f3460,color:#fff,stroke-width:1px style Q4 fill:#16213e,stroke:#0f3460,color:#fff,stroke-width:1px style Q5 fill:#16213e,stroke:#0f3460,color:#fff,stroke-width:1px style Q6 fill:#16213e,stroke:#0f3460,color:#fff,stroke-width:1px style Q7 fill:#16213e,stroke:#0f3460,color:#fff,stroke-width:1px style Q9 fill:#16213e,stroke:#0f3460,color:#fff,stroke-width:1px style D1_DESC fill:#132a22,stroke:#2d6a4f,color:#ccc,stroke-dasharray: 5 5 style D4_DESC fill:#132a22,stroke:#2d6a4f,color:#ccc,stroke-dasharray: 5 5 style D_COMMON fill:#1b4332,stroke:#2d6a4f,color:#fff style D3 fill:#1d3557,stroke:#457b9d,color:#fff style D7 fill:#1d3557,stroke:#457b9d,color:#fff style D6 fill:#462255,stroke:#7b2d8e,color:#fff style D8 fill:#462255,stroke:#7b2d8e,color:#fff style D5 fill:#5c4033,stroke:#8b6914,color:#fff

Real examples mapped to each outcome

Hiking safety copilot Exits at Q1 (No). Sensor data → risk thresholds → alert. Input is GPS, elevation, weather APIs — not language. Deterministic rules are faster and 100% reliable. START → Q1 → No → ✅ Deterministic code

Context-aware notifications Exits at Q3 (No). Generating creative, personalized meal reminders is open-ended text generation, not bounded function calling. Needs a larger model or cloud LLM. START → Q1 → Yes → Q3 → No → ☁️ Cloud LLM

Safety-critical outdoor alerts Exits at Q4 (No). "Should I turn back?" can't tolerate a 15% error rate. When lives are at stake, 85% accuracy isn't enough. START → Q1 → Yes → Q3 → Yes → Q4 → No → ✅ Deterministic code

Cycling mid-route re-planning Exits at Q7 (No). Language input is genuinely ambiguous ("find a flatter route"), but route computation requires a remote routing engine. If routing is simple enough for a few UI buttons, skip the on-device overhead. START → Q1 → Yes → Q3 → Yes → Q4 → Yes → Q5 → No → Q7 → No → ☁️ Cloud LLM directly

Mobile Actions / Tiny Garden Reaches D6. Google's demo use cases: bounded API surface, fully on-device, fine-tuned for specific tools. "Turn on the flashlight" and "Plant sunflowers in row 2" — the sweet spot. START → Q1 → Yes → Q3 → Yes → Q4 → Yes → Q5 → Yes → Q6 → Yes → Q9 → Yes → 📱 FunctionGemma

Complex voice commands → API routing Reaches D8. FunctionGemma classifies intent locally, Koog orchestrates tool calls, cloud handles execution. Useful when many API endpoints need natural language dispatch. START → Q1 → Yes → Q3 → Yes → Q4 → Yes → Q5 → No → Q7 → Yes → Q6 → Yes → Q9 → No → 📱☁️ Router

Any use case without fine-tuning Reaches D5. Base FunctionGemma scores 58% on Mobile Actions. Google's own docs say fine-tuning is required. Without it, the model isn't production-ready. START → ... → Q6 → No → ⏸️ Wait or reconsider

Should I use an on-device SLM for this mobile feature?

Real examples mapped to each outcome