On-Device Fitness Coach #2: On-Device AI Decision Scorecard

A practical framework for deciding whether an AI feature belongs on-device, in the cloud, or somewhere in between.

How to use this scorecard

Before evaluating models, SDKs, or runtimes, evaluate the feature itself. Score each dimension from 1–5 based on your specific use case :

• • 1–2: Strongly favors cloud inference
• • 3: Either approach could work
• • 4–5: Strongly favors on-device inference

There isn't a “correct” score. The goal is to make architectural trade-offs explicit before choosing a technology.

Interpreting Your Results

5–12 points → Cloud-first

Your feature is likely a better fit for cloud inference. That's not a lesser choice. Cloud provides flexibility, centralized updates, and access to larger models that may not realistically fit on mobile devices.

13–19 points → Hybrid

Your feature has competing requirements. This is where many production AI experiences live.

A common approach is :

• • Local inference for everyday interactions
• • Cloud inference for more complex reasoning
• • Graceful fallback when devices have limited capabilities
• • Clear user experience when switching between the two

Hybrid is often the most honest representation of the product you're building.

20–25 points → On-device

Your product requirements strongly support local inference.

The next architectural decision isn't whether to run on-device. It's how. That's where choosing the right Android AI stack becomes important - and exactly what we'll cover in Part 3 of this series.

Before You Build

Before writing a single line of AI code, answer these three questions.

1. Define the user promise

Complete this sentence :

For (user), given (input), the app generates (insight) so they can (benefit).

Example :

For runners, given the last two weeks of workouts, the app identifies consistency patterns so they can build more sustainable exercise habits.

If you can't clearly describe the user promise, the feature probably isn't well-defined yet.

2. Define what stays local

List every piece of data that should never leave the user's device. Examples :

• • Workout history
• • Heart rate
• • Sleep data
• • Photos
• • Voice recordings
• • Location history

Being intentional about what stays local is one of the simplest ways to build user trust.

3. Define the boundaries

Complete this sentence :

This feature will never...

Examples :

• • Diagnose illness
• • Replace professional advice
• • Present uncertain output as fact
• • Collect unnecessary personal data
• • Hide when cloud inference is being used

Good AI products aren't defined only by what they can do. They're also defined by what they intentionally choose not to do.

One Last Thought

Architecture decisions shape everything that follows. Choosing a model or SDK is important, but it's a second-order decision.

First, decide where the intelligence belongs. Only then decide how you'll build it.

What's Next?

In Part 3, we'll assume you've already made that decision. Now comes the next question every Android developer eventually asks : ML Kit? MediaPipe? LiteRT? Gemini Nano? We'll compare each option, understand where it fits, and build another practical decision framework to help you choose the right Android AI stack - not just the newest one.

TL;DR

• • Score Privacy, Latency, Availability, Cost at Scale, and Engineering Complexity from 1–5.
• • Use the live scorecard above; totals of 5–12, 13–19, and 20–25 map to cloud, hybrid, and on-device.
• • Define user promise, local-only data, and boundaries before writing AI code.
• • Part 3 : choosing the Android AI stack (ML Kit, MediaPipe, LiteRT, Gemini Nano).