On-Device Fitness Coach #3: Too Many On-Device AI Options? Here's How to Actually Choose

Open the Android AI docs for the first time and... it is a lot.

ML Kit. MediaPipe. LiteRT. LiteRT-LM. Gemini Nano. Plus whatever new name appears at the next Google I/O.

It is easy to freeze up. Or worse, pick whichever option had the flashiest demo and try to make the use case fit later.

I would rather flip that order.

Start with the task. Then let the task point you toward the technology.

None of these options is universally better. They live at different layers, solve different kinds of problems, and give developers very different levels of control. So before comparing them, there is one important distinction to clear up.

First, ML Kit and Gemini Nano Are Not Competing Options

ML Kit now covers two broad categories.

The first is its familiar set of task-specific APIs for problems such as text recognition, barcode scanning, face detection, language identification, and object detection.

The second is its GenAI APIs, which use Gemini Nano through Android's AICore service for on-device generative tasks such as summarization, rewriting, proofreading, image description, and custom prompting.

So the real comparison is not quite :

It is closer to :

1. ML Kit task-specific APIs
2. ML Kit GenAI APIs, powered by Gemini Nano
3. MediaPipe Tasks
4. LiteRT or LiteRT-LM

Small distinction. Big difference when making an architecture decision.

The Four Paths, in Plain Terms

1. ML Kit Task-Specific APIs

ML Kit is often the easiest place to begin when the problem is already common and clearly defined.

Think :

• • Scanning a barcode
• • Recognizing text in an image
• • Detecting a face
• • Identifying a language
• • Translating text

These APIs give you a higher-level interface, so you can focus more on the product experience and less on managing the model itself. This is a good fit when your task already has a supported API and you do not need deep control over the underlying model.

The tradeoff is that you are working within the task and output shapes the API provides.

2. ML Kit GenAI APIs, Powered by Gemini Nano

ML Kit's GenAI APIs are the higher-level path for supported on-device generative experiences.

The Prompt API can take text, or a combination of text and image input, and return text or structured output. Other GenAI APIs cover more specific tasks such as summarization, rewriting, proofreading, and image description.

This path fits use cases such as :

• • Summarizing a bounded piece of text
• • Rewriting a short message
• • Producing a brief natural-language insight
• • Generating structured output from a controlled prompt
• • Describing an image on-device

It lets you use Gemini Nano without selecting, packaging, optimizing, and shipping your own language model.

That simplicity comes with boundaries.

The Prompt API is currently in beta, availability depends on the device, and an app has to handle states such as available, downloadable, downloading, and unavailable. The APIs rely on AICore, and developers still need to validate latency, quality, supported devices, and fallback behavior for their own feature.

3. MediaPipe Tasks

MediaPipe Tasks is especially useful for customizable vision, audio, gesture, and real-time media pipelines.

Think :

• • Pose detection
• • Hand tracking
• • Image segmentation
• • Gesture recognition
• • Audio classification
• • Processing live camera frames

MediaPipe provides cross-platform task APIs, ready-to-run models, and room to configure or customize the pipeline for your application. For example, a workout app that analyzes a user's exercise form through the camera might use MediaPipe Pose Landmarker to detect body landmarks from images or live video.

MediaPipe has also offered an LLM Inference API, but that API is now in maintenance-only mode. Google recommends LiteRT-LM for new on-device LLM deployments.

So for a new generative text feature, MediaPipe would not be my starting point.

4. LiteRT and LiteRT-LM

LiteRT is Google's lower-level on-device runtime for deploying custom machine-learning and generative-AI models. This is the path for deeper control.

You might use LiteRT when :

• • No higher-level API covers the task
• • You already have a custom model
• • You need control over model inputs and outputs
• • You need to optimize for specific device hardware
• • You want ownership of model delivery and versioning
• • You need a model architecture tailored to your product

LiteRT does not mean you have to train a model from scratch. You can use, convert, or optimize an existing compatible model. What changes is how much of the model and runtime lifecycle your team owns.

For language models specifically, LiteRT-LM is the production-oriented orchestration layer built for running LLM pipelines with LiteRT across supported platforms and hardware.

This gives you flexibility, but flexibility has a price. Your team now owns more of the work around :

• • Model selection
• • Conversion and optimization
• • App size or model delivery
• • Hardware acceleration
• • Device compatibility
• • Performance testing
• • Version management
• • Runtime failures

Sometimes that ownership is exactly what the product needs. Sometimes it is simply more machinery than the first version requires.

Android On-Device AI Stack Cheat Sheet

Start with the task, not the SDK.

What Fitness Coach Actually Needs

Let us bring this back to the sample app running through this series.

Fitness Coach receives a bounded summary of recent activity, such as :

• • Workout frequency
• • Average workout duration
• • Active days
• • Rest days
• • Simple consistency patterns

It returns one short, supportive insight :

The task can be described in one sentence :

Structured, privacy-sensitive activity data goes in, and one short natural-language insight comes out.

That tells us quite a lot.

This is not primarily a vision task. It is not audio processing. It is not a barcode, recognition, extraction, or classification problem with a predefined output.

And for the first version, we do not need to train, optimize, package, and maintain our own language model.

The Fitness Coach Decision Record

Here is the decision more explicitly.

ML Kit task-specific APIs

Not selected.

The output is not a predefined classification, recognition, or extraction result. We need a short generated sentence shaped by a bounded activity summary.

MediaPipe Tasks

Not selected.

The feature is not centered on camera, audio, gesture, or real-time media processing. MediaPipe may become useful later if Fitness Coach adds features such as pose analysis or exercise-form feedback, but it does not match the current task.

LiteRT-LM

Deferred.

It could run a custom on-device language model and give us deeper control. For the first version, that would also mean taking on model selection, packaging, optimization, compatibility, and delivery before we have proven that the feature needs that level of ownership.

ML Kit GenAI Prompt API, Powered by Gemini Nano

Selected as the first candidate.

It matches the shape of the task :

• • Bounded input
• • Short generated text
• • Local processing
• • No custom model required
• • Lower model-lifecycle ownership for the first version

Notice that I am calling it the first candidate, not the permanent answer.

Before treating the decision as final, I would still validate :

• • Device coverage
• • Model availability behavior
• • Output quality
• • Response latency
• • Prompt and token constraints
• • Safety boundaries
• • Deterministic fallback behavior

A runtime can look perfect on paper and still be wrong for the real product.

One Thing Matters More Than the Pick Itself

Whatever runtime we choose, the ViewModel and UI do not need to know what runs underneath.

We can put the capability behind a simple interface :

interface FitnessInsightEngine {
    suspend fun generateInsight(
        summary: ActivitySummary
    ): FitnessInsight
}

The rest of the app depends on FitnessInsightEngine. Not Gemini Nano. Not ML Kit. Not LiteRT-LM.

That gives us room to begin with one implementation and change later without rewriting the entire application.

For example :

FitnessInsightEngine
        |
        ├── GeminiNanoFitnessInsightEngine
        ├── LiteRtFitnessInsightEngine
        ├── CloudFitnessInsightEngine
        └── RuleBasedFitnessInsightEngine

If device coverage is limited, the app can use the rule-based implementation. If a custom local model becomes worthwhile later, LiteRT-LM can sit behind the same interface. If the product eventually adds an opt-in cloud experience, that can become another implementation too.

The architecture holds even when the technology changes.

A Quick Decision Path

When evaluating your own Android AI feature, start here.

1. Is this a common, well-defined task?

Examples :

• • Text recognition
• • Barcode scanning
• • Translation
• • Face detection

Start with an ML Kit task-specific API.

2. Does the feature need short on-device generated text or structured output?

Examples :

• • Summarization
• • Rewriting
• • Guided insight generation
• • Short custom prompting

Evaluate ML Kit GenAI APIs powered by Gemini Nano.

3. Is it a customizable vision, audio, gesture, or live media pipeline?

Examples :

• • Pose detection
• • Segmentation
• • Gesture recognition
• • Audio classification

Evaluate MediaPipe Tasks.

4. Do you need to bring your own model or control the runtime deeply?

For a custom traditional ML model, evaluate LiteRT.

For an on-device language model, evaluate LiteRT-LM.

5. Does one path fail to cover all product and device requirements?

Consider a hybrid architecture, with each implementation behind the same product interface.

Before You Commit

Before writing code, capture four things.

Classify the task

Write one sentence describing :

• • The input
• • The output
• • Whether each is structured or open-ended
• • Whether the data is privacy-sensitive
• • Whether the feature needs to work offline

List the candidates

Include every option that could realistically perform the task. Do not include a runtime only because it is popular.

Record the decision

Write down :

• • What was selected
• • Why it fits
• • Why each alternative was deferred or rejected
• • What assumptions would cause the decision to change

The reasons matter as much as the final choice.

Hide the implementation

Create an interface around the product capability before connecting it to the UI. Your architecture will age much better than any individual SDK name.

The Takeaway

The Android AI ecosystem feels confusing when we begin with the list of tools. It becomes much simpler when we begin with the job.

For Fitness Coach, the first candidate is ML Kit's GenAI Prompt API powered by Gemini Nano because the feature needs bounded, private, on-device text generation without the complexity of owning a custom language model.

That may change as we test it. And that is okay.

The goal is not to choose the newest runtime or commit to one forever.

Choose the smallest amount of technology that honestly fits the task, and build the architecture so you can change your mind later.

What's Next?

In Part 4, we will take this decision and turn it into a production-minded Android architecture using Compose, ViewModel, StateFlow, coroutines, a model manager, and the FitnessInsightEngine abstraction. That is where the feature starts becoming a real app.

TL;DR

• • Start with the task, not the SDK list.
• • ML Kit task APIs, ML Kit GenAI + Gemini Nano, MediaPipe, and LiteRT/LiteRT-LM solve different layers of problems.
• • Fitness Coach's first candidate : ML Kit GenAI Prompt API powered by Gemini Nano.
• • Hide the runtime behind a product interface so you can swap implementations later.
• • Download the one-page cheat sheet for a quick reference.
• • Part 4 : production architecture with Compose, ViewModel, and FitnessInsightEngine.