Verify LiteRT Models On-Device with This Agent Skill

Where this fits

You convert a model to LiteRT, run it on a phone GPU, and get NaN while the host GPU was fine. Or the output is suspiciously identical to the CPU result—a sign of silent fallback. The on-device-verification skill automates the loop that proves a model actually runs on the accelerator you claim, checks numeric parity, and records the device, runtime version, and residency line.

Why agents benefit

• Confirms GPU residency via the CompiledModel API, so you know the model isn't silently running on CPU.
• Compares device output against the source model both numerically and on task-level gates (not just loss curves).
• Diagnoses device-only failures like fp16 range breaks or whole-graph compile ceilings that never appear on host GPUs.
• Produces reproducible records—device name, runtime version, and residency line—so a result is never a hand-wave.
• Integrates into your existing loop after conversion or quantization, making it a natural step for agent-driven pipelines.

Practical scenarios

Silent CPU fallback on a clean graph

Your model compiles and runs on the host GPU, but on a mobile device it falls back to CPU due to an unsupported op or memory pressure. The skill detects this by checking residency and flags the model as not actually GPU-accelerated.

NaN after fp16 quantization

Quantizing to fp16 can break ranges for certain activations. The skill compares device output against the source model's output, catching the NaN immediately and pinpointing the layer where precision diverges.

Whole-graph compile ceiling on mobile GPU

Some models compile fine on the host but fail on device because the device's shader compiler has a different memory or op limit. The skill reports the exact point of failure and log output, saving hours of manual debugging.

Add it to your agent workflow

Install the skill with:

npx skills add google-ai-edge/litert-samples --skill on-device-verification

In your agent loop, after conversion or quantization, invoke the skill to run the three-step verification: dump reference, check residency, compare outputs. If any check fails, the agent can automatically open a diagnostic report and suggest corrective actions. For context on how this fits into the broader agent ecosystem, see the OpenAgentSkill entry.

Compare before adopting

Host-side verification tools (like the CompiledModel checker used in gpu-clean-conversion) only exercise the host GPU—they miss the device's own shader compiler, precision behavior, and memory ceiling. Manual scripts lack the reproducibility the skill enforces. Before adopting, compare against your current verification workflow and ensure the skill's structured record format matches your reporting needs.

Why it is worth tracking

With 408 stars and active development under google-ai-edge, this skill addresses a real gap in the LiteRT toolchain. It's worth evaluating when you're shipping to devices and need confidence that your quantized model behaves identically on GPU. Its structured loop makes it a natural candidate for automation, and the project's momentum suggests it will keep pace with LiteRT updates—so track it now, and reevaluate as your model complexity grows.