Convert LLMs to LiteRT-LM Without Losing Quality

Where this fits

You've fine-tuned a 7B model on Hugging Face, and now you need it running on-device with the LiteRT-LM runtime. Your first export either crashes at engine creation, or the quantized version starts garbling answers. The litert-conversion-workflow skill turns this murky process into a repeatable pipeline: classify the architecture against known runtime walls, pick a recipe family, export, quantize, and gate output quality against the source model — before you ship.

Why agents benefit

• Diagnoses failure modes early: Instead of guessing why a bundle crashes, the agent checks architecture compatibility first, then walks through engine-load and generation gates.
• Applies the right recipe per model family: Dense, reasoning, hybrid SSM, and VLM models each need different export and quantization paths; the skill encodes those distinctions.
• Gates quality with parity checks, not smoke tests: It compares task-level outputs between source and converted models, rejecting bundles that "work" but answer worse.
• Handles multi-turn and backend constraints: The skill validates against target device and conversation scenarios, not just single-turn generation.
• Produces a publishable artifact: When all gates pass, you get a .litertlm bundle that's ready to deploy — with documented quality evidence.

Practical scenarios

Converting a new LLM to LiteRT-LM

Your team selects Llama-3.2-3B for an edge chatbot. The agent classifies it as a dense model, picks the dense recipe, exports, and quantizes to int8. Then it runs parity checks on a curated prompt set, comparing token-level perplexity and a few task scores. Only if those pass does it mark the conversion complete.

Diagnosing a crash at engine creation

Your existing converted bundle loads in the raw interpreter but dies when LiteRT-LM engine initializes. The agent re-runs classification and spots unsupported ops from the source model's custom attention implementation. It recommends switching to a hybrid SSM recipe or applying a fallback path, then re-exports.

Deciding whether a model is convertible at all

Before investing a day in export, the agent inspects the architecture against known runtime walls. It flags unsupported layers or exotic position embeddings. It might suggest a different checkpoint variant or a radical recipe change — saving you from a doomed conversion.

Add it to your agent workflow

Install the skill via npx:

npx skills add google-ai-edge/litert-samples --skill litert-conversion-workflow

Then instruct your agent with a task like:

Use litert-conversion-workflow to convert meta-llama/Llama-3.2-3B-Instruct to a .litertlm bundle for Android GPU. Gate against source and report the parity metrics.

The agent will read the skill's steps, invoke the tools, and give you a pass/fail verdict with evidence.

Compare before adopting

Before committing, compare it against your current export script or other conversion guides. Look at: whether it covers your exact model family (e.g., Mamba, Pixtral), how recent its recipe updates are, and if the quality gates match your production threshold. Also check if the skill's workflow fits your existing build system — does it expect a Linux environment or specific Python dependencies?

Why it is worth tracking

Google AI Edge maintains this skill in the litert-samples repo (408 stars, active commits). The workflow is grounded in real runtime constraints, not abstract theory — every recipe and gate addresses a known failure point. If you're shipping LLMs or VLMs on-device, this skill is a practical checklist that saves debugging hours. Re-evaluate it whenever LiteRT-LM updates its op support or quantization schemes. For a deeper dive, see the OpenAgentSkill listing for community usage patterns and version history.