accuracy-safe-quantization
Shrink a converted LiteRT model with ai-edge-quantizer (fp16 / int8 / int4) without losing accuracy, verifying parity against the float source after every step. Use when choosing a quantization recipe for a new model, when a quantized model fails to load, degrades on a task benchmark, or degenerates over long generations, or when deciding between dynamic-range, weight-only, and blockwise variants.
google-ai-edge · 已有静态检查记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add google-ai-edge/litert-samples --skill accuracy-safe-quantization技能说明
name: accuracy-safe-quantization description: Shrink a converted LiteRT model with ai-edge-quantizer (fp16 / int8 / int4) without losing accuracy, verifying parity against the float source after every step. Use when choosing a quantization recipe for a new model, when a quantized model fails to load, degrades on a task benchmark, or degenerates over long generations, or when deciding between dynamic-range, weight-only, and blockwise variants.
Accuracy-safe quantization
A quantization is done when three things hold, in this order:
- it exports and the file shrinks by what the recipe predicts,
- output parity with the float source holds on a task-level check, not just a smoke test,
- the quantized model still passes the deployment check on the target runtime and device.
Quantization rewrites the graph, so step 3 is a fresh obligation every time:
re-run the same CompiledModel verification you used to accept the float
conversion (see the gpu-clean-conversion skill), then the on-device
numerical check.
All recipes below are ai-edge-quantizer (pip install ai-edge-quantizer;
0.9.0 as of 2026-10, which pins ai-edge-litert 2.2.0 on the host),
plain Python, no build step. Worked examples live in this repo under
models/bonsai/bonsai_image_4b/converted/ and
models/qwen/qwen3_tts/converted/.
Choosing a lane
Start with the lightest recipe that meets the size budget, and move down only on evidence:
| Budget / model | Recipe |
|---|---|
| ~2× smaller, zero risk | fp16 float-casting. Weights cast to fp16, compute stays float. On a GPU that already computes in fp16 this is close to free numerically — verify anyway |
| ~4× smaller — encoders, conv nets, diffusion blocks | Dynamic-range int8 channelwise. int8 weights, float activations; this shape rides the GPU delegate |
| Dynamic int8 lost quality (conditioning, embeddings) | Weight-only, same bits. Inserts an explicit DEQUANTIZE so the matmul runs in float and activations are never quantized — more quality, some latency |
| ~7× smaller — LLM / autoregressive decoders | int4 blockwise-32 + OCTAV, embeddings int8. Never channelwise for a decoder: it looks fine on short outputs and degenerates over long generations. A blockwise recipe needs the quantized dimension divisible by the block (a 3-input linear fails block-32 with Quantized dimension 3 … is not divisible) |
| Data-free int4 still fails the task gate | Calibrated ingest. Take a GPTQ checkpoint and preserve its grid with DEQUANTIZED_WEIGHT_RECOVERY — see the routing table |
Full-integer static quantization (static_wi8_ai8 — quantized activations,
calibration data required) is a different lane aimed at NPU/AOT targets and
is not covered here.
Recipes
Recipes layer by regex: broad rule first, narrow overrides after — that is
how one file mixes lanes (bonsai's DiT puts everything at int8 channelwise,
then overrides .*TransformerBlock_.* to int4 blockwise).
fp16 float-casting:
from ai_edge_quantizer import quantizer, recipe_manager
from ai_edge_quantizer.recipe import AlgorithmName, qtyping
rm = recipe_manager.RecipeManager()
rm.add_quantization_config(
regex=".*", operation_name=qtyping.TFLOperationName.ALL_SUPPORTED,
op_config=qtyping.OpQuantizationConfig(
weight_tensor_config=qtyping.TensorQuantizationConfig(
num_bits=16, dtype=qtyping.TensorDataType.FLOAT),
compute_precision=qtyping.ComputePrecision.FLOAT),
algorithm_key=AlgorithmName.FLOAT_CASTING)
quantizer.Quantizer("model_fp32.tflite", rm.get_quantization_recipe()) \
.quantize().export_model("model_fp16.tflite")
Dynamic-range int (swap bits / granularity / algorithm per the table):
from ai_edge_quantizer.qtyping import QuantGranularity as G
from ai_edge_quantizer.qtyping import TFLOperationName as OP
rm = recipe_manager.RecipeManager()
rm.add_dynamic_config(regex=".*", operation_name=OP.FULLY_CONNECTED,
num_bits=4, granularity=G.BLOCKWISE_32,
algorithm_key=AlgorithmName.OCTAV)
rm.add_dynamic_config(regex=".*", operation_name=OP.EMBEDDING_LOOKUP,
num_bits=8, granularity=G.CHANNELWISE)
Weight-only uses the same signature via rm.add_weight_only_config(...) —
models/bonsai/bonsai_image_4b/converted/quantize_weight_only.py wraps it
as a reusable CLI.
ai_edge_quantizer.recipe also ships these as presets
(dynamic_wi8_afp32(), dynamic_wi4b32_afp32(), weight_only_wi8_afp32(),
…). The litert-torch LLM exporter accepts a preset name as its
quantization_recipe argument, and a custom recipe can be registered by
assigning a callable onto the module — the qwen3_tts talker recipe
(models/qwen/qwen3_tts/converted/export_talker.py) registers BOCTAV4
(blockwise-32 OCTAV int4 + int8 embeddings) that way.
Verify after every step
- Size first. fp16 ≈ ½, int8 ≈ ¼, int4 blockwise ≈ ⅐ of fp32 (block scales add overhead). If the file did not shrink as predicted, the regex did not match — fix that before measuring anything.
- Parity against the float reference. Same inputs through the float and quantized models; correlation on outputs plus the task-level check (argmax match, token-for-token greedy decode, IoU).
- A smoke gate is a floor, not a parity verdict. An LLM can pass most of a handful of chat prompts and still score near zero on a real benchmark. Before publishing an int4 decoder, run a task benchmark at real length (e.g. GSM8K-style, n≥100) against the float baseline.
- Long generations, specifically. Granularity problems do not show up in short outputs.
- On the target device. Host emulation of int kernels is pessimistic — int8 graphs have scored visibly worse on host CPU than the same graphs on the device GPU delegate. Never reject a recipe on desktop numbers alone; never accept one without device numbers.
When it breaks or degrades
| What you see | Knob to turn |
|---|