compiled-model-app-scaffolding
Build a new Android app (Kotlin, Compose) around a verified LiteRT model using the CompiledModel API - the app architecture, the inference-layer lifecycle rules, model delivery, and the UI traps that masquerade as model bugs. Use when turning a converted and device-verified model into a demo or product app, when an app's inference layer leaks memory or blocks the UI, or when a model that verified clean looks wrong inside an app.
google-ai-edge · 已有静态检查记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add google-ai-edge/litert-samples --skill compiled-model-app-scaffolding技能说明
name: compiled-model-app-scaffolding description: Build a new Android app (Kotlin, Compose) around a verified LiteRT model using the CompiledModel API - the app architecture, the inference-layer lifecycle rules, model delivery, and the UI traps that masquerade as model bugs. Use when turning a converted and device-verified model into a demo or product app, when an app's inference layer leaks memory or blocks the UI, or when a model that verified clean looks wrong inside an app.
CompiledModel app scaffolding
An app around a verified model is done when three things hold, in this order:
- the app reproduces the model recipe's verification numbers on the accelerator the recipe verified — before any UI exists,
- inference is confined and leak-free: one dispatcher owns the model, every buffer is closed, benchmarks include the readback,
- the inference code is liftable — another app could take the helper file unchanged.
Scope: this scaffolds a new app from a model recipe. Migrating an
existing TFLite Interpreter app to CompiledModel is a different task with
its own skill, litert-compiled-model-migration. LLMs are served through
the LiteRT-LM Engine rather than raw CompiledModel —
samples/litert/text_to_speech_lm is the reference for that case;
everything below is the non-LLM CompiledModel app.
Step 0: prove parity before building UI
The first milestone has no UI: a bare harness that loads the .tflite,
runs the recipe's verification input on the target accelerator, and
reproduces the numbers recorded in the recipe README (correlation,
residency). If they do not reproduce, stop — that is an
on-device-verification problem, and no amount of app code fixes it.
Only then build the ViewModel and the screen.
The shape
One sample = one standalone Gradle project, four layers:
app/src/main/java/<pkg>/
<Task>Helper.kt inference infra: owns the CompiledModel and its
buffers; pre/post-processing; NO Android UI types
MainViewModel.kt state machine: drives the helper on a confined
dispatcher, exposes UiState
UiState.kt one immutable data class
MainActivity.kt ComponentActivity + setContent, nothing else
view/ Screen.kt, Theme.kt, Color.kt — Compose only
app/src/main/res/values/ strings.xml etc. — no UI strings in Kotlin
The worked example of the full shape is
samples/litert/image_segmentation/kotlin_cpu_gpu/android (litert 2.3.0: the
GPU accelerator ships in litert-gpu, so declare litert:2.3.0 and
litert-gpu:2.3.0; without the second the runtime logs GPU accelerator could not be loaded and registered and a GPU + CPU set runs on the CPU — Galaxy
S26. The sample keeps Kotlin 2.2.21 with -Xskip-metadata-version-check;
the 2.3.0 classes carry Kotlin 2.4 metadata). The layer
boundary that matters most is the helper's: pre/post-processing is part of
the model contract (it must match what the recipe exported against), so it
lives with the model, not in the screen. Reusable inference code is worth
more than UI polish — a reader will lift <Task>Helper.kt and delete the
rest.
Inference-layer rules
- One serial dispatcher owns the model.
Dispatchers.IO.limitedParallelism(1, "ModelDispatcher")in the helper (the named form needs kotlinx-coroutines 1.9+;litert-api2.3.0 pulls in 1.8.0); create, run, and close the model only insidewithContext(modelDispatcher)— it serializes calls without pinning a thread. Not a bare executor in the Activity, and never the main thread. - Buffers are created once, reused every frame, and closed.
TensorBufferisAutoCloseable; forgetting the buffers leaks native memory even when the model itself is closed. Close inonCleared()on the dispatcher of rule 1 (viewModelScopeis already cancelled there, so alaunchnever runs). The vendoredCompiledModelRunner(below) gets the buffer lifecycle right. run()enqueues; the readback waits.run()may return before the GPU finishes — the output read is the synchronization point. Benchmark run + readback together, neverrun()alone.- Warm up once at init. The first GPU inference includes shader compilation. Run one inference on dummy input right after create, so the first user action is not the compile, and no first-run number ever gets quoted as latency.
- Ask for the strict accelerator the recipe verified.
Accelerator.GPUalone makescreatethrow when the GPU cannot take the graph (KotlinLiteRtExceptionFailed to compile model, Galaxy S26, litert 2.2.0 and 2.3.0). Surface it as a visible error and route it back to the model recipe; do not paper over it with a CPU fallback that makes a slow app look like a working one. The exception: a node the delegate declines while building (TopK requires src tensor C dimension to be divisible by 4, 2.2.0 and 2.3.0 wheels on a Mac) runs on the CPU with only a log line — also read the runtime'soperations will run on the GPUline. - Stateful and multi-graph models: move references, not data. Feed
step N's output buffers as step N+1's inputs (buffer ping-pong)
instead of copying state through the host. When one app creates many
CompiledModels (pipelines, chunked models), create oneEnvironmentand pass it to everyCompiledModel.createcall, and close per-run buffers. AnEnvironmentcreated and closed per model reloads the GPU accelerator library each time (one reload aborted the process, once in 50 loads, Galaxy S26, litert 2.2.0). Treat the GPU program-cache option (GpuOptionskeySERIALIZE_PROGRAM_CACHE) as untested per device, and note that the compiler-cache environment option targets NPU JIT, not GPU shader caching.
Vendor the helpers, don't rewrite them
utilities/common/kotlin/ holds the canonical copies of the code every
app needs and every