{"slug":"google-ai-edge-on-device-verification","name":"on-device-verification","description":"Prove a converted or quantized LiteRT model on the actual device via the CompiledModel API - confirm GPU residency, compare device output against the source model, and diagnose device-only failures such as silent CPU fallback, whole-graph compile ceilings, and fp16 range breaks. Use after conversion or quantization, when device output is wrong or NaN, when a clean graph fails to compile only on device, or when GPU and CPU outputs are suspiciously identical.","long_description":"---\nname: on-device-verification\ndescription: Prove a converted or quantized LiteRT model on the actual device via the CompiledModel API - confirm GPU residency, compare device output against the source model, and diagnose device-only failures such as silent CPU fallback, whole-graph compile ceilings, and fp16 range breaks. Use after conversion or quantization, when device output is wrong or NaN, when a clean graph fails to compile only on device, or when GPU and CPU outputs are suspiciously identical.\n---\n\n# On-device verification\n\nA device result is done when three things hold:\n\n1. the model compiles and runs on the accelerator you claim it runs on,\n2. the output matches the source model numerically **and** on a task-level\n   gate,\n3. the record names the device, the runtime version, and the residency\n   line. A number without those is not reproducible and not a result.\n\nHost-side checks (the CompiledModel checker used in `gpu-clean-conversion`)\nexercise the host GPU. The device has its own shader compiler, its own\nprecision behavior, and its own memory ceiling — every failure mode in the\ntable below was hit by a model that had already passed on the host.\n\n## Loop\n\n**1. Dump references from the source model, once.** Fixed inputs — one\nreal sample plus fixed-seed random — saved as `.npy` next to the recipe\n(`dump_*_ref.py`). These are ground truth for every later step; regenerate\nthem only when the source model changes.\n\n**2. Run the same inputs on the device: CPU first, then GPU.** One\nargument switches the accelerator:\n\n```python\nfrom ai_edge_litert.compiled_model import CompiledModel\nfrom ai_edge_litert.hardware_accelerator import HardwareAccelerator\n\nmodel = CompiledModel.from_file(\n    \"model.tflite\", hardware_accel=HardwareAccelerator.GPU)  # or .CPU\n```\n\nThe device-CPU run is the control. If it already diverges from the source\ndump, the problem is the conversion, not the GPU — go back to\n`gpu-clean-conversion`. A full worked example of the A/B lives in this\nrepo at `samples/litert/speech_recognition/convert/verify_tflite.py`.\n\nAsk for the strict accelerator. Compiling with\n`HardwareAccelerator.CPU | HardwareAccelerator.GPU` permits partial\ndelegation and hides fallback; use the combined mode only to discover\n*which* ops fell back after a strict GPU compile fails.\n\n**3. Read the delegate log before reading any numbers.**\n\n```\nReplacing N out of M node(s) with delegate ... X partitions\n```\n\nRecord `N/M` and the partition count. `N < M` or `X > 1` means part of the\ngraph runs on the CPU — decide whether that is acceptable before quoting\nany accuracy or latency number.\n\n**4. Gate the output three ways.**\n\n- **Numeric**: correlation and max abs diff against the source dump.\n  Expect genuine GPU fp16 to drift in the last digits — exact equality is\n  a symptom, not a pass (see below).\n- **Task**: argmax match, IoU, token-for-token greedy decode — whatever\n  the model is for.\n- **Artifact**, for generative models: the decoded text / audio / image.\n  An artifact gate catches argmax ties that numeric tolerance misses.\n\n**5. Record it**: device, runtime version, `N/M` + partitions,\ncorrelation, max abs diff, task result — one row per device in the recipe\nREADME.\n\n## The silent CPU fallback\n\nThe most common false positive: everything runs and the numbers match\nperfectly. If the GPU output is **bit-identical to the device CPU fp32\noutput, it almost certainly did not run on the GPU.** Perfect equality is\nthe tell, not the goal. Cross-check the residency line; only when `N/M`\nis full *and* the outputs drift in the last digits are you looking at a\nreal GPU run.\n\n## Device-only failures\n\n| What you see | What it is, what to do |\n|---|---|\n| GPU compile fails on device for a graph that is op-clean and passed the host check | A whole-graph compile ceiling, not a bad op — a fused graph can fail where each half compiles. Split at a natural block boundary (conv frontend / transformer encoder), verify split == monolith bit-exact on the host, ship the halves |\n| The file refuses to load at all | The >2 GB flatbuffer limit. Split, or quantize below it |\n| Full residency, wrong numbers | Bisect by intermediates: re-export with block-boundary tensors as extra outputs and find the first one that diverges. The cause is usually a reduction (mean, variance, Σx²) in fp16, not the op you suspect. If materializing a tap *fixes* the numbers, that localizes the bug to a fusion boundary — record it as a finding, not a nuisance |\n| Wrong-but-plausible output, and every hypothesis costs a slow re-export | Micro-probe instead: export ~1 KB graphs — the suspect subexpression and each candidate fix — and run them through the device harness you already have. Minutes per hypothesis instead of a re-export per hypothesis |\n| NaN or garbage only on the GPU, in a model with large-magnitude residuals or deep modulation paths | An fp16 range break. Confirm the attribution by forcing fp32 on the delegate where the API exposes it (~2× memory, slower); then fix it properly with the fp16-safe rewrites in `gpu-clean-conversion`, or keep the offending block on CPU and run the rest on GPU |\n| The process dies while loading or running a large float model | A memory ceiling, not a model bug. Weights + activations + delegate buffers must fit in available memory. Do not run a multi-GiB fp32 build in-process on a phone \"just to check\" — quantize or split first, then verify the smaller thing |\n| A deep transformer shipped as an fp16 graph is bit-exact on desktop and noise on the device CPU | Android ARM XNNPACK computes **native fp16**; desktop XNNPACK upcasts to fp32. Deep residual streams compound the difference to collapse. Ship fp32 graphs for CPU inference on device; fp16 is a GPU-side format |\n| Attention quality collapses only for a small-head submodule | head_dim is the fp16-fragile axis, not token count: the same graph at head_dim 64 held corr 0.998 where head_dim 16 fell to 0.86. Pad heads to ≥32 or keep the small-head module on CPU |\n| A `[1,N,C]` token tensor that is a graph output **and** feeds other consumers comes back corrupted | 3-D fan-out corruption — the later branch is clobbered, it cascades, and it reads exactly like an fp16 wall downstream (4-D NCHW maps with the same fan-out are fine). Keep token tensors as sole leaf outputs (or keep them 4-D) and push per-token heads to the host — exact, since per-token ops commute with the gather |\n| A recurrent/streaming graph gives correct output on call 1 and drifts on repeated calls | Fused-LSTM-style **variable tensors persist across `invoke()`** on a reused interpreter — and a fresh-interpreter-per-call verify script structurally cannot see it. Call `reset_all_variables()` before every invoke (cost ≈ 0). Related: the CompiledModel loader rejects variable tensors outright, so such graphs are Interpreter-only |\n\n## Watch for\n\n- **Two references, two verdicts.** The source-framework dump is the\n  truth; the device CPU run is the control that isolates the GPU. A\n  GPU-vs-CPU comparison alone can pass while both are wrong.\n- **The fp32-forcing knob is for attribution, not shipping.** It tells\n  you precision is the cause; the fix is a rewrite or hybrid placement.\n- **First inference includes shader compilation.** Correctness on the\n  first run is fine; never quote first-run latency, and never let a\n  latency number travel without its residency line.\n- **One device proves correctness, not portability.** GPU compilers\n  differ per vendor — a compile ceiling on one chip may not exist on\n  another. \"Runs on Android GPUs\" means a device matrix (your own\n  devices, or a farm service such as AI Edge Portal), recorded as one\n  row each.\n- **One runtime proves it for that runtime.** A delegate rejection or\n  miscompute is a fact about the runtime version you measured: ops have\n  been *dropped* between minor versions, a miscompute's victim output\n  has *moved* between versions, and mixing accelerator and core\n  libraries across versions silently falls back to CPU. When a wall\n  appears after an upgrade, bisect the runtime pin on the real graph —\n  micro-probes have repeatedly failed to reproduce walls that only fire\n  in full-graph context, so a negative micro-probe is not a refutation.\n- **Localize miscomputes with single-output graphs, never fan-out\n  taps.** A multi-output tapped probe is itself exposed to\n  output-aliasing bugs and has produced a confidently wrong culprit;\n  in single-output form every op was exact and the *assembly* was the\n  bug.\n- **Sweep the delegate options to classify a miscompute.** Run the same\n  graph across precision, buffer-storage, and backend options: a\n  bit-identical wrong result across all of them places the bug in the\n  shared graph-compilation layer and rules out precision/storage in one\n  pass. And know what the precision flag can do: forcing fp32 rescues\n  overflow→NaN cases only — it does **not** fix precision compounding\n  (the delegate still reduces in fp16), so \"fp32 didn't help\" does not\n  exonerate fp16.\n- **Time the enqueue and the readback as separate counters.** `run()`\n  is asynchronous; timing it alone has reported a 4× GPU win that did\n  not exist. A large readback time is usually the deferred compute, not\n  the transfer. Corollary economics: per-call overhead makes small\n  per-step graphs (KV-cache decoders re-uploading state every token) a\n  net GPU loss — estimate `calls × per-call overhead` against the CPU\n  time before re-exporting for GPU; the crossover sits around\n  hundreds of nodes per call.\n- **The desktop build is the CPU reference, not a GPU sieve** — desktop\n  Python runtimes exercise CPU/XNNPACK only, which is exactly what makes\n  them the right numerical reference. For the device loop, a minimal\n  push-run-pull binary (tflite in, output tensor out) iterates in\n  seconds without an app rebuild.\n\n## Output layout\n\nVerification is part of the model recipe, not a side script:\n\n```\nmodels/<family>/<model>/converted/\n  dump_*_ref.py            source-model reference dumps (.npy)\n  verify_*.py              parity vs those references, accelerator as a flag\n  README.md                per-device table: device | accelerator |\n                           N/M nodes, partitions | corr | max abs | task gate\n```\n\nKeep the dump and the verify separately runnable: references are dumped\nonce on the host, verification re-runs on every device and after every\nmodel change.\n","tagline":"Prove a converted or quantized LiteRT model on the actual device via the CompiledModel API - confirm GPU residency, compare device output against the source model, and diagnose device-only failures such as silent CPU fallback, whole-graph compile ceilings, and fp16 range breaks. ","category":"research","tags":["agent-skill"],"author":"google-ai-edge","verified":false,"attribution":{"status":"registry_indexed","statusLabel":"Registry indexed","shortLabel":"REGISTRY INDEXED","sourceLabel":"recursive skill source sync","sourceDetail":"google-ai-edge/litert-samples","creatorName":"google-ai-edge","creatorUrl":"https://github.com/google-ai-edge","sourceUrl":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification","indexedBy":"OpenAgentSkill community index","claimUrl":"https://www.openagentskill.com/skills/google-ai-edge-on-device-verification#claim-this-skill","claimCta":"Claim this skill","trustNote":"This listing was indexed from public sources and is not marked official until a maintainer claim is approved.","publicNote":"Attribution links to the public repository or creator profile. Creators can claim the listing to update ownership signals."},"stats":{"stars":416,"forks":116,"verified_installs":0,"successful_runs":0,"total_outcomes":0,"rating":0,"review_count":0,"quality_score":41.89},"quality":{"score":73,"tier":"strong","label":"Strong","summary":"Solid option that is likely worth shortlisting for production workflows.","signals":[{"label":"GitHub stars","value":"416","tone":"neutral"},{"label":"Freshness","value":"3d ago","tone":"positive"},{"label":"Install ready","value":"Yes","tone":"positive"},{"label":"License","value":"Apache-2.0","tone":"neutral"}],"warnings":[]},"trust":{"version":"trust-score-v5","score":68,"base_score":76,"outcome_confidence":0,"tier":"review","label":"Sandbox only","summary":"Useful candidate with missing or mixed trust signals. Keep it in an isolated workspace until the outcome loop proves task fit.","recommendedAction":"Run only in a sandbox and compare close alternatives before using it for real work.","decision":{"install_policy":"human_review_before_install","auto_install_allowed":false,"human_review_required":true,"sandbox_first":true,"agent_action":"Compare alternatives before installing.","reasoning":["68/100 Trust Score v5","76/100 Trust Score v4 baseline","Needs more real agent outcomes before unattended install","Install path is available","Review before production"],"review_required_when":["The workspace contains production secrets, payments, private customer data, or irreversible actions.","The install command requests shell, network, credential, database, or broad filesystem access.","Outcome evidence is missing, recently failed, or required human review.","Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface","Automatic installation in a production workspace"]},"dimensions":[{"id":"github_adoption","label":"GitHub adoption","score":62,"weight":0.13,"status":"info","detail":"416 GitHub stars"},{"id":"repo_activity","label":"Stars/forks activity","score":62,"weight":0.08,"status":"info","detail":"416 stars, 116 forks; issue activity unavailable in current metadata"},{"id":"maintenance","label":"Recent maintenance","score":100,"weight":0.14,"status":"pass","detail":"3d since push"},{"id":"license","label":"License clarity","score":86,"weight":0.09,"status":"pass","detail":"Apache-2.0"},{"id":"documentation","label":"README/SKILL.md completeness","score":86,"weight":0.14,"status":"pass","detail":"Metadata includes enough usage and workflow context"},{"id":"dependency_risk","label":"Dependency/runtime risk","score":64,"weight":0.12,"status":"info","detail":"credential or environment access, network or browser surface"},{"id":"installability","label":"Install availability","score":92,"weight":0.1,"status":"pass","detail":"npx skills add google-ai-edge/litert-samples --skill on-device-verification"},{"id":"install_safety","label":"Install command safety","score":92,"weight":0.1,"status":"pass","detail":"standard package or runtime install path"},{"id":"permission_surface","label":"Permission surface","score":46,"weight":0.07,"status":"warn","detail":"secrets or environment access, filesystem or document access"},{"id":"repository","label":"Repository evidence","score":86,"weight":0.04,"status":"pass","detail":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification"},{"id":"review_status","label":"Review status","score":88,"weight":0.05,"status":"pass","detail":"AI review data available"},{"id":"agent_outcomes","label":"Agent Proven outcomes","score":54,"weight":0.13,"status":"info","detail":"No agent outcome data yet"}],"checks":[{"status":"info","label":"GitHub adoption","detail":"416 GitHub stars"},{"status":"info","label":"Stars/forks activity","detail":"416 stars, 116 forks; issue activity unavailable in current metadata"},{"status":"pass","label":"Recent maintenance","detail":"3d since push"},{"status":"pass","label":"License clarity","detail":"Apache-2.0"},{"status":"pass","label":"README/SKILL.md completeness","detail":"Metadata includes enough usage and workflow context"},{"status":"info","label":"Dependency/runtime risk","detail":"credential or environment access, network or browser surface"},{"status":"pass","label":"Install availability","detail":"npx skills add google-ai-edge/litert-samples --skill on-device-verification"},{"status":"pass","label":"Install command safety","detail":"standard package or runtime install path"},{"status":"warn","label":"Permission surface","detail":"secrets or environment access, filesystem or document access"},{"status":"pass","label":"Repository evidence","detail":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification"},{"status":"pass","label":"Review status","detail":"AI review data available"},{"status":"info","label":"Agent Proven outcomes","detail":"No agent outcome data yet"},{"status":"warn","label":"Ownership","detail":"No approved owner claim yet"},{"status":"pass","label":"OpenAgentSkill usage","detail":"7 views, 0 install copies"},{"status":"info","label":"Agent outcomes","detail":"No agent outcome data yet"}],"strengths":["AI review approved","Install path is available","Repository evidence is available","Recently maintained repository","Install command has no obvious high-risk pattern","Outcome loop is ready but needs first real agent run"],"warnings":["Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access","Permission surface: secrets or environment access, filesystem or document access","No real agent outcome reports yet","Human review required before unattended installation"],"evidence":{"stars":"416 GitHub stars","repoActivity":"416 stars, 116 forks","lastPushed":"3d since push","license":"Apache-2.0","repository":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification","install":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","installSafety":"standard package or runtime install path","permissionSurface":"secrets or environment access, filesystem or document access","documentation":"Strong README/SKILL.md context","agentOutcomes":"No agent outcome data yet","agentProvenScore":0,"outcomeConfidence":"0%","installPolicy":"human_review_before_install"},"installReadiness":{"ready":true,"command":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","policy":"human_review_before_install","label":"Human review before install","notes":["Install path is available","Repository evidence is available","License is declared","No Agent Proven outcome evidence yet","3d since push","Financial domain: human review is required before use in a live investment workflow.","Trust Score v5 requires review or sandbox-only use before install."]},"agentCompatibility":["Codex","Claude Code","Cursor","OpenAgentSkill CLI"],"riskSummary":{"level":"medium","label":"Review before production","notes":["Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access","Permission surface: secrets or environment access, filesystem or document access"]},"outcomeEvidence":{"total":0,"successes":0,"failures":0,"notRelevant":0,"successRate":null,"installAttempts":0,"riskBlocked":0,"setupRequired":0,"installSuccessRate":null,"avgOutputQuality":null,"avgTimeToUsefulMs":null,"productionOutcomes":0,"humanReviewRequired":0,"recentSuccessRate":null,"recentFailureRate":null,"uniqueAgents":0,"agentProvenScore":0,"agentProvenLabel":"Needs first agent run","lastOutcomeAt":null,"label":"No agent outcome data yet"},"autoInstall":{"allowed":false,"sandboxRequired":true,"policy":"human_review_before_install","reason":"Compare alternatives before installing."},"outcome_loop":{"version":"openagentskill-agent-outcome-v4","required_after_install":true,"endpoint":"/api/agent/outcome","method":"POST","event_id_source":"feedback.event_id, install_receipt.resolve_event_id, or decision_packet.outcome_feedback.event_id","expected_outcomes":["success","failed","not_relevant","blocked_by_risk","setup_required"],"required_fields":["event_id","skill_slug","task"],"quality_fields":["task_success","output_quality","error_type","human_review_required","used_in_production","workspace","evidence_url","time_to_useful_ms","source_version"],"ranking_inputs_updated":["Trust Score v5 outcome confidence","Agent Proven Score","Resolve ranking task-fit evidence","Skill detail machine-readable metadata","Outcome leaderboard"]},"agent_contract":{"suited_tasks":["research","agent-skill"],"suited_agents":["Codex","Claude Code","Cursor","OpenAgentSkill CLI"],"install_command":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","trust_score":68,"trust_version":"trust-score-v5","risk_level":"medium","do_not_use_when":["Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface","Automatic installation in a production workspace","Autonomous investment, trading, tax, or suitability decisions without a qualified human review"],"before_install":["Read the audit page and machine-readable metadata.","Confirm the install command, license, and permission surface fit the workspace.","Get explicit human approval or choose an alternative before installing."],"after_run":["Report the outcome to /api/agent/outcome using the resolve event id.","Include output_quality, workspace, human_review_required, and evidence_url when available.","Re-resolve before broad production rollout."]},"bestFor":["research","agent-skill"],"doNotUseFor":["Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface","Automatic installation in a production workspace","Autonomous investment, trading, tax, or suitability decisions without a qualified human review"],"knownRisks":["Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access","Permission surface: secrets or environment access, filesystem or document access"],"backward_compatible":{"trust_score_v4":{"version":"trust-score-v4","score":76,"tier":"strong","label":"Strong shortlist","summary":"Good trust signals with a few areas worth checking before rollout."}}},"trust_score_v5":{"version":"trust-score-v5","score":68,"base_score":76,"outcome_confidence":0,"tier":"review","label":"Sandbox only","summary":"Useful candidate with missing or mixed trust signals. Keep it in an isolated workspace until the outcome loop proves task fit.","recommendedAction":"Run only in a sandbox and compare close alternatives before using it for real work.","decision":{"install_policy":"human_review_before_install","auto_install_allowed":false,"human_review_required":true,"sandbox_first":true,"agent_action":"Compare alternatives before installing.","reasoning":["68/100 Trust Score v5","76/100 Trust Score v4 baseline","Needs more real agent outcomes before unattended install","Install path is available","Review before production"],"review_required_when":["The workspace contains production secrets, payments, private customer data, or irreversible actions.","The install command requests shell, network, credential, database, or broad filesystem access.","Outcome evidence is missing, recently failed, or required human review.","Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface","Automatic installation in a production workspace"]},"dimensions":[{"id":"github_adoption","label":"GitHub adoption","score":62,"weight":0.13,"status":"info","detail":"416 GitHub stars"},{"id":"repo_activity","label":"Stars/forks activity","score":62,"weight":0.08,"status":"info","detail":"416 stars, 116 forks; issue activity unavailable in current metadata"},{"id":"maintenance","label":"Recent maintenance","score":100,"weight":0.14,"status":"pass","detail":"3d since push"},{"id":"license","label":"License clarity","score":86,"weight":0.09,"status":"pass","detail":"Apache-2.0"},{"id":"documentation","label":"README/SKILL.md completeness","score":86,"weight":0.14,"status":"pass","detail":"Metadata includes enough usage and workflow context"},{"id":"dependency_risk","label":"Dependency/runtime risk","score":64,"weight":0.12,"status":"info","detail":"credential or environment access, network or browser surface"},{"id":"installability","label":"Install availability","score":92,"weight":0.1,"status":"pass","detail":"npx skills add google-ai-edge/litert-samples --skill on-device-verification"},{"id":"install_safety","label":"Install command safety","score":92,"weight":0.1,"status":"pass","detail":"standard package or runtime install path"},{"id":"permission_surface","label":"Permission surface","score":46,"weight":0.07,"status":"warn","detail":"secrets or environment access, filesystem or document access"},{"id":"repository","label":"Repository evidence","score":86,"weight":0.04,"status":"pass","detail":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification"},{"id":"review_status","label":"Review status","score":88,"weight":0.05,"status":"pass","detail":"AI review data available"},{"id":"agent_outcomes","label":"Agent Proven outcomes","score":54,"weight":0.13,"status":"info","detail":"No agent outcome data yet"}],"checks":[{"status":"info","label":"GitHub adoption","detail":"416 GitHub stars"},{"status":"info","label":"Stars/forks activity","detail":"416 stars, 116 forks; issue activity unavailable in current metadata"},{"status":"pass","label":"Recent maintenance","detail":"3d since push"},{"status":"pass","label":"License clarity","detail":"Apache-2.0"},{"status":"pass","label":"README/SKILL.md completeness","detail":"Metadata includes enough usage and workflow context"},{"status":"info","label":"Dependency/runtime risk","detail":"credential or environment access, network or browser surface"},{"status":"pass","label":"Install availability","detail":"npx skills add google-ai-edge/litert-samples --skill on-device-verification"},{"status":"pass","label":"Install command safety","detail":"standard package or runtime install path"},{"status":"warn","label":"Permission surface","detail":"secrets or environment access, filesystem or document access"},{"status":"pass","label":"Repository evidence","detail":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification"},{"status":"pass","label":"Review status","detail":"AI review data available"},{"status":"info","label":"Agent Proven outcomes","detail":"No agent outcome data yet"},{"status":"warn","label":"Ownership","detail":"No approved owner claim yet"},{"status":"pass","label":"OpenAgentSkill usage","detail":"7 views, 0 install copies"},{"status":"info","label":"Agent outcomes","detail":"No agent outcome data yet"}],"strengths":["AI review approved","Install path is available","Repository evidence is available","Recently maintained repository","Install command has no obvious high-risk pattern","Outcome loop is ready but needs first real agent run"],"warnings":["Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access","Permission surface: secrets or environment access, filesystem or document access","No real agent outcome reports yet","Human review required before unattended installation"],"evidence":{"stars":"416 GitHub stars","repoActivity":"416 stars, 116 forks","lastPushed":"3d since push","license":"Apache-2.0","repository":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification","install":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","installSafety":"standard package or runtime install path","permissionSurface":"secrets or environment access, filesystem or document access","documentation":"Strong README/SKILL.md context","agentOutcomes":"No agent outcome data yet","agentProvenScore":0,"outcomeConfidence":"0%","installPolicy":"human_review_before_install"},"installReadiness":{"ready":true,"command":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","policy":"human_review_before_install","label":"Human review before install","notes":["Install path is available","Repository evidence is available","License is declared","No Agent Proven outcome evidence yet","3d since push","Financial domain: human review is required before use in a live investment workflow.","Trust Score v5 requires review or sandbox-only use before install."]},"agentCompatibility":["Codex","Claude Code","Cursor","OpenAgentSkill CLI"],"riskSummary":{"level":"medium","label":"Review before production","notes":["Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access","Permission surface: secrets or environment access, filesystem or document access"]},"outcomeEvidence":{"total":0,"successes":0,"failures":0,"notRelevant":0,"successRate":null,"installAttempts":0,"riskBlocked":0,"setupRequired":0,"installSuccessRate":null,"avgOutputQuality":null,"avgTimeToUsefulMs":null,"productionOutcomes":0,"humanReviewRequired":0,"recentSuccessRate":null,"recentFailureRate":null,"uniqueAgents":0,"agentProvenScore":0,"agentProvenLabel":"Needs first agent run","lastOutcomeAt":null,"label":"No agent outcome data yet"},"autoInstall":{"allowed":false,"sandboxRequired":true,"policy":"human_review_before_install","reason":"Compare alternatives before installing."},"outcome_loop":{"version":"openagentskill-agent-outcome-v4","required_after_install":true,"endpoint":"/api/agent/outcome","method":"POST","event_id_source":"feedback.event_id, install_receipt.resolve_event_id, or decision_packet.outcome_feedback.event_id","expected_outcomes":["success","failed","not_relevant","blocked_by_risk","setup_required"],"required_fields":["event_id","skill_slug","task"],"quality_fields":["task_success","output_quality","error_type","human_review_required","used_in_production","workspace","evidence_url","time_to_useful_ms","source_version"],"ranking_inputs_updated":["Trust Score v5 outcome confidence","Agent Proven Score","Resolve ranking task-fit evidence","Skill detail machine-readable metadata","Outcome leaderboard"]},"agent_contract":{"suited_tasks":["research","agent-skill"],"suited_agents":["Codex","Claude Code","Cursor","OpenAgentSkill CLI"],"install_command":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","trust_score":68,"trust_version":"trust-score-v5","risk_level":"medium","do_not_use_when":["Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface","Automatic installation in a production workspace","Autonomous investment, trading, tax, or suitability decisions without a qualified human review"],"before_install":["Read the audit page and machine-readable metadata.","Confirm the install command, license, and permission surface fit the workspace.","Get explicit human approval or choose an alternative before installing."],"after_run":["Report the outcome to /api/agent/outcome using the resolve event id.","Include output_quality, workspace, human_review_required, and evidence_url when available.","Re-resolve before broad production rollout."]},"bestFor":["research","agent-skill"],"doNotUseFor":["Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface","Automatic installation in a production workspace","Autonomous investment, trading, tax, or suitability decisions without a qualified human review"],"knownRisks":["Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access","Permission surface: secrets or environment access, filesystem or document access"],"backward_compatible":{"trust_score_v4":{"version":"trust-score-v4","score":76,"tier":"strong","label":"Strong shortlist","summary":"Good trust signals with a few areas worth checking before rollout."}}},"trust_score_v4":{"version":"trust-score-v4","score":76,"tier":"strong","label":"Strong shortlist","summary":"Good trust signals with a few areas worth checking before rollout.","recommendedAction":"Test in a sandbox workflow and compare its install path with close alternatives.","dimensions":[{"id":"github_adoption","label":"GitHub adoption","score":62,"weight":0.13,"status":"info","detail":"416 GitHub stars"},{"id":"repo_activity","label":"Stars/forks activity","score":62,"weight":0.08,"status":"info","detail":"416 stars, 116 forks; issue activity unavailable in current metadata"},{"id":"maintenance","label":"Recent maintenance","score":100,"weight":0.14,"status":"pass","detail":"3d since push"},{"id":"license","label":"License clarity","score":86,"weight":0.09,"status":"pass","detail":"Apache-2.0"},{"id":"documentation","label":"README/SKILL.md completeness","score":86,"weight":0.14,"status":"pass","detail":"Metadata includes enough usage and workflow context"},{"id":"dependency_risk","label":"Dependency/runtime risk","score":64,"weight":0.12,"status":"info","detail":"credential or environment access, network or browser surface"},{"id":"installability","label":"Install availability","score":92,"weight":0.1,"status":"pass","detail":"npx skills add google-ai-edge/litert-samples --skill on-device-verification"},{"id":"install_safety","label":"Install command safety","score":92,"weight":0.1,"status":"pass","detail":"standard package or runtime install path"},{"id":"permission_surface","label":"Permission surface","score":46,"weight":0.07,"status":"warn","detail":"secrets or environment access, filesystem or document access"},{"id":"repository","label":"Repository evidence","score":86,"weight":0.04,"status":"pass","detail":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification"},{"id":"review_status","label":"Review status","score":88,"weight":0.05,"status":"pass","detail":"AI review data available"},{"id":"agent_outcomes","label":"Agent Proven outcomes","score":54,"weight":0.13,"status":"info","detail":"No agent outcome data yet"}],"checks":[{"status":"info","label":"GitHub adoption","detail":"416 GitHub stars"},{"status":"info","label":"Stars/forks activity","detail":"416 stars, 116 forks; issue activity unavailable in current metadata"},{"status":"pass","label":"Recent maintenance","detail":"3d since push"},{"status":"pass","label":"License clarity","detail":"Apache-2.0"},{"status":"pass","label":"README/SKILL.md completeness","detail":"Metadata includes enough usage and workflow context"},{"status":"info","label":"Dependency/runtime risk","detail":"credential or environment access, network or browser surface"},{"status":"pass","label":"Install availability","detail":"npx skills add google-ai-edge/litert-samples --skill on-device-verification"},{"status":"pass","label":"Install command safety","detail":"standard package or runtime install path"},{"status":"warn","label":"Permission surface","detail":"secrets or environment access, filesystem or document access"},{"status":"pass","label":"Repository evidence","detail":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification"},{"status":"pass","label":"Review status","detail":"AI review data available"},{"status":"info","label":"Agent Proven outcomes","detail":"No agent outcome data yet"},{"status":"warn","label":"Ownership","detail":"No approved owner claim yet"},{"status":"pass","label":"OpenAgentSkill usage","detail":"7 views, 0 install copies"},{"status":"info","label":"Agent outcomes","detail":"No agent outcome data yet"}],"strengths":["AI review approved","Install path is available","Repository evidence is available","Recently maintained repository","Install command has no obvious high-risk pattern"],"warnings":["Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access","Permission surface: secrets or environment access, filesystem or document access"],"evidence":{"stars":"416 GitHub stars","repoActivity":"416 stars, 116 forks","lastPushed":"3d since push","license":"Apache-2.0","repository":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification","install":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","installSafety":"standard package or runtime install path","permissionSurface":"secrets or environment access, filesystem or document access","documentation":"Strong README/SKILL.md context","agentOutcomes":"No agent outcome data yet"},"installReadiness":{"ready":true,"command":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","policy":"human_review_before_install","label":"Human review before install","notes":["Install path is available","Repository evidence is available","License is declared","No Agent Proven outcome evidence yet","3d since push","Financial domain: human review is required before use in a live investment workflow."]},"agentCompatibility":["Codex","Claude Code","Cursor","OpenAgentSkill CLI"],"riskSummary":{"level":"medium","label":"Review before production","notes":["Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access","Permission surface: secrets or environment access, filesystem or document access"]},"outcomeEvidence":{"total":0,"successes":0,"failures":0,"notRelevant":0,"successRate":null,"installAttempts":0,"riskBlocked":0,"setupRequired":0,"installSuccessRate":null,"avgOutputQuality":null,"avgTimeToUsefulMs":null,"productionOutcomes":0,"humanReviewRequired":0,"recentSuccessRate":null,"recentFailureRate":null,"uniqueAgents":0,"agentProvenScore":0,"agentProvenLabel":"Needs first agent run","lastOutcomeAt":null,"label":"No agent outcome data yet"},"autoInstall":{"allowed":false,"sandboxRequired":true,"policy":"human_review_before_install","reason":"Human review or sandbox validation is required before automatic installation."},"bestFor":["research","agent-skill"],"doNotUseFor":["Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface","Automatic installation in a production workspace","Autonomous investment, trading, tax, or suitability decisions without a qualified human review"],"knownRisks":["Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access","Permission surface: secrets or environment access, filesystem or document access"]},"agent_proven":{"version":"agent-proven-v1","score":0,"tier":"unproven","label":"Needs first agent run","summary":"No agent outcome reports yet. Use Resolve, run one narrow sandbox task, then report the result.","metrics":{"totalOutcomes":0,"successfulOutcomes":0,"failedOutcomes":0,"installAttempts":0,"installSuccessRate":null,"successRate":null,"recentSuccessRate":null,"recentFailureRate":null,"riskBlocked":0,"setupRequired":0,"notRelevant":0,"avgOutputQuality":null,"avgTimeToUsefulMs":null,"productionOutcomes":0,"humanReviewRequired":0,"uniqueAgents":0,"lastOutcomeAt":null},"signals":[],"penalties":["No real agent outcome evidence yet"]},"outcome_stats":null,"safety":{"score":49,"level":"avoid_auto_install","label":"Avoid automatic install","safety_tier":{"tier":"experimental","label":"Experimental","badge":"EXPERIMENTAL","summary":"Sparse or mixed signals. Useful for discovery, but not for autonomous installation.","recommended_action":"Test manually in an isolated workspace and compare against safer alternatives.","auto_install_policy":"review","reasons":["High-risk permission hints: Secrets or environment access","49/100 agent safety score"]},"auto_install_allowed":false,"human_review_required":true,"blocked":false,"audit_risk":"needs_review","permission_hints":[{"id":"browser","label":"Browser automation","reason":"Skill may drive a browser or interact with web pages.","severity":"medium"},{"id":"network","label":"Network access","reason":"Skill likely fetches remote pages, APIs, repositories, or external services.","severity":"medium"},{"id":"filesystem","label":"Filesystem access","reason":"Skill may read or write project files, documents, generated artifacts, or local workspace state.","severity":"medium"},{"id":"secrets","label":"Secrets or environment access","reason":"Skill metadata references credentials, tokens, environment variables, or secret-bearing workflows.","severity":"high"}],"policy_warnings":["High-risk permission hints: Secrets or environment access","Permission surface may require sandboxing"],"constraints_applied":{"max_risk":"medium","needs_install_command":true,"min_stars":0}},"safety_gate":{"tier":"experimental","label":"Experimental","badge":"EXPERIMENTAL","auto_install_policy":"review","auto_install_allowed":false,"blocked":false,"human_review_required":true,"recommended_action":"Test manually in an isolated workspace and compare against safer alternatives.","reasons":["High-risk permission hints: Secrets or environment access","49/100 agent safety score"]},"eval":{"version":"openagentskill-skill-eval-v1","status":"failed","score":71,"risk_level":"high","decision":{"recommendation":"do_not_auto_install","reason":"Permission surface: secrets or environment access, filesystem or document access","auto_install_allowed":false,"policy":"block","human_review_required":true},"blockers":["Permission surface: secrets or environment access, filesystem or document access"],"warnings":["Trust score: Good trust signals with a few areas worth checking before rollout.","Audit score: Needs review","Agent safety gate: Sparse or mixed signals. Useful for discovery, but not for autonomous installation.","High-risk permission hints: Secrets or environment access","Permission surface may require sandboxing","Financial research output is not financial advice; require human review before any live investment decision","Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access","Permission surface: secrets or environment access, filesystem or document access"],"validation_plan":["Inspect repository, README/SKILL.md, license, and recent commits before production use.","Install in an isolated workspace or sandbox with no production secrets available.","Run the smallest representative task and record files touched, commands run, network access, and outputs.","Compare the selected skill against at least one alternative when the eval status is review or failed.","Promote only after the agent reports a successful verification result and unresolved warnings are accepted."],"checks":[{"id":"task_fit","label":"Task fit","status":"pass","score":84,"required_for_auto_install":true,"detail":"Task wording matches this skill metadata.","evidence":["Evaluate on-device-verification before installing it in an agent workflow","research","Research agents workflows; Claude Code teams; builders willing to evaluate younger projects"]},{"id":"install_path","label":"Install path","status":"pass","score":92,"required_for_auto_install":true,"detail":"Install handoff is available.","evidence":["npx skills add google-ai-edge/litert-samples --skill on-device-verification"]},{"id":"install_safety","label":"Install command safety","status":"pass","score":92,"required_for_auto_install":true,"detail":"standard package or runtime install path","evidence":["npx skills add google-ai-edge/litert-samples --skill on-device-verification"]},{"id":"trust_score","label":"Trust score","status":"warn","score":76,"required_for_auto_install":true,"detail":"Good trust signals with a few areas worth checking before rollout.","evidence":["Strong shortlist","416 GitHub stars","Apache-2.0"]},{"id":"audit_score","label":"Audit score","status":"warn","score":81,"required_for_auto_install":true,"detail":"Needs review","evidence":["Permission surface may require sandboxing"]},{"id":"agent_safety_gate","label":"Agent safety gate","status":"warn","score":49,"required_for_auto_install":true,"detail":"Sparse or mixed signals. Useful for discovery, but not for autonomous installation.","evidence":["Test manually in an isolated workspace and compare against safer alternatives.","High-risk permission hints: Secrets or environment access"]},{"id":"readme_skillmd_completeness","label":"README/SKILL.md completeness","status":"pass","score":86,"required_for_auto_install":false,"detail":"Metadata includes enough usage and workflow context","evidence":["Strong README/SKILL.md context"]},{"id":"license_clarity","label":"License clarity","status":"pass","score":86,"required_for_auto_install":true,"detail":"Apache-2.0","evidence":["Apache-2.0"]},{"id":"recent_maintenance","label":"Recent maintenance","status":"pass","score":100,"required_for_auto_install":false,"detail":"3d since push","evidence":["3d since push"]},{"id":"permission_surface","label":"Permission surface","status":"fail","score":46,"required_for_auto_install":true,"detail":"secrets or environment access, filesystem or document access","evidence":["Browser automation: medium","Network access: medium","Filesystem access: medium"]},{"id":"alternatives","label":"Alternatives available","status":"info","score":55,"required_for_auto_install":false,"detail":"No close alternatives were found in the current shortlist.","evidence":[]}],"endpoints":{"web":"https://www.openagentskill.com/skills/google-ai-edge-on-device-verification/evals","api":"/api/agent/evals?slug=google-ai-edge-on-device-verification","text":"/api/agent/evals?slug=google-ai-edge-on-device-verification&format=text"}},"agent_readable_metadata":{"version":"openagentskill-agent-metadata-v2","skill":{"slug":"google-ai-edge-on-device-verification","name":"on-device-verification","description":"Prove a converted or quantized LiteRT model on the actual device via the CompiledModel API - confirm GPU residency, compare device output against the source model, and diagnose device-only failures such as silent CPU fallback, whole-graph compile ceilings, and fp16 range breaks. Use after conversion or quantization, when device output is wrong or NaN, when a clean graph fails to compile only on device, or when GPU and CPU outputs are suspiciously identical.","category":"research","url":"https://www.openagentskill.com/skills/google-ai-edge-on-device-verification","repository":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification","github_repo":"google-ai-edge/litert-samples"},"suited_tasks":["Research agents workflows","Claude Code teams","builders willing to evaluate younger projects","Search sources","Extract claims","Synthesize findings","Navigate local resources","Run repeatable desktop actions"],"suited_agents":["Codex","Claude Code","Cursor","OpenAgentSkill CLI","CLI"],"install":{"command":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","ready":true,"targets":[{"id":"openagentskill-cli","label":"CLI","kind":"command","value":"npx --yes https://github.com/Leon-Drq/openagentskill/releases/download/cli-v0.3.0/openagentskill-0.3.0.tgz add google-ai-edge-on-device-verification"},{"id":"codex","label":"Codex","kind":"agent-prompt","value":"Install the \"on-device-verification\" agent skill from https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification. Read its SKILL.md or equivalent instructions first, install only the files needed for this workspace, and summarize any required setup before using it. Skill purpose: Prove a converted or quantized LiteRT model on the actual device via the CompiledModel API - confirm GPU residency, compare device output against the source model, and diagnose device-only failures such as silent CPU fallback, whole-graph compile ceilings, and fp16 range breaks. Use after conversion or quantization, when device output is wrong or NaN, when a clean graph fails to compile only on device, or when GPU and CPU outputs are suspiciously identical. After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"google-ai-edge-on-device-verification\",\"task\":\"Install on-device-verification\",\"agent\":\"codex\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes."},{"id":"claude-code","label":"Claude Code","kind":"agent-prompt","value":"Add \"on-device-verification\" as a Claude Code skill from https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification. Inspect the skill instructions, place the reusable skill files in the appropriate local skills location for this project, and report the activation steps. Skill purpose: Prove a converted or quantized LiteRT model on the actual device via the CompiledModel API - confirm GPU residency, compare device output against the source model, and diagnose device-only failures such as silent CPU fallback, whole-graph compile ceilings, and fp16 range breaks. Use after conversion or quantization, when device output is wrong or NaN, when a clean graph fails to compile only on device, or when GPU and CPU outputs are suspiciously identical. After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"google-ai-edge-on-device-verification\",\"task\":\"Install on-device-verification\",\"agent\":\"claude-code\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes."},{"id":"cursor","label":"Cursor","kind":"agent-prompt","value":"Turn \"on-device-verification\" from https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification into a reusable Cursor project rule or agent instruction. Preserve the core workflow, adapt paths to this repo, and keep the rule scoped to tasks where it is relevant. Skill purpose: Prove a converted or quantized LiteRT model on the actual device via the CompiledModel API - confirm GPU residency, compare device output against the source model, and diagnose device-only failures such as silent CPU fallback, whole-graph compile ceilings, and fp16 range breaks. Use after conversion or quantization, when device output is wrong or NaN, when a clean graph fails to compile only on device, or when GPU and CPU outputs are suspiciously identical. After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"google-ai-edge-on-device-verification\",\"task\":\"Install on-device-verification\",\"agent\":\"cursor\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes."}],"handoff_url":"https://www.openagentskill.com/api/skills/google-ai-edge-on-device-verification/install","manifest_url":"https://www.openagentskill.com/api/registry/manifest/google-ai-edge-on-device-verification"},"trust":{"score":76,"label":"Strong shortlist","version":"trust-score-v4","install_policy":"human_review_before_install","evidence":{"stars":"416 GitHub stars","repoActivity":"416 stars, 116 forks","lastPushed":"3d since push","license":"Apache-2.0","repository":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification","install":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","installSafety":"standard package or runtime install path","permissionSurface":"secrets or environment access, filesystem or document access","documentation":"Strong README/SKILL.md context","agentOutcomes":"No agent outcome data yet"},"outcome_evidence":{"total":0,"successes":0,"failures":0,"not_relevant":0,"success_rate":null,"recent_success_rate":null,"recent_failure_rate":null,"install_attempts":0,"install_success_rate":null,"risk_blocked":0,"setup_required":0,"avg_output_quality":null,"production_outcomes":0,"last_outcome_at":null,"label":"No agent outcome data yet"},"auto_install":{"allowed":false,"sandbox_required":true,"reason":"Human review or sandbox validation is required before automatic installation."},"best_for":["research","agent-skill"],"known_risks":["Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access","Permission surface: secrets or environment access, filesystem or document access"]},"agent_proven":{"version":"agent-proven-v1","score":0,"tier":"unproven","label":"Needs first agent run","summary":"No agent outcome reports yet. Use Resolve, run one narrow sandbox task, then report the result.","metrics":{"totalOutcomes":0,"successfulOutcomes":0,"failedOutcomes":0,"installAttempts":0,"installSuccessRate":null,"successRate":null,"recentSuccessRate":null,"recentFailureRate":null,"riskBlocked":0,"setupRequired":0,"notRelevant":0,"avgOutputQuality":null,"avgTimeToUsefulMs":null,"productionOutcomes":0,"humanReviewRequired":0,"uniqueAgents":0,"lastOutcomeAt":null},"signals":[],"penalties":["No real agent outcome evidence yet"]},"audit":{"score":81,"risk_level":"needs_review","risk_label":"Needs review","warnings":["Permission surface may require sandboxing","Financial research output is not financial advice; require human review before any live investment decision","Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access","Permission surface: secrets or environment access, filesystem or document access"]},"safety_gate":{"tier":"experimental","label":"Experimental","auto_install_policy":"review","auto_install_allowed":false,"human_review_required":true,"blocked":false,"recommended_action":"Test manually in an isolated workspace and compare against safer alternatives."},"quality":{"score":73,"label":"Strong"},"supply":{"track":"Research and knowledge work","scenario":"Research agents","maintenance":"3d since push","risk":"Needs review"},"alternative_skills":[],"do_not_use_when":["teams that need a vendor-supported SLA","high-compliance environments without internal security review","No major risk signals from current metadata","High-risk permission hints: Secrets or environment access","Permission surface may require sandboxing","Financial research output is not financial advice; require human review before any live investment decision","Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review"],"agent_contract":{"task_input":"Use on-device-verification in an agent workflow","recommended_action":"Test manually in an isolated workspace and compare against safer alternatives.","install_policy":"review","minimum_review_before_use":["Trust: 76/100 Strong shortlist","Audit: 81/100 Needs review","Safety: 49/100 Avoid automatic install","Review repository, license, install command, and permission surface before production use."],"expected_agent_output":{"selected_skill":"google-ai-edge-on-device-verification (on-device-verification)","install_command":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","risk_summary":"Needs review; Experimental; Review before production","verification_result":"Report the smallest successful task, files touched, warnings, and any missing setup."}},"outcome_feedback":{"endpoint":"https://www.openagentskill.com/api/agent/outcome","method":"POST","requires_resolve_event_id":true,"event_id_source":"Use install_receipt.outcome_feedback.event_id or feedback.event_id returned by /api/agent/resolve for the current task.","expected_outcomes":["success","failed","not_relevant","blocked_by_risk","setup_required"],"payload_template":{"event_id":"<install_receipt.outcome_feedback.event_id or feedback.event_id from /api/agent/resolve>","skill_slug":"google-ai-edge-on-device-verification","task":"Use on-device-verification in an agent workflow","agent":"codex","outcome":"success","install_used":true,"risk_blocked":false,"setup_required":false,"task_success":true,"output_quality":4,"error_type":null,"human_review_required":false,"workspace":"sandbox","time_to_useful_ms":120000,"notes":"Report the smallest successful task, setup friction, files touched, and risk notes."}},"endpoints":{"web":"https://www.openagentskill.com/skills/google-ai-edge-on-device-verification","api":"https://www.openagentskill.com/api/agent/skills/google-ai-edge-on-device-verification","audit":"https://www.openagentskill.com/skills/google-ai-edge-on-device-verification/audit","eval":"https://www.openagentskill.com/api/agent/evals?slug=google-ai-edge-on-device-verification&task=Use%20on-device-verification%20in%20an%20agent%20workflow&max_risk=medium","resolve":"https://www.openagentskill.com/api/agent/resolve?task=Use%20on-device-verification%20in%20an%20agent%20workflow&agent=codex&max_risk=medium","receipt":"https://www.openagentskill.com/api/agent/receipt?task=Use%20on-device-verification%20in%20an%20agent%20workflow&agent=codex&max_risk=medium&format=text","install":"https://www.openagentskill.com/api/skills/google-ai-edge-on-device-verification/install","manifest":"https://www.openagentskill.com/api/registry/manifest/google-ai-edge-on-device-verification"}},"machine_metadata":{"version":"openagentskill-agent-metadata-v2","skill":{"slug":"google-ai-edge-on-device-verification","name":"on-device-verification","description":"Prove a converted or quantized LiteRT model on the actual device via the CompiledModel API - confirm GPU residency, compare device output against the source model, and diagnose device-only failures such as silent CPU fallback, whole-graph compile ceilings, and fp16 range breaks. Use after conversion or quantization, when device output is wrong or NaN, when a clean graph fails to compile only on device, or when GPU and CPU outputs are suspiciously identical.","category":"research","url":"https://www.openagentskill.com/skills/google-ai-edge-on-device-verification","repository":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification","github_repo":"google-ai-edge/litert-samples"},"suited_tasks":["Research agents workflows","Claude Code teams","builders willing to evaluate younger projects","Search sources","Extract claims","Synthesize findings","Navigate local resources","Run repeatable desktop actions"],"suited_agents":["Codex","Claude Code","Cursor","OpenAgentSkill CLI","CLI"],"install":{"command":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","ready":true,"targets":[{"id":"openagentskill-cli","label":"CLI","kind":"command","value":"npx --yes https://github.com/Leon-Drq/openagentskill/releases/download/cli-v0.3.0/openagentskill-0.3.0.tgz add google-ai-edge-on-device-verification"},{"id":"codex","label":"Codex","kind":"agent-prompt","value":"Install the \"on-device-verification\" agent skill from https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification. Read its SKILL.md or equivalent instructions first, install only the files needed for this workspace, and summarize any required setup before using it. Skill purpose: Prove a converted or quantized LiteRT model on the actual device via the CompiledModel API - confirm GPU residency, compare device output against the source model, and diagnose device-only failures such as silent CPU fallback, whole-graph compile ceilings, and fp16 range breaks. Use after conversion or quantization, when device output is wrong or NaN, when a clean graph fails to compile only on device, or when GPU and CPU outputs are suspiciously identical. After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"google-ai-edge-on-device-verification\",\"task\":\"Install on-device-verification\",\"agent\":\"codex\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes."},{"id":"claude-code","label":"Claude Code","kind":"agent-prompt","value":"Add \"on-device-verification\" as a Claude Code skill from https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification. Inspect the skill instructions, place the reusable skill files in the appropriate local skills location for this project, and report the activation steps. Skill purpose: Prove a converted or quantized LiteRT model on the actual device via the CompiledModel API - confirm GPU residency, compare device output against the source model, and diagnose device-only failures such as silent CPU fallback, whole-graph compile ceilings, and fp16 range breaks. Use after conversion or quantization, when device output is wrong or NaN, when a clean graph fails to compile only on device, or when GPU and CPU outputs are suspiciously identical. After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"google-ai-edge-on-device-verification\",\"task\":\"Install on-device-verification\",\"agent\":\"claude-code\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes."},{"id":"cursor","label":"Cursor","kind":"agent-prompt","value":"Turn \"on-device-verification\" from https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification into a reusable Cursor project rule or agent instruction. Preserve the core workflow, adapt paths to this repo, and keep the rule scoped to tasks where it is relevant. Skill purpose: Prove a converted or quantized LiteRT model on the actual device via the CompiledModel API - confirm GPU residency, compare device output against the source model, and diagnose device-only failures such as silent CPU fallback, whole-graph compile ceilings, and fp16 range breaks. Use after conversion or quantization, when device output is wrong or NaN, when a clean graph fails to compile only on device, or when GPU and CPU outputs are suspiciously identical. After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"google-ai-edge-on-device-verification\",\"task\":\"Install on-device-verification\",\"agent\":\"cursor\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes."}],"handoff_url":"https://www.openagentskill.com/api/skills/google-ai-edge-on-device-verification/install","manifest_url":"https://www.openagentskill.com/api/registry/manifest/google-ai-edge-on-device-verification"},"trust":{"score":76,"label":"Strong shortlist","version":"trust-score-v4","install_policy":"human_review_before_install","evidence":{"stars":"416 GitHub stars","repoActivity":"416 stars, 116 forks","lastPushed":"3d since push","license":"Apache-2.0","repository":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification","install":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","installSafety":"standard package or runtime install path","permissionSurface":"secrets or environment access, filesystem or document access","documentation":"Strong README/SKILL.md context","agentOutcomes":"No agent outcome data yet"},"outcome_evidence":{"total":0,"successes":0,"failures":0,"not_relevant":0,"success_rate":null,"recent_success_rate":null,"recent_failure_rate":null,"install_attempts":0,"install_success_rate":null,"risk_blocked":0,"setup_required":0,"avg_output_quality":null,"production_outcomes":0,"last_outcome_at":null,"label":"No agent outcome data yet"},"auto_install":{"allowed":false,"sandbox_required":true,"reason":"Human review or sandbox validation is required before automatic installation."},"best_for":["research","agent-skill"],"known_risks":["Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access","Permission surface: secrets or environment access, filesystem or document access"]},"agent_proven":{"version":"agent-proven-v1","score":0,"tier":"unproven","label":"Needs first agent run","summary":"No agent outcome reports yet. Use Resolve, run one narrow sandbox task, then report the result.","metrics":{"totalOutcomes":0,"successfulOutcomes":0,"failedOutcomes":0,"installAttempts":0,"installSuccessRate":null,"successRate":null,"recentSuccessRate":null,"recentFailureRate":null,"riskBlocked":0,"setupRequired":0,"notRelevant":0,"avgOutputQuality":null,"avgTimeToUsefulMs":null,"productionOutcomes":0,"humanReviewRequired":0,"uniqueAgents":0,"lastOutcomeAt":null},"signals":[],"penalties":["No real agent outcome evidence yet"]},"audit":{"score":81,"risk_level":"needs_review","risk_label":"Needs review","warnings":["Permission surface may require sandboxing","Financial research output is not financial advice; require human review before any live investment decision","Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access","Permission surface: secrets or environment access, filesystem or document access"]},"safety_gate":{"tier":"experimental","label":"Experimental","auto_install_policy":"review","auto_install_allowed":false,"human_review_required":true,"blocked":false,"recommended_action":"Test manually in an isolated workspace and compare against safer alternatives."},"quality":{"score":73,"label":"Strong"},"supply":{"track":"Research and knowledge work","scenario":"Research agents","maintenance":"3d since push","risk":"Needs review"},"alternative_skills":[],"do_not_use_when":["teams that need a vendor-supported SLA","high-compliance environments without internal security review","No major risk signals from current metadata","High-risk permission hints: Secrets or environment access","Permission surface may require sandboxing","Financial research output is not financial advice; require human review before any live investment decision","Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review"],"agent_contract":{"task_input":"Use on-device-verification in an agent workflow","recommended_action":"Test manually in an isolated workspace and compare against safer alternatives.","install_policy":"review","minimum_review_before_use":["Trust: 76/100 Strong shortlist","Audit: 81/100 Needs review","Safety: 49/100 Avoid automatic install","Review repository, license, install command, and permission surface before production use."],"expected_agent_output":{"selected_skill":"google-ai-edge-on-device-verification (on-device-verification)","install_command":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","risk_summary":"Needs review; Experimental; Review before production","verification_result":"Report the smallest successful task, files touched, warnings, and any missing setup."}},"outcome_feedback":{"endpoint":"https://www.openagentskill.com/api/agent/outcome","method":"POST","requires_resolve_event_id":true,"event_id_source":"Use install_receipt.outcome_feedback.event_id or feedback.event_id returned by /api/agent/resolve for the current task.","expected_outcomes":["success","failed","not_relevant","blocked_by_risk","setup_required"],"payload_template":{"event_id":"<install_receipt.outcome_feedback.event_id or feedback.event_id from /api/agent/resolve>","skill_slug":"google-ai-edge-on-device-verification","task":"Use on-device-verification in an agent workflow","agent":"codex","outcome":"success","install_used":true,"risk_blocked":false,"setup_required":false,"task_success":true,"output_quality":4,"error_type":null,"human_review_required":false,"workspace":"sandbox","time_to_useful_ms":120000,"notes":"Report the smallest successful task, setup friction, files touched, and risk notes."}},"endpoints":{"web":"https://www.openagentskill.com/skills/google-ai-edge-on-device-verification","api":"https://www.openagentskill.com/api/agent/skills/google-ai-edge-on-device-verification","audit":"https://www.openagentskill.com/skills/google-ai-edge-on-device-verification/audit","eval":"https://www.openagentskill.com/api/agent/evals?slug=google-ai-edge-on-device-verification&task=Use%20on-device-verification%20in%20an%20agent%20workflow&max_risk=medium","resolve":"https://www.openagentskill.com/api/agent/resolve?task=Use%20on-device-verification%20in%20an%20agent%20workflow&agent=codex&max_risk=medium","receipt":"https://www.openagentskill.com/api/agent/receipt?task=Use%20on-device-verification%20in%20an%20agent%20workflow&agent=codex&max_risk=medium&format=text","install":"https://www.openagentskill.com/api/skills/google-ai-edge-on-device-verification/install","manifest":"https://www.openagentskill.com/api/registry/manifest/google-ai-edge-on-device-verification"}},"supply_profile":{"track":{"slug":"research","label":"Research and knowledge work","shortLabel":"Research","description":"Deep research, source comparison, literature review, RAG, knowledge search, and reports."},"scenario":{"label":"Research agents","description":"I need my agent to research a topic, compare sources, and produce a concise report.","useCases":[{"slug":"research-agents","title":"Research agents"},{"slug":"local-desktop","title":"Local desktop"},{"slug":"multimodal-media","title":"Multimodal media"}]},"applicableAgents":["Claude Code","CLI","Codex","Cursor"],"install":{"ready":true,"command":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","primaryTarget":"CLI","targetCount":4},"githubQuality":{"stars":416,"starsLabel":"416","forks":116,"license":"Apache-2.0","qualityScore":73,"trustScore":76,"auditScore":81},"maintenance":{"status":"fresh","label":"3d since push","daysSincePush":3,"lastPushedAt":"2026-09-03T21:51:36+00:00"},"risk":{"level":"needs_review","label":"Needs review","requiresReview":true,"notes":["Permission surface may require sandboxing","Financial research output is not financial advice; require human review before any live investment decision","Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access"]},"coverageTags":["Research","Research agents","agent-skill"]},"audit":{"audit_score":81,"risk_level":"needs_review","risk_label":"Needs review","quality_score":73,"trust_score":76,"maintenance_score":100,"security_score":80,"install_score":92,"warnings":["Permission surface may require sandboxing","Financial research output is not financial advice; require human review before any live investment decision","Financial research output is not financial advice; require human review before any live investment decision.","Quality score needs review","Permission surface needs review: secrets or environment access, filesystem or document access","Permission surface: secrets or environment access, filesystem or document access"]},"quality_signals":{"model":"v2","star_score":18.34,"usage_score":0,"review_score":5.55,"metadata_score":3,"freshness_score":15},"platforms":["Claude Code"],"use_cases":[{"slug":"research-agents","title":"Research agents","url":"https://www.openagentskill.com/use-cases/research-agents"},{"slug":"local-desktop","title":"Local desktop","url":"https://www.openagentskill.com/use-cases/local-desktop"},{"slug":"multimodal-media","title":"Multimodal media","url":"https://www.openagentskill.com/use-cases/multimodal-media"},{"slug":"rag-knowledge","title":"RAG and knowledge","url":"https://www.openagentskill.com/use-cases/rag-knowledge"}],"stacks":[{"slug":"research-report-agent","title":"Research report agent","url":"https://www.openagentskill.com/collections/research-report-agent"},{"slug":"rag-knowledge-base","title":"RAG knowledge base","url":"https://www.openagentskill.com/collections/rag-knowledge-base"},{"slug":"browser-qa-agent","title":"Browser QA agent","url":"https://www.openagentskill.com/collections/browser-qa-agent"}],"install":"npx skills add google-ai-edge/litert-samples --skill on-device-verification","install_targets":[{"id":"openagentskill-cli","label":"CLI","title":"OpenAgentSkill CLI","kind":"command","value":"npx --yes https://github.com/Leon-Drq/openagentskill/releases/download/cli-v0.3.0/openagentskill-0.3.0.tgz add google-ai-edge-on-device-verification","description":"Resolve policy, run the source installer safely, and report a verified install receipt.","copyLabel":"Copy command"},{"id":"codex","label":"Codex","title":"Codex install prompt","kind":"agent-prompt","value":"Install the \"on-device-verification\" agent skill from https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification. Read its SKILL.md or equivalent instructions first, install only the files needed for this workspace, and summarize any required setup before using it. Skill purpose: Prove a converted or quantized LiteRT model on the actual device via the CompiledModel API - confirm GPU residency, compare device output against the source model, and diagnose device-only failures such as silent CPU fallback, whole-graph compile ceilings, and fp16 range breaks. Use after conversion or quantization, when device output is wrong or NaN, when a clean graph fails to compile only on device, or when GPU and CPU outputs are suspiciously identical. After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"google-ai-edge-on-device-verification\",\"task\":\"Install on-device-verification\",\"agent\":\"codex\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes.","description":"Give Codex a repo-aware install prompt when the skill is not available through a local CLI.","copyLabel":"Copy prompt"},{"id":"claude-code","label":"Claude Code","title":"Claude Code skill prompt","kind":"agent-prompt","value":"Add \"on-device-verification\" as a Claude Code skill from https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification. Inspect the skill instructions, place the reusable skill files in the appropriate local skills location for this project, and report the activation steps. Skill purpose: Prove a converted or quantized LiteRT model on the actual device via the CompiledModel API - confirm GPU residency, compare device output against the source model, and diagnose device-only failures such as silent CPU fallback, whole-graph compile ceilings, and fp16 range breaks. Use after conversion or quantization, when device output is wrong or NaN, when a clean graph fails to compile only on device, or when GPU and CPU outputs are suspiciously identical. After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"google-ai-edge-on-device-verification\",\"task\":\"Install on-device-verification\",\"agent\":\"claude-code\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes.","description":"Use this prompt to ask Claude Code to add the skill and explain the local activation steps.","copyLabel":"Copy prompt"},{"id":"cursor","label":"Cursor","title":"Cursor rule prompt","kind":"agent-prompt","value":"Turn \"on-device-verification\" from https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification into a reusable Cursor project rule or agent instruction. Preserve the core workflow, adapt paths to this repo, and keep the rule scoped to tasks where it is relevant. Skill purpose: Prove a converted or quantized LiteRT model on the actual device via the CompiledModel API - confirm GPU residency, compare device output against the source model, and diagnose device-only failures such as silent CPU fallback, whole-graph compile ceilings, and fp16 range breaks. Use after conversion or quantization, when device output is wrong or NaN, when a clean graph fails to compile only on device, or when GPU and CPU outputs are suspiciously identical. After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"google-ai-edge-on-device-verification\",\"task\":\"Install on-device-verification\",\"agent\":\"cursor\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes.","description":"Use this when installing as Cursor project rules or reusable agent instructions.","copyLabel":"Copy prompt"}],"repository":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification","github_repo":"google-ai-edge/litert-samples","version":"1.0.0","license":"Apache-2.0","urls":{"web":"https://www.openagentskill.com/skills/google-ai-edge-on-device-verification","repository":"https://github.com/google-ai-edge/litert-samples/tree/main/skills/on-device-verification","api":"/api/agent/skills/google-ai-edge-on-device-verification","install_api":"/api/skills/google-ai-edge-on-device-verification/install"},"meta":{"created_at":"2026-08-26T20:36:31.384588+00:00","updated_at":"2026-09-04T03:47:54.419714+00:00","agent_friendly":true}}