local-ai-app-integration
Integrates local AI capabilities into applications using Embeddable Lemonade. Use when the user wants to add local AI, offline AI, private AI, on-device AI, a local LLM, local chat, embeddings, image generation, speech-to-text, or text-to-speech to an existing app; replace or supplement OpenAI, Anthropic, Ollama, or other cloud AI APIs with a local backend; only use to convert user apps. Do not use when the user just wants the agent itself to generate images, transcribe, or speak locally in the current workspace, even to cut their own API bill.
amd · 已有 AI 审核记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add amd/skills --skill local-ai-app-integration技能说明
name: local-ai-app-integration description: >- Integrates local AI capabilities into applications using Embeddable Lemonade. Use when the user wants to add local AI, offline AI, private AI, on-device AI, a local LLM, local chat, embeddings, image generation, speech-to-text, or text-to-speech to an existing app; replace or supplement OpenAI, Anthropic, Ollama, or other cloud AI APIs with a local backend; only use to convert user apps. Do not use when the user just wants the agent itself to generate images, transcribe, or speak locally in the current workspace, even to cut their own API bill.
Local AI App Integration (Embeddable Lemonade)
Add a local AI mode to an existing app that already talks to a cloud AI API
(OpenAI, Anthropic, or Ollama-compatible). The app launches lemond, the
Embeddable Lemonade binary, as a private subprocess and the existing client
talks to it on http://localhost:PORT/api/v1. The user gets local, private,
hardware-optimized inference (CPU, AMD iGPU/dGPU, XDNA2 NPU) with no separate
install.
What you'll end up with: one new launcher module (~30 lines), three mandatory changes to the existing HTTP client (base_url, api_key, and a 120-second HTTP timeout), one vendored binary under vendor/lemonade/.
When this skill is the right tool
Use this skill when all of the following are true:
- The app already calls a cloud AI service over HTTP (OpenAI Chat Completions, Anthropic Messages, or Ollama).
- The user wants that AI to run on the end-user's PC, with the AI engine bundled into the app, not as a separate user install.
- The target platform is Windows x64 or Linux x64 (macOS embeddable is in beta).
If the user instead wants a system-wide Lemonade Server (one install,
shared across apps), do not use this skill; point them at
https://lemonade-server.ai/install_options.html and the standard OpenAI base
URL http://localhost:13305/api/v1.
The opinionated path
This skill follows one fixed sequence. Do not deviate without a stated reason.
[ ] 1. Survey the app's current AI integration
[ ] 2. Pick a model + backend profile
[ ] 3. Place Embeddable Lemonade in the app's tree (full package, not just the binary)
[ ] 4. Add a `lemond` launcher (subprocess + API key + port + per-stage logging)
[ ] 5. Re-point the existing client at lemond (base_url, api_key, 120s timeout — all three required)
[ ] 6. Wait for /api/v1/health, install backend, then PULL the model before first use
[ ] 7. Wire shutdown and error recovery
Track progress against this checklist. Move on only when each step verifies.
Log every stage. A local integration has many silent failure points — spawn, health, backend install, model download, first inference. Without a log line at each transition, "nothing happened" is indistinguishable from "broke at stage 3." Emit one clear line per stage as you build (see Step 4); the most common dead-end in this integration — a blank result with no error — is invisible without them.
Step 1: Survey the app
Find every place the app currently calls a cloud AI API. Search the repo for:
openai,OpenAI(,chat.completions,responses.createanthropic,Anthropic(,messages.createapi.openai.com,api.anthropic.com,localhost:11434(Ollama)OPENAI_API_KEY,ANTHROPIC_API_KEY
Record three things before continuing:
- Client library and language (e.g.,
openai-python,openai-node,@anthropic-ai/sdk,go-openai, rawfetch). - Modalities used: text chat, tool calling, embeddings, image gen, transcription, TTS. This drives the model + backend choice in Step 2.
- One single place where the base URL and API key are constructed. If there isn't one, refactor to one before going further. Local-mode toggling must flip exactly one config object.
- Any API-key gating that blocks the app before a key is entered (onboarding walls, validators that reject empty keys, startup checks that disable AI until a key exists). Note each one — Step 5 bypasses them in local mode.
Step 2: Pick a model + backend profile
Choose one default profile based on the app's primary modality. Do not ship a buffet. Ship one good default and document how the user can override it.
| App's primary need | Default model | Recipe | Why |
|---|---|---|---|
| General chat / assistant | Qwen3-4B-GGUF | llamacpp | Small, fast, good tool calling, fits 8GB systems |
| Coding assistant | Qwen2.5-Coder-7B-Instruct-GGUF | llamacpp | Strong code, runs on iGPU |
| Vision / multimodal chat | Gemma-4-E2B-it-GGUF | llamacpp | Small multimodal default |
| NPU-first on Ryzen AI | Llama-3.2-3B-Instruct-Hybrid | ryzenai-llm | XDNA2 NPU on Windows |
| Speech-to-text (Windows) | Whisper-Large-v3-Turbo | whispercpp | One model; probe picks NPU → iGPU/dGPU → CPU automatically |
| Speech-to-text (Linux NPU) | whisper-v3-turbo-FLM | flm | Linux NPU path; falls back to whispercpp iGPU/CPU off-NPU |
| Text-to-speech | kokoro-v1 | kokoro | CPU-only, low latency |
| Image generation | SDXL-Turbo | sd-cpp | Single-step generation |
For the LLM backend, default to llamacpp and let lemond pick
rocm → vulkan → cpu automatically by leaving llamacpp_backend
unset. Override only if the app has hard hardware requirements.
Scope: this skill selects a backend once at integration time on the
developer's machine. Runtime fallback based on the end user's hardware is
out of scope. Bundle vulkan as the universal fallback so the app works on
any machine. If the dev machine has an NPU and the chosen recipe supports it,
the skill will use the NPU backend — otherwise it falls back to vulkan.
阅读完整来源 ↗Note: having an NPU does not mean every recipe supports NPU. Confirm the recipe/backend pair is
installedorinstallableviaGET /api/v1/system-infobefore committing to it. See [reference.md](re