complete-ai-agent-stack-deployment-self-hosted-from-scratch
Sequences a complete, end-to-end, fully self-hosted AI agent stack deployment from scratch — GPU procurement/sizing for open-weight model serving, self-hosted LLM serving (vLLM/TGI), agent control-flow architecture, a self-hosted vector database for RAG, self-hosted MCP servers for tool access, and an evaluation/guardrails harness — with no managed LLM API or managed vector database anywhere in the stack. An integration/orchestration skill that sequences existing tool-specific skills in the right order and flags handoff points, explicit about the added GPU-procurement and operational burden versus a cloud-managed agent stack. Use when a user asks to "build a self-hosted AI agent stack with open-weight models," "run our agent on our own GPUs with no managed LLM API," "stand up a self-hosted vector database and MCP servers for an agent platform," or "give me the end-to-end sequence for a fully self-hosted agent deployment from GPU procurement to production."
selvarajmurugesan90 · 已有静态检查记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add selvarajmurugesan90/ops-engineering-skills --skill complete-ai-agent-stack-deployment-self-hosted-from-scratch技能说明
name: complete-ai-agent-stack-deployment-self-hosted-from-scratch description: > Sequences a complete, end-to-end, fully self-hosted AI agent stack deployment from scratch — GPU procurement/sizing for open-weight model serving, self-hosted LLM serving (vLLM/TGI), agent control-flow architecture, a self-hosted vector database for RAG, self-hosted MCP servers for tool access, and an evaluation/guardrails harness — with no managed LLM API or managed vector database anywhere in the stack. An integration/orchestration skill that sequences existing tool-specific skills in the right order and flags handoff points, explicit about the added GPU-procurement and operational burden versus a cloud-managed agent stack. Use when a user asks to "build a self-hosted AI agent stack with open-weight models," "run our agent on our own GPUs with no managed LLM API," "stand up a self-hosted vector database and MCP servers for an agent platform," or "give me the end-to-end sequence for a fully self-hosted agent deployment from GPU procurement to production." license: Apache-2.0 compatibility: "Claude Code, GitHub Copilot, OpenAI Codex, Cursor, Gemini CLI" metadata: domain: ai-agent maturity: stable
Complete AI Agent Stack Deployment (Self-Hosted) From Scratch
Purpose
The cloud-managed AI agent path in this skill family leans on managed LLM provider APIs and a managed vector database, trading infrastructure ownership for per-token pricing and someone else's on-call rotation. This skill is the opposite path: serving an open-weight model on GPU infrastructure the team itself procures and operates, paired with a self-hosted vector database and self-hosted MCP servers, with no managed LLM API or managed vector database anywhere in the stack. The tradeoff is real, and the sequencing risk is sharper than on the managed path: GPU capacity has to be sized and provisioned before the agent's latency budget is even meaningfully designable (unlike a managed API, where provider-side scaling is someone else's problem), and every durability concern a managed vector database absorbs — replication, backup, upgrade — becomes this team's responsibility from day one. This skill sequences that whole path — GPU procurement through evaluation — and is explicit throughout about where the self-hosted burden actually lands.
When to use
- Standing up a production AI agent with a hard requirement of no managed LLM API or managed vector database — data residency, air-gapped deployment, fixed-cost GPU amortization, or model-customization reasons all commonly drive this.
- Deciding whether a team genuinely has the GPU procurement and operational capacity to self-host an agent stack, versus one of the cloud-managed alternatives in this skill family.
- Auditing an existing self-hosted agent deployment for a skipped or out-of-order phase (e.g. an agent's latency budget designed before real serving latency was measured on actual hardware, or a self-hosted vector database with no replication running in production for months).
- Rebuilding a reference self-hosted agent architecture for a second team or environment that should follow the same proven sequence as a known-good first deployment.
- Honestly comparing the total cost and operational burden of this path against the cloud-managed alternative before committing to it.
Prerequisites & environment
- GPU infrastructure already provisioned or provisionable — on Kubernetes, this means the NVIDIA GPU Operator and dedicated serving-shaped GPU node pools per gpu-accelerator-infrastructure-for-ml-training (that skill's title says "for ML training" but its GPU Operator/MIG/ node-pool guidance applies identically to inference-serving GPU capacity). Whether this capacity is on-prem, colocated, or cloud-rented-as-raw-compute, this team owns its procurement lead time and scaling — there is no managed API absorbing a traffic spike on its own.
- A self-hosting-capable serving runtime (vLLM or TGI) and the open-weight model checkpoint(s) already selected and downloaded, with a plan for where model weights are versioned and stored (not just "a directory on the serving node").
- A self-hosted vector database deployment target (Weaviate or Milvus, self-managed on Kubernetes) and its own dedicated compute/storage — distinct from the GPU serving nodes, since vector search is typically CPU/memory-bound, not GPU-bound.
kubectl/helmif deploying on Kubernetes, and a realistic estimate of expected concurrent request volume and sequence length before sizing either the GPU serving fleet or the vector database cluster — sizing either without real numbers produces guesses that fail under real load.- A decision, made deliberately and with realistic staffing in mind, about whether this team can actually operate GPU capacity planning, model serving upgrades, and vector database durability long-term — see the honest tradeoff called out in Common pitfalls.
Step-by-step guidance
This is the phase sequence. Each phase links to the skill that covers its full depth; the text here covers only the self-hosted-specific sequencing and the operational burden each phase adds versus a managed alternative.
- Phase 1 — GPU infrastructure procurement and sizing. Before
anything else, size and provision the GPU capacity this stack will
run on, per
gpu-accelerator-infrastructure-for-ml-training:
install the NVIDIA GPU Operator, and design a dedicated serving GPU
node pool (separate from any training capacity that may share the
cluster) sized to the model's memory footprint plus KV-cache headroom
at expected concurrency:
helm install gpu-operator nvidia/gpu-operator \ --namespace gpu-operator --create-name