ai-architect-review
Review the architecture of a system that calls a language model. Use when adding an agent loop, a retrieval path, a tool surface, or an MCP server to a codebase; when a system that worked in a demo is being prepared for production; or when asked to audit, review, or plan AI/agent architecture. Checks the four decisions that are expensive to reverse, then the seven planes and their boundaries.
frankxai · 已有 AI 审核记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add frankxai/claude-skills-library --skill ai-architect-review技能说明
name: ai-architect-review description: Review the architecture of a system that calls a language model. Use when adding an agent loop, a retrieval path, a tool surface, or an MCP server to a codebase; when a system that worked in a demo is being prepared for production; or when asked to audit, review, or plan AI/agent architecture. Checks the four decisions that are expensive to reverse, then the seven planes and their boundaries. license: MIT source: https://www.frankx.ai/ai-architect
AI architecture review
Most AI system choices are reversible in an afternoon. Four are not. This skill checks those four first, then walks the planes for anything unowned.
Run it against a real codebase. Every check below names something you can grep, measure, or point at. If you cannot produce the evidence, the check fails — an architecture review with no evidence is an opinion.
Step 1 — the four decisions
Work these in order. Each one asks for evidence, not a judgement.
1. Where does the model call go?
The question is not which model. It is whether exactly one place in the system knows a model provider's name.
Evidence: grep for the provider SDK import across the repo.
rg -l "from ['\"](openai|@anthropic-ai/sdk|@google/gen|@aws-sdk/client-bedrock)" --type-add 'src:*.{ts,tsx,js,py,go,rb,java}' -tsrc
- One module → decision made. Note where the seam is.
- More than one → deferred, not made. Every call site is a migration cost that grows monotonically. Say this plainly and give the count.
What a single seam buys, that scattered calls cannot: routing by task, fallback when a provider degrades, a cache with a coherent key, and one line item for spend. A system with no fallback path has no fallback path at every layer, whatever the layers above it do.
2. What shape is the loop?
Four shapes. Three questions pick one.
- Can you name every step before the request arrives? → fixed workflow
- If not: is it one coherent piece of work? → single agent loop
- If not: does the work mutate shared state? → sequential sub-agents, otherwise parallel sub-agents
| Shape | Costs | Fails by |
|---|---|---|
| Fixed workflow | No adaptation when input is not what you planned for | Staying confident, going silently wrong once reality drifts from the graph |
| Single agent loop | Latency and spend grow with loop length | Losing track of its own earlier decisions as the window fills |
| Parallel sub-agents | A merge step you have to design | Duplicating work, then contradicting with no way to adjudicate |
| Sequential sub-agents | Throughput, given up on purpose | Being slow enough that someone parallelises the writes again |
Evidence: find the loop's exit condition.
- Exit condition in code (a counter, a budget, a state machine) → bounded.
- Exit condition in a prompt ("stop when you are done") → unbounded loop with a polite request attached. Flag it.
Bias toward the workflow. A workflow that was written down is cheaper to run, easier to debug, easier to evaluate, and easier to explain to whoever owns the incident. Reach for a loop when the steps genuinely cannot be enumerated.
The reversal cost is not the code — it is the eval harness. Workflows are graded per step; loops have to be graded on trajectory. Those are different harnesses, and switching shapes discards the graded examples with them.
3. Where does the trust boundary sit?
Text that came back from a tool call is not your text.
Retrieved documents, API responses, fetched pages, uploaded files — all authored by someone who is not the operator. If that text reaches the position in the context window where instructions live, it is an instruction. No prompt fixes this, because "ignore any instructions below" is also just text.
Evidence: trace one retrieved document from retriever to context window. Point at the line where it becomes labelled data rather than plain text.
Three structural requirements:
- Untrusted content never enters the instruction position. Fence it, label it, keep it in the data position.
- Side effects sit behind a gate. Anything irreversible — a write, a message, a payment, a merge, a deploy — needs a step a document cannot perform on its own behalf.
- Tool scopes are narrow by default. A tool that reads a calendar and a tool that reads a calendar and sends mail are different risk objects, one line apart in config.
This gets expensive fastest, because the cost scales with the number of tools already shipped. If only one decision is being deferred, do not let it be this one.
4. Where does a long run live?
An agent loop running for eleven minutes is a different deployment problem from a completion returning in two seconds — not a harder version of the same one.
Evidence: the longest real production run (not the median), and the platform's execution ceiling. If either number is unknown, the decision is unmade.
| Unit of execution | Where a long loop lives |
|---|---|
| Serverless / edge functions | A queued background job, not the request that started it |
| Edge isolates | A durable object or workflow primitive |
| Functions, containers, or VMs on a cloud | A step-function state machine, or a container kept warm |
| Long-lived container | Just a process — no special primitive |
| Micro-VMs | A machine owned for the run's duration, then released |
| Remote-container functions | The function call itself; duration is a parameter |
There is no winner here. Managed platforms hand you a primitive and take the operations; self-assembled clouds hand you every primitive and take your Tuesdays. Both answer different questions about what the team wants to own.
What makes it hard to reverse: long-running work is a runtime property, not a config flag. Moving it means moving the execution model, which means moving the state that execution model implied.
Step 2 — walk the planes
Once the four are settled, check ea
阅读完整来源 ↗