跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
数据与分析

designing-distributed-system-tests

Design claim-driven test plans for distributed or stateful systems with persistence, replication, consensus, retries, idempotency, async messaging, multi-tenancy, or partial failure. Supports change-scoped and project-wide plans, including existing-test inventories and gap analysis. Use for stability, release-validation, durability, partition, crash-recovery, upgrade, linearizability, deterministic-simulation, SQL semantics or compatibility, tenant isolation, authz, namespace boundaries, fairness, and noisy-neighbor testing. Investigate claimed guarantees and design scenarios to falsify them during normal operation and under relevant faults. Also use for "what should we test for this change", "are my tenants isolated", or "make a holistic test plan".

shenli · 已有静态检查记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add shenli/distributed-system-testing --skill designing-distributed-system-tests

技能说明


name: designing-distributed-system-tests description: >- Design claim-driven test plans for distributed or stateful systems with persistence, replication, consensus, retries, idempotency, async messaging, multi-tenancy, or partial failure. Supports change-scoped and project-wide plans, including existing-test inventories and gap analysis. Use for stability, release-validation, durability, partition, crash-recovery, upgrade, linearizability, deterministic-simulation, SQL semantics or compatibility, tenant isolation, authz, namespace boundaries, fairness, and noisy-neighbor testing. Investigate claimed guarantees and design scenarios to falsify them during normal operation and under relevant faults. Also use for "what should we test for this change", "are my tenants isolated", or "make a holistic test plan".

Designing Distributed-System Tests

The default for testing distributed and stateful systems — write a few integration tests and call it done — finds a small fraction of the bugs that actually break these systems in production. This skill enforces an opinionated workflow: scope the change, generate failure-mode hypotheses that cover the categories the literature says matter most, pick techniques from a curated catalog, and emit a structured plan file that the executing-distributed-system-tests skill (or a human) can run.

Plan modes

This skill produces two shapes of plan. Decide which one applies before you start; the steps below branch on it.

  • Change-scoped — the default. Use when the caller names a commit, PR, branch-diff, or feature. The plan covers what this change could regress, scoped by its blast radius.
  • Project-wide — use when the caller asks for a "release-validation plan", "stability plan for the whole system", "test plan to enough coverage", "what should we be testing", or otherwise frames the request without a specific change. The plan covers what the system should be tested for, with an explicit inventory of existing tests and a gap analysis driving the new-scenario list.

If the framing is ambiguous, ask once before starting — the modes diverge enough that retrofitting one into the other wastes work.

Process

Follow these steps in order. Do not skip; the order matters because later steps depend on artifacts the earlier steps produce.

1. Scope the system

Read the project's entry points: README, AGENTS.md or CLAUDE.md, top-level docs/, any existing test-plan or runbook files. Note:

  • Tenancy / isolation model
  • Persistence model (what is durable, fsync contract)
  • Replication / consensus protocol, quorum, leadership
  • Ordering guarantee exposed to clients
  • Network boundaries (which RPCs / streams)
  • Retry / idempotency contract
  • Observability (logs, metrics, traces) available to an oracle

Write this as a one-paragraph SUT model. If anything is ambiguous from the repo, ask the user before proceeding — do not invent guarantees.

1b. Extract claims and guarantees

A good test plan exists to falsify what the product claims. Before generating hypotheses, write down what the SUT promises its users. This is the spine the rest of the plan hangs off — every hypothesis, every scenario, every oracle should be traceable back to a claim it either confirms or refutes.

Sources to mine:

  • README "guarantees" / "what we offer" sections
  • API docs / reference manuals
  • ARCHITECTURE / DESIGN docs (claims about consistency, durability, replication, fault tolerance)
  • Public blog posts, talks, marketing material (if any)
  • The code itself: function names, doc-comments on public APIs, error types (IdempotencyConflict, StaleRead, etc.) imply guarantees the system claims to enforce
  • Existing test names (a test called linearizable_under_partition implies a linearizability claim under partition)

Categorise each claim:

  • Safety — "the system never returns a stale read", "no acknowledged write is ever lost", "linearizable per key"
  • Liveness — "every accepted operation eventually commits", "leader election completes within N seconds of crash"
  • Durability — "fsync'd writes survive crash", "replicated writes survive single-AZ loss"
  • Performance / SLO — "p99 append latency ≤ X ms at Y ops/s per session"
  • Operational — "rolling upgrade is non-disruptive", "configuration changes are atomic"
  • Idempotency / dedup — "same idempotency key never produces two committed effects"
  • Isolation — "tenant A's reads never observe tenant B's writes", "no read returns data from a transaction that has not yet committed"
  • Ordering — "consumers always see messages in the order the producer sent them", "every reader sees a prefix of the global log order"
  • Membership — "a node that fails its liveness probe is removed from the cluster membership view within N seconds", "every joined member appears in the membership table exactly once"
  • Boundary — access-boundary semantics: "tenant A's data is never reachable from tenant B on any surface", "a request scoped to namespace X never routes to namespace Y". Subsumes tenancy / authz / namespace / routing / multi-protocol; do not file those as separate categories. Triggers the §7.M.S surface-decomposition discipline (see step 3 and references/boundary-and-isolation-testing.md).
  • Fairness — per-group performance and noisy-neighbor isolation: "no tenant can starve another for throughput", "one shard's load does not blow another shard's p99". Group can be tenant, shard, queue, partition, region, priority class, user, table, or workload class. Also triggers §7.M.S.

If the project does NOT explicitly document a claim that appears in the code, write it as an inferred claim and mark it as such — inferred claims are still testable, and surfacing them often catches places where the docs lie or are silent about real guarantees the implementation depends on.

Bind safety-critical claims to a version and config

阅读完整来源 ↗