跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
数据与分析

data-storage

This skill should be used when the user asks "SQL or NoSQL", "which database", how to design a "data model" or "schema design", picks an "indexing" strategy, needs "sharding" or "partitioning", sets up "replication" (leader-follower / multi-leader), defines a "primary key"/"sort key", asks whether to "denormalize", or weighs "polyglot persistence". Use it whenever a design must decide where records live, how they are keyed and accessed, and how the store scales past one node — even if the user just says "store the data".

proyecto26 · 已有静态检查记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add proyecto26/system-design-skills --skill data-storage

技能说明


name: data-storage description: This skill should be used when the user asks "SQL or NoSQL", "which database", how to design a "data model" or "schema design", picks an "indexing" strategy, needs "sharding" or "partitioning", sets up "replication" (leader-follower / multi-leader), defines a "primary key"/"sort key", asks whether to "denormalize", or weighs "polyglot persistence". Use it whenever a design must decide where records live, how they are keyed and accessed, and how the store scales past one node — even if the user just says "store the data".

Data Storage

Choose where records live, how they are keyed and queried, and how the store grows past a single machine. Storage is the hardest layer to change later: a wrong data model or shard key calcifies into a scaling ceiling, and getting replication wrong silently serves stale or lost data.

When to reach for this

Any system that persists state: picking SQL vs NoSQL, designing a schema and its access paths, adding indexes, splitting a hot table, distributing data across nodes (sharding/partitioning), adding read replicas, or deciding what to denormalize. Reach here the moment "store the data" needs a concrete key and query shape.

When NOT to

Don't shard, add replicas, or reach for NoSQL before a number forces it (YAGNI). A single well-indexed relational node handles ~1k QPS and tens of GB to low TB comfortably — most systems never outgrow it. Sharding multiplies operational cost and breaks joins/transactions; add it only when one node's write throughput or dataset size is genuinely exceeded (→ back-of-the-envelope). Caching reads (→ caching) and adding read replicas are cheaper first moves than sharding.

Clarify first

Answer these before choosing a store or topology — they decide the design:

  • Data shape & relationships — flat key-value? rich relations needing joins? document blobs? a graph of connections? Drives SQL vs NoSQL.
  • Access patterns — how is data read and written, not just where it lives. Point lookups by key, range scans, ad-hoc queries, aggregations? Model the store around the queries it must serve.
  • Read:write ratio & scale — QPS each way, total size now and at retention. (→ back-of-the-envelope for QPS, storage, and shard counts.)
  • Consistency need — must reads see the latest write, or is eventual OK? Are multi-record transactions required? (CAP/consistency theory → consistency-coordination.)
  • Latency & durability targets — p99 read/write budget, and how much recent data the system can afford to lose on a node failure.

The options

Relational (SQL — Postgres, MySQL): strict schema, joins, ACID transactions. Use when data is relational, integrity matters, and queries are varied/ad-hoc — the safe default until a number rules it out.

Document (MongoDB, etc.): flexible schema, self-contained JSON-ish documents queried by structure. Use when records are read/written as a whole and the schema evolves; relationships are few.

Key-value (Redis, DynamoDB, Riak): O(1) get/put by key, no rich queries. Use when access is purely by a known key and massive throughput is needed.

Wide-column (Cassandra, Bigtable, HBase): rows keyed by partition, columns sparse, keys kept sorted for range scans. Use when writes are huge and the table can be designed around a few known query patterns.

Graph (Neo4j): nodes and edges. Use when the core queries traverse many-to-many relationships (social graph, recommendations).

Scaling moves (apply on top of any store):

  • Indexing — add a secondary structure so a query stops scanning. First lever.
  • Read replicas (leader-follower) — copy writes to followers that serve reads. Use when reads dominate and slight staleness is OK.
  • Federation — split DBs by function (users / products / forums). Use when functional domains scale independently and rarely join.
  • Sharding/partitioning — split one logical table across nodes by a shard key. Use when a single node's writes or dataset are exceeded. (This skill owns it.)
  • Denormalization — store redundant copies to skip joins. Use when reads vastly outnumber writes and joins are the bottleneck.

Polyglot persistence: use more than one of the above, each for what it's best at (e.g. Postgres for orders, Redis for sessions, a search index for full-text). The cost is operating and reconciling several stores.

Trade-offs

OptionWhat it solvesWhat it worsensChange it when
Relational/SQLJoins, ACID, ad-hoc queries, integritySingle-node write ceiling; schema migrations; harder horizontal scaleWrites/size exceed one node, or schema is truly fluid → NoSQL/shard
DocumentSchema flexibility; whole-object readsNo joins; cross-document consistency is manual; query engine weakerData turns relational or needs multi-doc transactions → SQL
Key-valueExtreme throughput, simple opsOnly key access; no range/secondary queriesQueries beyond the key are needed → document/wide-column
Wide-columnWrite-heavy scale, range scans on sorted keysMust know queries up front; rigid once keyed; eventual by defaultAccess patterns are unknown/varied → relational
GraphCheap deep relationship traversalNiche tooling; hard to shard; weak for bulk scansRelationships are shallow → relational/document
IndexingTurns scans into lookupsSlower writes; more storage; index bloatWrite amplification hurts more than the read win
Read replicasOffloads reads; redundancyReplication lag → stale reads; failover/promotion logicStale reads unacceptable → read-from-leader / consistency-coordination
FederationPer-domain scale, smaller working setsCross-domain joins break; app routing logicA single domain still won't fit → shard that domain
ShardingHorizontal write + storage scaleCross-shard joins/txns hard; resharding pain; hot shards
阅读完整来源 ↗