data-storage
This skill should be used when the user asks "SQL or NoSQL", "which database", how to design a "data model" or "schema design", picks an "indexing" strategy, needs "sharding" or "partitioning", sets up "replication" (leader-follower / multi-leader), defines a "primary key"/"sort key", asks whether to "denormalize", or weighs "polyglot persistence". Use it whenever a design must decide where records live, how they are keyed and accessed, and how the store scales past one node — even if the user just says "store the data".
proyecto26 · 已有静态检查记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add proyecto26/system-design-skills --skill data-storage技能说明
name: data-storage description: This skill should be used when the user asks "SQL or NoSQL", "which database", how to design a "data model" or "schema design", picks an "indexing" strategy, needs "sharding" or "partitioning", sets up "replication" (leader-follower / multi-leader), defines a "primary key"/"sort key", asks whether to "denormalize", or weighs "polyglot persistence". Use it whenever a design must decide where records live, how they are keyed and accessed, and how the store scales past one node — even if the user just says "store the data".
Data Storage
Choose where records live, how they are keyed and queried, and how the store grows past a single machine. Storage is the hardest layer to change later: a wrong data model or shard key calcifies into a scaling ceiling, and getting replication wrong silently serves stale or lost data.
When to reach for this
Any system that persists state: picking SQL vs NoSQL, designing a schema and its access paths, adding indexes, splitting a hot table, distributing data across nodes (sharding/partitioning), adding read replicas, or deciding what to denormalize. Reach here the moment "store the data" needs a concrete key and query shape.
When NOT to
Don't shard, add replicas, or reach for NoSQL before a number forces it (YAGNI).
A single well-indexed relational node handles ~1k QPS and tens of GB to low TB
comfortably — most systems never outgrow it. Sharding multiplies operational
cost and breaks joins/transactions; add it only when one node's write throughput
or dataset size is genuinely exceeded (→ back-of-the-envelope). Caching reads
(→ caching) and adding read replicas are cheaper first moves than sharding.
Clarify first
Answer these before choosing a store or topology — they decide the design:
- Data shape & relationships — flat key-value? rich relations needing joins? document blobs? a graph of connections? Drives SQL vs NoSQL.
- Access patterns — how is data read and written, not just where it lives. Point lookups by key, range scans, ad-hoc queries, aggregations? Model the store around the queries it must serve.
- Read:write ratio & scale — QPS each way, total size now and at retention.
(→
back-of-the-envelopefor QPS, storage, and shard counts.) - Consistency need — must reads see the latest write, or is eventual OK? Are
multi-record transactions required? (CAP/consistency theory →
consistency-coordination.) - Latency & durability targets — p99 read/write budget, and how much recent data the system can afford to lose on a node failure.
The options
Relational (SQL — Postgres, MySQL): strict schema, joins, ACID transactions. Use when data is relational, integrity matters, and queries are varied/ad-hoc — the safe default until a number rules it out.
Document (MongoDB, etc.): flexible schema, self-contained JSON-ish documents queried by structure. Use when records are read/written as a whole and the schema evolves; relationships are few.
Key-value (Redis, DynamoDB, Riak): O(1) get/put by key, no rich queries. Use when access is purely by a known key and massive throughput is needed.
Wide-column (Cassandra, Bigtable, HBase): rows keyed by partition, columns sparse, keys kept sorted for range scans. Use when writes are huge and the table can be designed around a few known query patterns.
Graph (Neo4j): nodes and edges. Use when the core queries traverse many-to-many relationships (social graph, recommendations).
Scaling moves (apply on top of any store):
- Indexing — add a secondary structure so a query stops scanning. First lever.
- Read replicas (leader-follower) — copy writes to followers that serve reads. Use when reads dominate and slight staleness is OK.
- Federation — split DBs by function (users / products / forums). Use when functional domains scale independently and rarely join.
- Sharding/partitioning — split one logical table across nodes by a shard key. Use when a single node's writes or dataset are exceeded. (This skill owns it.)
- Denormalization — store redundant copies to skip joins. Use when reads vastly outnumber writes and joins are the bottleneck.
Polyglot persistence: use more than one of the above, each for what it's best at (e.g. Postgres for orders, Redis for sessions, a search index for full-text). The cost is operating and reconciling several stores.
Trade-offs
| Option | What it solves | What it worsens | Change it when |
|---|---|---|---|
| Relational/SQL | Joins, ACID, ad-hoc queries, integrity | Single-node write ceiling; schema migrations; harder horizontal scale | Writes/size exceed one node, or schema is truly fluid → NoSQL/shard |
| Document | Schema flexibility; whole-object reads | No joins; cross-document consistency is manual; query engine weaker | Data turns relational or needs multi-doc transactions → SQL |
| Key-value | Extreme throughput, simple ops | Only key access; no range/secondary queries | Queries beyond the key are needed → document/wide-column |
| Wide-column | Write-heavy scale, range scans on sorted keys | Must know queries up front; rigid once keyed; eventual by default | Access patterns are unknown/varied → relational |
| Graph | Cheap deep relationship traversal | Niche tooling; hard to shard; weak for bulk scans | Relationships are shallow → relational/document |
| Indexing | Turns scans into lookups | Slower writes; more storage; index bloat | Write amplification hurts more than the read win |
| Read replicas | Offloads reads; redundancy | Replication lag → stale reads; failover/promotion logic | Stale reads unacceptable → read-from-leader / consistency-coordination |
| Federation | Per-domain scale, smaller working sets | Cross-domain joins break; app routing logic | A single domain still won't fit → shard that domain |
| Sharding | Horizontal write + storage scale | Cross-shard joins/txns hard; resharding pain; hot shards |