using-dbt-state
Use when a user is enabling, configuring, optimizing, or debugging dbt State (the server-backed reuse mechanism that clones or skips nodes instead of rebuilding them). Use when they conflate dbt State with the `state:modified` selector or `--state` deferral. Use when asked about models rebuilding unexpectedly, views with `select *` rebuilding, volatile SQL (`current_timestamp()`, `random()`) rebuilding or not, cross-developer cloning, lag_tolerance.
dbt-labs · 已有 AI 审核记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add dbt-labs/dbt-agent-skills --skill using-dbt-state技能说明
name: using-dbt-state
description: Use when a user is enabling, configuring, optimizing, or debugging dbt State (the server-backed reuse mechanism that clones or skips nodes instead of rebuilding them). Use when they conflate dbt State with the state:modified selector or --state deferral. Use when asked about models rebuilding unexpectedly, views with select * rebuilding, volatile SQL (current_timestamp(), random()) rebuilding or not, cross-developer cloning, lag_tolerance.
user-invocable: false
metadata:
author: dbt-labs
Using dbt State
dbt State is a server-backed reuse mechanism. It should not be conflated with dbt's state:modified selector or --state deferral.
Before building each selected node, dbt asks the dbt State server whether the object can be skipped (reuse from the target schema), cloned (reuse from another schema), or must be built. It is the successor to State-Aware Orchestration, but works in dbt Core, in development, and in CI — not just Fusion in production.
dbt State is a paid product, but it does not require a dbt platform (fka dbt Cloud) subscription.
Common Misconceptions
| Misconception | Reality |
|---|---|
"dbt State is just state:modified / --state" | No. state:modified hashes file contents against a manifest you manage (and must keep fresh — e.g. via dbt parse or similar) and rebuilds state:modified+ (all descendants). dbt State manages state automatically on a server and does not require maintaining a fresh comparison manifest — it parses SQL into a syntax tree and compares semantic hashes, considers upstream data freshness, and rebuilds a descendant only if it actually depends on the change (not the whole + subtree). |
| "It's Fusion-only / production-only" | Works in dbt Core, the dbt platform, and Fusion, across dev, CI, and production, with any orchestrator. |
| "dbt Core users can't use it" | They can. dbt Core 1.7–1.11 require pip install dbt-state. It's baked into dbt Core 1.12 / v2.0 and Fusion. |
| "It's free / it's local" | It calls the dbt State server and requires authentication via a dbt platform account or a standalone dbt State account (app.state.dbt.com). Reuse is metered in DATTs — daily active target tables (see Billing below). |
| "It sends my data to dbt Labs" | It sends last-modified timestamps and SQL text. The SQL is hashed then discarded — dbt Labs cannot read query contents after hashing, and can never access raw data. |
How the reuse decision works
For each selected node, dbt State picks the cheapest valid option:
- Skip — object exists in the target schema, its semantic hash is unchanged, and no parent has fresher data beyond
lag_tolerance. Does nothing. - Clone — a matching object (same hash, fresh data) exists in another schema (e.g. production, or a teammate's dev schema). Clones it, marked Reused. Uses zero-copy clone if supported by the warehouse, or runs a CTAS statement to copy the transformed data from elsewhere if not. Test results are reused too — a failing test still surfaces even though it wasn't re-executed.
- Build — no valid reuse. Builds normally, auto-deferring unselected upstream nodes.
If a node is selected for execution but its inputs do not exist in the target schema, dbt State uses deferral as normal. If a manifest.json is present it will use that, otherwise it will make a best-effort guess at the correct FQN based on the generate_*_name macros. Deferral does not consume DATTs. The defer_to_target config in profiles.yml can be used to specify which schema to defer to for self-managed users. It is not necessary for dbt platform users.
To get freshness, dbt fetches warehouse metadata (or loaded_at_field/loaded_at_query) for each input relation. For views without a loaded_at config, it traverses upstream until it finds a real table.
Query normalization & why models rebuild
dbt State hashes a parsed syntax tree, so it ignores cosmetic changes — whitespace, comments, table aliases, dbt lint --fix reformatting. A model rebuilds only when its logic or data changes.
Volatile SQL (current_timestamp(), getdate(), random()): by default treated as logic — the hash uses the function name, not its runtime value, so it does not invalidate the model every run (otherwise nothing downstream of getdate() could ever be reused). To make a model rebuild when the value changes:
- Set
evaluate_volatile_sql: true(preferred — covers all functions in the model, inheritable like any config). dbt State emulates the function's value into the hash. - Or use a Jinja equivalent (e.g.
{{ run_started_at }}) — Jinja renders before parsing, so it changes the compiled SQL each run.
Non-deterministic Jinja (e.g. dbt_utils.get_relations_by_pattern returning relations in varying order) produces a different compiled hash and triggers rebuilds even when logic is unchanged.
Config changes: only build-relevant configs affect the hash (materialized, on_schema_change, severity, …). Cosmetic configs (meta, tags) are ignored. If a post-hook mutates tables based on ignored fields (e.g. applying meta as warehouse tags), set execute_hooks_on_any_reuse: true so hooks run on reuse.
Configs quick reference
Set under models: +state: in dbt_project.yml, in schema.yml config.state, or in {{ config(state={...}) }}.
| Config | Default | Purpose |
|---|---|---|
lag_tolerance | 45m | How stale data may be before a node is eligible to rebuild. Data freshness only — SQL changes rebuild regardless. |
require_fresh_data_from | any | Whether any or all direct parents need fresh data to trigger a rebuild. |
evaluate_volatile_sql | `false |