数据与分析
Aqueduct
LLM-driven data engineering agent that automates SQL generation and ETL workflows from natural language requirements.
JohnnyQ-commits · 已有 AI 审核记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add JohnnyQ-commits/Aqueduct技能说明
Aqueduct
Data Engineering Automation Agent Framework
Overview
The problem: Data engineers spend 60-70% of their time on repetitive work — understanding requirements, writing boilerplate SQL, creating DDL, writing DQC tests, generating documentation. The creative part (business logic) is small; the mechanical part is huge.
Aqueduct automates the mechanical part.
Give it a requirement document. Get back 11 standardized deliverables: DDL, ETL SQL, DQC test cases, field-level lineage, design documents, and a comprehensive report.
Requirement (.md) --> Design (.md) --> DDL (.sql) --> SQL (.sql) --> Review --> DQC (.sql) --> Report (.md) --> Deliverables (11 files)
What makes it different
| Principle | What it means |
|---|---|
| Framework, not tool | Embed into Claude Code, LangChain, or your own app via from aqueduct import Aqueduct |
| 7-layer architecture | Clean separation: MCP / LLM / Tools / Skills / Engine / Memory / Config |
| Platform agnostic | Connects to any data platform via standard MCP protocol (.mcp.json) |
| Ontology knowledge | Business domains modeled as typed JSON — entities, relationships, metrics, axioms |
| DAG orchestration | StateGraph-based workflow with interactive checkpoints and error recovery |
| Review-fix loop | Code review finds bugs → auto-fix SQL → re-review → production-ready quality |
| Full observability | Per-task log files, phase timing, LLM call tracing, tool execution audit |
| Prompt-code decoupled | Prompts as .tpl.md files — edit without touching code |