Star0

E2e Data Engineering

STRONG · 72
Community indexed

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.

Downloads0
Stars331
Version1.0.0
Quality46/100 · Needs review
Trust72/100 · Strong shortlist
Audit64/100 · Needs review

Supply asset profile

Data, BI, and analytics

CSV, SQL, notebooks, dashboards, data pipelines, BI, ETL, and spreadsheet analysis.

Browse track

Scenario

Coding agents

I need a coding agent that can understand a repository, edit code, and review pull requests.

Agent fit

Claude Code + CLI + Codex

Codex, Claude Code, Cursor, CLI, or custom agents.

Install

Ready

npx skills add airscholar/e2e-data-engineering

Maintenance

stale

1y since push

Risk

Needs review

License is unclear

GitHub quality

331

46/100 quality · 72/100 trust

Coverage tags

DataCoding agentsdata-analysisdata-pipelineautomation

Review notes

License is unclear · Repository appears stale

Agent adoption scorecard

Trust, audit, and install readiness at a glance

These scores combine public repository metadata, OpenAgentSkill review signals, maintenance freshness, and install readiness. They are a shortlist signal, not a replacement for human review.

Quality

Needs review
46

Inspect the repository carefully before adding it to an agent workflow.

Trust

Strong shortlist
72

Good trust signals with a few areas worth checking before rollout.

Audit

Needs review
64

Install readiness, security metadata, maintenance, and adoption risk.

Trust Score v3

Human review before install

Test in a sandbox workflow and compare its install path with close alternatives.

PythonData PipelineCodexClaude CodeCursor

Stars

331 GitHub stars

Repo activity

331 stars, 150 forks

Maintenance

1y since push

License

Unknown

Install

npx skills add airscholar/e2e-data-engineering

Install safety

standard package or runtime install path

Permission surface

filesystem or document access

Docs

Strong README/SKILL.md context

Risk summary

Review before production

  • License is unclear
  • Repository looks stale
  • Quality score needs review
  • Recent maintenance: 1y since push

Install readiness

Install path available

  • Install path is available
  • Repository evidence is available
  • License is unclear
  • 1y since push

Agent-readable metadata

Machine-readable decision data for this skill.

Use this block or the embedded JSON to decide whether an agent should install this skill, choose an alternative, or ask for human review first.

Open JSON

Suited tasks

  • Coding agents workflows
  • Claude Code teams
  • builders willing to evaluate younger projects
  • Inspect source files
  • Explain architecture

Suited agents

PythonData PipelineCodexClaude CodeCursorOpenAgentSkill CLICLI

Trust and risk

Trust score
72/100
Risk level
Needs review
Auto install
review

Install command

npx skills add airscholar/e2e-data-engineering
Public auditResolve APIInstall handoff

Do not use when

  • teams that require actively maintained dependencies
  • production agents without a repository review
  • Repository looks stale
  • No OpenAgentSkill engagement data yet
  • License is unclear

Alternative

D3

113.1K stars

npx skills add d3/d3

Alternative

Grafana

74.4K stars

npx skills add grafana/grafana

Alternative

Superset

73.3K stars

npx skills add apache/superset

Alternative

Echarts

66.6K stars

npx skills add apache/echarts

Agent safety v2

48/100 · Avoid automatic install

Experimentalreview

Sparse or mixed signals. Useful for discovery, but not for autonomous installation.

Test manually in an isolated workspace and compare against safer alternatives.

Resolve via API

medium

Network access

Skill likely fetches remote pages, APIs, repositories, or external services.

medium

Filesystem access

Skill may read or write project files, documents, generated artifacts, or local workspace state.

  • License is unclear

Install targets

Install this skill in your agent workflow

Copy the registry command or an agent-specific install prompt for Codex, Claude Code, and Cursor.

skill install

OpenAgentSkill CLI

Use the registry command when your workflow supports the OpenAgentSkill installer.

$ npx skills add airscholar/e2e-data-engineering

Agent resolve plan

Let an agent verify fit before installing.

The Resolve API returns the selected skill, alternatives, safety policy, audit notes, install target, and copy-paste prompt an agent can follow without scraping this page.

Open text plan

Resolve JSON

/api/agent/resolve?task=Use%20E2e%20Data%20Engineering%20for%20an%20agent%20workflow&agent=codex&max_risk=medium

Resolve text

/api/agent/resolve?task=Use%20E2e%20Data%20Engineering%20for%20an%20agent%20workflow&agent=codex&max_risk=medium&format=text

Install handoff

/api/skills/airscholar-e2e-data-engineering/install

Agent should check

  • Task fit and alternatives from Resolve API.
  • Audit score, trust score, and safety policy warnings.
  • Install target compatibility for Codex, Claude Code, Cursor, or CLI.

Copy prompt

Task: Use E2e Data Engineering in this workspace.
Resolve first: https://www.openagentskill.com/api/agent/resolve?task=Use%20E2e%20Data%20Engineering%20for%20an%20agent%20workflow&agent=codex&max_risk=medium
Review install handoff: https://www.openagentskill.com/api/skills/airscholar-e2e-data-engineering/install
Install command: npx skills add airscholar/e2e-data-engineering
Before running it, summarize audit warnings, required permissions, and the fallback skill if install is risky.

Agent handoff

Give an agent the install path, not another directory page.

Use the public install endpoint to fetch the command, safety checklist, target prompts, and canonical links for this skill.

Open install API

Install handoff

/api/skills/airscholar-e2e-data-engineering/install

LLM text format

/api/skills/airscholar-e2e-data-engineering/install?format=text

Find alternatives

/api/skills/search?q=E2e%20Data%20Engineering&limit=3

Agent prompt

Use E2e Data Engineering for this task. Review https://www.openagentskill.com/api/skills/airscholar-e2e-data-engineering/install, then install with: npx skills add airscholar/e2e-data-engineering

Registry metadata

Agent-readable profile for automatic skill selection.

This page exposes the same decision, trust, audit, use-case, and install signals through the Registry API, so agents can rank this skill without scraping the UI.

Open manifest

Manifest

/api/registry/manifest/airscholar-e2e-data-engineering

LLM text

/api/registry/manifest/airscholar-e2e-data-engineering?format=text

Install alias

/api/registry/install/airscholar-e2e-data-engineering

Recommend

/api/registry/recommend?task=Use%20E2e%20Data%20Engineering%20in%20an%20agent%20workflow&limit=3

Agent fit

36/100

Coding agents

Use-case tags

Coding agentsRAG and knowledgeBrowser automation

Platforms

Python, Data Pipeline, Claude Code

Audit report

Needs review · 64/100

Review install readiness, maintenance, trust, quality, and metadata warnings before adding this skill to an agent workflow.

View audit report

Agent decision cockpit

Needs validation for Coding agents

Do a manual repository review before adding this to an agent workflow.

36
Readiness
Review
Stage

Role in stack

Needs validation

Primary fit

Coding agents

Trust label

Needs manual review

Install path

Command ready

Use when

  • Coding agents workflows
  • Claude Code teams
  • builders willing to evaluate younger projects

Evidence

  • install command or GitHub repo available
  • 46/100 quality profile

Review first

  • Repository looks stale
  • No OpenAgentSkill engagement data yet

Implementation path

  1. 1Install it in a sandbox agent and run one Coding agents task end to end.
  2. 2Compare output quality, latency, and failure behavior against at least one alternative.
  3. 3Promote it into production only after reviewing repository permissions, license, and maintenance signals.

Trust profile

Strong shortlist

Good trust signals with a few areas worth checking before rollout.

72
Trust score

GitHub adoption

INFO

331 GitHub stars

Stars/forks activity

INFO

331 stars, 150 forks; issue activity unavailable in current metadata

Recent maintenance

FIX

1y since push

License clarity

CHECK

Unknown

Good signals

  • AI review approved
  • Install path is available
  • Repository evidence is available
  • Install command has no obvious high-risk pattern

Review before install

  • License is unclear
  • Repository looks stale
  • Quality score needs review
  • Recent maintenance: 1y since push
  • License clarity: Unknown

Recommended action

Test in a sandbox workflow and compare its install path with close alternatives.

Quality profile

Needs review candidate for agent workflows

Inspect the repository carefully before adding it to an agent workflow.

46
GitHub stars
331
Freshness
1y ago
Install ready
Yes
License
Unknown
Check before install: Repository looks stale

Workflow fit

Use this skill in these scenarios

Build and ship code

Coding agents

I need a coding agent that can understand a repository, edit code, and review pull requests.

Search private knowledge

RAG and knowledge

I need my agent to build a RAG workflow over documents and retrieve reliable context.

Operate web apps

Browser automation

I need my agent to control a browser, fill forms, and verify web app workflows.

Stack fit

Add it to a complete workflow

Turn skills into distribution

Content growth agent

A stack for turning newly indexed skills into SEO briefs, social drafts, comparison pages, and reusable publishing workflows.

Inspect, patch, and verify code

Coding review agent

A stack for software agents that inspect repositories, review pull requests, generate tests, and turn findings into shippable patches.

Ingest, retrieve, and cite

RAG knowledge base

A stack for document-heavy agents that ingest files, create searchable knowledge, retrieve relevant context, and answer with grounded sources.

Alternative shortlist

Compare before you install

Similar skills in this category, ranked with the same readiness and quality signals.

Compare all

D3

Bring data to life with SVG, Canvas and HTML. :bar_chart::chart_with_upwards_trend::tada:

Adopt
Ready100
Quality100
Stars113.1K

Grafana

The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.

Adopt
Ready100
Quality100
Stars74.4K

Superset

Apache Superset is a Data Visualization and Data Exploration Platform

Adopt
Ready100
Quality100
Stars73.3K

Echarts

Apache ECharts is a powerful, interactive charting and data visualization library for browser

Adopt
Ready100
Quality100
Stars66.6K

Overview

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.

Imported by the skill-only GitHub discovery pipeline because it matches agent skill, automation, domain workflow, RAG, document-processing, data, finance, security, or developer-tool signals. Protocol-server projects are excluded from automated imports.

Platform Compatibility

pythonFULL
data-pipelineFULL

Technical Details

Version
1.0.0
License
Unknown
Last Updated
6/16/2026
Published
6/16/2026

Frameworks & Tools

PythonData Pipeline

Decision snapshot

Needs validation

36
Ready
Review
Stage

install command or GitHub repo available

Audit snapshot

Install review

Install and adoption review

64
Needs review
Security
80/100
Maintenance
38/100
Install
92/100
Open full audit

Install

Add to agent workflow

Free and open source. Review the audit before production use.

Compare AlternativesAuto-resolve PlanView on GitHubDocumentation

Growth loop

Share kit

X

Scenario-led draft for E2e Data Engineering, ready for a manual X post.

OpenAgentSkill Update
OpenAgentSkill Update
Today: E2e Data Engineering

Use it when your agent needs to turn docs, data, or knowledge bases into answers and actions.

331 stars - data-analysis
Link: https://www.openagentskill.com/skills/airscholar-e2e-data-engineering?ref=x
#AIAgents #OpenAgentSkill
Open X draft
Optional reply with install command
Link for E2e Data Engineering:
https://www.openagentskill.com/skills/airscholar-e2e-data-engineering?ref=x

Install: npx skills add airscholar/e2e-data-engineering
Open reply draft

Listing source

Community indexed

Claimable

This listing was indexed from public sources and is not marked official until a maintainer claim is approved.

Creator
airscholar
Indexed by
OpenAgentSkill community index

Attribution links to the public repository or creator profile. Creators can claim the listing to update ownership signals.

Claim this skill

Owner claim

Claim this skill listing

This community indexed listing is attributed to airscholar but is not marked official yet. Claim it to add a verified owner signal and make future launch, install, and audit updates easier to trust.

README badge

Add this badge to your GitHub README to show the listing, trust score, and install handoff.

[![OpenAgentSkill](https://www.openagentskill.com/api/badge/airscholar-e2e-data-engineering)](https://www.openagentskill.com/skills/airscholar-e2e-data-engineering)
Preview badgeAudit badge

Author

A

airscholar

@airscholar

Tags

data-pipelineautomationapache-airflowapache-kafkaapache-sparkapache-zookeeperbig-datacassandracontainerizationdata-engineering

Platform Fit

Claude Code

Health Signals

GitHub stars
331
Quality score
36/100
Last GitHub push
Feb 14, 2025
Framework hints
2
OpenAgentSkill views
0
Install copies
0
Outbound clicks
0

Community Signal

Share whether this skill looks useful for your agent workflow. Aggregated feedback improves rankings over time.

Trust & Safety

Strong shortlist

72
  • GitHub adoption331 GitHub starsINFO
  • Stars/forks activity331 stars, 150 forks; issue activity unavailable in current metadataINFO
  • Recent maintenance1y since pushFIX
  • License clarityUnknownCHECK
  • README/SKILL.md completenessMetadata includes enough usage and workflow contextPASS
  • Dependency/runtime riskexternal package install surfaceINFO

Related Skills

D3

Bring data to life with SVG, Canvas and HTML. :bar_chart::chart_with_upwards_trend::tada:

113.1K stars · 0 installs

Grafana

The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.

74.4K stars · 0 installs

Superset

Apache Superset is a Data Visualization and Data Exploration Platform

73.3K stars · 0 installs

Echarts

Apache ECharts is a powerful, interactive charting and data visualization library for browser

66.6K stars · 0 installs