@crawlab-team
Creator · crawlab-team
Last updated · Sep 1, 2026
Crawlab
STRONG · 86Community indexed
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
Creator · crawlab-team
Last updated · Sep 1, 2026
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
Creator · crawlab-team
Last updated · Sep 1, 2026
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
Creator · crawlab-team
Last updated · Sep 1, 2026
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
Creator · crawlab-team
Last updated · Sep 1, 2026
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
Review then install
Install targets
Codex install prompt
Install the "Crawlab" agent skill from https://github.com/crawlab-team/crawlab. Read its SKILL.md or equivalent instructions first, install only the files needed for this workspace, and summarize any required setup before using it. Skill purpose: Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架 After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {"event_id":"install_<unique-id>","skill_slug":"crawlab-team-crawlab","task":"Install Crawlab","agent":"codex","outcome":"success","install_used":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes.Supply asset profile
Code review, repo analysis, testing, CI, GitHub, DevOps, and developer workflow skills.
Scenario
Coding agents
I need a coding agent that can understand a repository, edit code, and review pull requests.
Agent fit
Claude Code + CLI + Codex
Codex, Claude Code, Cursor, CLI, or custom agents.
Install
Ready
npx skills add crawlab-team/crawlab
Maintenance
stable
7mo since push
Risk
Safe to try
No major risk signals from available metadata
GitHub quality
12K
96/100 Quality · 89/100 Trust
Coverage tags
Review notes
No major risk signals from available metadata
Agent adoption scorecard
These scores combine public repository metadata, OpenAgentSkill review signals, maintenance freshness, and install readiness. They are a shortlist signal, not a replacement for human review.
Quality
ExcellentHigh-confidence pick with strong adoption and healthy maintenance signals.
Trust
Review then installGood shortlist signal, but the agent should review audit notes, install policy, and outcome evidence before running it.
Audit
Safe to tryA machine-readable review of install readiness, security metadata, maintenance, and adoption risk.
OpenAgentSkill Trust Score v5
Use as the primary candidate after human or sandbox review.
Stars
12K GitHub stars
Repo activity
12K stars, 1.9K forks
Maintenance
7mo since push
License
BSD-3-Clause
Install
npx skills add crawlab-team/crawlab
Install safety
Agent-readable metadata
Use this block or the embedded JSON to decide whether an agent should install this skill, choose an alternative, or ask for human review first.
Suited tasks
Suited agents
Install decision
Trust and risk
Outcome loop
Do not use when
Agent safety v2
Strong metadata, audit, install, and review signals. Suitable for agent shortlists after normal workspace review.
Allow agent install in a sandbox or low-risk workspace, then promote after one successful narrow task.
medium
Skill likely fetches remote pages, APIs, repositories, or external services.
Agent resolve plan
The Resolve API returns the selected skill, alternatives, safety policy, audit notes, install target, and copy-paste prompt an agent can follow without scraping this page.
Open JSON
/api/agent/resolve?task=Use%20Crawlab%20for%20an%20agent%20workflow&agent=codex&max_risk=medium
Resolve text
/api/agent/resolve?task=Use%20Crawlab%20for%20an%20agent%20workflow&agent=codex&max_risk=medium&format=text
Install handoff
/api/skills/crawlab-team-crawlab/install
Agent should check
Copy prompt
Task: Use Crawlab in this workspace.
Resolve first: https://www.openagentskill.com/api/agent/resolve?task=Use%20Crawlab%20for%20an%20agent%20workflow&agent=codex&max_risk=medium
Review install handoff: https://www.openagentskill.com/api/skills/crawlab-team-crawlab/install
Install command: npx skills add crawlab-team/crawlab
Before running it, summarize audit warnings, required permissions, and the fallback skill if install is risky.Agent handoff
Use the public install endpoint to fetch the command, safety checklist, target prompts, and canonical links for this skill.
Install handoff
/api/skills/crawlab-team-crawlab/install
LLM text format
/api/skills/crawlab-team-crawlab/install?format=text
Find alternatives
/api/skills/search?q=Crawlab&limit=3
Agent prompt
Use Crawlab for this task. Review https://www.openagentskill.com/api/skills/crawlab-team-crawlab/install, then install with: npx skills add crawlab-team/crawlabRegistry metadata
This page exposes the same decision, trust, audit, use-case, and install signals through the Registry API, so agents can rank this skill without scraping the UI.
Manifest
/api/registry/manifest/crawlab-team-crawlab
LLM text
/api/registry/manifest/crawlab-team-crawlab?format=text
Install alias
/api/registry/install/crawlab-team-crawlab
Recommend
/api/registry/recommend?task=Use%20Crawlab%20in%20an%20agent%20workflow&limit=3
Agent fit
Web scraping
Use-case tags
Platforms
Go, Crawler, Claude Code
Audit report
A machine-readable review of install readiness, security metadata, maintenance, and adoption risk.
Agent decision cockpit
Use this as a leading candidate, then validate the README and install path in your own agent stack.
Role in stack
Primary pick
Primary fit
Web scraping
Trust label
Production-ready
Install path
Command ready
Use when
Evidence
review first
Implementation path
Trust profile
Good shortlist signal, but the agent should review audit notes, install policy, and outcome evidence before running it.
GitHub adoption
PASS12K GitHub stars
Stars/forks activity
PASS12K stars, 1.9K forks; issue activity unavailable in current metadata
Recent maintenance
INFO7mo since push
License clarity
PASSBSD-3-Clause
Good signals
Review before install
Recommended action
Use as the primary candidate after human or sandbox review.
Quality profile
High-confidence pick with strong adoption and healthy maintenance signals.
Workflow fit
Collect structured data
I need my agent to scrape websites and extract structured data from pages.
Operate web apps
I need my agent to control a browser, fill forms, and verify web app workflows.
Analyze matches
I need my agent to analyze football matches, World Cup data, xG, players, teams, and predictions.
Workflow fit
Scrape, clean, and reuse web data
A practical workflow for agents that crawl public pages, extract clean content, normalize data, and hand it to downstream research or RAG workflows.
Operate and verify web apps
A workflow for agents that navigate products, fill forms, take screenshots, and verify real user flows across web applications.
Inspect, patch, and verify code
A workflow for software agents that inspect repositories, review pull requests, generate tests, and turn findings into shippable patches.
Alternative shortlist
Similar skills that may fit this task.
Open-source LLM-friendly web crawler and scraper
Python scraper based on AI
Control web interfaces with natural language agents
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
Imported by the skill-only GitHub discovery pipeline because it matches agent skill, automation, RAG, or developer-tool signals. Protocol-server projects are excluded from automated imports.
Frameworks & tools
Decision snapshot
12,221 GitHub stars
Audit
Install and adoption review
Agent-proven evidence
Outcome reports after resolve, review, install, and one narrow run.
No agent outcome data yet. The first agent run can report success, setup needs, risk blocks, failure, or not-relevant through /api/agent/outcome.
Install
Free and open source. Review the report before installing into production agents.
Growth loop
Scenario-led draft for Crawlab, ready for a manual X post.
Crawlab: Distributed web crawler admin platform for spiders management regardless of languages and fra... 12.2K stars https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=x
Listing + install path for Crawlab: https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=x Install: npx skills add crawlab-team/crawlab
Listing source
This listing was indexed from public sources and is not marked official until a maintainer claim is approved.
Attribution links to the public repository or creator profile. Creators can claim the listing to update ownership signals.
Claim this skillOwner claim
This Community indexed listing is attributed to crawlab-team but is not marked official yet. Claim it to add a verified owner signal and make future launch, install, and audit updates easier to trust.
Creator backlink kit
Show the canonical listing, current trust and audit signals, and real Agent-Proven evidence where developers evaluate the repository.
[](https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=github&utm_source=github&utm_medium=referral&utm_campaign=creator_badge)
[](https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=github&utm_source=github&utm_medium=referral&utm_campaign=creator_badge)
[](https://www.openagentskill.com/skills/crawlab-team-crawlab/audit)
[](https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=github&utm_source=github&utm_medium=referral&utm_campaign=creator_badge)crawlab-team✓
@crawlab-team
Share whether this skill looks useful for your agent workflow. Aggregated feedback improves rankings over time.
Review then install
Crawl4AI
Open-source LLM-friendly web crawler and scraper
73.1K StarsScrapegraph AI
Python scraper based on AI
27.2K StarsPage Agent
Control web interfaces with natural language agents
28.6K StarsMaxun
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
16.2K StarsReview then install
Install targets
Codex install prompt
Install the "Crawlab" agent skill from https://github.com/crawlab-team/crawlab. Read its SKILL.md or equivalent instructions first, install only the files needed for this workspace, and summarize any required setup before using it. Skill purpose: Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架 After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {"event_id":"install_<unique-id>","skill_slug":"crawlab-team-crawlab","task":"Install Crawlab","agent":"codex","outcome":"success","install_used":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes.Supply asset profile
Code review, repo analysis, testing, CI, GitHub, DevOps, and developer workflow skills.
Scenario
Coding agents
I need a coding agent that can understand a repository, edit code, and review pull requests.
Agent fit
Claude Code + CLI + Codex
Codex, Claude Code, Cursor, CLI, or custom agents.
Install
Ready
npx skills add crawlab-team/crawlab
Maintenance
stable
7mo since push
Risk
Safe to try
No major risk signals from available metadata
GitHub quality
12K
96/100 Quality · 89/100 Trust
Coverage tags
Review notes
No major risk signals from available metadata
Agent adoption scorecard
These scores combine public repository metadata, OpenAgentSkill review signals, maintenance freshness, and install readiness. They are a shortlist signal, not a replacement for human review.
Quality
ExcellentHigh-confidence pick with strong adoption and healthy maintenance signals.
Trust
Review then installGood shortlist signal, but the agent should review audit notes, install policy, and outcome evidence before running it.
Audit
Safe to tryA machine-readable review of install readiness, security metadata, maintenance, and adoption risk.
OpenAgentSkill Trust Score v5
Use as the primary candidate after human or sandbox review.
Stars
12K GitHub stars
Repo activity
12K stars, 1.9K forks
Maintenance
7mo since push
License
BSD-3-Clause
Install
npx skills add crawlab-team/crawlab
Install safety
Agent-readable metadata
Use this block or the embedded JSON to decide whether an agent should install this skill, choose an alternative, or ask for human review first.
Suited tasks
Suited agents
Install decision
Trust and risk
Outcome loop
Do not use when
Agent safety v2
Strong metadata, audit, install, and review signals. Suitable for agent shortlists after normal workspace review.
Allow agent install in a sandbox or low-risk workspace, then promote after one successful narrow task.
medium
Skill likely fetches remote pages, APIs, repositories, or external services.
Agent resolve plan
The Resolve API returns the selected skill, alternatives, safety policy, audit notes, install target, and copy-paste prompt an agent can follow without scraping this page.
Open JSON
/api/agent/resolve?task=Use%20Crawlab%20for%20an%20agent%20workflow&agent=codex&max_risk=medium
Resolve text
/api/agent/resolve?task=Use%20Crawlab%20for%20an%20agent%20workflow&agent=codex&max_risk=medium&format=text
Install handoff
/api/skills/crawlab-team-crawlab/install
Agent should check
Copy prompt
Task: Use Crawlab in this workspace.
Resolve first: https://www.openagentskill.com/api/agent/resolve?task=Use%20Crawlab%20for%20an%20agent%20workflow&agent=codex&max_risk=medium
Review install handoff: https://www.openagentskill.com/api/skills/crawlab-team-crawlab/install
Install command: npx skills add crawlab-team/crawlab
Before running it, summarize audit warnings, required permissions, and the fallback skill if install is risky.Agent handoff
Use the public install endpoint to fetch the command, safety checklist, target prompts, and canonical links for this skill.
Install handoff
/api/skills/crawlab-team-crawlab/install
LLM text format
/api/skills/crawlab-team-crawlab/install?format=text
Find alternatives
/api/skills/search?q=Crawlab&limit=3
Agent prompt
Use Crawlab for this task. Review https://www.openagentskill.com/api/skills/crawlab-team-crawlab/install, then install with: npx skills add crawlab-team/crawlabRegistry metadata
This page exposes the same decision, trust, audit, use-case, and install signals through the Registry API, so agents can rank this skill without scraping the UI.
Manifest
/api/registry/manifest/crawlab-team-crawlab
LLM text
/api/registry/manifest/crawlab-team-crawlab?format=text
Install alias
/api/registry/install/crawlab-team-crawlab
Recommend
/api/registry/recommend?task=Use%20Crawlab%20in%20an%20agent%20workflow&limit=3
Agent fit
Web scraping
Use-case tags
Platforms
Go, Crawler, Claude Code
Audit report
A machine-readable review of install readiness, security metadata, maintenance, and adoption risk.
Agent decision cockpit
Use this as a leading candidate, then validate the README and install path in your own agent stack.
Role in stack
Primary pick
Primary fit
Web scraping
Trust label
Production-ready
Install path
Command ready
Use when
Evidence
review first
Implementation path
Trust profile
Good shortlist signal, but the agent should review audit notes, install policy, and outcome evidence before running it.
GitHub adoption
PASS12K GitHub stars
Stars/forks activity
PASS12K stars, 1.9K forks; issue activity unavailable in current metadata
Recent maintenance
INFO7mo since push
License clarity
PASSBSD-3-Clause
Good signals
Review before install
Recommended action
Use as the primary candidate after human or sandbox review.
Quality profile
High-confidence pick with strong adoption and healthy maintenance signals.
Workflow fit
Collect structured data
I need my agent to scrape websites and extract structured data from pages.
Operate web apps
I need my agent to control a browser, fill forms, and verify web app workflows.
Analyze matches
I need my agent to analyze football matches, World Cup data, xG, players, teams, and predictions.
Workflow fit
Scrape, clean, and reuse web data
A practical workflow for agents that crawl public pages, extract clean content, normalize data, and hand it to downstream research or RAG workflows.
Operate and verify web apps
A workflow for agents that navigate products, fill forms, take screenshots, and verify real user flows across web applications.
Inspect, patch, and verify code
A workflow for software agents that inspect repositories, review pull requests, generate tests, and turn findings into shippable patches.
Alternative shortlist
Similar skills that may fit this task.
Open-source LLM-friendly web crawler and scraper
Python scraper based on AI
Control web interfaces with natural language agents
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
Imported by the skill-only GitHub discovery pipeline because it matches agent skill, automation, RAG, or developer-tool signals. Protocol-server projects are excluded from automated imports.
Frameworks & tools
Decision snapshot
12,221 GitHub stars
Audit
Install and adoption review
Agent-proven evidence
Outcome reports after resolve, review, install, and one narrow run.
No agent outcome data yet. The first agent run can report success, setup needs, risk blocks, failure, or not-relevant through /api/agent/outcome.
Install
Free and open source. Review the report before installing into production agents.
Growth loop
Scenario-led draft for Crawlab, ready for a manual X post.
Crawlab: Distributed web crawler admin platform for spiders management regardless of languages and fra... 12.2K stars https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=x
Listing + install path for Crawlab: https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=x Install: npx skills add crawlab-team/crawlab
Listing source
This listing was indexed from public sources and is not marked official until a maintainer claim is approved.
Attribution links to the public repository or creator profile. Creators can claim the listing to update ownership signals.
Claim this skillOwner claim
This Community indexed listing is attributed to crawlab-team but is not marked official yet. Claim it to add a verified owner signal and make future launch, install, and audit updates easier to trust.
Creator backlink kit
Show the canonical listing, current trust and audit signals, and real Agent-Proven evidence where developers evaluate the repository.
[](https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=github&utm_source=github&utm_medium=referral&utm_campaign=creator_badge)
[](https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=github&utm_source=github&utm_medium=referral&utm_campaign=creator_badge)
[](https://www.openagentskill.com/skills/crawlab-team-crawlab/audit)
[](https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=github&utm_source=github&utm_medium=referral&utm_campaign=creator_badge)crawlab-team✓
@crawlab-team
Share whether this skill looks useful for your agent workflow. Aggregated feedback improves rankings over time.
Review then install
Crawl4AI
Open-source LLM-friendly web crawler and scraper
73.1K StarsScrapegraph AI
Python scraper based on AI
27.2K StarsPage Agent
Control web interfaces with natural language agents
28.6K StarsMaxun
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
16.2K StarsReview then install
Install targets
Codex install prompt
Install the "Crawlab" agent skill from https://github.com/crawlab-team/crawlab. Read its SKILL.md or equivalent instructions first, install only the files needed for this workspace, and summarize any required setup before using it. Skill purpose: Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架 After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {"event_id":"install_<unique-id>","skill_slug":"crawlab-team-crawlab","task":"Install Crawlab","agent":"codex","outcome":"success","install_used":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes.Supply asset profile
Code review, repo analysis, testing, CI, GitHub, DevOps, and developer workflow skills.
Scenario
Coding agents
I need a coding agent that can understand a repository, edit code, and review pull requests.
Agent fit
Claude Code + CLI + Codex
Codex, Claude Code, Cursor, CLI, or custom agents.
Install
Ready
npx skills add crawlab-team/crawlab
Maintenance
stable
7mo since push
Risk
Safe to try
No major risk signals from available metadata
GitHub quality
12K
96/100 Quality · 89/100 Trust
Coverage tags
Review notes
No major risk signals from available metadata
Agent adoption scorecard
These scores combine public repository metadata, OpenAgentSkill review signals, maintenance freshness, and install readiness. They are a shortlist signal, not a replacement for human review.
Quality
ExcellentHigh-confidence pick with strong adoption and healthy maintenance signals.
Trust
Review then installGood shortlist signal, but the agent should review audit notes, install policy, and outcome evidence before running it.
Audit
Safe to tryA machine-readable review of install readiness, security metadata, maintenance, and adoption risk.
OpenAgentSkill Trust Score v5
Use as the primary candidate after human or sandbox review.
Stars
12K GitHub stars
Repo activity
12K stars, 1.9K forks
Maintenance
7mo since push
License
BSD-3-Clause
Install
npx skills add crawlab-team/crawlab
Install safety
Agent-readable metadata
Use this block or the embedded JSON to decide whether an agent should install this skill, choose an alternative, or ask for human review first.
Suited tasks
Suited agents
Install decision
Trust and risk
Outcome loop
Do not use when
Agent safety v2
Strong metadata, audit, install, and review signals. Suitable for agent shortlists after normal workspace review.
Allow agent install in a sandbox or low-risk workspace, then promote after one successful narrow task.
medium
Skill likely fetches remote pages, APIs, repositories, or external services.
Agent resolve plan
The Resolve API returns the selected skill, alternatives, safety policy, audit notes, install target, and copy-paste prompt an agent can follow without scraping this page.
Open JSON
/api/agent/resolve?task=Use%20Crawlab%20for%20an%20agent%20workflow&agent=codex&max_risk=medium
Resolve text
/api/agent/resolve?task=Use%20Crawlab%20for%20an%20agent%20workflow&agent=codex&max_risk=medium&format=text
Install handoff
/api/skills/crawlab-team-crawlab/install
Agent should check
Copy prompt
Task: Use Crawlab in this workspace.
Resolve first: https://www.openagentskill.com/api/agent/resolve?task=Use%20Crawlab%20for%20an%20agent%20workflow&agent=codex&max_risk=medium
Review install handoff: https://www.openagentskill.com/api/skills/crawlab-team-crawlab/install
Install command: npx skills add crawlab-team/crawlab
Before running it, summarize audit warnings, required permissions, and the fallback skill if install is risky.Agent handoff
Use the public install endpoint to fetch the command, safety checklist, target prompts, and canonical links for this skill.
Install handoff
/api/skills/crawlab-team-crawlab/install
LLM text format
/api/skills/crawlab-team-crawlab/install?format=text
Find alternatives
/api/skills/search?q=Crawlab&limit=3
Agent prompt
Use Crawlab for this task. Review https://www.openagentskill.com/api/skills/crawlab-team-crawlab/install, then install with: npx skills add crawlab-team/crawlabRegistry metadata
This page exposes the same decision, trust, audit, use-case, and install signals through the Registry API, so agents can rank this skill without scraping the UI.
Manifest
/api/registry/manifest/crawlab-team-crawlab
LLM text
/api/registry/manifest/crawlab-team-crawlab?format=text
Install alias
/api/registry/install/crawlab-team-crawlab
Recommend
/api/registry/recommend?task=Use%20Crawlab%20in%20an%20agent%20workflow&limit=3
Agent fit
Web scraping
Use-case tags
Platforms
Go, Crawler, Claude Code
Audit report
A machine-readable review of install readiness, security metadata, maintenance, and adoption risk.
Agent decision cockpit
Use this as a leading candidate, then validate the README and install path in your own agent stack.
Role in stack
Primary pick
Primary fit
Web scraping
Trust label
Production-ready
Install path
Command ready
Use when
Evidence
review first
Implementation path
Trust profile
Good shortlist signal, but the agent should review audit notes, install policy, and outcome evidence before running it.
GitHub adoption
PASS12K GitHub stars
Stars/forks activity
PASS12K stars, 1.9K forks; issue activity unavailable in current metadata
Recent maintenance
INFO7mo since push
License clarity
PASSBSD-3-Clause
Good signals
Review before install
Recommended action
Use as the primary candidate after human or sandbox review.
Quality profile
High-confidence pick with strong adoption and healthy maintenance signals.
Workflow fit
Collect structured data
I need my agent to scrape websites and extract structured data from pages.
Operate web apps
I need my agent to control a browser, fill forms, and verify web app workflows.
Analyze matches
I need my agent to analyze football matches, World Cup data, xG, players, teams, and predictions.
Workflow fit
Scrape, clean, and reuse web data
A practical workflow for agents that crawl public pages, extract clean content, normalize data, and hand it to downstream research or RAG workflows.
Operate and verify web apps
A workflow for agents that navigate products, fill forms, take screenshots, and verify real user flows across web applications.
Inspect, patch, and verify code
A workflow for software agents that inspect repositories, review pull requests, generate tests, and turn findings into shippable patches.
Alternative shortlist
Similar skills that may fit this task.
Open-source LLM-friendly web crawler and scraper
Python scraper based on AI
Control web interfaces with natural language agents
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
Imported by the skill-only GitHub discovery pipeline because it matches agent skill, automation, RAG, or developer-tool signals. Protocol-server projects are excluded from automated imports.
Frameworks & tools
Decision snapshot
12,221 GitHub stars
Audit
Install and adoption review
Agent-proven evidence
Outcome reports after resolve, review, install, and one narrow run.
No agent outcome data yet. The first agent run can report success, setup needs, risk blocks, failure, or not-relevant through /api/agent/outcome.
Install
Free and open source. Review the report before installing into production agents.
Growth loop
Scenario-led draft for Crawlab, ready for a manual X post.
Crawlab: Distributed web crawler admin platform for spiders management regardless of languages and fra... 12.2K stars https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=x
Listing + install path for Crawlab: https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=x Install: npx skills add crawlab-team/crawlab
Listing source
This listing was indexed from public sources and is not marked official until a maintainer claim is approved.
Attribution links to the public repository or creator profile. Creators can claim the listing to update ownership signals.
Claim this skillOwner claim
This Community indexed listing is attributed to crawlab-team but is not marked official yet. Claim it to add a verified owner signal and make future launch, install, and audit updates easier to trust.
Creator backlink kit
Show the canonical listing, current trust and audit signals, and real Agent-Proven evidence where developers evaluate the repository.
[](https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=github&utm_source=github&utm_medium=referral&utm_campaign=creator_badge)
[](https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=github&utm_source=github&utm_medium=referral&utm_campaign=creator_badge)
[](https://www.openagentskill.com/skills/crawlab-team-crawlab/audit)
[](https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=github&utm_source=github&utm_medium=referral&utm_campaign=creator_badge)crawlab-team✓
@crawlab-team
Share whether this skill looks useful for your agent workflow. Aggregated feedback improves rankings over time.
Review then install
Crawl4AI
Open-source LLM-friendly web crawler and scraper
73.1K StarsScrapegraph AI
Python scraper based on AI
27.2K StarsPage Agent
Control web interfaces with natural language agents
28.6K StarsMaxun
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
16.2K StarsReview then install
Install targets
Codex install prompt
Install the "Crawlab" agent skill from https://github.com/crawlab-team/crawlab. Read its SKILL.md or equivalent instructions first, install only the files needed for this workspace, and summarize any required setup before using it. Skill purpose: Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架 After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {"event_id":"install_<unique-id>","skill_slug":"crawlab-team-crawlab","task":"Install Crawlab","agent":"codex","outcome":"success","install_used":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes.Supply asset profile
Code review, repo analysis, testing, CI, GitHub, DevOps, and developer workflow skills.
Scenario
Coding agents
I need a coding agent that can understand a repository, edit code, and review pull requests.
Agent fit
Claude Code + CLI + Codex
Codex, Claude Code, Cursor, CLI, or custom agents.
Install
Ready
npx skills add crawlab-team/crawlab
Maintenance
stable
7mo since push
Risk
Safe to try
No major risk signals from available metadata
GitHub quality
12K
96/100 Quality · 89/100 Trust
Coverage tags
Review notes
No major risk signals from available metadata
Agent adoption scorecard
These scores combine public repository metadata, OpenAgentSkill review signals, maintenance freshness, and install readiness. They are a shortlist signal, not a replacement for human review.
Quality
ExcellentHigh-confidence pick with strong adoption and healthy maintenance signals.
Trust
Review then installGood shortlist signal, but the agent should review audit notes, install policy, and outcome evidence before running it.
Audit
Safe to tryA machine-readable review of install readiness, security metadata, maintenance, and adoption risk.
OpenAgentSkill Trust Score v5
Use as the primary candidate after human or sandbox review.
Stars
12K GitHub stars
Repo activity
12K stars, 1.9K forks
Maintenance
7mo since push
License
BSD-3-Clause
Install
npx skills add crawlab-team/crawlab
Install safety
Agent-readable metadata
Use this block or the embedded JSON to decide whether an agent should install this skill, choose an alternative, or ask for human review first.
Suited tasks
Suited agents
Install decision
Trust and risk
Outcome loop
Do not use when
Agent safety v2
Strong metadata, audit, install, and review signals. Suitable for agent shortlists after normal workspace review.
Allow agent install in a sandbox or low-risk workspace, then promote after one successful narrow task.
medium
Skill likely fetches remote pages, APIs, repositories, or external services.
Agent resolve plan
The Resolve API returns the selected skill, alternatives, safety policy, audit notes, install target, and copy-paste prompt an agent can follow without scraping this page.
Open JSON
/api/agent/resolve?task=Use%20Crawlab%20for%20an%20agent%20workflow&agent=codex&max_risk=medium
Resolve text
/api/agent/resolve?task=Use%20Crawlab%20for%20an%20agent%20workflow&agent=codex&max_risk=medium&format=text
Install handoff
/api/skills/crawlab-team-crawlab/install
Agent should check
Copy prompt
Task: Use Crawlab in this workspace.
Resolve first: https://www.openagentskill.com/api/agent/resolve?task=Use%20Crawlab%20for%20an%20agent%20workflow&agent=codex&max_risk=medium
Review install handoff: https://www.openagentskill.com/api/skills/crawlab-team-crawlab/install
Install command: npx skills add crawlab-team/crawlab
Before running it, summarize audit warnings, required permissions, and the fallback skill if install is risky.Agent handoff
Use the public install endpoint to fetch the command, safety checklist, target prompts, and canonical links for this skill.
Install handoff
/api/skills/crawlab-team-crawlab/install
LLM text format
/api/skills/crawlab-team-crawlab/install?format=text
Find alternatives
/api/skills/search?q=Crawlab&limit=3
Agent prompt
Use Crawlab for this task. Review https://www.openagentskill.com/api/skills/crawlab-team-crawlab/install, then install with: npx skills add crawlab-team/crawlabRegistry metadata
This page exposes the same decision, trust, audit, use-case, and install signals through the Registry API, so agents can rank this skill without scraping the UI.
Manifest
/api/registry/manifest/crawlab-team-crawlab
LLM text
/api/registry/manifest/crawlab-team-crawlab?format=text
Install alias
/api/registry/install/crawlab-team-crawlab
Recommend
/api/registry/recommend?task=Use%20Crawlab%20in%20an%20agent%20workflow&limit=3
Agent fit
Web scraping
Use-case tags
Platforms
Go, Crawler, Claude Code
Audit report
A machine-readable review of install readiness, security metadata, maintenance, and adoption risk.
Agent decision cockpit
Use this as a leading candidate, then validate the README and install path in your own agent stack.
Role in stack
Primary pick
Primary fit
Web scraping
Trust label
Production-ready
Install path
Command ready
Use when
Evidence
review first
Implementation path
Trust profile
Good shortlist signal, but the agent should review audit notes, install policy, and outcome evidence before running it.
GitHub adoption
PASS12K GitHub stars
Stars/forks activity
PASS12K stars, 1.9K forks; issue activity unavailable in current metadata
Recent maintenance
INFO7mo since push
License clarity
PASSBSD-3-Clause
Good signals
Review before install
Recommended action
Use as the primary candidate after human or sandbox review.
Quality profile
High-confidence pick with strong adoption and healthy maintenance signals.
Workflow fit
Collect structured data
I need my agent to scrape websites and extract structured data from pages.
Operate web apps
I need my agent to control a browser, fill forms, and verify web app workflows.
Analyze matches
I need my agent to analyze football matches, World Cup data, xG, players, teams, and predictions.
Workflow fit
Scrape, clean, and reuse web data
A practical workflow for agents that crawl public pages, extract clean content, normalize data, and hand it to downstream research or RAG workflows.
Operate and verify web apps
A workflow for agents that navigate products, fill forms, take screenshots, and verify real user flows across web applications.
Inspect, patch, and verify code
A workflow for software agents that inspect repositories, review pull requests, generate tests, and turn findings into shippable patches.
Alternative shortlist
Similar skills that may fit this task.
Open-source LLM-friendly web crawler and scraper
Python scraper based on AI
Control web interfaces with natural language agents
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
Imported by the skill-only GitHub discovery pipeline because it matches agent skill, automation, RAG, or developer-tool signals. Protocol-server projects are excluded from automated imports.
Frameworks & tools
Decision snapshot
12,221 GitHub stars
Audit
Install and adoption review
Agent-proven evidence
Outcome reports after resolve, review, install, and one narrow run.
No agent outcome data yet. The first agent run can report success, setup needs, risk blocks, failure, or not-relevant through /api/agent/outcome.
Install
Free and open source. Review the report before installing into production agents.
Growth loop
Scenario-led draft for Crawlab, ready for a manual X post.
Crawlab: Distributed web crawler admin platform for spiders management regardless of languages and fra... 12.2K stars https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=x
Listing + install path for Crawlab: https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=x Install: npx skills add crawlab-team/crawlab
Listing source
This listing was indexed from public sources and is not marked official until a maintainer claim is approved.
Attribution links to the public repository or creator profile. Creators can claim the listing to update ownership signals.
Claim this skillOwner claim
This Community indexed listing is attributed to crawlab-team but is not marked official yet. Claim it to add a verified owner signal and make future launch, install, and audit updates easier to trust.
Creator backlink kit
Show the canonical listing, current trust and audit signals, and real Agent-Proven evidence where developers evaluate the repository.
[](https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=github&utm_source=github&utm_medium=referral&utm_campaign=creator_badge)
[](https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=github&utm_source=github&utm_medium=referral&utm_campaign=creator_badge)
[](https://www.openagentskill.com/skills/crawlab-team-crawlab/audit)
[](https://www.openagentskill.com/skills/crawlab-team-crawlab?ref=github&utm_source=github&utm_medium=referral&utm_campaign=creator_badge)crawlab-team✓
@crawlab-team
Share whether this skill looks useful for your agent workflow. Aggregated feedback improves rankings over time.
Review then install
Crawl4AI
Open-source LLM-friendly web crawler and scraper
73.1K StarsScrapegraph AI
Python scraper based on AI
27.2K StarsPage Agent
Control web interfaces with natural language agents
28.6K StarsMaxun
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
16.2K StarsPermission surface
no high-risk permission surface in public metadata
Agent outcomes
No agent outcome data yet
Docs
Strong README/SKILL.md context
Risk summary
Install readiness
Permission surface
no high-risk permission surface in public metadata
Agent outcomes
No agent outcome data yet
Docs
Strong README/SKILL.md context
Risk summary
Install readiness
Permission surface
no high-risk permission surface in public metadata
Agent outcomes
No agent outcome data yet
Docs
Strong README/SKILL.md context
Risk summary
Install readiness
Permission surface
no high-risk permission surface in public metadata
Agent outcomes
No agent outcome data yet
Docs
Strong README/SKILL.md context
Risk summary
Install readiness