Star200+

Crawl4AI

REVIEW · 74
Registry indexed

Open-source LLM-friendly web crawler and scraper for agent workflows.

Verified installs0
Stars66.0K
Version1.0.0
Quality100/100 · Excellent
Trust74/100 · Sandbox only
Audit89/100 · Safe to try

Supply asset profile

Research and knowledge work

Deep research, source comparison, literature review, RAG, knowledge search, and reports.

Browse track

Scenario

Research agents

I need my agent to research a topic, compare sources, and produce a concise report.

Agent fit

Claude Code + OpenAI Agents + Cursor

Codex, Claude Code, Cursor, CLI, or custom agents.

Install

Ready

npx skills add unclecode/crawl4ai

Maintenance

active

3mo since push

Risk

Safe to try

Documentation summary is thin

GitHub quality

66K

100/100 Quality · 82/100 Trust

Coverage tags

ResearchResearch agentsWeb Scrapingcrawlermarkdown

Review notes

Documentation summary is thin · README/SKILL.md completeness: Public metadata needs stronger README/SKILL.md context

Agent adoption scorecard

Trust, audit, and install readiness at a glance

These scores combine public repository metadata, OpenAgentSkill review signals, maintenance freshness, and install readiness. They are a shortlist signal, not a replacement for human review.

Quality

Excellent
100

High-confidence pick with strong adoption and healthy maintenance signals.

Trust

Sandbox only
74

Useful candidate with missing or mixed trust signals. Keep it in an isolated workspace until the outcome loop proves task fit.

Audit

Safe to try
89

A machine-readable review of install readiness, security metadata, maintenance, and adoption risk.

OpenAgentSkill Trust Score v5

Human review before install

Run only in a sandbox and compare close alternatives before using it for real work.

CodexClaude CodeCursorOpenAgentSkill CLI

Stars

66K GitHub stars

Repo activity

66K stars, 0 forks

Maintenance

3mo since push

License

Apache-2.0

Install

npx skills add unclecode/crawl4ai

Install safety

standard package or runtime install path

Permission surface

no high-risk permission surface in public metadata

Agent outcomes

No agent outcome data yet

Docs

Thin public metadata

Risk summary

Review before production

  • Documentation summary is thin
  • README/SKILL.md completeness: Public metadata needs stronger README/SKILL.md context

Install readiness

Install path available

  • Install path is available
  • Repository evidence is available
  • License is declared
  • No Agent Proven outcome evidence yet

Agent-readable metadata

Machine-readable decision data for this skill.

Use this block or the embedded JSON to decide whether an agent should install this skill, choose an alternative, or ask for human review first.

Open JSON

Suited tasks

  • Web scraping workflows
  • Claude Code teams
  • teams that value GitHub adoption signals
  • Crawl target URLs

Suited agents

CodexClaude CodeCursorOpenAgentSkill CLIOpenAI AgentsCLI

Install decision

Command
npx skills add unclecode/crawl4ai
Policy
review
Human review
yes

Trust and risk

Trust
74/100
Audit
89/100
Risk level
Safe to try

Outcome loop

Endpoint
/api/agent/outcome
Event ID
resolve
Outcomes
5

Install command

npx skills add unclecode/crawl4ai
Public auditEval reportResolve APIInstall handoff

Do not use when

  • teams that need a vendor-supported SLA
  • high-compliance environments without internal security review
  • No OpenAgentSkill engagement data yet
  • Documentation summary is thin
  • README/SKILL.md completeness: Public metadata needs stronger README/SKILL.md context
No close alternatives indexed yet.

Agent safety v2

73/100 · Review before install

Reviewedreview

Good audit and safety signals with no high-risk permission hints in public metadata.

Review the audit page, then allow agent install in a sandboxed workflow.

Resolve via API

medium

Network access

Skill likely fetches remote pages, APIs, repositories, or external services.

medium

Filesystem access

Skill may read or write project files, documents, generated artifacts, or local workspace state.

  • Documentation summary is thin

Install targets

Install this skill in your agent workflow

Use the public install endpoint to fetch the command, safety checklist, target prompts, and canonical links for this skill.

skill install

OpenAgentSkill CLI

Resolve policy, run the source installer safely, and report a verified install receipt.

$ npx --yes https://github.com/Leon-Drq/openagentskill/releases/download/cli-v0.2.1/openagentskill-0.2.1.tgz install crawl4ai

Agent resolve plan

Let an agent verify fit before installing.

The Resolve API returns the selected skill, alternatives, safety policy, audit notes, install target, and copy-paste prompt an agent can follow without scraping this page.

Open text plan

Open JSON

/api/agent/resolve?task=Use%20Crawl4AI%20for%20an%20agent%20workflow&agent=codex&max_risk=medium

Resolve text

/api/agent/resolve?task=Use%20Crawl4AI%20for%20an%20agent%20workflow&agent=codex&max_risk=medium&format=text

Install handoff

/api/skills/crawl4ai/install

Agent should check

  • Task fit and alternatives from Resolve API.
  • Audit score, trust score, and safety policy warnings.
  • Install target compatibility for Codex, Claude Code, Cursor, or CLI.

Copy prompt

Task: Use Crawl4AI in this workspace.
Resolve first: https://www.openagentskill.com/api/agent/resolve?task=Use%20Crawl4AI%20for%20an%20agent%20workflow&agent=codex&max_risk=medium
Review install handoff: https://www.openagentskill.com/api/skills/crawl4ai/install
Install command: npx skills add unclecode/crawl4ai
Before running it, summarize audit warnings, required permissions, and the fallback skill if install is risky.

Agent handoff

Give an agent the install path, not another directory page.

Use the public install endpoint to fetch the command, safety checklist, target prompts, and canonical links for this skill.

Open install API

Install handoff

/api/skills/crawl4ai/install

LLM text format

/api/skills/crawl4ai/install?format=text

Find alternatives

/api/skills/search?q=Crawl4AI&limit=3

Agent prompt

Use Crawl4AI for this task. Review https://www.openagentskill.com/api/skills/crawl4ai/install, then install with: npx skills add unclecode/crawl4ai

Registry metadata

Agent-readable profile for automatic skill selection.

This page exposes the same decision, trust, audit, use-case, and install signals through the Registry API, so agents can rank this skill without scraping the UI.

Open manifest

Manifest

/api/registry/manifest/crawl4ai

LLM text

/api/registry/manifest/crawl4ai?format=text

Install alias

/api/registry/install/crawl4ai

Recommend

/api/registry/recommend?task=Use%20Crawl4AI%20in%20an%20agent%20workflow&limit=3

Agent fit

100/100

Web scraping

Use-case tags

Web scrapingResearch agentsCoding agents

Platforms

Codex, Claude Code, Cursor, OpenAI Agents

Audit report

Safe to try · 89/100

A machine-readable review of install readiness, security metadata, maintenance, and adoption risk.

View audit reportView eval report

Agent decision cockpit

Primary pick for Web scraping

Use this as a leading candidate, then validate the README and install path in your own agent stack.

100
Readiness
Adopt
Stage

Role in stack

Primary pick

Primary fit

Web scraping

Trust label

Production-ready

Install path

Command ready

Use when

  • Web scraping workflows
  • Claude Code teams
  • teams that value GitHub adoption signals

Evidence

  • 66,000 GitHub stars
  • recent repository activity
  • install command or GitHub repo available
  • 100/100 quality profile

review first

  • No OpenAgentSkill engagement data yet

Implementation path

  1. 1Install it in a sandbox agent and run one Web scraping task end to end.
  2. 2Compare output quality, latency, and failure behavior against at least one alternative.
  3. 3Promote it into production only after reviewing repository permissions, license, and maintenance signals.

Trust profile

Sandbox only

Useful candidate with missing or mixed trust signals. Keep it in an isolated workspace until the outcome loop proves task fit.

74
OpenAgentSkill Trust Score

GitHub adoption

PASS

66K GitHub stars

Stars/forks activity

INFO

66K stars, 0 forks; issue activity unavailable in current metadata

Recent maintenance

PASS

3mo since push

License clarity

PASS

Apache-2.0

Good signals

  • AI review approved
  • Install path is available
  • Repository evidence is available
  • Recently maintained repository
  • Large GitHub adoption signal
  • High quality score
  • Install command has no obvious high-risk pattern
  • Outcome loop is ready but needs first real agent run

Review before install

  • Documentation summary is thin
  • README/SKILL.md completeness: Public metadata needs stronger README/SKILL.md context
  • No real agent outcome reports yet
  • Human review required before unattended installation

Recommended action

Run only in a sandbox and compare close alternatives before using it for real work.

Quality profile

Excellent candidate for agent workflows

High-confidence pick with strong adoption and healthy maintenance signals.

100
GitHub stars
66K
Freshness
3mo ago
Install ready
Yes
License
Apache-2.0

Workflow fit

Use this skill in these scenarios

Collect structured data

Web scraping

I need my agent to scrape websites and extract structured data from pages.

Investigate faster

Research agents

I need my agent to research a topic, compare sources, and produce a concise report.

Build and ship code

Coding agents

I need a coding agent that can understand a repository, edit code, and review pull requests.

Workflow fit

Add it to a complete workflow

Scrape, clean, and reuse web data

Web data pipeline

A practical workflow for agents that crawl public pages, extract clean content, normalize data, and hand it to downstream research or RAG workflows.

Find, compare, and synthesize

Research report agent

A workflow for agents that gather sources, compare claims, summarize long material, and draft useful research briefs.

Turn skills into distribution

Content growth agent

A workflow for turning newly indexed skills into SEO briefs, social drafts, comparison pages, and reusable publishing workflows.

Overview

Open-source LLM-friendly web crawler and scraper for agent workflows.

Platform compatibility

codexFULL
claude-codeFULL
cursorFULL

Technical details

Version
1.0.0
License
Apache-2.0
Last updated
Jun 1, 2026
Published
Jun 1, 2026

Frameworks & tools

CodexClaude CodeCursor

Decision snapshot

Primary pick

100
Ready
Adopt
Stage

66,000 GitHub stars

Audit

Install review

Install and adoption review

89
Safe to try
Security
85/100
Maintenance
88/100
Install
92/100
Open full auditView eval report

Agent-proven evidence

Agent-proven evidence

Outcome reports after resolve, review, install, and one narrow run.

0
Proven
Needs first agent runAuto-install: review firstLast: Unknown
Success rate
—
Recent failure
—
Outcomes
0
Output quality
—
Failed
0
Not relevant
0
Installs
0
Risk blocked
0
Setup needed
0
Production
0

No agent outcome data yet. The first agent run can report success, setup needs, risk blocks, failure, or not-relevant through /api/agent/outcome.

Agent-Proven rankingOutcome contract

Install

Add to agent workflow

Free and open source. Review the report before installing into production agents.

Compare AlternativesAuto-resolve PlanView on GitHubDocumentation

Growth loop

Share kit

X

Scenario-led draft for Crawl4AI, ready for a manual X post.

Curator note
A practical pick for a web workflow:

Crawl4AI: Open-source LLM-friendly web crawler and scraper for agent workflows.

66.0K stars

https://www.openagentskill.com/skills/crawl4ai?ref=x
Open X draft
Optional reply with install command
Listing + install path for Crawl4AI:
https://www.openagentskill.com/skills/crawl4ai?ref=x

Install: npx skills add unclecode/crawl4ai
Open reply draft

Listing source

Registry indexed

Claimable

This listing was indexed from public sources and is not marked official until a maintainer claim is approved.

Creator
unclecode
Indexed by
OpenAgentSkill community index

Attribution links to the public repository or creator profile. Creators can claim the listing to update ownership signals.

Claim this skill

Owner claim

Claim this skill listing

This Registry indexed listing is attributed to unclecode but is not marked official yet. Claim it to add a verified owner signal and make future launch, install, and audit updates easier to trust.

Creator backlink kit

Add the evidence badges to your README

Show the canonical listing, current trust and audit signals, and real Agent-Proven evidence where developers evaluate the repository.

[![Listed on OpenAgentSkill](https://www.openagentskill.com/api/badge/crawl4ai?metric=listed&label=Listed)](https://www.openagentskill.com/skills/crawl4ai)
[![OpenAgentSkill Trust](https://www.openagentskill.com/api/badge/crawl4ai?metric=trust&label=Trust)](https://www.openagentskill.com/skills/crawl4ai)
[![OpenAgentSkill Audit](https://www.openagentskill.com/api/badge/crawl4ai?metric=audit&label=Audit)](https://www.openagentskill.com/skills/crawl4ai/audit)
[![Agent Proven](https://www.openagentskill.com/api/badge/crawl4ai?metric=proven&label=Agent%20Proven)](https://www.openagentskill.com/skills/crawl4ai)
Preview badge Open audit Creator kit

Author

U

unclecode

@unclecode

Tags

web scrapingcrawlerresearchmarkdown

Platform fit

Claude CodeOpenAI AgentsCursor

Health signals

GitHub stars
66.0K
Quality score
96/100
Last GitHub push
Jun 1, 2026
Framework hints
3
OpenAgentSkill views
0
Install copies
0
Outbound clicks
0

Community signal

Share whether this skill looks useful for your agent workflow. Aggregated feedback improves rankings over time.

Trust & safety

Sandbox only

74
  • GitHub adoption66K GitHub starsPASS
  • Stars/forks activity66K stars, 0 forks; issue activity unavailable in current metadataINFO
  • Recent maintenance3mo since pushPASS
  • License clarityApache-2.0PASS
  • README/SKILL.md completenessPublic metadata needs stronger README/SKILL.md contextCHECK
  • Dependency/runtime riskno major dependency risk hints in public metadataPASS
InstallView on GitHub