Newcrawler
Free Web Scraping Tool with Java
Supply asset profile
Research and knowledge work
Deep research, source comparison, literature review, RAG, knowledge search, and reports.
Scenario
RAG and knowledge
I need my agent to build a RAG workflow over documents and retrieve reliable context.
Agent fit
Claude Code + CLI + Codex
Codex, Claude Code, Cursor, CLI, or custom agents.
Install
Ready
npx skills add speed/newcrawler
Maintenance
stale
3y since push
Risk
Needs review
License is unclear
GitHub quality
587
49/100 Quality · 69/100 Trust
Coverage tags
Review notes
License is unclear · Repository appears stale
Agent adoption scorecard
Trust, audit, and install readiness at a glance
These scores combine public repository metadata, OpenAgentSkill review signals, maintenance freshness, and install readiness. They are a shortlist signal, not a replacement for human review.
Quality
Needs reviewInspect the repository carefully before adding it to an agent workflow.
Trust
Sandbox onlyUseful candidate with missing or mixed trust signals. Keep it in an isolated workspace until the outcome loop proves task fit.
Audit
Needs reviewA machine-readable review of install readiness, security metadata, maintenance, and adoption risk.
OpenAgentSkill Trust Score v5
Human review before install
Run only in a sandbox and compare close alternatives before using it for real work.
Stars
587 GitHub stars
Repo activity
587 stars, 112 forks
Maintenance
3y since push
License
Unknown
Install
npx skills add speed/newcrawler
Install safety
standard package or runtime install path
Permission surface
filesystem or document access
Agent outcomes
No agent outcome data yet
Docs
Usable metadata, review docs
Risk summary
Review before production
- License is unclear
- Repository looks stale
- Quality score needs review
- Recent maintenance: 3y since push
Install readiness
Install path available
- Install path is available
- Repository evidence is available
- License is unclear
- No Agent Proven outcome evidence yet
Agent-readable metadata
Machine-readable decision data for this skill.
Use this block or the embedded JSON to decide whether an agent should install this skill, choose an alternative, or ask for human review first.
Suited tasks
- Web scraping workflows
- Claude Code teams
- teams that value GitHub adoption signals
- Crawl target URLs
Suited agents
Install decision
- Command
- npx skills add speed/newcrawler
- Policy
- review
- Human review
- yes
Trust and risk
- Trust
- 61/100
- Audit
- 61/100
- Risk level
- Needs review
Outcome loop
- Endpoint
- /api/agent/outcome
- Event ID
- resolve
- Outcomes
- 5
Do not use when
- teams that require actively maintained dependencies
- production agents without a repository review
- Repository looks stale
- License is unclear
- Repository appears stale
Agent safety v2
45/100 · Avoid automatic install
Sparse or mixed signals. Useful for discovery, but not for autonomous installation.
Test manually in an isolated workspace and compare against safer alternatives.
medium
Network access
Skill likely fetches remote pages, APIs, repositories, or external services.
medium
Filesystem access
Skill may read or write project files, documents, generated artifacts, or local workspace state.
- License is unclear
Install targets
Install this skill in your agent workflow
Use the public install endpoint to fetch the command, safety checklist, target prompts, and canonical links for this skill.
OpenAgentSkill CLI
Use the registry command when your workflow supports the OpenAgentSkill installer.
$ npx skills add speed/newcrawlerAgent resolve plan
Let an agent verify fit before installing.
The Resolve API returns the selected skill, alternatives, safety policy, audit notes, install target, and copy-paste prompt an agent can follow without scraping this page.
Open JSON
/api/agent/resolve?task=Use%20Newcrawler%20for%20an%20agent%20workflow&agent=codex&max_risk=medium
Resolve text
/api/agent/resolve?task=Use%20Newcrawler%20for%20an%20agent%20workflow&agent=codex&max_risk=medium&format=text
Install handoff
/api/skills/speed-newcrawler/install
Agent should check
- Task fit and alternatives from Resolve API.
- Audit score, trust score, and safety policy warnings.
- Install target compatibility for Codex, Claude Code, Cursor, or CLI.
Copy prompt
Task: Use Newcrawler in this workspace.
Resolve first: https://www.openagentskill.com/api/agent/resolve?task=Use%20Newcrawler%20for%20an%20agent%20workflow&agent=codex&max_risk=medium
Review install handoff: https://www.openagentskill.com/api/skills/speed-newcrawler/install
Install command: npx skills add speed/newcrawler
Before running it, summarize audit warnings, required permissions, and the fallback skill if install is risky.Agent handoff
Give an agent the install path, not another directory page.
Use the public install endpoint to fetch the command, safety checklist, target prompts, and canonical links for this skill.
Install handoff
/api/skills/speed-newcrawler/install
LLM text format
/api/skills/speed-newcrawler/install?format=text
Find alternatives
/api/skills/search?q=Newcrawler&limit=3
Agent prompt
Use Newcrawler for this task. Review https://www.openagentskill.com/api/skills/speed-newcrawler/install, then install with: npx skills add speed/newcrawlerRegistry metadata
Agent-readable profile for automatic skill selection.
This page exposes the same decision, trust, audit, use-case, and install signals through the Registry API, so agents can rank this skill without scraping the UI.
Manifest
/api/registry/manifest/speed-newcrawler
LLM text
/api/registry/manifest/speed-newcrawler?format=text
Install alias
/api/registry/install/speed-newcrawler
Recommend
/api/registry/recommend?task=Use%20Newcrawler%20in%20an%20agent%20workflow&limit=3
Agent fit
Web scraping
Use-case tags
Platforms
JavaScript, Crawler, Claude Code
Audit report
Needs review · 61/100
A machine-readable review of install readiness, security metadata, maintenance, and adoption risk.
Agent decision cockpit
Needs validation for Web scraping
Do a manual repository review before adding this to an agent workflow.
Role in stack
Needs validation
Primary fit
Web scraping
Trust label
Needs manual review
Install path
Command ready
Use when
- Web scraping workflows
- Claude Code teams
- teams that value GitHub adoption signals
Evidence
- 587 GitHub stars
- install command or GitHub repo available
- 49/100 quality profile
- 5 OpenAgentSkill engagement events
review first
- Repository looks stale
Implementation path
- 1Install it in a sandbox agent and run one Web scraping task end to end.
- 2Compare output quality, latency, and failure behavior against at least one alternative.
- 3Promote it into production only after reviewing repository permissions, license, and maintenance signals.
Trust profile
Sandbox only
Useful candidate with missing or mixed trust signals. Keep it in an isolated workspace until the outcome loop proves task fit.
GitHub adoption
INFO587 GitHub stars
Stars/forks activity
INFO587 stars, 112 forks; issue activity unavailable in current metadata
Recent maintenance
FIX3y since push
License clarity
CHECKUnknown
Good signals
- AI review approved
- Install path is available
- Repository evidence is available
- Meaningful GitHub adoption signal
- Install command has no obvious high-risk pattern
- Outcome loop is ready but needs first real agent run
Review before install
- License is unclear
- Repository looks stale
- Quality score needs review
- Recent maintenance: 3y since push
- License clarity: Unknown
- No real agent outcome reports yet
- Human review required before unattended installation
Recommended action
Run only in a sandbox and compare close alternatives before using it for real work.
Quality profile
Needs review candidate for agent workflows
Inspect the repository carefully before adding it to an agent workflow.
Workflow fit
Use this skill in these scenarios
Collect structured data
Web scraping
I need my agent to scrape websites and extract structured data from pages.
Build and ship code
Coding agents
I need a coding agent that can understand a repository, edit code, and review pull requests.
Search private knowledge
RAG and knowledge
I need my agent to build a RAG workflow over documents and retrieve reliable context.
Workflow fit
Add it to a complete workflow
Scrape, clean, and reuse web data
Web data pipeline
A practical workflow for agents that crawl public pages, extract clean content, normalize data, and hand it to downstream research or RAG workflows.
Turn skills into distribution
Content growth agent
A workflow for turning newly indexed skills into SEO briefs, social drafts, comparison pages, and reusable publishing workflows.
Inspect, patch, and verify code
Coding review agent
A workflow for software agents that inspect repositories, review pull requests, generate tests, and turn findings into shippable patches.
Alternative shortlist
Compare before you install
Similar skills that may fit this task.
Crawl4AI
Open-source LLM-friendly web crawler and scraper
Scrapling
Adaptive web scraping for agent data collection
Scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
EasySpider
A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。
Overview
Free Web Scraping Tool with Java
Imported by the skill-only GitHub discovery pipeline because it matches agent skill, automation, domain workflow, RAG, document-processing, data, finance, security, or developer-tool signals. Protocol-server projects are excluded from automated imports.
Platform compatibility
Technical details
- Version
- 1.0.0
- License
- Unknown
- Last updated
- Jul 19, 2026
- Published
- May 23, 2026
Frameworks & tools
Decision snapshot
Needs validation
587 GitHub stars
Audit
Install review
Install and adoption review
- Security
- 80/100
- Maintenance
- 20/100
- Install
- 92/100
Agent-proven evidence
Agent-proven evidence
Outcome reports after resolve, review, install, and one narrow run.
- Success rate
- —
- Recent failure
- —
- Outcomes
- 0
- Output quality
- —
- Failed
- 0
- Not relevant
- 0
- Installs
- 0
- Risk blocked
- 0
- Setup needed
- 0
- Production
- 0
No agent outcome data yet. The first agent run can report success, setup needs, risk blocks, failure, or not-relevant through /api/agent/outcome.
Install
Add to agent workflow
Free and open source. Review the report before installing into production agents.
Growth loop
Share kit
Scenario-led draft for Newcrawler, ready for a manual X post.
Before you hand an agent a web workflow, give it a repeatable starting point. Newcrawler: Free Web Scraping Tool with Java 587 stars https://www.openagentskill.com/skills/speed-newcrawler?ref=x
Optional reply with install command
Listing + install path for Newcrawler: https://www.openagentskill.com/skills/speed-newcrawler?ref=x Install: npx skills add speed/newcrawler
Listing source
Community indexed
This listing was indexed from public sources and is not marked official until a maintainer claim is approved.
- Creator
- speed
- Source
- speed/newcrawler
- Indexed by
- OpenAgentSkill community index
Attribution links to the public repository or creator profile. Creators can claim the listing to update ownership signals.
Claim this skillOwner claim
Claim this skill listing
This Community indexed listing is attributed to speed but is not marked official yet. Claim it to add a verified owner signal and make future launch, install, and audit updates easier to trust.
Creator backlink kit
Add the evidence badges to your README
Show the canonical listing, current trust and audit signals, and real Agent-Proven evidence where developers evaluate the repository.
[](https://www.openagentskill.com/skills/speed-newcrawler)
[](https://www.openagentskill.com/skills/speed-newcrawler)
[](https://www.openagentskill.com/skills/speed-newcrawler/audit)
[](https://www.openagentskill.com/skills/speed-newcrawler)Author
speed
@speed
Platform fit
Health signals
- GitHub stars
- 587
- Quality score
- 38/100
- Last GitHub push
- Nov 25, 2023
- Framework hints
- 2
- OpenAgentSkill views
- 3
- Install copies
- 0
- Outbound clicks
- 0
Community signal
Share whether this skill looks useful for your agent workflow. Aggregated feedback improves rankings over time.
Trust & safety
Sandbox only
- GitHub adoption587 GitHub starsINFO
- Stars/forks activity587 stars, 112 forks; issue activity unavailable in current metadataINFO
- Recent maintenance3y since pushFIX
- License clarityUnknownCHECK
- README/SKILL.md completenessPublic metadata needs stronger README/SKILL.md contextINFO
- Dependency/runtime riskexternal package install surfaceINFO
Related skills
Crawl4AI
Open-source LLM-friendly web crawler and scraper
73.1K Stars · 31.0K InstallsScrapling
Adaptive web scraping for agent data collection
70.0K Stars · 0 InstallsScrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
62.5K Stars · 0 InstallsEasySpider
A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。
44.1K Stars · 0 Installs