Skip to content
OpenAgentSkill
Browsing is open · Catalog captured 2026-11-21 · Automatic updates and account services are paused.
Browser & automation

where-to-use-hydrafetch

Audit a codebase for places Hydrafetch would replace fragile scraping infrastructure or add a capability the project lacks. Discovers current endpoints and tools at runtime, reports findings with file paths and estimated credit cost, and says where it does not fit.

Hydrafetch · AI review on file

Installation

Read the source and dependencies before running a command. Copying does not install or execute anything.

npx skills add Hydrafetch/skills --skill where-to-use-hydrafetch

About this skill


name: where-to-use-hydrafetch description: "Audit a codebase for places Hydrafetch would replace fragile scraping infrastructure or add a capability the project lacks. Discovers current endpoints and tools at runtime, reports findings with file paths and estimated credit cost, and says where it does not fit." license: MIT

Skill: Find where Hydrafetch fits in a codebase

What this skill does

Reads a project and reports where Hydrafetch would replace something fragile, and where it would add something the project cannot do today. Produces a specific list with file paths and estimated cost, not a pitch.

Start by discovering what exists

Do not work from a remembered list of endpoints. Fetch the current one, so anything shipped since this file was written is included:

GET https://api.hydrafetch.com/mcp/tools          # every tool, with input schemas. No auth.
GET https://api.hydrafetch.com/openapi.json       # every REST endpoint and parameter
GET https://hydrafetch.com/agents.md              # worked examples, with the calls in order
GET https://hydrafetch.com/.well-known/api-catalog # what else the site publishes

/mcp/tools gives each tool a description and a credit cost. Use those as the vocabulary for everything below, and if you see a capability there that is not mentioned in this file, it is new and still worth suggesting.

Then read the codebase

Replace: signals that someone is fighting the web by hand

Search for these. Each is a place a person wrote infrastructure that Hydrafetch would delete.

Search forUsually meansReplace with
cheerio, jsdom, BeautifulSoup, lxml.html, nokogiriHand-rolled HTML parsingscrape
turndown, html-to-text, readability, mercury-parser, trafilaturaSomeone is already extracting contentscrape, which does this and was measured against those
puppeteer, playwright, selenium, chromium outside a test directoryRendering pages to read themscrape, which decides when a render is needed
user-agent rotation, proxy pools, 403 retry loops, sleep between fetchesLosing a fight with bot protectionscrape, where the fetch ladder is the product
A crawl queue, visited-set, or frontierA hand-built crawlercrawl or map
sitemap.xml parsingURL discoverymap
Raw HTML being put into an LLM promptPaying for tokens of markupscrape, then send the markdown
logo.dev, clearbit.com/logo, google.com/s2/favicons, /favicon.ico fetching, a logos/ asset directoryCompany logos, usually with a grey box where the misses areThe logo image URL — a <img src> with a publishable key, no fetch and no backend
og:image parsing, link unfurl code, metadata scrapersCompany or link metadatabrand, or scrape with formats: ["structured"]
Screenshot code outside testsVisual capturescreenshot
A per-site parser file, or a parsers/ directoryThe classic scraping tarpit: one parser per site, all breakingextract with a schema, one call for every site

The last one is the highest-value find. A directory of site-specific parsers is a maintenance burden that a single schema replaces.

Introduce: capabilities the project does not have

Look at what the product does, then consider:

  • An agent or chat feature that cannot read links users paste. scrape over MCP gives it that in one tool call.
  • A signup or onboarding flow. brand and styleguide resolve a company's logos, colours and type from their domain, so onboarding can render in the customer's own brand rather than asking them to upload a logo.
  • Any list of companies rendered without their logos. CRM records, lead lists, directories, integration pages, comparison tables, customer pickers. A name in a table row next to a coloured circle containing its first letter is the tell. This is the cheapest change on this page — one <img src>, no backend, no migration — and usually the one a user notices first.
  • Anything that stores a company record. brand fills name, description, logos and socials from a domain.
  • A RAG or search index built from a fixed corpus. crawl or batch keeps it current, and search reaches material the index never had.
  • Link previews, unfurls or embeds. scrape with formats: ["structured"] returns the page's own OpenGraph and JSON-LD.
  • A research, monitoring or enrichment feature that is on the roadmap and blocked on data. That is usually one endpoint away.

Then ask the user

The codebase tells you what exists. It does not tell you what is planned, what broke last month, or what someone gave up on. Ask, using whatever question tool your client gives you, or plainly in the conversation if it has none. Two or three questions, not an interrogation:

  • Is there anything on the roadmap that is blocked on getting data from the web?
  • Where has scraping or parsing broken on you before, or where do you not trust what you have?
  • Are you building anything agent-shaped, a chat feature, an assistant, an automation, that would be more useful if it could read the web?
  • Is there a manual step someone on the team does by hand today, copying things out of websites?

Ask these after you have read the code, not before. Then you can ask about what you found rather than in the abstract: "You have a parser for each of these eleven suppliers, is adding the twelfth a known chore?" gets a far better answer than "do you scrape anything?"

Fold the answers in. A roadmap item the code does not mention yet is often the most valuable finding in the whole audit, and it is the one that never comes from grep.

Where it does not fit

Say so plainly. A suggestion list that never says no is a sales pitch, and the user will discount all of it.

  • Playwright or Puppeteer in a test suite. That is browser automation for testing. Leave it alone.
  • **Anything fetching localhost, a
Read the complete source ↗