Harvest WeChat Articles into Clean, Resumable Datasets

Where this fits

You're building an agent that needs to analyze public WeChat articles—say, tracking a competitor's tutorials or aggregating industry insights over the past year. Manually scraping mp.weixin.qq.com is brittle: login expirations, anti-bot walls, and inconsistent HTML break naive scripts. This skill turns that messy process into a structured pipeline: it finds the account via wcx, pulls metadata, filters by date and title, extracts full text with Playwright, and exports Markdown/JSON/CSV—all with checkpointing so you can resume after rate limits or captchas.

Why agents benefit

• Resumable batch harvesting: --batch then --resume restores account, date range, and fulltext policy, so long historical pulls survive interruptions without re-scraping.
• Selective filtering: --title-regex lets agents zero in on tutorial-style posts (e.g., '(教程|手把手|教你|技巧)') before committing to full-text extraction.
• Graceful degradation: When verification or risk-control pages appear, the browser circuit-breaker stops further requests; old trusted bodies are kept and new ones marked partial instead of failing silently.
• Auditable output: Every article's extraction state is recorded—fulltext, partial, or skipped—so downstream pipelines know exactly what's trustworthy.
• Privacy-first defaults: Tokens never enter process args, cookies stay out of command lines, and paid Metaso fallback only runs with explicit --allow-metaso authorization.

Practical scenarios

Build a year-in-review dataset for a niche account

Run harvest_wxmp.py --account "AI前线" --from-date 2024-01-01 --to-date 2024-12-31 --fulltext --batch. After cooldowns, resume with --resume until the local folder contains every article with intact image order. Export to JSON for downstream topic modeling.

Filter only actionable tutorials from a feed

Add --title-regex '(教程|手把手|实操)' to a date-range command. The agent can then feed only those URLs into a summarization pipeline, cutting noise from news posts.

Save a single link without full scraping

Use browser_reader.py <url> to capture one article's content and metadata into your local knowledge base—useful when a client drops a link mid-conversation.

Add it to your agent workflow

Install the skill:

npx skills add zjp1997720/zhijian-skills --skill wxmp-article-harvester

Run the preflight to check dependencies (it won't auto-install anything):

python3 <skill-root>/scripts/preflight.py --json

Then list recent articles without full text:

python3 scripts/harvest_wxmp.py --account "SomeAccount" --limit 50 --no-fulltext

For a deeper workflow, check the skill's page at OpenAgentSkill for community notes and version history.

Compare before adopting

• Quality signals: Does the skill handle WeChat's anti-bot responses gracefully? Does it preserve image order and mark partial extractions?
• Maintenance freshness: Last commit date, open issues about wcx or Playwright breakage, and whether the author responds to PRs.
• Alternatives: Generic wechat-scraper libraries or manual Selenium scripts—compare setup time and resume behavior.
• Workflow fit: Does it output the formats you need (Markdown/JSON/CSV)? Does the --allow-metaso gating align with your cost/security policies?

Why it is worth tracking

With 576 stars and an active maintainer (zjp1997720), this skill shows real community traction for a niche but painful scraping problem. Its design—preflight checks, circuit breakers, resumable batches, and explicit paid-fallback authorization—signals maturity beyond a quick hack. Evaluate it when you need reliable WeChat article harvesting at scale, or when you want a reference for building robust, privacy-aware scraper skills of your own.