跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
文档处理

cp-skill-snapshot-web

Snapshot a URL into the local kb/sources/.snapshots/ cache, routing GitHub, X/Twitter, PDF, and ordinary web sources to the appropriate capture path.

zby · 已有 AI 审核记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add zby/commonplace --skill cp-skill-snapshot-web

技能说明


name: cp-skill-snapshot-web description: Snapshot a URL into the local kb/sources/.snapshots/ cache, routing GitHub, X/Twitter, PDF, and ordinary web sources to the appropriate capture path. type: kb/types/instruction.md user-invocable: true allowed-tools: Read, Write, Grep, Glob, Bash context: fork model: sonnet argument-hint: "[url] — URL to snapshot (web page, PDF, GitHub issue/PR, or X/Twitter post)"

EXECUTE NOW

Target: $ARGUMENTS

If no URL provided, ask the user for one.

If URL provided, start Step 1 immediately.

START NOW.


Step 1: Verify Local Storage and Check for Duplicates

Keep the provided URL as source_url. Verify that kb/sources/.snapshots/ is ignored by the project. The shipped scaffold does this through kb/sources/.gitignore. If the directory is not ignored, stop before writing and report the missing rule.

Use Grep to search for an exact frontmatter source: {source_url} in existing Markdown files in kb/sources/.snapshots/. If found, compute the SHA-256 of the exact file bytes, tell the user, and stop:

Already snapshotted: kb/sources/.snapshots/{filename} SHA-256: {64-character lowercase checksum}

Step 2: Route by URL Type

Detect the source_url type and branch:

  • GitHub issue/PR (github.com/.../issues/N or github.com/.../pull/N) → Step 2a
  • X/Twitter (x.com/.../status/... or twitter.com/.../status/...) → Step 2b
  • arXiv abstract page (arxiv.org/abs/...) → Step 2c
  • PDF (URL ends in .pdf, or arxiv.org/pdf/) → Step 2c
  • Everything else → Step 2d

Step 2a: GitHub Issue/PR

Run:

commonplace-github-snapshot "{source_url}"

Parse either the Snapshot saved: or Already snapshotted: line from the output to get the file path. Tell the user and stop — the script handles metadata, formatting, and saving.

Step 2b: X/Twitter Post

Run:

commonplace-x-snapshot "{source_url}"

Parse either the Snapshot saved: or Already snapshotted: line from the output to get the file path. Tell the user and stop — the script handles metadata, formatting, and saving.

Step 2c: Resolve and Fetch PDF

Verify that the PDF capture prerequisites are available:

command -v curl
command -v pdfinfo
command -v pdftotext

If any command is missing, go to Step 3. Do not probe for an alternative converter.

Set pdf_url:

  • For an arXiv abstract URL, replace /abs/ with /pdf/ and discard any query string or fragment. Preserve an explicit terminal version such as v1. If the abstract URL has no terminal version, leave the PDF URL unversioned so arXiv serves the latest paper version. For example, https://arxiv.org/abs/2606.03979 becomes https://arxiv.org/pdf/2606.03979. Do not route the abstract page through ordinary HTML capture.
  • For an existing PDF URL, use source_url unchanged.

Run this as one Bash invocation. Retain the printed directory path as {snapshot_tmp}:

set -e
snapshot_tmp=$(mktemp -d)
printf 'Snapshot temp: %s\n' "$snapshot_tmp"
curl -fsSL -o "$snapshot_tmp/source.pdf" "{pdf_url}"
pdfinfo -isodates "$snapshot_tmp/source.pdf" > "$snapshot_tmp/pdfinfo.txt"
pdfinfo -meta "$snapshot_tmp/source.pdf" > "$snapshot_tmp/pdfmeta.xml" || true
pdftotext -enc UTF-8 -eol unix -nopgbrk \
  "$snapshot_tmp/source.pdf" "$snapshot_tmp/extracted.txt"

Use Read to inspect pdfinfo.txt, non-empty pdfmeta.xml, and a bounded beginning of extracted.txt. DOI metadata inspection is best effort and its failure does not make an otherwise successful capture fail. Treat PDF metadata fields as leads, not as authority: confirm the title and authors against the document text when available. Use Grep plus bounded Read ranges to locate an abstract, executive summary, introduction, or the source document's own DOI when the beginning does not supply enough metadata. Do not treat a DOI found only in the references as the paper's DOI. Do not read the whole extracted file merely to copy it. If extracted.txt is empty or contains no substantive text, go to Step 3.

Set capture_method to pdftotext, set body_file to {snapshot_tmp}/extracted.txt, and go to Step 4.

Step 2d: Fetch Web Page

Verify that the HTML capture prerequisites are available:

command -v trafilatura

If the command is missing, go to Step 3. Do not probe for another HTML converter.

Run this as one Bash invocation to download and extract the page. Retain the printed directory path as {snapshot_tmp}:

set -e
snapshot_tmp=$(mktemp -d)
printf 'Snapshot temp: %s\n' "$snapshot_tmp"
trafilatura -u "{source_url}" \
  --markdown --with-metadata --links --no-comments --recall \
  --backup-dir "$snapshot_tmp/raw" \
  > "$snapshot_tmp/extracted.md"

Use Read to inspect only the leading metadata and a bounded beginning of extracted.md. Its leading YAML block, when present, is Trafilatura metadata: retain it as input to Step 4 but do not copy that block into the snapshot body. Trafilatura also retains its downloaded HTML as a gzip file under {snapshot_tmp}/raw/. When gzip is available, decompress that file within {snapshot_tmp} and use Grep with bounded output to inspect article-level DOI metadata such as citation_doi, dc.identifier, prism.doi, or a JSON-LD doi property. DOI inspection is best effort: inability to inspect the raw HTML does not make an otherwise successful capture fail. Strip that block locally without re-emitting the document:

awk '
NR == 1 && $0 == "---" { in_metadata = 1; next }
in_metadata && $0 == "---" { in_metadata = 0; next }
!in_metadata { print }
' "{snapshot_tmp}/extracted.md" > "{snapshot_tmp}/body.md"

If body.md is empty or contains no substantive main content, go to Step 3.

Set capture_method to trafilatura, set body_file to {snapshot_tmp}/body.md, and go to Step 4.

Step 3: Handle Failures

If any fetch or extraction method fails (missing prerequisite, curl e

阅读完整来源 ↗