review-clawhub-profile-proposal
Use when an OpenClaw maintainer or owner is reviewing a ClawHub malicious-skill profile proposal PR: checking `proposals/<GHSA-ID>/clawscan.yml`, reading the private vulnerability context without leaking it, running the SkillTrustBench Profile Gate or equivalent local benchmark, updating the accepted baseline, and promoting an accepted candidate into `internal/profiles/clawhub/clawscan.yml`.
openclaw · 已有静态检查记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add openclaw/clawscan --skill review-clawhub-profile-proposal技能说明
name: review-clawhub-profile-proposal
description: >-
Use when an OpenClaw maintainer or owner is reviewing a ClawHub malicious-skill
profile proposal PR: checking proposals/<GHSA-ID>/clawscan.yml, reading the
private vulnerability context without leaking it, running the SkillTrustBench
Profile Gate or equivalent local benchmark, updating the accepted baseline,
and promoting an accepted candidate into internal/profiles/clawhub/clawscan.yml.
Review ClawHub Profile Proposal
Overview
Use this owner-side workflow for a public PR that proposes a candidate ClawHub
profile at proposals/<GHSA-ID>/clawscan.yml after a private vulnerability
report. The goal is to validate the candidate, keep sensitive details private,
and, if accepted, promote the public profile behavior into the bundled
internal/profiles/clawhub/clawscan.yml.
Safety Boundary
- Treat the GitHub private vulnerability report as the source for sensitive malicious skill details.
- Do not paste live exploit details, private report text, private artifacts, or suspicious skill payloads into public PR comments, public docs, commit messages, or committed baseline summaries.
- The public proposal PR should start with only
proposals/<GHSA-ID>/clawscan.yml. Do not trust it until reviewed. - Do not run the suspicious skill. ClawScan scans skill files as data; it should not execute the skill's behavior.
- Do not promote a candidate unless the private report, proposal diff, and benchmark proof all line up.
Review Workflow
-
Read the current repo and PR state.
- Check
git status --short --branchbefore editing. - Fetch the PR branch and inspect the changed files.
- Confirm the public PR initially contains only
proposals/<GHSA-ID>/clawscan.ymlplus any baseline summary commit produced by the maintainer gate. - Confirm the proposal file defines a
clawhubprofile. - Confirm the proposal does not edit official bundled profile files yet:
internal/profiles/clawhub/clawscan.yml,internal/profiles/clawhub/prompt.md, orinternal/profiles/clawhub/output.schema.json.
- Check
-
Read the private report privately.
Use the private vulnerability report only to understand:
- what malicious behavior must be caught
- why the current built-in
clawhubprofile misses or under-detects it - what evidence should be preserved privately
- whether any proposal text or config comments leak sensitive details
Keep public comments high-level, such as "validated against the private report" or "needs private-case follow-up"; do not quote private details.
-
Validate the candidate profile.
Prefer the manual GitHub Actions workflow when available:
SkillTrustBench Profile GateDispatch it with the PR number and proposal path. It should run:
clawscan benchmark SkillTrustBench \ --ids https://huggingface.co/datasets/cuhk-zhuque/SkillTrustBench-results/resolve/main/data/evaluation_subset_10pct.jsonl \ --config proposals/<GHSA-ID>/clawscan.yml \ --profile clawhub \ --output ./artifacts/skilltrustbench-candidate.jsonIf running locally, use the same command. The
--idssource is the public SkillTrustBench leaderboard subset and is mutually exclusive with--limitand--offset. -
Review the benchmark artifact.
Check:
- the artifact is full JSON and preserved as a workflow artifact or private maintainer artifact
benchmark.idiscuhk-zhuque/SkillTrustBenchbenchmark.splitisbenchmarkbenchmark.idsCountis556benchmark.idsSha256matches the planned subset hash903a036e4b7b16ee28e22d5d9db57a00b3764cfe41e43144acad67921e5196c2- scanner and judge statuses are acceptable
- evaluation metrics are not an unacceptable regression
- the candidate catches the private reported behavior when private proof is available
-
Update the accepted baseline summary.
Use the repo script so the compact baseline is generated from the full candidate artifact:
go run ./scripts/update-skilltrustbench-baseline \ --artifact ./artifacts/skilltrustbench-candidate.json \ --output benchmarks/skilltrustbench-leaderboard-10pct/<YYYY-MM-DD>.json \ --profile clawhub \ --profile-source proposals/<GHSA-ID>/clawscan.yml \ --subset-case-ids-sha256 903a036e4b7b16ee28e22d5d9db57a00b3764cfe41e43144acad67921e5196c2 \ --workflow-url <workflow-url>If the PR merges, the newest dated JSON file in
benchmarks/skilltrustbench-leaderboard-10pct/is the latest accepted baseline for the bundledclawhubprofile. The script fails if the candidate artifact's selected-ID hash does not match the planned subset. No post-merge rerun is required. -
Decide.
If rejected:
- leave a public PR comment with non-sensitive reasons
- keep details that identify the malicious payload in the private report
- do not edit official bundled profile files
If accepted:
- promote the accepted public
clawhubprofile behavior intointernal/profiles/clawhub/clawscan.yml - add a dated baseline under
benchmarks/skilltrustbench-leaderboard-10pct/from the candidate artifact in the same PR - preserve or remove
proposals/<GHSA-ID>/clawscan.ymlaccording to the issue/PR instruction; default to preserving it as public proposal trail unless the maintainer explicitly chooses to remove it - keep prompt/schema changes maintainer-owned; edit
internal/profiles/clawhub/prompt.mdorinternal/profiles/clawhub/output.schema.jsononly when that is the accepted change
-
Verify the promoted built-in profile.
Run at least:
go test -count=1 ./... go vet ./... go run ./cmd/clawscan profiles -v go run ./cmd/clawscan --helpFor benchmark proof after promotion, run:
clawscan benchmark SkillTrustBench