Caliper
Lightweight evaluation harness for agent skills to track success rates across multiple agent runtimes.
edonadei · 已有 AI 审核记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add edonadei/caliper技能说明
Caliper: Know if your agent skill actually works
Caliper is a lightweight evaluation harness for agent skills. Write a short spec of what "good" looks like, run it again and again, and get a success rate you can track. It also runs every task without the skill, so you learn whether the skill is doing the work or the base agent would have passed anyway. Works with the agent you already use: Claude Code, Codex, Pi, or Hermes.
Teach your agent to evaluate:
npx skills@latest add edonadei/caliper
Or run it yourself:
caliper run commit-commands.eval.yaml --k 3 --baseline
You write a spec, a few lines of YAML describing what "working" means, which you hand-write or have /grill-skill generate for you. With --baseline, Caliper runs each task with and without the skill and diffs the two runs task by task:
Agent skills are hard to test. A skill that works on your machine, on this prompt, today, might fail tomorrow after a model update or a one-line prompt edit. Caliper makes reliability measurable: define what success looks like, run the skill repeatedly, and get a success rate you can track over time.
Use Caliper to answer questions like:
- Did my prompt edit actually improve the skill?
- Is the skill doing the work, or would the base agent pass without it?
- Does it still pa