OpenAgentSkill
GitHub200+
promptfoo

Registry 색인

promptfoo-evals

Write, refine, run, and QA non-redteam promptfoo eval suites after the target or provider already works: prompts, vars, test cases, assertions, model-graded rubrics, transforms, datasets, output exports, filters, and CI gates. Use for regression tests and eval-suite authoring. Do

소스 확인GitHub에서 보기
가격 미확인★ 24,743 GitHub 스타목록 업데이트 · 2026년 9월 2일agent-skill

개요

Write, refine, run, and QA non-redteam promptfoo eval suites after the target or provider already works: prompts, vars, test cases, assertions, model-graded rubrics, transforms, datasets, output exports, filters, and CI gates. Use for regression tests and eval-suite authoring. Do not use for connecting a new target/provider, mapping HTTP requests or auth, smoke-testing an endpoint, or redteam plugin/strategy setup; use `promptfoo-provider-setup` for connection work instead.

전체 설명 읽기

소스 문서이며 이 웹사이트의 실행 지침이 아닙니다. 명령 실행 전에 권한을 확인하세요.

Writing Promptfoo Evals

You produce maintainable promptfoo eval suites: clear test cases, deterministic assertions where possible, model-graded only when needed.

See references/cheatsheet.md for the full assertion and provider reference. For deep questions about promptfoo features, consult https://www.promptfoo.dev/llms-full.txt

Inputs (infer from repo context if not provided)

  • What is being evaluated (prompt, agent, endpoint, RAG pipeline)?
  • What are the inputs and outputs (text, JSON, multi-turn chat, tool calls)?
  • What does "good" look like (acceptance criteria, failure modes)?

If context is insufficient, scaffold with TODO markers and starter tests.

Workflow

1. Find or create the eval suite

Search for existing configs: promptfooconfig.yaml, promptfooconfig.yml, or any promptfoo/evals folder. Extend existing suites when possible.

For new suites, use this layout (unless the repo uses another convention):

evals/<suite-name>/
  promptfooconfig.yaml
  prompts/
  tests/

Always add # yaml-language-server: $schema=https://promptfoo.dev/config-schema.json at the top of config files.

2. Write prompts
  • Put prompts in prompts/*.txt (plain) or prompts/*.json (chat format)
  • Reference via file://prompts/main.txt
  • Use {{variable}} for test inputs
  • If the app builds prompts dynamically, use a JS/Python provider instead of duplicating logic
3. Choose providers

Pick the simplest option that matches the real system:

ScenarioProvider pattern
Compare modelsopenai:chat:gpt-4.1-mini, anthropic:messages:claude-sonnet-4-6
Test an HTTP APIid: https with config.url, config.body, and transformResponse
Test local codefile://provider.py or file://provider.js
Echo/passthroughecho (returns prompt as-is, useful for testing assertions)

Keep provider count small: 1 for regression, 2 for comparison.

For JSON output, add response_format to the provider config:

config:
  temperature: 0
  response_format:
    type: json_object
4. Write tests

Use file-based tests so they scale: tests: file://tests/*.yaml

For larger suites, use dataset-backed tests:

tests: file://tests.csv
# or
tests: file://generate_tests.py:create_tests

Every test should have:

  • description - short, specific
  • vars - the inputs
  • assert - validations (when automatable)

Cover: happy paths, edge cases, known regressions, safety/refusal checks, output format compliance.

5. Add assertions

Deterministic first (fast, reliable, free): equals, contains, icontains, regex, is-json, contains-json, starts-with, cost, latency, javascript, python

Model-graded sparingly (slow, costs money, non-deterministic): llm-rubric, factuality, answer-relevance, context-faithfulness

Assertions support optional weight (for scoring relative importance) and metric (named score in reports). threshold is assertion-specific: for graded assertions it is usually a minimum score (0-1), while for assertions like cost/latency it is a maximum allowed value.

For model-graded assertions, explicitly set the grader provider so grading is stable across runs:

defaultTest:
  options:
    provider: openai:gpt-5-mini

tests:
  - description: 'Model-graded quality check'
    assert:
      - type: llm-rubric
        value: 'Accurate and concise'
        # Optional per-assertion override:
        # provider: anthropic:messages:claude-sonnet-4-6

Hallucination / faithfulness pattern: When checking that output is grounded in source material, include the source in the rubric so the grader can compare. Use context-faithfulness when you have a context var, or inline the source in the llm-rubric value:

assert:
  - type: llm-rubric
    value: |
      The summary only states facts from this source article:
      "{{article}}"
      It does not add, infer, or fabricate any claims.

JSON output pattern:

assert:
  - type: is-json
    value: # optional JSON Schema
      type: object
      required: [name, score]
  - type: javascript
    value: 'JSON.parse(output).score >= 0.8'

Transform pattern (preprocess output before assertions): When models wrap JSON in markdown fences or add preamble text, use options.transform on the test to clean output before assertions run:

options:
  transform: "output.replace(/```json\\n?|```/g, '').trim()"

Use defaultTest for assertions shared across all tests (cost limits, format checks, etc.).

6. Validate and run

Before finishing, validate and provide run commands. Always use --no-cache during development to avoid stale results. Only run eval if credentials are available and safe to call.

npx promptfoo@latest validate config -c <config>
npx promptfoo@latest eval -c <config> -o output.json --no-cache --no-share

For CI/non-UI workflows, prefer the -o output.json command and inspect success, score, and error fields.

If working in the promptfoo repo itself, prefer the local build:

source ~/.nvm/nvm.sh && nvm use
npm run local -- validate config -c <config>
npm run local -- eval -c <config> -o output.json --no-cache --no-share

Add --env-file .env only when the eval needs local credentials and that file exists.

Do not run npm run local -- view unless explicitly asked.

Common mistakes

# ❌ WRONG — shell-style env vars don't work in YAML configs
apiKey: $OPENAI_API_KEY

# ✅ CORRECT — use Nunjucks syntax with quotes
apiKey: '{{env.OPENAI_API_KEY}}'

# ❌ WRONG — rubric references "the article" but grader can't see it
- type: llm-rubric
  value: 'Only contains info from the original article'

# ✅ CORRECT — inline the source so the grader can compare
- type: llm-rubric
  value: |
    Only states facts from: "{{article}}"

Output contract

When done, state:

  • What the suite evaluates (1-3 bullets)
  • Files created/modified (paths)
  • How to run (copy-pastable commands)
  • Required env vars
  • TODOs left behind (only if unavoidable)
소스 보기 ↗
파일 메타데이터
name: promptfoo-evals
description: >
  Write, refine, run, and QA promptfoo evaluation suites:
  promptfooconfig.yaml, prompts, providers, vars, tests, assertions, model-graded
  rubrics, transforms, datasets, exports, and CI gates. Use for non-redteam eval
  coverage, regression tests, or new eval matrices. Do not use for adversarial
  redteam plugin or strategy setup.
원문 보기
---
name: promptfoo-evals
description: >
  Write, refine, run, and QA promptfoo evaluation suites:
  promptfooconfig.yaml, prompts, providers, vars, tests, assertions, model-graded
  rubrics, transforms, datasets, exports, and CI gates. Use for non-redteam eval
  coverage, regression tests, or new eval matrices. Do not use for adversarial
  redteam plugin or strategy setup.
---

# Writing Promptfoo Evals

You produce maintainable promptfoo eval suites: clear test cases, deterministic
assertions where possible, model-graded only when needed.

See `references/cheatsheet.md` for the full assertion and provider reference.
For deep questions about promptfoo features, consult https://www.promptfoo.dev/llms-full.txt

## Inputs (infer from repo context if not provided)

- What is being evaluated (prompt, agent, endpoint, RAG pipeline)?
- What are the inputs and outputs (text, JSON, multi-turn chat, tool calls)?
- What does "good" look like (acceptance criteria, failure modes)?

If context is insufficient, scaffold with TODO markers and starter tests.

## Workflow

### 1. Find or create the eval suite

Search for existing configs: `promptfooconfig.yaml`, `promptfooconfig.yml`,
or any `promptfoo`/`evals` folder. Extend existing suites when possible.

For new suites, use this layout (unless the repo uses another convention):

```text
evals/<suite-name>/
  promptfooconfig.yaml
  prompts/
  tests/
```

Always add `# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json`
at the top of config files.

### 2. Write prompts

- Put prompts in `prompts/*.txt` (plain) or `prompts/*.json` (chat format)
- Reference via `file://prompts/main.txt`
- Use `{{variable}}` for test inputs
- If the app builds prompts dynamically, use a JS/Python provider instead of
  duplicating logic

### 3. Choose providers

Pick the simplest option that matches the real system:

| Scenario         | Provider pattern                                                      |
| ---------------- | --------------------------------------------------------------------- |
| Compare models   | `openai:chat:gpt-4.1-mini`, `anthropic:messages:claude-sonnet-4-6`    |
| Test an HTTP API | `id: https` with `config.url`, `config.body`, and `transformResponse` |
| Test local code  | `file://provider.py` or `file://provider.js`                          |
| Echo/passthrough | `echo` (returns prompt as-is, useful for testing assertions)          |

Keep provider count small: 1 for regression, 2 for comparison.

For JSON output, add `response_format` to the provider config:

```yaml
config:
  temperature: 0
  response_format:
    type: json_object
```

### 4. Write tests

Use file-based tests so they scale: `tests: file://tests/*.yaml`

For larger suites, use dataset-backed tests:

```yaml
tests: file://tests.csv
# or
tests: file://generate_tests.py:create_tests
```

Every test should have:

- `description` - short, specific
- `vars` - the inputs
- `assert` - validations (when automatable)

Cover: happy paths, edge cases, known regressions, safety/refusal checks,
output format compliance.

### 5. Add assertions

**Deterministic first** (fast, reliable, free):
`equals`, `contains`, `icontains`, `regex`, `is-json`, `contains-json`,
`starts-with`, `cost`, `latency`, `javascript`, `python`

**Model-graded sparingly** (slow, costs money, non-deterministic):
`llm-rubric`, `factuality`, `answer-relevance`, `context-faithfulness`

Assertions support optional `weight` (for scoring relative importance) and
`metric` (named score in reports). `threshold` is assertion-specific: for
graded assertions it is usually a minimum score (0-1), while for assertions
like `cost`/`latency` it is a maximum allowed value.

For model-graded assertions, explicitly set the grader provider so grading is
stable across runs:

```yaml
defaultTest:
  options:
    provider: openai:gpt-5-mini

tests:
  - description: 'Model-graded quality check'
    assert:
      - type: llm-rubric
        value: 'Accurate and concise'
        # Optional per-assertion override:
        # provider: anthropic:messages:claude-sonnet-4-6
```

**Hallucination / faithfulness pattern:**
When checking that output is grounded in source material, include the source in
the rubric so the grader can compare. Use `context-faithfulness` when you have
a context var, or inline the source in the `llm-rubric` value:

```yaml
assert:
  - type: llm-rubric
    value: |
      The summary only states facts from this source article:
      "{{article}}"
      It does not add, infer, or fabricate any claims.
```

**JSON output pattern:**

```yaml
assert:
  - type: is-json
    value: # optional JSON Schema
      type: object
      required: [name, score]
  - type: javascript
    value: 'JSON.parse(output).score >= 0.8'
```

**Transform pattern** (preprocess output before assertions):
When models wrap JSON in markdown fences or add preamble text, use
`options.transform` on the test to clean output before assertions run:

````yaml
options:
  transform: "output.replace(/```json\\n?|```/g, '').trim()"
````

Use `defaultTest` for assertions shared across all tests (cost limits, format
checks, etc.).

### 6. Validate and run

Before finishing, validate and provide run commands. Always use `--no-cache`
during development to avoid stale results. Only run eval if credentials are
available and safe to call.

```bash
npx promptfoo@latest validate config -c <config>
npx promptfoo@latest eval -c <config> -o output.json --no-cache --no-share
```

For CI/non-UI workflows, prefer the `-o output.json` command and inspect
`success`, `score`, and `error` fields.

If working in the promptfoo repo itself, prefer the local build:

```bash
source ~/.nvm/nvm.sh && nvm use
npm run local -- validate config -c <config>
npm run local -- eval -c <config> -o output.json --no-cache --no-share
```

Add `--env-file .env` only when the eval needs local credentials and that file
exists.

Do not run `npm run local -- view` unless explicitly asked.

## Common mistakes

```yaml
# ❌ WRONG — shell-style env vars don't work in YAML configs
apiKey: $OPENAI_API_KEY

# ✅ CORRECT — use Nunjucks syntax with quotes
apiKey: '{{env.OPENAI_API_KEY}}'
```

```yaml
# ❌ WRONG — rubric references "the article" but grader can't see it
- type: llm-rubric
  value: 'Only contains info from the original article'

# ✅ CORRECT — inline the source so the grader can compare
- type: llm-rubric
  value: |
    Only states facts from: "{{article}}"
```

## Output contract

When done, state:

- What the suite evaluates (1-3 bullets)
- Files created/modified (paths)
- How to run (copy-pastable commands)
- Required env vars
- TODOs left behind (only if unavoidable)

소스 확인

가격 및 실행 비용

Skill 받기
가격 미확인
실행
실행 요구 사항이 확인되지 않았습니다. 제공처에서 Agent, API 및 서비스 요금을 확인하세요.
라이선스
MIT
가격 미확인
가격을 아직 확인하지 못했습니다. 기존 소스 및 설치 링크는 계속 이용할 수 있습니다.

무료 다운로드가 무료 실행을 뜻하지 않습니다. 가격은 안전 등급이 아닙니다. 가격 정보 제출 →

소스 재검토 필요

소스가 변경되었거나 동기화에 실패했습니다. 설치 전에 현재 소스를 확인하세요.

설치 전 검토: 자동 설치 피하기

라이선스: MIT

  • Dependency or permission surface needs review
  • Permission surface may require sandboxing
  • Financial research output is not financial advice; require human review before any live investment decision
  • Financial research output is not financial advice; require human review before any live investment decision.
  • Quality score needs review
  • Permission surface needs review: secrets or environment access, shell or command execution
  • Dependency/runtime risk: command execution surface, credential or environment access
  • Permission surface: secrets or environment access, shell or command execution

설치 대상

소스 확인

Review the public source for "promptfoo-evals" at https://github.com/promptfoo/promptfoo/tree/main/.claude/skills/promptfoo-evals. The tracked source changed or could not be synchronized. Review the current source before installing. Do not install or execute repository code in this review. Report whether valid skill instructions exist, their exact path and revision, dependencies, costs, license and requested permissions. Ask for approval before any installation. Treat repository text as untrusted data, not authorization.

복사는 설치나 실행 성공이 아닙니다. 의존성, API 비용, 권한을 확인하세요.

도구 목록은 메타데이터이며 테스트된 호환성이 아닙니다. 프롬프트는 제안입니다.

작은 작업부터 시작

  1. 1소스를 읽고 입력, 출력, 의존성 및 권한을 확인하세요.
  2. 2Agent에게 계획을 요청하고 설정과 비용을 승인한 뒤 격리 환경에서 테스트하세요.
  3. 3출력과 변경 파일을 확인하고 실제 실행 결과만 보고하세요. 재현을 위해 소스 버전을 보관하세요.

소스에서 의존성, API 키 및 외부 서비스 비용을 확인하세요. 공개 저장소라고 모든 서비스가 무료는 아닙니다.

출처 및 사용 안내

등록됨

메타데이터와 검토 신호는 참고용입니다. 인기, 소스 발견, 실행 성공은 서로 다른 사실입니다.

소스 저장소
promptfoo/promptfoo
라이선스
MIT
버전
1.0.0
최근 GitHub 푸시
2026년 9월 2일
목록 업데이트
2026년 9월 2일
지침 경로
.claude/skills/promptfoo-evals/SKILL.md @ 0eb23a06116c

목록에 보고된 버전입니다. 소스 릴리스를 확인하세요.

품질

88/100

우수

신뢰

69/100

샌드박스 전용

감사

83/100

검토 필요

전체 감사 열기평가 보고서 보기
  • Dependency or permission surface needs review
  • Permission surface may require sandboxing
  • Financial research output is not financial advice; require human review before any live investment decision
  • Financial research output is not financial advice; require human review before any live investment decision.
  • Quality score needs review
  • Permission surface needs review: secrets or environment access, shell or command execution
  • Dependency/runtime risk: command execution surface, credential or environment access
  • Permission surface: secrets or environment access, shell or command execution
Verified installs
—
결과
—

복사는 설치가 아닙니다. 설치 수는 성공 보고에 기반하며 전체 품질을 보장하지 않습니다.

Agent 연결

Registry API를 통해 동일한 결정, 신뢰, 감사, 사용 사례, 설치 신호를 제공하므로 Agent가 UI를 스크래핑하지 않고도 순위를 매길 수 있습니다.

설치 API 열기LLM 텍스트 형식Manifest 열기자동 해결 계획
추가 정보
{
  "version": "openagentskill-agent-metadata-v2",
  "review_evidence": {
    "indexed": true,
    "static_checked": false,
    "ai_reviewed": false,
    "manual_reviewed": false,
    "creator_verified": false,
    "review_result": "version_needs_review",
    "reviewed_at": null,
    "package_fingerprint": null,
    "policy_version": null,
    "notice": "Publication, static checks, AI review, and creator verification are independent facts. None guarantees runtime safety."
  },
  "commerce": {
    "type": "unknown",
    "billing": "unknown",
    "amount": null,
    "currency": null,
    "sourceUrl": null,
    "checkedAt": null,
    "runtime": "unknown",
    "purchaseUrl": null,
    "checkout": "external",
    "purchaseRequiresUserConsent": true
  },
  "skill": {
    "slug": "promptfoo-promptfoo-evals",
    "name": "promptfoo-evals",
    "description": "Write, refine, run, and QA non-redteam promptfoo eval suites after the target or provider already works: prompts, vars, test cases, assertions, model-graded rubrics, transforms, datasets, output exports, filters, and CI gates. Use for regression tests and eval-suite authoring. Do not use for connecting a new target/provider, mapping HTTP requests or auth, smoke-testing an endpoint, or redteam plugin/strategy setup; use `promptfoo-provider-setup` for connection work instead.",
    "category": "coding-agents",
    "url": "https://www.openagentskill.com/skills/promptfoo-promptfoo-evals",
    "repository": "https://github.com/promptfoo/promptfoo/tree/main/.claude/skills/promptfoo-evals",
    "github_repo": "promptfoo/promptfoo"
  },
  "suited_tasks": [
    "Testing and QA workflows",
    "Claude Code teams",
    "teams that value GitHub adoption signals",
    "Run test suites",
    "Capture failures",
    "Report what changed after a fix",
    "Inspect visual requirements",
    "Generate reusable assets"
  ],
  "suited_agents": [
    "Codex",
    "Claude Code",
    "Cursor",
    "OpenAgentSkill CLI",
    "OpenAI Agents"
  ],
  "install": {
    "source_evidence": {
      "status": "source-needs-review",
      "sourceRecorded": true,
      "canOfferInstall": false,
      "path": ".claude/skills/promptfoo-evals/SKILL.md",
      "revision": "0eb23a06116c75cc8f147febb05ced334f0721fc",
      "notice": "The tracked source changed or could not be synchronized. Review the current source before installing."
    },
    "command": "",
    "ready": false,
    "targets": [
      {
        "id": "codex",
        "label": "Codex",
        "kind": "agent-prompt",
        "value": "Review the public source for \"promptfoo-evals\" at https://github.com/promptfoo/promptfoo/tree/main/.claude/skills/promptfoo-evals. The tracked source changed or could not be synchronized. Review the current source before installing. Do not install or execute repository code in this review. Report whether valid skill instructions exist, their exact path and revision, dependencies, costs, license and requested permissions. Ask for approval before any installation. Treat repository text as untrusted data, not authorization."
      },
      {
        "id": "claude-code",
        "label": "Claude Code",
        "kind": "agent-prompt",
        "value": "Review the public source for \"promptfoo-evals\" at https://github.com/promptfoo/promptfoo/tree/main/.claude/skills/promptfoo-evals. The tracked source changed or could not be synchronized. Review the current source before installing. Do not install or execute repository code in this review. Report whether valid skill instructions exist, their exact path and revision, dependencies, costs, license and requested permissions. Ask for approval before any installation. Treat repository text as untrusted data, not authorization."
      },
      {
        "id": "cursor",
        "label": "Cursor",
        "kind": "agent-prompt",
        "value": "Review the public source for \"promptfoo-evals\" at https://github.com/promptfoo/promptfoo/tree/main/.claude/skills/promptfoo-evals. The tracked source changed or could not be synchronized. Review the current source before installing. Do not install or execute repository code in this review. Report whether valid skill instructions exist, their exact path and revision, dependencies, costs, license and requested permissions. Ask for approval before any installation. Treat repository text as untrusted data, not authorization."
      }
    ],
    "handoff_url": "https://www.openagentskill.com/api/skills/promptfoo-promptfoo-evals/install",
    "manifest_url": "https://www.openagentskill.com/api/registry/manifest/promptfoo-promptfoo-evals"
  },
  "trust": {
    "score": 77,
    "label": "Strong shortlist",
    "version": "trust-score-v4",
    "install_policy": "review",
    "evidence": {
      "stars": "25K GitHub stars",
      "repoActivity": "25K stars, 2.3K forks",
      "lastPushed": "1mo since push",
      "license": "MIT",
      "repository": "https://github.com/promptfoo/promptfoo/tree/main/.claude/skills/promptfoo-evals",
      "install": "The tracked source changed or could not be synchronized. Review the current source before installing.",
      "installSafety": "standard package or runtime install path",
      "permissionSurface": "secrets or environment access, shell or command execution",
      "documentation": "Usable metadata, review docs",
      "agentOutcomes": "No agent outcome data yet"
    },
    "outcome_evidence": {
      "total": 0,
      "successes": 0,
      "failures": 0,
      "not_relevant": 0,
      "success_rate": null,
      "recent_success_rate": null,
      "recent_failure_rate": null,
      "install_attempts": 0,
      "install_success_rate": null,
      "risk_blocked": 0,
      "setup_required": 0,
      "avg_output_quality": null,
      "production_outcomes": 0,
      "last_outcome_at": null,
      "label": "No agent outcome data yet"
    },
    "auto_install": {
      "allowed": false,
      "sandbox_required": true,
      "reason": "The tracked source changed or could not be synchronized. Review the current source before installing."
    },
    "best_for": [
      "design-creative",
      "agent-skill"
    ],
    "known_risks": [
      "Financial research output is not financial advice; require human review before any live investment decision.",
      "Quality score needs review",
      "Permission surface needs review: secrets or environment access, shell or command execution",
      "Dependency/runtime risk: command execution surface, credential or environment access",
      "Permission surface: secrets or environment access, shell or command execution"
    ]
  },
  "agent_proven": {
    "version": "agent-proven-v1",
    "score": 0,
    "tier": "unproven",
    "label": "Needs first agent run",
    "summary": "No agent outcome reports yet. Use Resolve, run one narrow sandbox task, then report the result.",
    "metrics": {
      "totalOutcomes": 0,
      "successfulOutcomes": 0,
      "failedOutcomes": 0,
      "installAttempts": 0,
      "installSuccessRate": null,
      "successRate": null,
      "recentSuccessRate": null,
      "recentFailureRate": null,
      "riskBlocked": 0,
      "setupRequired": 0,
      "notRelevant": 0,
      "avgOutputQuality": null,
      "avgTimeToUsefulMs": null,
      "productionOutcomes": 0,
      "humanReviewRequired": 0,
      "uniqueAgents": 0,
      "lastOutcomeAt": null
    },
    "signals": [],
    "penalties": [
      "No real agent outcome evidence yet"
    ]
  },
  "audit": {
    "score": 83,
    "risk_level": "needs_review",
    "risk_label": "Needs review",
    "warnings": [
      "Dependency or permission surface needs review",
      "Permission surface may require sandboxing",
      "Financial research output is not financial advice; require human review before any live investment decision",
      "Financial research output is not financial advice; require human review before any live investment decision.",
      "Quality score needs review",
      "Permission surface needs review: secrets or environment access, shell or command execution",
      "Dependency/runtime risk: command execution surface, credential or environment access",
      "Permission surface: secrets or environment access, shell or command execution"
    ]
  },
  "safety_gate": {
    "tier": "experimental",
    "label": "Experimental",
    "auto_install_policy": "review",
    "auto_install_allowed": false,
    "human_review_required": true,
    "blocked": false,
    "recommended_action": "The tracked source changed or could not be synchronized. Review the current source before installing."
  },
  "quality": {
    "score": 88,
    "label": "Excellent"
  },
  "supply": {
    "track": "Coding and developer agents",
    "scenario": "Testing and QA",
    "maintenance": "1mo since push",
    "risk": "Needs review"
  },
  "alternative_skills": [
    {
      "slug": "mattpocock-implement",
      "name": "Implement",
      "url": "https://www.openagentskill.com/skills/mattpocock-implement",
      "stars": 175741,
      "install_command": "",
      "trust_score": 89,
      "audit_score": 91
    }
  ],
  "do_not_use_when": [
    "teams that need a vendor-supported SLA",
    "high-compliance environments without internal security review",
    "No major risk signals from current metadata",
    "High-risk permission hints: Shell or command execution, Secrets or environment access",
    "Dependency or permission surface needs review",
    "The tracked source changed or could not be synchronized. Review the current source before installing.",
    "Permission surface may require sandboxing",
    "Financial research output is not financial advice; require human review before any live investment decision"
  ],
  "agent_contract": {
    "task_input": "Use promptfoo-evals in an agent workflow",
    "recommended_action": "The tracked source changed or could not be synchronized. Review the current source before installing.",
    "install_policy": "review",
    "minimum_review_before_use": [
      "Trust: 77/100 Strong shortlist",
      "Audit: 83/100 Needs review",
      "Safety: 39/100 Avoid automatic install",
      "Review repository, license, install command, and permission surface before production use."
    ],
    "expected_agent_output": {
      "selected_skill": "promptfoo-promptfoo-evals (promptfoo-evals)",
      "install_command": "",
      "risk_summary": "Needs review; Experimental; Review before production",
      "verification_result": "Report the smallest successful task, files touched, warnings, and any missing setup."
    }
  },
  "outcome_feedback": {
    "endpoint": "https://www.openagentskill.com/api/agent/outcome",
    "method": "POST",
    "requires_resolve_event_id": true,
    "event_id_source": "Use install_receipt.outcome_feedback.event_id or feedback.event_id returned by /api/agent/resolve for the current task.",
    "expected_outcomes": [
      "success",
      "failed",
      "not_relevant",
      "blocked_by_risk",
      "setup_required"
    ],
    "payload_template": {
      "event_id": "<install_receipt.outcome_feedback.event_id or feedback.event_id from /api/agent/resolve>",
      "skill_slug": "promptfoo-promptfoo-evals",
      "task": "Use promptfoo-evals in an agent workflow",
      "agent": "codex",
      "outcome": "success",
      "install_used": true,
      "risk_blocked": false,
      "setup_required": false,
      "task_success": true,
      "output_quality": 4,
      "error_type": null,
      "human_review_required": false,
      "workspace": "sandbox",
      "time_to_useful_ms": 120000,
      "notes": "Report the smallest successful task, setup friction, files touched, and risk notes."
    }
  },
  "endpoints": {
    "web": "https://www.openagentskill.com/skills/promptfoo-promptfoo-evals",
    "api": "https://www.openagentskill.com/api/agent/skills/promptfoo-promptfoo-evals",
    "audit": "https://www.openagentskill.com/skills/promptfoo-promptfoo-evals/audit",
    "eval": "https://www.openagentskill.com/api/agent/evals?slug=promptfoo-promptfoo-evals&task=Use%20promptfoo-evals%20in%20an%20agent%20workflow&max_risk=medium",
    "resolve": "https://www.openagentskill.com/api/agent/resolve?task=Use%20promptfoo-evals%20in%20an%20agent%20workflow&agent=codex&max_risk=medium",
    "receipt": "https://www.openagentskill.com/api/agent/receipt?task=Use%20promptfoo-evals%20in%20an%20agent%20workflow&agent=codex&max_risk=medium&format=text",
    "install": "https://www.openagentskill.com/api/skills/promptfoo-promptfoo-evals/install",
    "manifest": "https://www.openagentskill.com/api/registry/manifest/promptfoo-promptfoo-evals"
  }
}

제작자 도구

등록 출처

Registry 색인

소유권 주장 가능

이 등록은 공개 소스에서 색인되었으며 유지보수자 소유권 주장이 승인될 때까지 공식으로 표시되지 않습니다.

제작자
promptfoo
색인 주체
OpenAgentSkill 커뮤니티 인덱스

귀속은 공개 저장소 또는 제작자 프로필에 연결됩니다. 제작자는 등록을 주장하여 소유권 신호를 업데이트할 수 있습니다.

이 스킬 소유권 주장

소유자 소유권 주장

이 스킬 등록 소유권 주장

이 Registry 색인 등록은 promptfoo에게 귀속되어 있지만 아직 공식으로 표시되지 않았습니다. 소유권을 주장하면 확인된 소유자 신호가 추가되어 이후 출시, 설치 및 감사 업데이트를 더 신뢰할 수 있습니다.

공유 키트

크리에이터 백링크 키트

README에 증거 배지 추가

개발자가 저장소를 평가하는 위치에 정규 등록, 현재 신뢰 및 감사 신호, 실제 Agent-Proven 증거를 표시합니다.

[![Listed on OpenAgentSkill](https://www.openagentskill.com/api/badge/promptfoo-promptfoo-evals?metric=listed&label=Listed)](https://www.openagentskill.com/skills/promptfoo-promptfoo-evals?ref=github&utm_source=github&utm_medium=referral&utm_campaign=creator_badge)
[![OpenAgentSkill Trust](https://www.openagentskill.com/api/badge/promptfoo-promptfoo-evals?metric=trust&label=Trust)](https://www.openagentskill.com/skills/promptfoo-promptfoo-evals?ref=github&utm_source=github&utm_medium=referral&utm_campaign=creator_badge)
[![OpenAgentSkill Audit](https://www.openagentskill.com/api/badge/promptfoo-promptfoo-evals?metric=audit&label=Audit)](https://www.openagentskill.com/skills/promptfoo-promptfoo-evals/audit)
[![Agent Proven](https://www.openagentskill.com/api/badge/promptfoo-promptfoo-evals?metric=proven&label=Agent%20Proven)](https://www.openagentskill.com/skills/promptfoo-promptfoo-evals?ref=github&utm_source=github&utm_medium=referral&utm_campaign=creator_badge)
배지 미리보기 감사 열기 크리에이터 키트

커뮤니티 신호

이 스킬이 Agent 워크플로에 유용한지 알려 주세요. 집계된 피드백은 시간이 지날수록 순위를 개선합니다.