{"slug":"github-agentic-eval","name":"agentic-eval","description":"Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when:\n- Implementing self-critique and reflection loops\n- Building evaluator-optimizer pipelines for quality-critical generation\n- Creating test-driven code refinement workflows\n- Designing rubric-based or LLM-as-judge evaluation systems\n- Adding iterative improvement to agent outputs (code, reports, analysis)\n- Measuring and improving agent response quality","long_description":"---\nname: agentic-eval\ndescription: |\n  Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when:\n  - Implementing self-critique and reflection loops\n  - Building evaluator-optimizer pipelines for quality-critical generation\n  - Creating test-driven code refinement workflows\n  - Designing rubric-based or LLM-as-judge evaluation systems\n  - Adding iterative improvement to agent outputs (code, reports, analysis)\n  - Measuring and improving agent response quality\n---\n\n# Agentic Evaluation Patterns\n\nPatterns for self-improvement through iterative evaluation and refinement.\n\n## Overview\n\nEvaluation patterns enable agents to assess and improve their own outputs, moving beyond single-shot generation to iterative refinement loops.\n\n```\nGenerate → Evaluate → Critique → Refine → Output\n    ↑                              │\n    └──────────────────────────────┘\n```\n\n## When to Use\n\n- **Quality-critical generation**: Code, reports, analysis requiring high accuracy\n- **Tasks with clear evaluation criteria**: Defined success metrics exist\n- **Content requiring specific standards**: Style guides, compliance, formatting\n\n---\n\n## Pattern 1: Basic Reflection\n\nAgent evaluates and improves its own output through self-critique.\n\n```python\ndef reflect_and_refine(task: str, criteria: list[str], max_iterations: int = 3) -> str:\n    \"\"\"Generate with reflection loop.\"\"\"\n    output = llm(f\"Complete this task:\\n{task}\")\n    \n    for i in range(max_iterations):\n        # Self-critique\n        critique = llm(f\"\"\"\n        Evaluate this output against criteria: {criteria}\n        Output: {output}\n        Rate each: PASS/FAIL with feedback as JSON.\n        \"\"\")\n        \n        critique_data = json.loads(critique)\n        all_pass = all(c[\"status\"] == \"PASS\" for c in critique_data.values())\n        if all_pass:\n            return output\n        \n        # Refine based on critique\n        failed = {k: v[\"feedback\"] for k, v in critique_data.items() if v[\"status\"] == \"FAIL\"}\n        output = llm(f\"Improve to address: {failed}\\nOriginal: {output}\")\n    \n    return output\n```\n\n**Key insight**: Use structured JSON output for reliable parsing of critique results.\n\n---\n\n## Pattern 2: Evaluator-Optimizer\n\nSeparate generation and evaluation into distinct components for clearer responsibilities.\n\n```python\nclass EvaluatorOptimizer:\n    def __init__(self, score_threshold: float = 0.8):\n        self.score_threshold = score_threshold\n    \n    def generate(self, task: str) -> str:\n        return llm(f\"Complete: {task}\")\n    \n    def evaluate(self, output: str, task: str) -> dict:\n        return json.loads(llm(f\"\"\"\n        Evaluate output for task: {task}\n        Output: {output}\n        Return JSON: {{\"overall_score\": 0-1, \"dimensions\": {{\"accuracy\": ..., \"clarity\": ...}}}}\n        \"\"\"))\n    \n    def optimize(self, output: str, feedback: dict) -> str:\n        return llm(f\"Improve based on feedback: {feedback}\\nOutput: {output}\")\n    \n    def run(self, task: str, max_iterations: int = 3) -> str:\n        output = self.generate(task)\n        for _ in range(max_iterations):\n            evaluation = self.evaluate(output, task)\n            if evaluation[\"overall_score\"] >= self.score_threshold:\n                break\n            output = self.optimize(output, evaluation)\n        return output\n```\n\n---\n\n## Pattern 3: Code-Specific Reflection\n\nTest-driven refinement loop for code generation.\n\n```python\nclass CodeReflector:\n    def reflect_and_fix(self, spec: str, max_iterations: int = 3) -> str:\n        code = llm(f\"Write Python code for: {spec}\")\n        tests = llm(f\"Generate pytest tests for: {spec}\\nCode: {code}\")\n        \n        for _ in range(max_iterations):\n            result = run_tests(code, tests)\n            if result[\"success\"]:\n                return code\n            code = llm(f\"Fix error: {result['error']}\\nCode: {code}\")\n        return code\n```\n\n---\n\n## Evaluation Strategies\n\n### Outcome-Based\nEvaluate whether output achieves the expected result.\n\n```python\ndef evaluate_outcome(task: str, output: str, expected: str) -> str:\n    return llm(f\"Does output achieve expected outcome? Task: {task}, Expected: {expected}, Output: {output}\")\n```\n\n### LLM-as-Judge\nUse LLM to compare and rank outputs.\n\n```python\ndef llm_judge(output_a: str, output_b: str, criteria: str) -> str:\n    return llm(f\"Compare outputs A and B for {criteria}. Which is better and why?\")\n```\n\n### Rubric-Based\nScore outputs against weighted dimensions.\n\n```python\nRUBRIC = {\n    \"accuracy\": {\"weight\": 0.4},\n    \"clarity\": {\"weight\": 0.3},\n    \"completeness\": {\"weight\": 0.3}\n}\n\ndef evaluate_with_rubric(output: str, rubric: dict) -> float:\n    scores = json.loads(llm(f\"Rate 1-5 for each dimension: {list(rubric.keys())}\\nOutput: {output}\"))\n    return sum(scores[d] * rubric[d][\"weight\"] for d in rubric) / 5\n```\n\n---\n\n## Best Practices\n\n| Practice | Rationale |\n|----------|-----------|\n| **Clear criteria** | Define specific, measurable evaluation criteria upfront |\n| **Iteration limits** | Set max iterations (3-5) to prevent infinite loops |\n| **Convergence check** | Stop if output score isn't improving between iterations |\n| **Log history** | Keep full trajectory for debugging and analysis |\n| **Structured output** | Use JSON for reliable parsing of evaluation results |\n\n---\n\n## Quick Start Checklist\n\n```markdown\n## Evaluation Implementation Checklist\n\n### Setup\n- [ ] Define evaluation criteria/rubric\n- [ ] Set score threshold for \"good enough\"\n- [ ] Configure max iterations (default: 3)\n\n### Implementation\n- [ ] Implement generate() function\n- [ ] Implement evaluate() function with structured output\n- [ ] Implement optimize() function\n- [ ] Wire up the refinement loop\n\n### Safety\n- [ ] Add convergence detection\n- [ ] Log all iterations for debugging\n- [ ] Handle evaluation parse failures gracefully\n```\n","tagline":"Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when:\n- Implementing self-critique and reflection loops\n- Building evaluator-optimizer pipelines for quality-critical generation\n- Creating test-driven code refinement workflows\n- Designing rubr","category":"design-creative","tags":["agent-skill"],"author":"github","verified":false,"attribution":{"status":"registry_indexed","statusLabel":"Registry indexed","shortLabel":"REGISTRY INDEXED","sourceLabel":"github fast track","sourceDetail":"github/awesome-copilot","creatorName":"github","creatorUrl":"https://github.com/github","sourceUrl":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval","indexedBy":"OpenAgentSkill community index","claimUrl":"https://www.openagentskill.com/skills/github-agentic-eval#claim-this-skill","claimCta":"Claim this skill","trustNote":"This listing was indexed from public sources and is not marked official until a maintainer claim is approved.","publicNote":"Attribution links to the public repository or creator profile. Creators can claim the listing to update ownership signals."},"stats":{"stars":38524,"forks":4869,"verified_installs":0,"successful_runs":0,"total_outcomes":0,"rating":0,"review_count":0,"quality_score":55.2},"quality":{"score":92,"tier":"excellent","label":"Excellent","summary":"High-confidence pick with strong adoption and healthy maintenance signals.","signals":[{"label":"GitHub stars","value":"39K","tone":"positive"},{"label":"Freshness","value":"4d ago","tone":"positive"},{"label":"Install ready","value":"Yes","tone":"positive"},{"label":"License","value":"MIT","tone":"neutral"}],"warnings":[]},"trust":{"version":"trust-score-v5","score":83,"base_score":86,"outcome_confidence":0,"tier":"strong","label":"Review then install","summary":"Good shortlist signal, but the agent should review audit notes, install policy, and outcome evidence before running it.","recommendedAction":"Use as the primary candidate after human or sandbox review.","decision":{"install_policy":"agent_install_candidate","auto_install_allowed":false,"human_review_required":true,"sandbox_first":true,"agent_action":"Ask for approval or run a sandbox-only trial before installing.","reasoning":["83/100 Trust Score v5","86/100 Trust Score v4 baseline","Needs more real agent outcomes before unattended install","Install path is available","Low metadata risk"],"review_required_when":["The workspace contains production secrets, payments, private customer data, or irreversible actions.","The install command requests shell, network, credential, database, or broad filesystem access.","Outcome evidence is missing, recently failed, or required human review.","Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface"]},"dimensions":[{"id":"github_adoption","label":"GitHub adoption","score":100,"weight":0.13,"status":"pass","detail":"39K GitHub stars"},{"id":"repo_activity","label":"Stars/forks activity","score":97,"weight":0.08,"status":"pass","detail":"39K stars, 4.9K forks; issue activity unavailable in current metadata"},{"id":"maintenance","label":"Recent maintenance","score":100,"weight":0.14,"status":"pass","detail":"4d since push"},{"id":"license","label":"License clarity","score":86,"weight":0.09,"status":"pass","detail":"MIT"},{"id":"documentation","label":"README/SKILL.md completeness","score":76,"weight":0.14,"status":"info","detail":"Public metadata needs stronger README/SKILL.md context"},{"id":"dependency_risk","label":"Dependency/runtime risk","score":90,"weight":0.12,"status":"pass","detail":"no major dependency risk hints in public metadata"},{"id":"installability","label":"Install availability","score":92,"weight":0.1,"status":"pass","detail":"npx skills add github/awesome-copilot --skill agentic-eval"},{"id":"install_safety","label":"Install command safety","score":68,"weight":0.1,"status":"info","detail":"dynamic command execution, standard package or runtime install path"},{"id":"permission_surface","label":"Permission surface","score":100,"weight":0.07,"status":"pass","detail":"no high-risk permission surface in public metadata"},{"id":"repository","label":"Repository evidence","score":86,"weight":0.04,"status":"pass","detail":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval"},{"id":"review_status","label":"Review status","score":88,"weight":0.05,"status":"pass","detail":"AI review data available"},{"id":"agent_outcomes","label":"Agent Proven outcomes","score":54,"weight":0.13,"status":"info","detail":"No agent outcome data yet"}],"checks":[{"status":"pass","label":"GitHub adoption","detail":"39K GitHub stars"},{"status":"pass","label":"Stars/forks activity","detail":"39K stars, 4.9K forks; issue activity unavailable in current metadata"},{"status":"pass","label":"Recent maintenance","detail":"4d since push"},{"status":"pass","label":"License clarity","detail":"MIT"},{"status":"info","label":"README/SKILL.md completeness","detail":"Public metadata needs stronger README/SKILL.md context"},{"status":"pass","label":"Dependency/runtime risk","detail":"no major dependency risk hints in public metadata"},{"status":"pass","label":"Install availability","detail":"npx skills add github/awesome-copilot --skill agentic-eval"},{"status":"info","label":"Install command safety","detail":"dynamic command execution, standard package or runtime install path"},{"status":"pass","label":"Permission surface","detail":"no high-risk permission surface in public metadata"},{"status":"pass","label":"Repository evidence","detail":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval"},{"status":"pass","label":"Review status","detail":"AI review data available"},{"status":"info","label":"Agent Proven outcomes","detail":"No agent outcome data yet"},{"status":"warn","label":"Ownership","detail":"No approved owner claim yet"},{"status":"pass","label":"OpenAgentSkill usage","detail":"2 views, 0 install copies"},{"status":"info","label":"Agent outcomes","detail":"No agent outcome data yet"}],"strengths":["AI review approved","Install path is available","Repository evidence is available","Recently maintained repository","Large GitHub adoption signal","Install command has no obvious high-risk pattern","Outcome loop is ready but needs first real agent run"],"warnings":["No real agent outcome reports yet","Human review required before unattended installation"],"evidence":{"stars":"39K GitHub stars","repoActivity":"39K stars, 4.9K forks","lastPushed":"4d since push","license":"MIT","repository":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval","install":"npx skills add github/awesome-copilot --skill agentic-eval","installSafety":"dynamic command execution, standard package or runtime install path","permissionSurface":"no high-risk permission surface in public metadata","documentation":"Usable metadata, review docs","agentOutcomes":"No agent outcome data yet","agentProvenScore":0,"outcomeConfidence":"0%","installPolicy":"agent_install_candidate"},"installReadiness":{"ready":true,"command":"npx skills add github/awesome-copilot --skill agentic-eval","policy":"agent_install_candidate","label":"Agent install candidate","notes":["Install path is available","Repository evidence is available","License is declared","No Agent Proven outcome evidence yet","4d since push","Trust Score v5 requires review or sandbox-only use before install."]},"agentCompatibility":["Codex","Claude Code","Cursor","OpenAgentSkill CLI"],"riskSummary":{"level":"low","label":"Low metadata risk","notes":["No major trust warnings detected from available metadata"]},"outcomeEvidence":{"total":0,"successes":0,"failures":0,"notRelevant":0,"successRate":null,"installAttempts":0,"riskBlocked":0,"setupRequired":0,"installSuccessRate":null,"avgOutputQuality":null,"avgTimeToUsefulMs":null,"productionOutcomes":0,"humanReviewRequired":0,"recentSuccessRate":null,"recentFailureRate":null,"uniqueAgents":0,"agentProvenScore":0,"agentProvenLabel":"Needs first agent run","lastOutcomeAt":null,"label":"No agent outcome data yet"},"autoInstall":{"allowed":false,"sandboxRequired":true,"policy":"agent_install_candidate","reason":"Ask for approval or run a sandbox-only trial before installing."},"outcome_loop":{"version":"openagentskill-agent-outcome-v4","required_after_install":true,"endpoint":"/api/agent/outcome","method":"POST","event_id_source":"feedback.event_id, install_receipt.resolve_event_id, or decision_packet.outcome_feedback.event_id","expected_outcomes":["success","failed","not_relevant","blocked_by_risk","setup_required"],"required_fields":["event_id","skill_slug","task"],"quality_fields":["task_success","output_quality","error_type","human_review_required","used_in_production","workspace","evidence_url","time_to_useful_ms","source_version"],"ranking_inputs_updated":["Trust Score v5 outcome confidence","Agent Proven Score","Resolve ranking task-fit evidence","Skill detail machine-readable metadata","Outcome leaderboard"]},"agent_contract":{"suited_tasks":["design-creative","agent-skill"],"suited_agents":["Codex","Claude Code","Cursor","OpenAgentSkill CLI"],"install_command":"npx skills add github/awesome-copilot --skill agentic-eval","trust_score":83,"trust_version":"trust-score-v5","risk_level":"low","do_not_use_when":["Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface"],"before_install":["Read the audit page and machine-readable metadata.","Confirm the install command, license, and permission surface fit the workspace.","Get explicit human approval or choose an alternative before installing."],"after_run":["Report the outcome to /api/agent/outcome using the resolve event id.","Include output_quality, workspace, human_review_required, and evidence_url when available.","Re-resolve before broad production rollout."]},"bestFor":["design-creative","agent-skill"],"doNotUseFor":["Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface"],"knownRisks":[],"backward_compatible":{"trust_score_v4":{"version":"trust-score-v4","score":86,"tier":"production","label":"Production candidate","summary":"Strong OpenAgentSkill Trust Score across adoption, recent maintenance, license clarity, documentation, dependency/runtime risk, install safety, permission surface, and install availability."}}},"trust_score_v5":{"version":"trust-score-v5","score":83,"base_score":86,"outcome_confidence":0,"tier":"strong","label":"Review then install","summary":"Good shortlist signal, but the agent should review audit notes, install policy, and outcome evidence before running it.","recommendedAction":"Use as the primary candidate after human or sandbox review.","decision":{"install_policy":"agent_install_candidate","auto_install_allowed":false,"human_review_required":true,"sandbox_first":true,"agent_action":"Ask for approval or run a sandbox-only trial before installing.","reasoning":["83/100 Trust Score v5","86/100 Trust Score v4 baseline","Needs more real agent outcomes before unattended install","Install path is available","Low metadata risk"],"review_required_when":["The workspace contains production secrets, payments, private customer data, or irreversible actions.","The install command requests shell, network, credential, database, or broad filesystem access.","Outcome evidence is missing, recently failed, or required human review.","Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface"]},"dimensions":[{"id":"github_adoption","label":"GitHub adoption","score":100,"weight":0.13,"status":"pass","detail":"39K GitHub stars"},{"id":"repo_activity","label":"Stars/forks activity","score":97,"weight":0.08,"status":"pass","detail":"39K stars, 4.9K forks; issue activity unavailable in current metadata"},{"id":"maintenance","label":"Recent maintenance","score":100,"weight":0.14,"status":"pass","detail":"4d since push"},{"id":"license","label":"License clarity","score":86,"weight":0.09,"status":"pass","detail":"MIT"},{"id":"documentation","label":"README/SKILL.md completeness","score":76,"weight":0.14,"status":"info","detail":"Public metadata needs stronger README/SKILL.md context"},{"id":"dependency_risk","label":"Dependency/runtime risk","score":90,"weight":0.12,"status":"pass","detail":"no major dependency risk hints in public metadata"},{"id":"installability","label":"Install availability","score":92,"weight":0.1,"status":"pass","detail":"npx skills add github/awesome-copilot --skill agentic-eval"},{"id":"install_safety","label":"Install command safety","score":68,"weight":0.1,"status":"info","detail":"dynamic command execution, standard package or runtime install path"},{"id":"permission_surface","label":"Permission surface","score":100,"weight":0.07,"status":"pass","detail":"no high-risk permission surface in public metadata"},{"id":"repository","label":"Repository evidence","score":86,"weight":0.04,"status":"pass","detail":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval"},{"id":"review_status","label":"Review status","score":88,"weight":0.05,"status":"pass","detail":"AI review data available"},{"id":"agent_outcomes","label":"Agent Proven outcomes","score":54,"weight":0.13,"status":"info","detail":"No agent outcome data yet"}],"checks":[{"status":"pass","label":"GitHub adoption","detail":"39K GitHub stars"},{"status":"pass","label":"Stars/forks activity","detail":"39K stars, 4.9K forks; issue activity unavailable in current metadata"},{"status":"pass","label":"Recent maintenance","detail":"4d since push"},{"status":"pass","label":"License clarity","detail":"MIT"},{"status":"info","label":"README/SKILL.md completeness","detail":"Public metadata needs stronger README/SKILL.md context"},{"status":"pass","label":"Dependency/runtime risk","detail":"no major dependency risk hints in public metadata"},{"status":"pass","label":"Install availability","detail":"npx skills add github/awesome-copilot --skill agentic-eval"},{"status":"info","label":"Install command safety","detail":"dynamic command execution, standard package or runtime install path"},{"status":"pass","label":"Permission surface","detail":"no high-risk permission surface in public metadata"},{"status":"pass","label":"Repository evidence","detail":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval"},{"status":"pass","label":"Review status","detail":"AI review data available"},{"status":"info","label":"Agent Proven outcomes","detail":"No agent outcome data yet"},{"status":"warn","label":"Ownership","detail":"No approved owner claim yet"},{"status":"pass","label":"OpenAgentSkill usage","detail":"2 views, 0 install copies"},{"status":"info","label":"Agent outcomes","detail":"No agent outcome data yet"}],"strengths":["AI review approved","Install path is available","Repository evidence is available","Recently maintained repository","Large GitHub adoption signal","Install command has no obvious high-risk pattern","Outcome loop is ready but needs first real agent run"],"warnings":["No real agent outcome reports yet","Human review required before unattended installation"],"evidence":{"stars":"39K GitHub stars","repoActivity":"39K stars, 4.9K forks","lastPushed":"4d since push","license":"MIT","repository":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval","install":"npx skills add github/awesome-copilot --skill agentic-eval","installSafety":"dynamic command execution, standard package or runtime install path","permissionSurface":"no high-risk permission surface in public metadata","documentation":"Usable metadata, review docs","agentOutcomes":"No agent outcome data yet","agentProvenScore":0,"outcomeConfidence":"0%","installPolicy":"agent_install_candidate"},"installReadiness":{"ready":true,"command":"npx skills add github/awesome-copilot --skill agentic-eval","policy":"agent_install_candidate","label":"Agent install candidate","notes":["Install path is available","Repository evidence is available","License is declared","No Agent Proven outcome evidence yet","4d since push","Trust Score v5 requires review or sandbox-only use before install."]},"agentCompatibility":["Codex","Claude Code","Cursor","OpenAgentSkill CLI"],"riskSummary":{"level":"low","label":"Low metadata risk","notes":["No major trust warnings detected from available metadata"]},"outcomeEvidence":{"total":0,"successes":0,"failures":0,"notRelevant":0,"successRate":null,"installAttempts":0,"riskBlocked":0,"setupRequired":0,"installSuccessRate":null,"avgOutputQuality":null,"avgTimeToUsefulMs":null,"productionOutcomes":0,"humanReviewRequired":0,"recentSuccessRate":null,"recentFailureRate":null,"uniqueAgents":0,"agentProvenScore":0,"agentProvenLabel":"Needs first agent run","lastOutcomeAt":null,"label":"No agent outcome data yet"},"autoInstall":{"allowed":false,"sandboxRequired":true,"policy":"agent_install_candidate","reason":"Ask for approval or run a sandbox-only trial before installing."},"outcome_loop":{"version":"openagentskill-agent-outcome-v4","required_after_install":true,"endpoint":"/api/agent/outcome","method":"POST","event_id_source":"feedback.event_id, install_receipt.resolve_event_id, or decision_packet.outcome_feedback.event_id","expected_outcomes":["success","failed","not_relevant","blocked_by_risk","setup_required"],"required_fields":["event_id","skill_slug","task"],"quality_fields":["task_success","output_quality","error_type","human_review_required","used_in_production","workspace","evidence_url","time_to_useful_ms","source_version"],"ranking_inputs_updated":["Trust Score v5 outcome confidence","Agent Proven Score","Resolve ranking task-fit evidence","Skill detail machine-readable metadata","Outcome leaderboard"]},"agent_contract":{"suited_tasks":["design-creative","agent-skill"],"suited_agents":["Codex","Claude Code","Cursor","OpenAgentSkill CLI"],"install_command":"npx skills add github/awesome-copilot --skill agentic-eval","trust_score":83,"trust_version":"trust-score-v5","risk_level":"low","do_not_use_when":["Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface"],"before_install":["Read the audit page and machine-readable metadata.","Confirm the install command, license, and permission surface fit the workspace.","Get explicit human approval or choose an alternative before installing."],"after_run":["Report the outcome to /api/agent/outcome using the resolve event id.","Include output_quality, workspace, human_review_required, and evidence_url when available.","Re-resolve before broad production rollout."]},"bestFor":["design-creative","agent-skill"],"doNotUseFor":["Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface"],"knownRisks":[],"backward_compatible":{"trust_score_v4":{"version":"trust-score-v4","score":86,"tier":"production","label":"Production candidate","summary":"Strong OpenAgentSkill Trust Score across adoption, recent maintenance, license clarity, documentation, dependency/runtime risk, install safety, permission surface, and install availability."}}},"trust_score_v4":{"version":"trust-score-v4","score":86,"tier":"production","label":"Production candidate","summary":"Strong OpenAgentSkill Trust Score across adoption, recent maintenance, license clarity, documentation, dependency/runtime risk, install safety, permission surface, and install availability.","recommendedAction":"Shortlist for production use, then run a normal repository and dependency review.","dimensions":[{"id":"github_adoption","label":"GitHub adoption","score":100,"weight":0.13,"status":"pass","detail":"39K GitHub stars"},{"id":"repo_activity","label":"Stars/forks activity","score":97,"weight":0.08,"status":"pass","detail":"39K stars, 4.9K forks; issue activity unavailable in current metadata"},{"id":"maintenance","label":"Recent maintenance","score":100,"weight":0.14,"status":"pass","detail":"4d since push"},{"id":"license","label":"License clarity","score":86,"weight":0.09,"status":"pass","detail":"MIT"},{"id":"documentation","label":"README/SKILL.md completeness","score":76,"weight":0.14,"status":"info","detail":"Public metadata needs stronger README/SKILL.md context"},{"id":"dependency_risk","label":"Dependency/runtime risk","score":90,"weight":0.12,"status":"pass","detail":"no major dependency risk hints in public metadata"},{"id":"installability","label":"Install availability","score":92,"weight":0.1,"status":"pass","detail":"npx skills add github/awesome-copilot --skill agentic-eval"},{"id":"install_safety","label":"Install command safety","score":68,"weight":0.1,"status":"info","detail":"dynamic command execution, standard package or runtime install path"},{"id":"permission_surface","label":"Permission surface","score":100,"weight":0.07,"status":"pass","detail":"no high-risk permission surface in public metadata"},{"id":"repository","label":"Repository evidence","score":86,"weight":0.04,"status":"pass","detail":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval"},{"id":"review_status","label":"Review status","score":88,"weight":0.05,"status":"pass","detail":"AI review data available"},{"id":"agent_outcomes","label":"Agent Proven outcomes","score":54,"weight":0.13,"status":"info","detail":"No agent outcome data yet"}],"checks":[{"status":"pass","label":"GitHub adoption","detail":"39K GitHub stars"},{"status":"pass","label":"Stars/forks activity","detail":"39K stars, 4.9K forks; issue activity unavailable in current metadata"},{"status":"pass","label":"Recent maintenance","detail":"4d since push"},{"status":"pass","label":"License clarity","detail":"MIT"},{"status":"info","label":"README/SKILL.md completeness","detail":"Public metadata needs stronger README/SKILL.md context"},{"status":"pass","label":"Dependency/runtime risk","detail":"no major dependency risk hints in public metadata"},{"status":"pass","label":"Install availability","detail":"npx skills add github/awesome-copilot --skill agentic-eval"},{"status":"info","label":"Install command safety","detail":"dynamic command execution, standard package or runtime install path"},{"status":"pass","label":"Permission surface","detail":"no high-risk permission surface in public metadata"},{"status":"pass","label":"Repository evidence","detail":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval"},{"status":"pass","label":"Review status","detail":"AI review data available"},{"status":"info","label":"Agent Proven outcomes","detail":"No agent outcome data yet"},{"status":"warn","label":"Ownership","detail":"No approved owner claim yet"},{"status":"pass","label":"OpenAgentSkill usage","detail":"2 views, 0 install copies"},{"status":"info","label":"Agent outcomes","detail":"No agent outcome data yet"}],"strengths":["AI review approved","Install path is available","Repository evidence is available","Recently maintained repository","Large GitHub adoption signal","Install command has no obvious high-risk pattern"],"warnings":[],"evidence":{"stars":"39K GitHub stars","repoActivity":"39K stars, 4.9K forks","lastPushed":"4d since push","license":"MIT","repository":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval","install":"npx skills add github/awesome-copilot --skill agentic-eval","installSafety":"dynamic command execution, standard package or runtime install path","permissionSurface":"no high-risk permission surface in public metadata","documentation":"Usable metadata, review docs","agentOutcomes":"No agent outcome data yet"},"installReadiness":{"ready":true,"command":"npx skills add github/awesome-copilot --skill agentic-eval","policy":"agent_install_candidate","label":"Agent install candidate","notes":["Install path is available","Repository evidence is available","License is declared","No Agent Proven outcome evidence yet","4d since push"]},"agentCompatibility":["Codex","Claude Code","Cursor","OpenAgentSkill CLI"],"riskSummary":{"level":"low","label":"Low metadata risk","notes":["No major trust warnings detected from available metadata"]},"outcomeEvidence":{"total":0,"successes":0,"failures":0,"notRelevant":0,"successRate":null,"installAttempts":0,"riskBlocked":0,"setupRequired":0,"installSuccessRate":null,"avgOutputQuality":null,"avgTimeToUsefulMs":null,"productionOutcomes":0,"humanReviewRequired":0,"recentSuccessRate":null,"recentFailureRate":null,"uniqueAgents":0,"agentProvenScore":0,"agentProvenLabel":"Needs first agent run","lastOutcomeAt":null,"label":"No agent outcome data yet"},"autoInstall":{"allowed":true,"sandboxRequired":true,"policy":"agent_install_candidate","reason":"Trust Score v4 allows sandbox-first agent installation after normal workspace review."},"bestFor":["design-creative","agent-skill"],"doNotUseFor":["Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface"],"knownRisks":[]},"agent_proven":{"version":"agent-proven-v1","score":0,"tier":"unproven","label":"Needs first agent run","summary":"No agent outcome reports yet. Use Resolve, run one narrow sandbox task, then report the result.","metrics":{"totalOutcomes":0,"successfulOutcomes":0,"failedOutcomes":0,"installAttempts":0,"installSuccessRate":null,"successRate":null,"recentSuccessRate":null,"recentFailureRate":null,"riskBlocked":0,"setupRequired":0,"notRelevant":0,"avgOutputQuality":null,"avgTimeToUsefulMs":null,"productionOutcomes":0,"humanReviewRequired":0,"uniqueAgents":0,"lastOutcomeAt":null},"signals":[],"penalties":["No real agent outcome evidence yet"]},"outcome_stats":null,"safety":{"score":82,"level":"safe_to_install","label":"Safe to install with normal review","safety_tier":{"tier":"reviewed","label":"Reviewed","badge":"REVIEWED","summary":"Good audit and safety signals with no high-risk permission hints in public metadata.","recommended_action":"Review the audit page, then allow agent install in a sandboxed workflow.","auto_install_policy":"allow","reasons":["Safe-to-try audit","82/100 agent safety score"]},"auto_install_allowed":true,"human_review_required":false,"blocked":false,"audit_risk":"safe_to_try","permission_hints":[{"id":"network","label":"Network access","reason":"Skill likely fetches remote pages, APIs, repositories, or external services.","severity":"medium"},{"id":"filesystem","label":"Filesystem access","reason":"Skill may read or write project files, documents, generated artifacts, or local workspace state.","severity":"medium"}],"policy_warnings":[],"constraints_applied":{"max_risk":"medium","needs_install_command":true,"min_stars":0}},"safety_gate":{"tier":"reviewed","label":"Reviewed","badge":"REVIEWED","auto_install_policy":"allow","auto_install_allowed":true,"blocked":false,"human_review_required":false,"recommended_action":"Review the audit page, then allow agent install in a sandboxed workflow.","reasons":["Safe-to-try audit","82/100 agent safety score"]},"eval":{"version":"openagentskill-skill-eval-v1","status":"review","score":87,"risk_level":"medium","decision":{"recommendation":"manual_review","reason":"Review the audit page, then allow agent install in a sandboxed workflow.","auto_install_allowed":false,"policy":"allow","human_review_required":true},"blockers":[],"warnings":["Install command safety: dynamic command execution, standard package or runtime install path","README/SKILL.md completeness: Public metadata needs stronger README/SKILL.md context"],"validation_plan":["Inspect repository, README/SKILL.md, license, and recent commits before production use.","Install in an isolated workspace or sandbox with no production secrets available.","Run the smallest representative task and record files touched, commands run, network access, and outputs.","Compare the selected skill against at least one alternative when the eval status is review or failed.","Promote only after the agent reports a successful verification result and unresolved warnings are accepted."],"checks":[{"id":"task_fit","label":"Task fit","status":"pass","score":84,"required_for_auto_install":true,"detail":"Task wording matches this skill metadata.","evidence":["Evaluate agentic-eval before installing it in an agent workflow","design-creative","Coding agents workflows; Claude Code teams; teams that value GitHub adoption signals"]},{"id":"install_path","label":"Install path","status":"pass","score":92,"required_for_auto_install":true,"detail":"Install handoff is available.","evidence":["npx skills add github/awesome-copilot --skill agentic-eval"]},{"id":"install_safety","label":"Install command safety","status":"warn","score":68,"required_for_auto_install":true,"detail":"dynamic command execution, standard package or runtime install path","evidence":["npx skills add github/awesome-copilot --skill agentic-eval"]},{"id":"trust_score","label":"Trust score","status":"pass","score":86,"required_for_auto_install":true,"detail":"Strong OpenAgentSkill Trust Score across adoption, recent maintenance, license clarity, documentation, dependency/runtime risk, install safety, permission surface, and install availability.","evidence":["Production candidate","39K GitHub stars","MIT"]},{"id":"audit_score","label":"Audit score","status":"pass","score":90,"required_for_auto_install":true,"detail":"Safe to try","evidence":["No major audit warning from metadata."]},{"id":"agent_safety_gate","label":"Agent safety gate","status":"pass","score":82,"required_for_auto_install":true,"detail":"Good audit and safety signals with no high-risk permission hints in public metadata.","evidence":["Review the audit page, then allow agent install in a sandboxed workflow.","Safe-to-try audit"]},{"id":"readme_skillmd_completeness","label":"README/SKILL.md completeness","status":"warn","score":76,"required_for_auto_install":false,"detail":"Public metadata needs stronger README/SKILL.md context","evidence":["Usable metadata, review docs"]},{"id":"license_clarity","label":"License clarity","status":"pass","score":86,"required_for_auto_install":true,"detail":"MIT","evidence":["MIT"]},{"id":"recent_maintenance","label":"Recent maintenance","status":"pass","score":100,"required_for_auto_install":false,"detail":"4d since push","evidence":["4d since push"]},{"id":"permission_surface","label":"Permission surface","status":"pass","score":100,"required_for_auto_install":true,"detail":"no high-risk permission surface in public metadata","evidence":["Network access: medium","Filesystem access: medium"]},{"id":"alternatives","label":"Alternatives available","status":"info","score":55,"required_for_auto_install":false,"detail":"No close alternatives were found in the current shortlist.","evidence":[]}],"endpoints":{"web":"https://www.openagentskill.com/skills/github-agentic-eval/evals","api":"/api/agent/evals?slug=github-agentic-eval","text":"/api/agent/evals?slug=github-agentic-eval&format=text"}},"agent_readable_metadata":{"version":"openagentskill-agent-metadata-v2","skill":{"slug":"github-agentic-eval","name":"agentic-eval","description":"Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when:\n- Implementing self-critique and reflection loops\n- Building evaluator-optimizer pipelines for quality-critical generation\n- Creating test-driven code refinement workflows\n- Designing rubric-based or LLM-as-judge evaluation systems\n- Adding iterative improvement to agent outputs (code, reports, analysis)\n- Measuring and improving agent response quality","category":"design-creative","url":"https://www.openagentskill.com/skills/github-agentic-eval","repository":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval","github_repo":"github/awesome-copilot"},"suited_tasks":["Coding agents workflows","Claude Code teams","teams that value GitHub adoption signals","Inspect source files","Explain architecture","Patch bugs and verify changes","Navigate pages","Click and type safely"],"suited_agents":["Codex","Claude Code","Cursor","OpenAgentSkill CLI","CLI"],"install":{"command":"npx skills add github/awesome-copilot --skill agentic-eval","ready":true,"targets":[{"id":"openagentskill-cli","label":"CLI","kind":"command","value":"npx --yes https://github.com/Leon-Drq/openagentskill/releases/download/cli-v0.3.0/openagentskill-0.3.0.tgz add github-agentic-eval"},{"id":"codex","label":"Codex","kind":"agent-prompt","value":"Install the \"agentic-eval\" agent skill from https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval. Read its SKILL.md or equivalent instructions first, install only the files needed for this workspace, and summarize any required setup before using it. Skill purpose: Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-judge evaluation systems - Adding iterative improvement to agent outputs (code, reports, analysis) - Measuring and improving agent response quality After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"github-agentic-eval\",\"task\":\"Install agentic-eval\",\"agent\":\"codex\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes."},{"id":"claude-code","label":"Claude Code","kind":"agent-prompt","value":"Add \"agentic-eval\" as a Claude Code skill from https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval. Inspect the skill instructions, place the reusable skill files in the appropriate local skills location for this project, and report the activation steps. Skill purpose: Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-judge evaluation systems - Adding iterative improvement to agent outputs (code, reports, analysis) - Measuring and improving agent response quality After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"github-agentic-eval\",\"task\":\"Install agentic-eval\",\"agent\":\"claude-code\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes."},{"id":"cursor","label":"Cursor","kind":"agent-prompt","value":"Turn \"agentic-eval\" from https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval into a reusable Cursor project rule or agent instruction. Preserve the core workflow, adapt paths to this repo, and keep the rule scoped to tasks where it is relevant. Skill purpose: Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-judge evaluation systems - Adding iterative improvement to agent outputs (code, reports, analysis) - Measuring and improving agent response quality After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"github-agentic-eval\",\"task\":\"Install agentic-eval\",\"agent\":\"cursor\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes."}],"handoff_url":"https://www.openagentskill.com/api/skills/github-agentic-eval/install","manifest_url":"https://www.openagentskill.com/api/registry/manifest/github-agentic-eval"},"trust":{"score":86,"label":"Production candidate","version":"trust-score-v4","install_policy":"agent_install_candidate","evidence":{"stars":"39K GitHub stars","repoActivity":"39K stars, 4.9K forks","lastPushed":"4d since push","license":"MIT","repository":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval","install":"npx skills add github/awesome-copilot --skill agentic-eval","installSafety":"dynamic command execution, standard package or runtime install path","permissionSurface":"no high-risk permission surface in public metadata","documentation":"Usable metadata, review docs","agentOutcomes":"No agent outcome data yet"},"outcome_evidence":{"total":0,"successes":0,"failures":0,"not_relevant":0,"success_rate":null,"recent_success_rate":null,"recent_failure_rate":null,"install_attempts":0,"install_success_rate":null,"risk_blocked":0,"setup_required":0,"avg_output_quality":null,"production_outcomes":0,"last_outcome_at":null,"label":"No agent outcome data yet"},"auto_install":{"allowed":true,"sandbox_required":true,"reason":"Trust Score v4 allows sandbox-first agent installation after normal workspace review."},"best_for":["design-creative","agent-skill"],"known_risks":[]},"agent_proven":{"version":"agent-proven-v1","score":0,"tier":"unproven","label":"Needs first agent run","summary":"No agent outcome reports yet. Use Resolve, run one narrow sandbox task, then report the result.","metrics":{"totalOutcomes":0,"successfulOutcomes":0,"failedOutcomes":0,"installAttempts":0,"installSuccessRate":null,"successRate":null,"recentSuccessRate":null,"recentFailureRate":null,"riskBlocked":0,"setupRequired":0,"notRelevant":0,"avgOutputQuality":null,"avgTimeToUsefulMs":null,"productionOutcomes":0,"humanReviewRequired":0,"uniqueAgents":0,"lastOutcomeAt":null},"signals":[],"penalties":["No real agent outcome evidence yet"]},"audit":{"score":90,"risk_level":"safe_to_try","risk_label":"Safe to try","warnings":[]},"safety_gate":{"tier":"reviewed","label":"Reviewed","auto_install_policy":"allow","auto_install_allowed":true,"human_review_required":false,"blocked":false,"recommended_action":"Review the audit page, then allow agent install in a sandboxed workflow."},"quality":{"score":92,"label":"Excellent"},"supply":{"track":"Coding and developer agents","scenario":"Coding agents","maintenance":"4d since push","risk":"Safe to try"},"alternative_skills":[],"do_not_use_when":["teams that need a vendor-supported SLA","high-compliance environments without internal security review","No major risk signals from current metadata","No major trust warnings detected from available metadata","Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface"],"agent_contract":{"task_input":"Use agentic-eval in an agent workflow","recommended_action":"Review the audit page, then allow agent install in a sandboxed workflow.","install_policy":"allow","minimum_review_before_use":["Trust: 86/100 Production candidate","Audit: 90/100 Safe to try","Safety: 82/100 Safe to install with normal review","Review repository, license, install command, and permission surface before production use."],"expected_agent_output":{"selected_skill":"github-agentic-eval (agentic-eval)","install_command":"npx skills add github/awesome-copilot --skill agentic-eval","risk_summary":"Safe to try; Reviewed; Low metadata risk","verification_result":"Report the smallest successful task, files touched, warnings, and any missing setup."}},"outcome_feedback":{"endpoint":"https://www.openagentskill.com/api/agent/outcome","method":"POST","requires_resolve_event_id":true,"event_id_source":"Use install_receipt.outcome_feedback.event_id or feedback.event_id returned by /api/agent/resolve for the current task.","expected_outcomes":["success","failed","not_relevant","blocked_by_risk","setup_required"],"payload_template":{"event_id":"<install_receipt.outcome_feedback.event_id or feedback.event_id from /api/agent/resolve>","skill_slug":"github-agentic-eval","task":"Use agentic-eval in an agent workflow","agent":"codex","outcome":"success","install_used":true,"risk_blocked":false,"setup_required":false,"task_success":true,"output_quality":4,"error_type":null,"human_review_required":false,"workspace":"sandbox","time_to_useful_ms":120000,"notes":"Report the smallest successful task, setup friction, files touched, and risk notes."}},"endpoints":{"web":"https://www.openagentskill.com/skills/github-agentic-eval","api":"https://www.openagentskill.com/api/agent/skills/github-agentic-eval","audit":"https://www.openagentskill.com/skills/github-agentic-eval/audit","eval":"https://www.openagentskill.com/api/agent/evals?slug=github-agentic-eval&task=Use%20agentic-eval%20in%20an%20agent%20workflow&max_risk=medium","resolve":"https://www.openagentskill.com/api/agent/resolve?task=Use%20agentic-eval%20in%20an%20agent%20workflow&agent=codex&max_risk=medium","receipt":"https://www.openagentskill.com/api/agent/receipt?task=Use%20agentic-eval%20in%20an%20agent%20workflow&agent=codex&max_risk=medium&format=text","install":"https://www.openagentskill.com/api/skills/github-agentic-eval/install","manifest":"https://www.openagentskill.com/api/registry/manifest/github-agentic-eval"}},"machine_metadata":{"version":"openagentskill-agent-metadata-v2","skill":{"slug":"github-agentic-eval","name":"agentic-eval","description":"Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when:\n- Implementing self-critique and reflection loops\n- Building evaluator-optimizer pipelines for quality-critical generation\n- Creating test-driven code refinement workflows\n- Designing rubric-based or LLM-as-judge evaluation systems\n- Adding iterative improvement to agent outputs (code, reports, analysis)\n- Measuring and improving agent response quality","category":"design-creative","url":"https://www.openagentskill.com/skills/github-agentic-eval","repository":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval","github_repo":"github/awesome-copilot"},"suited_tasks":["Coding agents workflows","Claude Code teams","teams that value GitHub adoption signals","Inspect source files","Explain architecture","Patch bugs and verify changes","Navigate pages","Click and type safely"],"suited_agents":["Codex","Claude Code","Cursor","OpenAgentSkill CLI","CLI"],"install":{"command":"npx skills add github/awesome-copilot --skill agentic-eval","ready":true,"targets":[{"id":"openagentskill-cli","label":"CLI","kind":"command","value":"npx --yes https://github.com/Leon-Drq/openagentskill/releases/download/cli-v0.3.0/openagentskill-0.3.0.tgz add github-agentic-eval"},{"id":"codex","label":"Codex","kind":"agent-prompt","value":"Install the \"agentic-eval\" agent skill from https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval. Read its SKILL.md or equivalent instructions first, install only the files needed for this workspace, and summarize any required setup before using it. Skill purpose: Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-judge evaluation systems - Adding iterative improvement to agent outputs (code, reports, analysis) - Measuring and improving agent response quality After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"github-agentic-eval\",\"task\":\"Install agentic-eval\",\"agent\":\"codex\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes."},{"id":"claude-code","label":"Claude Code","kind":"agent-prompt","value":"Add \"agentic-eval\" as a Claude Code skill from https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval. Inspect the skill instructions, place the reusable skill files in the appropriate local skills location for this project, and report the activation steps. Skill purpose: Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-judge evaluation systems - Adding iterative improvement to agent outputs (code, reports, analysis) - Measuring and improving agent response quality After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"github-agentic-eval\",\"task\":\"Install agentic-eval\",\"agent\":\"claude-code\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes."},{"id":"cursor","label":"Cursor","kind":"agent-prompt","value":"Turn \"agentic-eval\" from https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval into a reusable Cursor project rule or agent instruction. Preserve the core workflow, adapt paths to this repo, and keep the rule scoped to tasks where it is relevant. Skill purpose: Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-judge evaluation systems - Adding iterative improvement to agent outputs (code, reports, analysis) - Measuring and improving agent response quality After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"github-agentic-eval\",\"task\":\"Install agentic-eval\",\"agent\":\"cursor\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes."}],"handoff_url":"https://www.openagentskill.com/api/skills/github-agentic-eval/install","manifest_url":"https://www.openagentskill.com/api/registry/manifest/github-agentic-eval"},"trust":{"score":86,"label":"Production candidate","version":"trust-score-v4","install_policy":"agent_install_candidate","evidence":{"stars":"39K GitHub stars","repoActivity":"39K stars, 4.9K forks","lastPushed":"4d since push","license":"MIT","repository":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval","install":"npx skills add github/awesome-copilot --skill agentic-eval","installSafety":"dynamic command execution, standard package or runtime install path","permissionSurface":"no high-risk permission surface in public metadata","documentation":"Usable metadata, review docs","agentOutcomes":"No agent outcome data yet"},"outcome_evidence":{"total":0,"successes":0,"failures":0,"not_relevant":0,"success_rate":null,"recent_success_rate":null,"recent_failure_rate":null,"install_attempts":0,"install_success_rate":null,"risk_blocked":0,"setup_required":0,"avg_output_quality":null,"production_outcomes":0,"last_outcome_at":null,"label":"No agent outcome data yet"},"auto_install":{"allowed":true,"sandbox_required":true,"reason":"Trust Score v4 allows sandbox-first agent installation after normal workspace review."},"best_for":["design-creative","agent-skill"],"known_risks":[]},"agent_proven":{"version":"agent-proven-v1","score":0,"tier":"unproven","label":"Needs first agent run","summary":"No agent outcome reports yet. Use Resolve, run one narrow sandbox task, then report the result.","metrics":{"totalOutcomes":0,"successfulOutcomes":0,"failedOutcomes":0,"installAttempts":0,"installSuccessRate":null,"successRate":null,"recentSuccessRate":null,"recentFailureRate":null,"riskBlocked":0,"setupRequired":0,"notRelevant":0,"avgOutputQuality":null,"avgTimeToUsefulMs":null,"productionOutcomes":0,"humanReviewRequired":0,"uniqueAgents":0,"lastOutcomeAt":null},"signals":[],"penalties":["No real agent outcome evidence yet"]},"audit":{"score":90,"risk_level":"safe_to_try","risk_label":"Safe to try","warnings":[]},"safety_gate":{"tier":"reviewed","label":"Reviewed","auto_install_policy":"allow","auto_install_allowed":true,"human_review_required":false,"blocked":false,"recommended_action":"Review the audit page, then allow agent install in a sandboxed workflow."},"quality":{"score":92,"label":"Excellent"},"supply":{"track":"Coding and developer agents","scenario":"Coding agents","maintenance":"4d since push","risk":"Safe to try"},"alternative_skills":[],"do_not_use_when":["teams that need a vendor-supported SLA","high-compliance environments without internal security review","No major risk signals from current metadata","No major trust warnings detected from available metadata","Production credentials, payments, or irreversible account changes without explicit human review","Sensitive private data before reviewing repository code, license, and permission surface"],"agent_contract":{"task_input":"Use agentic-eval in an agent workflow","recommended_action":"Review the audit page, then allow agent install in a sandboxed workflow.","install_policy":"allow","minimum_review_before_use":["Trust: 86/100 Production candidate","Audit: 90/100 Safe to try","Safety: 82/100 Safe to install with normal review","Review repository, license, install command, and permission surface before production use."],"expected_agent_output":{"selected_skill":"github-agentic-eval (agentic-eval)","install_command":"npx skills add github/awesome-copilot --skill agentic-eval","risk_summary":"Safe to try; Reviewed; Low metadata risk","verification_result":"Report the smallest successful task, files touched, warnings, and any missing setup."}},"outcome_feedback":{"endpoint":"https://www.openagentskill.com/api/agent/outcome","method":"POST","requires_resolve_event_id":true,"event_id_source":"Use install_receipt.outcome_feedback.event_id or feedback.event_id returned by /api/agent/resolve for the current task.","expected_outcomes":["success","failed","not_relevant","blocked_by_risk","setup_required"],"payload_template":{"event_id":"<install_receipt.outcome_feedback.event_id or feedback.event_id from /api/agent/resolve>","skill_slug":"github-agentic-eval","task":"Use agentic-eval in an agent workflow","agent":"codex","outcome":"success","install_used":true,"risk_blocked":false,"setup_required":false,"task_success":true,"output_quality":4,"error_type":null,"human_review_required":false,"workspace":"sandbox","time_to_useful_ms":120000,"notes":"Report the smallest successful task, setup friction, files touched, and risk notes."}},"endpoints":{"web":"https://www.openagentskill.com/skills/github-agentic-eval","api":"https://www.openagentskill.com/api/agent/skills/github-agentic-eval","audit":"https://www.openagentskill.com/skills/github-agentic-eval/audit","eval":"https://www.openagentskill.com/api/agent/evals?slug=github-agentic-eval&task=Use%20agentic-eval%20in%20an%20agent%20workflow&max_risk=medium","resolve":"https://www.openagentskill.com/api/agent/resolve?task=Use%20agentic-eval%20in%20an%20agent%20workflow&agent=codex&max_risk=medium","receipt":"https://www.openagentskill.com/api/agent/receipt?task=Use%20agentic-eval%20in%20an%20agent%20workflow&agent=codex&max_risk=medium&format=text","install":"https://www.openagentskill.com/api/skills/github-agentic-eval/install","manifest":"https://www.openagentskill.com/api/registry/manifest/github-agentic-eval"}},"supply_profile":{"track":{"slug":"coding","label":"Coding and developer agents","shortLabel":"Coding","description":"Code review, repo analysis, testing, CI, GitHub, DevOps, and developer workflow skills."},"scenario":{"label":"Coding agents","description":"I need a coding agent that can understand a repository, edit code, and review pull requests.","useCases":[{"slug":"coding-agents","title":"Coding agents"},{"slug":"browser-automation","title":"Browser automation"},{"slug":"research-agents","title":"Research agents"}]},"applicableAgents":["Claude Code","CLI","Codex","Cursor"],"install":{"ready":true,"command":"npx skills add github/awesome-copilot --skill agentic-eval","primaryTarget":"CLI","targetCount":4},"githubQuality":{"stars":38524,"starsLabel":"39K","forks":4869,"license":"MIT","qualityScore":92,"trustScore":86,"auditScore":90},"maintenance":{"status":"fresh","label":"4d since push","daysSincePush":4,"lastPushedAt":"2026-09-01T20:53:42+00:00"},"risk":{"level":"safe_to_try","label":"Safe to try","requiresReview":false,"notes":["No major risk signals from available metadata"]},"coverageTags":["Coding","Coding agents","design-creative","agent-skill"]},"audit":{"audit_score":90,"risk_level":"safe_to_try","risk_label":"Safe to try","quality_score":92,"trust_score":86,"maintenance_score":100,"security_score":86,"install_score":92,"warnings":[]},"quality_signals":{"model":"v2","star_score":32.1,"usage_score":0,"review_score":5.1,"metadata_score":3,"freshness_score":15},"platforms":["Claude Code"],"use_cases":[{"slug":"coding-agents","title":"Coding agents","url":"https://www.openagentskill.com/use-cases/coding-agents"},{"slug":"browser-automation","title":"Browser automation","url":"https://www.openagentskill.com/use-cases/browser-automation"},{"slug":"research-agents","title":"Research agents","url":"https://www.openagentskill.com/use-cases/research-agents"},{"slug":"github-automation","title":"GitHub automation","url":"https://www.openagentskill.com/use-cases/github-automation"}],"stacks":[{"slug":"coding-review-agent","title":"Coding review agent","url":"https://www.openagentskill.com/collections/coding-review-agent"},{"slug":"browser-qa-agent","title":"Browser QA agent","url":"https://www.openagentskill.com/collections/browser-qa-agent"},{"slug":"frontend-product-ui","title":"Frontend and UI","url":"https://www.openagentskill.com/collections/frontend-product-ui"}],"install":"npx skills add github/awesome-copilot --skill agentic-eval","install_targets":[{"id":"openagentskill-cli","label":"CLI","title":"OpenAgentSkill CLI","kind":"command","value":"npx --yes https://github.com/Leon-Drq/openagentskill/releases/download/cli-v0.3.0/openagentskill-0.3.0.tgz add github-agentic-eval","description":"Resolve policy, run the source installer safely, and report a verified install receipt.","copyLabel":"Copy command"},{"id":"codex","label":"Codex","title":"Codex install prompt","kind":"agent-prompt","value":"Install the \"agentic-eval\" agent skill from https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval. Read its SKILL.md or equivalent instructions first, install only the files needed for this workspace, and summarize any required setup before using it. Skill purpose: Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-judge evaluation systems - Adding iterative improvement to agent outputs (code, reports, analysis) - Measuring and improving agent response quality After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"github-agentic-eval\",\"task\":\"Install agentic-eval\",\"agent\":\"codex\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes.","description":"Give Codex a repo-aware install prompt when the skill is not available through a local CLI.","copyLabel":"Copy prompt"},{"id":"claude-code","label":"Claude Code","title":"Claude Code skill prompt","kind":"agent-prompt","value":"Add \"agentic-eval\" as a Claude Code skill from https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval. Inspect the skill instructions, place the reusable skill files in the appropriate local skills location for this project, and report the activation steps. Skill purpose: Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-judge evaluation systems - Adding iterative improvement to agent outputs (code, reports, analysis) - Measuring and improving agent response quality After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"github-agentic-eval\",\"task\":\"Install agentic-eval\",\"agent\":\"claude-code\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes.","description":"Use this prompt to ask Claude Code to add the skill and explain the local activation steps.","copyLabel":"Copy prompt"},{"id":"cursor","label":"Cursor","title":"Cursor rule prompt","kind":"agent-prompt","value":"Turn \"agentic-eval\" from https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval into a reusable Cursor project rule or agent instruction. Preserve the core workflow, adapt paths to this repo, and keep the rule scoped to tasks where it is relevant. Skill purpose: Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-judge evaluation systems - Adding iterative improvement to agent outputs (code, reports, analysis) - Measuring and improving agent response quality After the install attempt, report the result to https://www.openagentskill.com/api/agent/outcome with POST JSON {\"event_id\":\"install_<unique-id>\",\"skill_slug\":\"github-agentic-eval\",\"task\":\"Install agentic-eval\",\"agent\":\"cursor\",\"outcome\":\"success\",\"install_used\":true}. Replace event_id with a unique value and outcome with success or failed. Report success only after the skill is installed and a minimal verification passes.","description":"Use this when installing as Cursor project rules or reusable agent instructions.","copyLabel":"Copy prompt"}],"repository":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval","github_repo":"github/awesome-copilot","version":"1.0.0","license":"MIT","urls":{"web":"https://www.openagentskill.com/skills/github-agentic-eval","repository":"https://github.com/github/awesome-copilot/tree/main/skills/agentic-eval","api":"/api/agent/skills/github-agentic-eval","install_api":"/api/skills/github-agentic-eval/install"},"meta":{"created_at":"2026-09-01T22:43:27.850025+00:00","updated_at":"2026-09-01T22:43:28.005548+00:00","agent_friendly":true}}