OpenAgentSkill Registry Manifest
Skill: annotator-input-parity-check
Slug: kennethkhoocy-annotator-input-parity-check
Category: security
Description: Before designing, training, or auditing ANY model that replicates
human-annotated labels, audit the annotation protocol's INPUT — the exact
document/evidence the human labelers consulted — and give the model that
same input. Use when: (1) designing a classifier/LLM extractor whose target
is a hand-coded label set, (2) a label-replication model shows low recall
concentrated in a label subset and the diagnosis on offer is "the label's
information is not in the features", (3) reviewers propose construct splits
(e.g. "designation vs record-evident"), adjudication sittings, or per-domain
stop rules to explain residual disagreement with gold, (4) validating an
extraction pipeline against labels transcribed from a source document.
Symptom of the underlying failure: elaborate theory accumulates to explain
why gold is "partially unpredictable" when the model was simply never shown
the document the annotators read.

Agent fit:
- Decision: 63/100 Prototype first
- Primary fit: Research agents
- Role: Fallback candidate

Supply profile:
- Track: Research and knowledge work
- Scenario: Research agents
- Applicable agents: Claude Code, CLI, Codex, Cursor
- Maintenance: Pushed today
- Risk: Needs review

Trust:
- Trust score: 71/100 Manual review
- Audit: 77/100 Needs review

Attribution:
- Status: Registry indexed
- Source: recursive skill source sync
- Creator: Claude Code
- Claim URL: https://www.openagentskill.com/skills/kennethkhoocy-annotator-input-parity-check#claim-this-skill

Install:
npx skills add kennethkhoocy/applied-micro-skills --skill annotator-input-parity-check

URLs:
- Web: https://www.openagentskill.com/skills/kennethkhoocy-annotator-input-parity-check
- API: https://www.openagentskill.com/api/agent/skills/kennethkhoocy-annotator-input-parity-check
- Install API: https://www.openagentskill.com/api/skills/kennethkhoocy-annotator-input-parity-check/install
- Repository: https://github.com/kennethkhoocy/applied-micro-skills/tree/main/plugins/applied-micro/skills/annotator-input-parity-check