Setup Skill: Onboard Non-Technical Users to Multi-Judge Agents

Where this fits

You've built an agent that uses multiple judge models (Codex, OpenCode, OpenRouter, Gemini) to evaluate outputs. Your end user isn't a developer—they just want to run the system. But every model needs a connection, and they don't know where to start. The setup skill turns that painful onboarding into a guided, step-by-step conversation that even the least technical person can follow without feeling lost.

Why agents benefit

• Probes first, explains later: The skill runs a probe script to detect which backends are missing, then tailors instructions to exactly what the user needs to do—no generic docs.
• Plain‑language guidance: It deliberately avoids jargon, walls of text, and assumes zero technical background. Each model is connected one at a time, in a natural order.
• Self‑contained: Uses only allowed tools (Bash, Read, Grep, Glob) to locate its own scripts and execute them, so it works across different installations.
• Reduces support load: Instead of the user filing a ticket, the agent resolves configuration gaps autonomously, freeing you from hand‑holding.
• Idempotent and safe: The probe runs once, the skill reads the output without re‑running, and it never modifies anything without explicit user action.

Practical scenarios

Onboarding a new evaluator to a multi‑agent review system

A product manager wants to use your agent for code review but has no clue about API keys. They say "set up multi". The skill probes, finds three of four backends missing, and walks them through generating keys and pasting them into the right config files—one model at a time.

Recovering from a failed probe after a backend API key change

One of your judge models expires its token. The next run triggers a probe that flags the missing backend. Instead of a cryptic error, the agent invokes setup and guides the user through updating just that one key, without touching the others.

Adding a new judge model without reading manual docs

You want to add Gemini as a fourth judge. The skill detects it's unconfigured and provides the exact steps to obtain an API key and set it up, including where to place it in the config—no need to dig through READMEs.

Add it to your agent workflow

Install the skill with:

npx skills add szarkans/multi --skill setup

Then let your agent trigger it when the probe shows missing backends or when the user asks for "multi setup". For example, in your agent's system prompt: "If the user says 'configure judges' or 'set up multi', run the setup skill." The skill itself requires no arguments—just run it and let the probe do the work.

Compare before adopting

Before adding this to your stack, compare it against alternatives. Check the GitHub repo (szarkans/multi) for stars, recent commits, and open issues. Ask: Does the probe logic match your current backend set? Is the skill actively maintained? Could a simpler inline script achieve the same? Also consider whether your users are technical—if they are, this skill might be overkill.

Why it is worth tracking

With only 4 stars, this is an early‑stage skill, but its design is notably thoughtful: it solves a real pain point (non‑technical onboarding) in a way that scales across multiple judge models. The probe‑first approach and plain‑language instructions are strong signals. Keep an eye on its GitHub activity and community adoption—if it gains traction, it could become a standard part of multi‑agent evaluation setups. Evaluate it now if you're already using multiple judge backends and want to reduce onboarding friction. For more context, see OpenAgentSkill's listing.