rag-answerer: Keep Agent Answers Inside Your Context

Where this fits

You have a support agent that summarizes internal policy documents. A user asks about refunds, your retrieval step returns the refund policy snippet, but the model adds "you can also claim via email" from training memory. That unverified addition shifts the answer from grounded to risky. rag-answerer is a small skill that locks the model to the supplied context: answer from it, or say the context doesn't specify.

Why agents benefit

• Answers can be attributed: The skill asks the model to cite relevant phrases, so a user or downstream validator can trace claims back to the retrieved passage.
• Unsupported questions produce a clear not-in-context signal instead of a plausible guess, which makes it easier to trigger a new retrieval or human handoff.
• It reduces prompt drifting across agent runs because the hard rule is encoded in a reusable skill, not a one-off instruction.
• Short answers lower the chance of wrapping retrieved facts in filler or contradictory background knowledge.
• It composes with any RAG pipeline: the skill only constrains how the model answers, so retrieval and reranking stay under your control.

Practical scenarios

Internal wiki Q&A

An employee-facing agent searches Confluence and passes top chunks to rag-answerer. If a policy has no relevant section, the agent says so instead of inventing a policy.

Contract clause lookup

A legal team assistant pulls clauses from approved contracts. The skill forces the assistant to quote the exact clause language for dates, liability caps, or notice periods.

Customer support copilot

A support agent gets the knowledge base article most similar to the ticket. rag-answerer keeps the reply inside that article, flagging when the issue isn't covered so the human agent can escalate.

Add it to your agent workflow

Install the skill in your agent project:

npx skills add lizhiyao/oh-my-knowledge --skill rag-answerer

Then attach it to the agent prompt or skill list and include context:

Use the rag-answerer skill.
Context: [retrieved passages for user question]
Question: [user question]

The agent will answer from the context only, and if the context doesn't contain an answer, it will state that explicitly.

Compare before adopting

Before depending on rag-answerer, compare a few things:

• Quality signals: Does it consistently produce not-in-context statements for out-of-scope queries? Test it on your own retrieval failures, not just its README examples.
• Maintenance freshness: The repository has 18 stars and a narrow, stable rule set. Check whether the author actively responds to issues or updates the skill for new agent runtimes.
• Alternatives: You might already enforce context-only behavior with a system prompt. Compare the skill's explicit hardRules and command-based install against your template's maintainability.
• Workflow fit: Does your team already have a skill registry? Is a one-line install easier than updating a shared prompt? If you need strict citation formatting, the skill may need adaptation. You can also browse its OpenAgentSkill listing at https://www.openagentskill.com/skills/lizhiyao-rag-answerer for metadata and provenance.

Why it is worth tracking

This skill is a good signal for the broader trend of "small, constrained skills" that fix one risky behavior rather than adding agentic freedom. Its 18 stars are modest, but the rule is sharp: answer from context or say no. That clarity makes it easy to evaluate in an afternoon. Track it alongside retrieval quality improvements; when your RAG pipeline returns better context, a strict answerer skill converts that gain directly into safer responses. Re-evaluate if the repository stops being updated or if a more flexible citation mode becomes necessary.