Self-Evolving Skills: Run WikiSkill's Loop on Your Agent

Where this fits

You've been running a customer-support agent for weeks, and it still fumbles the same edge cases—missing the refund policy, misinterpreting a return request. You know the answers are buried in its own session logs, but manually extracting them into reusable skills is tedious and error-prone. wikiskill-evolve automates that loop: it takes raw agent traces, distills failure patterns into a persistent wiki, proposes candidate skills, and only accepts them when they measurably improve validation scores. This is the exact workflow from Google's WikiSkill paper (arXiv:2608.27454), implemented as a practical skill for Hermes workspaces.

Why agents benefit

• Self-evolution from real experience – The agent learns from its own failures and successes, not from hand-crafted examples, so skills stay aligned with actual task distributions.
• Strict statistical gating – A candidate skill is accepted only if R_val > R_best; otherwise, the system does a git rollback. No more blind trust in 'it feels better.'
• Persistent knowledge accumulation – The maintainer writes distilled insights to a wiki, so knowledge compounds across sessions and doesn't vanish after a single run.
• Reproducible protocol – Faithfully implements the paper's raw→maintainer→proposer→gate pipeline, so you can compare results with published benchmarks.
• Safe, revertible changes – Every evolution step is tracked in git, giving you full auditability and the ability to revert to a known-good state.

Practical scenarios

1. Customer-support agent that learns from ticket histories

Feed it past conversation logs; the loop identifies recurring complaint patterns, proposes a skill for handling each, and validates that the new skill actually reduces unresolved tickets.

2. Code-generation agent that improves from test failures

After each coding task, the agent's failed attempts become raw sessions. The maintainer extracts common pitfalls (e.g., missing null checks), and the proposer writes a skill that preempts them, gated by pass-rate improvements on held-out tests.

3. Research assistant refining tool-use strategies

For an agent that calls external APIs, the loop learns which call sequences yield better results (e.g., query formatting, retry logic) and codifies them into skills, validated against a small evaluation set.

Add it to your agent workflow

Install the skill via the OpenAgentSkill registry:

npx skills add ashutoshsinghpr7/wikiskill --skill wikiskill-evolve

Or directly via pip (requires Python ≥ 3.10):

pip install wikiskill

Initialize a workspace and run the evolution loop:

wikiskill init myws
wikiskill evolve myws --sessions ./raw_logs --eval ./heldout_tasks

The loop will produce a wiki, candidate skills, and a validation report showing whether each candidate was accepted or rolled back.

Compare before adopting

Before making this a permanent part of your stack, compare it against your current skill-management approach. Look at quality signals – does the R_val > R_best gate align with your own success metrics? Check maintenance freshness – how often does the wiki get updated, and does that cadence match your agent's learning needs? Consider alternatives like manual skill curation or simpler prompt tweaking, and weigh the overhead of running a multi-stage pipeline. Finally, assess workflow fit – this skill is designed for Hermes workspaces; if your agent runs on a different backend, you'll need to adapt the integration.

Why it is worth tracking

With 40 stars and a direct link to a peer-reviewed protocol, wikiskill-evolve is a credible implementation of a research idea that's rapidly gaining traction in the agent community. It's worth tracking because it addresses a core pain point – turning raw agent experience into validated, reusable skills – and it does so with statistical rigor rather than vibes. The project is still young, so expect API changes and feature additions. Evaluate it now if you're building self-improving agents; the pattern it implements may become a standard component in agent toolchains. For more details and updates, see the skill page on OpenAgentSkill.