Skip to content
OpenAgentSkill
Browsing is open · Catalog captured 2026-11-21 · Automatic updates and account services are paused.
Security

input-token-overheads

Use when context window is filling up too fast or input token cost is too high. Audits overhead sources.

moonlight-lupin · Static checks on file

Installation

Read the source and dependencies before running a command. Copying does not install or execute anything.

npx skills add moonlight-lupin/agent-skills --skill input-token-overheads

About this skill


name: input-token-overheads description: "Use when context window is filling up too fast or input token cost is too high. Audits overhead sources." license: MIT metadata: version: 1.4.0 author: moonlight-lupin platforms: [linux, macos, windows] tags: [tokens, overhead, context, optimization, agent-ops] related_skills: [skill-maintainer]

Input Token Overheads

Audit every source of per-turn input token cost on a Hermes Agent instance. Measure each, rank by cost, act on the top consumers.

When to Use

  • User says "token overhead", "context too large", "why is input so expensive"
  • Model output quality degrades from context dilution
  • Cost optimization — fewer input tokens per turn means lower API spend
  • After adding skills, plugins, or tools — verify the overhead delta

The Overhead Map

Every turn, Hermes injects these blocks into the system prompt before the user's message:

BlockWhen loadedCost model
Skill descriptionsEvery turn (skill-retrieval top-K)~200 chars per description, K per turn
Memory (personal notes)Every turnStatic, grows with usage
User profileEvery turnStatic, grows as preferences accumulate
Memory provider contextEvery turn (if memory plugin active)Dynamic, 5 memories recalled by default
Tool schemas (direct)Every turnFull JSON schema per enabled tool
Deferred tool catalogEvery turn (if configured)Name + description only
Mandatory skillsEvery turn (if configured)Full SKILL.md body
Platform formatting rulesEvery turnFixed, platform-specific
Behavioral rulesEvery turnFixed system prompt text
Full skill bodyOn-demand (skill_view)Only when a skill is loaded
Compression summaryAfter thresholdReplaces older messages with a summary

On-demand (not per-turn): full SKILL.md via skill_view, deferred tool schemas via tool_describe, reference files via skill_view(file_path=...).

Health Ratio

The health metric is overhead ratio: overhead tokens divided by the model's context window. The absolute number matters for cost; the ratio matters for quality.

RatioRatingNotes
< 5%ExcellentMost of the window available for conversation
5-15%HealthyNormal for a capable agent with tools, skills, memory
15-25%AcceptableApproaching the limit. Consider trimming.
> 25%UnhealthyEats conversation capacity. Cost and quality risk.

Why the ratio matters: Three studies confirm that input length degrades model performance independent of content quality:

  1. Lost in the Middle (Liu et al., TACL 2023) — Models follow a U-shaped curve: best recall at the start and end of context, severe degradation in the middle. Overhead sits at the top of every turn, but it pushes conversation history into the degradation zone. arxiv.org/abs/2307.03172

  2. Same Task, More Tokens (Levy et al., ACL 2024) — Reasoning performance degrades at input lengths far shorter than the model's stated maximum. The degradation appears even when the extra tokens are padding with no distracting content. The model's technical context window is not its effective context window. aclanthology.org/2024.acl-long.818

  3. Context Length Alone Hurts (Du et al., EMNLP 2025) — Performance degrades 14-85% as input length increases, even when retrieval is perfect, irrelevant tokens are replaced with whitespace, or all tokens except relevant ones are masked. The sheer length of the input is itself a limitation. aclanthology.org/2025.findings-emnlp.1264

Cost compounding: Overhead is paid every turn. At 10k tokens over 100 turns, that is 1M input tokens spent on overhead alone. Reducing overhead by 2k tokens saves 200k tokens per 100-turn session.

Mitigations from the research:

FindingSourceAction
Models recall start and end of context best; middle degradesLiu et al. 2023Keep overhead at the top (Hermes already does this). Avoid pushing critical conversation history into the middle — lower compression threshold if history is being compressed too aggressively
Reasoning degrades well below the stated context window maximumLevy et al. 2024Treat the effective context window as 50-70% of the advertised maximum. Target an overhead ratio under 10% of the advertised window, not the effective one
Sheer input length hurts even with perfect retrieval and no distractionDu et al. 2025Reduce overhead aggressively. Every 1k tokens of overhead removed improves reasoning quality, not just cost. The study's mitigation: prompt the model to recite key evidence before solving — equivalent to Hermes compression summarizing relevant context
Tool calling degrades 7-85% as tool catalog grows from 8k to 120k tokensLongFuncEval (arxiv 2505.10570)Keep the enabled toolset count low. Prefer deferred tools (loaded on demand) over always-on schemas. Disable unused toolsets

Procedure

1. Measure each overhead source

Run the audit script to get real numbers:

python3 -c "
import yaml, pathlib, glob, os, re

# --- Skill descriptions (skill-retrieval index) ---
files = glob.glob(os.path.expanduser('~/.hermes/skills/**/SKILL.md'), recursive=True)
total_desc = 0; count = 0; by_cat = {}
for f in files:
    try:
        text = pathlib.Path(f).read_text()
        m = re.match(r'^---\n(.*?)\n---\n', text, re.DOTALL)
        if not m: continue
        fm = yaml.safe_load(m.group(1))
        if not fm: continue
        desc = fm.get('description', '')
        if not desc: continue
        cat = f.split('/skills/')[1].split('/')[0]
        by_cat.setdefault(cat, [0,0]); by_cat[cat][0] += len(desc); by_cat[cat][1] += 1
        total_desc += len(desc); count += 1
 
Read the complete source ↗