Debug Empty Agent Traces: launch failures vs behavior

Where this fits

You're debugging an agent evaluation run that produced four traces with only the initial prompt — no tool calls, no API calls, no final answer. Before you blame your agent's reasoning loop, you need to confirm the harness actually launched the agent. This skill encodes that first diagnostic step: empty session traces usually mean a launch failure, not a behavioral failure.

Why agents benefit

• Prevents false positives — Stops your agent from analyzing non-existent behavior and drawing conclusions from noise.
• Saves debugging cycles — Instead of inspecting prompt chains and tool outputs, you check three fields (api_call_count, tool_call_count, stdout.txt) and immediately know if the run died at launch.
• Improves evaluation reliability — By filtering out dead runs, you ensure your agent only learns from sessions where real interaction happened.
• Automates root-cause triage — The skill's logic can be embedded in your analysis pipeline to flag launch failures before any deeper trace analysis runs.
• Aligns scores with actual behavior — It reminds you that scores in meta.json may come from grading deliverables, not from agent execution, so you don't over-trust them.

Practical scenarios

CI evaluation with silent launch failures

Your nightly batch of 100 agent runs shows 4 sessions with zero tool calls. The skill's check immediately reveals stdout.txt contains "HTTP 400: default is not a valid model ID" — the harness misconfigured the model. You fix the config and rerun, avoiding days of misdiagnosis.

Comparing agent versions on a benchmark

You're A/B testing two prompt strategies. One version shows suspiciously low tool-call counts. Instead of assuming the prompts are bad, you apply this skill to confirm the low counts come from launch crashes, not from the agent's decision to skip tools.

Auditing third-party evaluation datasets

You inherit a dataset with pre-recorded traces and scores. Before using it for fine-tuning, you run this skill to detect any empty sessions that might have inflated or deflated scores. This protects your model from learning from phantom interactions.

Add it to your agent workflow

Install the skill via npx:

npx skills add ashutoshsinghpr7/wikiskill --skill trace-harness-launch-failure

Then, in your trace analysis script, call the skill's function on each session directory:

from trace_harness_launch_failure import check_launch

result = check_launch("runs/task_003")
if result.launch_failed:
    print(f"Skipping session: {result.reason}")
else:
    analyze_agent_behavior(result.trace)

Compare before adopting

Before integrating, compare against your current debugging workflow: does it already check api_call_count and stdout.txt? If yes, this skill may be redundant. Also evaluate maintenance freshness — check the GitHub repo's last commit and open issues. Finally, consider whether your harness might have different failure modes (e.g., timeouts vs. model ID errors) that this skill doesn't cover. For a broader view of agent skills, you can also browse related skills at OpenAgentSkill.

Why it is worth tracking

With 40 stars and an active author, this skill addresses a common but under-documented failure mode in agent evaluation harnesses. Its value lies not in complex logic but in encoding a critical sanity check that most developers learn the hard way. Track it if you build or maintain evaluation pipelines; it's a cheap insurance policy against silent data corruption. Re-evaluate when your harness changes or when you adopt a new trace format — the core principle (verify launch before analyzing behavior) will remain relevant.