validate-genai-spans
Use this to check that your LLM tracing actually emits complete, spec-compliant spans, so cost/latency/model dashboards downstream are not full of holes. Trigger on "are my traces complete", "validate my instrumentation", "my spans are missing fields", "lint my OTel GenAI spans", "test my tracing". Ships a runnable, tested validator you can drop into your instrumentation tests.
ContextJet-ai · AI review on file
Installation
Read the source and dependencies before running a command. Copying does not install or execute anything.
npx skills add ContextJet-ai/awesome-llm-observability --skill validate-genai-spansAbout this skill
name: validate-genai-spans description: Use this to check that your LLM tracing actually emits complete, spec-compliant spans, so cost/latency/model dashboards downstream are not full of holes. Trigger on "are my traces complete", "validate my instrumentation", "my spans are missing fields", "lint my OTel GenAI spans", "test my tracing". Ships a runnable, tested validator you can drop into your instrumentation tests. license: CC0-1.0
Validate GenAI spans
Broken instrumentation fails silently: the app works, the dashboards just quietly miss the model, the tokens, or the cost. This skill ships a validator that asserts your spans carry the required OpenTelemetry gen_ai.* fields, so you catch gaps in a test instead of in a half-empty dashboard.
Use the bundled script
scripts/validate_span.py is pure Python, no install needed:
from validate_span import validate_gen_ai_span, is_valid_gen_ai_span
problems = validate_gen_ai_span(span_attributes) # [] means valid
validate_gen_ai_span(span_attributes, strict=True) # also flag recommended fields
assert is_valid_gen_ai_span(span_attributes) # use in an instrumentation test
It checks the required fields (gen_ai.provider.name, gen_ai.operation.name, gen_ai.request.model) and, in strict mode, the recommended usage fields (gen_ai.usage.input_tokens/output_tokens, gen_ai.response.model) that cost analysis depends on. Spans still using the pre-2025 gen_ai.system name pass but get a deprecation note, so you can migrate at your own pace.
How to apply it
- Add an instrumentation test: capture the span your app emits for a sample call (most SDKs have an in-memory span exporter for tests) and assert
is_valid_gen_ai_span(attrs). - Run it in CI so a change that breaks instrumentation fails the build, not production.
- Use strict mode once basic spans pass, to push toward full cost/usage coverage.
Pairs with instrument-llm-observability (which gets the spans emitted in the first place).
Validation
Run the tests: pytest skills/validate-genai-spans/tests/. They cover a valid span, each missing required field, empty/wrong-typed values, the deprecated gen_ai.system alias, strict-mode recommended checks, and non-dict input.
Anti-patterns
- Assuming "traces show up" means they are complete (missing token fields break cost dashboards).
- Only checking spans by eye once, instead of asserting them in CI.
- Ignoring the response model id (you cannot break cost down by served model without it).