analyze-skill-issues
Query the integration-test storage account to find why a specific skill's tests are failing. Reads blob-stored test result files and surfaces error details. TRIGGERS: why is skill failing, skill test failures, debug skill tests, skill failing tests, analyze skill failures, why are tests failing for skill, skill test errors, investigate skill issues. DO NOT USE FOR: analyzing a GitHub Actions run report or comparing test runs across runs (use analyze-test-run).
microsoft · AI review on file
Installation
Read the source and dependencies before running a command. Copying does not install or execute anything.
npx skills add microsoft/GitHub-Copilot-for-Azure --skill analyze-skill-issuesAbout this skill
name: analyze-skill-issues description: "Query the integration-test storage account to find why a specific skill's tests are failing. Reads blob-stored test result files and surfaces error details. TRIGGERS: why is skill failing, skill test failures, debug skill tests, skill failing tests, analyze skill failures, why are tests failing for skill, skill test errors, investigate skill issues. DO NOT USE FOR: analyzing a GitHub Actions run report or comparing test runs across runs (use analyze-test-run)." license: MIT metadata: author: Microsoft version: "1.0.1"
Analyze Skill Issues
Queries the strdashboarddevveobvk Azure Storage account that stores all integration test results and retrieves error details for a given skill.
Quick Reference
| Property | Value |
|---|---|
| Storage account | strdashboarddevveobvk |
| Container | integration-reports |
| Blob path pattern | {date}/{run_id}/{skill_name}/[{test_name}/]<file> — see Blob Path Layout |
| Blob discovery | mcp_azure_mcp_storage_blob_get — list blobs to find the right paths |
| Blob content download | az storage blob download ... --file "$env:TEMP\<filename>" — local path reported in summary |
| Best for | Diagnosing why a skill's integration tests are failing |
TOOL USAGE — MANDATORY: Use
mcp_azure_mcp_storage_blob_getto enumerate and identify the correct blob paths. Then useaz storage blob downloadto download and read the content of those blobs. These tools serve different purposes and are both always required — they are not alternatives.
When to Use
- User asks why a skill's tests are failing
- User asks to debug or investigate a skill's recent test failures
- User wants to see errors from integration test runs for a specific skill
- User asks what errors or exceptions caused skill test failures
Skill Name Mapping
Resolve the user's skill name to its canonical directory name using the Skill Name Mapping.
MCP Tools
MCP tools are used exclusively for blob discovery — finding which blob paths exist. All calls require account: "strdashboarddevveobvk" and container: "integration-reports". Use az storage blob download (see Phase 2) to read blob content.
| Tool | Purpose | Key parameters |
|---|---|---|
mcp_azure_mcp_storage_blob_get | List blob names and metadata (does not return file content) | account, container; omit blob to list all |
mcp_azure_mcp_storage_blob_container_get | Verify the container exists | account, container |
Output Requirements
Every response MUST include all of the following. Do not omit any item:
- Root cause explanation of why the skill has low confidence or failing tests
- Per-test breakdown (test name, pass/fail, confidence or error)
- Referenced Files section listing every downloaded file as a
vscode://file/link
⚠️ MANDATORY: The Referenced Files section is not optional. Always resolve
$env:TEMPto the actual path (runecho $env:TEMP) and include a clickable link for every file downloaded. If this section is missing from your response, the output is incomplete.
Workflow
TOOL RESPONSIBILITIES:
mcp_azure_mcp_storage_blob_getis used exclusively for blob discovery (listing available paths).az storage blob downloadis used exclusively for reading blob content. Both are always required.
Phase 1 — Enumerate Recent Blobs for the Skill
-
Resolve the skill name using the Skill Name Mapping.
-
List all blobs in the container to discover available date/run paths:
mcp_azure_mcp_storage_blob_get({ account: "strdashboarddevveobvk", container: "integration-reports" })This returns a flat list of all blob names. Filter for entries that:
- Start with a date string matching
yyyy-mm-ddwithin the last 7 days - Contain
/{skill_name}/in the path - Do not end with
token-usage.jsonoragent-metadata.json
- Start with a date string matching
-
Group the matching blobs by date (descending) and by run ID (the second path segment).
-
Limit scope: If the user specified a date in their prompt, use only that date. Otherwise, take the 3 most recent dates from the filtered results and discard the rest. Do not process more than 3 dates.
Phase 2 — Read Test Result Files
For each matching blob path identified in Phase 1, download it to a local temp file and read its content:
az storage blob download --account-name strdashboarddevveobvk --container-name integration-reports `
--name "<full-blob-path>" --file "$env:TEMP\<filename>" --auth-mode login --no-progress
Use just the last path segment as <filename>.
⚠️ REQUIRED: After each download, immediately append the resolved local path to your working list of referenced files. Run
echo $env:TEMPonce to resolve the actual temp directory. Every downloaded file must appear in the Referenced Files section of your final response — this is not optional.
Priority order for files to read (most useful first):
- Any
*.jsonfiles that are NOTtoken-usage.jsonoragent-metadata.json— these contain test pass/fail results and error messages *-SKILL-REPORT.mdor*report*.jsonfiles — contain skill invocation details and test narrativesjunit.xmlblobs if present — contain structured test failure messages
Phase 3 — Extract Failure Information
For each test result file, extract:
- Test name — the test case identifier
- Status — passed / failed / error / skipped
- Error message — the exception or assertion failure text
- Stack trace — if present in the file
- Date and run ID — for traceability
Look specifically for fields such as:
"status": "failed"or"passed": false"error","errorMessage","failureMessage", `"message