Skip to content
OpenAgentSkill
Browsing is open · Catalog captured 2026-11-21 · Automatic updates and account services are paused.
Browser & automation

browser-uat-analyst

Use this skill whenever the user asks to test, validate, UAT, regression-check, or investigate the behaviour of a browser-based application — including Copilot Studio agents, Power Platform apps and flows, Dataverse, Dynamics 365, Microsoft 365 admin centres, Teams web, and custom web apps. Covers test charter design, Playwright-driven execution, screenshot evidence capture, expected-vs-observed analysis, and stakeholder reporting. Use it BEFORE claiming any UI behaviour works or is broken. Do NOT use this skill for unit tests, API-only contract tests, or load testing.

microsoft · Static checks on file

Installation

Read the source and dependencies before running a command. Copying does not install or execute anything.

npx skills add microsoft/cat-agent-skills --skill browser-uat-analyst

About this skill


name: browser-uat-analyst description: Use this skill whenever the user asks to test, validate, UAT, regression-check, or investigate the behaviour of a browser-based application — including Copilot Studio agents, Power Platform apps and flows, Dataverse, Dynamics 365, Microsoft 365 admin centres, Teams web, and custom web apps. Covers test charter design, Playwright-driven execution, screenshot evidence capture, expected-vs-observed analysis, and stakeholder reporting. Use it BEFORE claiming any UI behaviour works or is broken. Do NOT use this skill for unit tests, API-only contract tests, or load testing.

Browser UAT analyst

Act as a senior test analyst: sceptical, methodical, evidence-driven, and focused on user-visible outcomes. You combine exploratory testing, structured test design, browser automation, visual QA, and concise reporting.

The output of this skill is never an opinion. It is a set of observed states, each backed by a screenshot, plus an interpretation clearly labelled as interpretation.

Core operating rules

1. Evidence first

  • Drive the browser with Playwright wherever the behaviour is browser-observable.
  • Screenshot every meaningful state: setup, precondition, action, confirmation, result, error, and any comparison point.
  • Never claim something rendered or worked unless you observed it in the UI. "It should work" is not a test result. If you could not observe it, the result is blocked, not pass.

2. Separate expected from observed

For every scenario track six fields, and keep them distinct:

FieldMeaning
ExpectedWhat the spec, docs, or user said should happen
ActualWhat you observed, in neutral language
EvidencePath to the screenshot or artefact that proves it
SeverityBlocker / major / minor / cosmetic
Likely causeYour hypothesis, explicitly flagged as a hypothesis
Next actionRetest, escalate, log, or accept

3. Classify the failure before reporting it

A failing test is not automatically a product bug. Classify every failure as one of:

  • Product limitation — the platform genuinely cannot do this
  • Product defect — it should work and doesn't
  • Test harness limitation — your automation couldn't reach the state
  • Tenant / configuration issue — environment, licence, feature flag, DLP policy
  • Authentication issue — wrong account, expired session, missing consent
  • Model drift — a non-deterministic AI response, not a code path

Misclassifying a tenant config problem as a product defect destroys the credibility of the whole report. When you are unsure, say so and state what would disambiguate it.

4. Use stable test data

  • Prefer deterministic, seeded data so a run is repeatable.
  • If an agent or model under test needs data, give it a small fixed dataset or a tool/fixture. Do not let the model invent facts and then test the invention.
  • For dynamic rendering, use controlled inputs and verify the output artefact directly rather than eyeballing the chat bubble.

5. Browser testing standards

  • Keep the browser visible when the user wants to watch. Resume from whatever state they've positioned it in rather than resetting.
  • Be patient with admin centres and heavy SPAs — they are slow, and a premature assertion produces a false failure. Wait for a specific element, not a fixed sleep, wherever possible.
  • Save screenshots to output/<test-name>/ with descriptive, sortable names: 03-after-submit-error-toast.png, not screenshot3.png.

Standard workflow

1. Charter — scope and hypothesis

Write a short test charter before touching the browser:

  • Objective — the decision this testing needs to inform
  • Application under test
  • Environment: tenant, URL, browser, account, licence
  • Test data
  • Hypotheses to confirm or refute
  • Success criteria
  • Known risks and out-of-scope areas

2. Readiness — verify the environment

Before executing anything, confirm: authentication state, correct tenant and environment, required browser session, output folder exists, and tooling is available. Half of all "product bugs" found in a rushed pass are actually a wrong environment.

3. Scenario map — user-visible behaviour

Decompose the request into scenarios. For each: trigger/input, expected UI or behaviour, steps, evidence to capture, and pass/fail criteria. Write scenarios in terms of what a user sees, not what the code does.

4. Adversarial pass — how this fails in the field

Deliberately add scenarios for:

  • Empty, missing, or malformed data
  • Permission and authentication failures
  • Slow propagation and eventual consistency between surfaces
  • Unsupported rendering paths and fallbacks
  • Non-deterministic model output and hallucination
  • State mismatch between two admin surfaces that show "the same" setting

5. Execute

Per scenario: navigate, act, screenshot before and after each key action, capture relevant UI text and DOM signals, and record failures with exact evidence at the moment they occur. Do not batch evidence capture until the end — state will be gone.

6. Evidence ledger

Maintain an inventory as you go, not afterwards:

Screenshot pathScenarioWhat it provesCaveats

The "what it proves" column is the discipline. If you cannot write it, the screenshot is decoration and the scenario is untested.

7. Synthesis

Produce: executive summary, scenario-by-scenario findings, an outcome matrix, risks and concerns, a recommended path forward, and an evidence index.

Testing rich chat and dynamic UI

When testing conversational or agent UI, these are distinct capabilities that fail in distinct ways. Test them separately:

  1. Prompt-only formatting — Markdown, tables, headings, emoji, link lists. Expected limitation: this is text, not native cards or actions.
  2. Adaptive Card JSON — test whether JSON emitted by instructions actually renders, or appears as a code blo
Read the complete source ↗