ai-risk-management
Apply the NIST AI Risk Management Framework (AI RMF 1.0) and adjacent guidance to AI / ML systems — model lifecycle governance, fairness and bias evaluation, robustness, transparency, accountability, third-party model risk, monitoring for drift, and AI incident response. Broader than prompt-injection (which is the security slice). Use when the user mentions 'AI risk,' 'AI governance,' 'NIST AI RMF,' 'AI compliance,' 'ML governance,' 'model risk management,' 'AI fairness,' 'AI bias,' 'algorithmic accountability,' 'AI Bill of Rights,' 'EU AI Act,' 'AI transparency,' 'model card,' 'AI red-team program,' 'AI safety,' 'responsible AI,' 'model drift,' 'concept drift,' 'AI monitoring,' 'AI incident,' or needs to assess or govern an AI / ML system.
briiirussell · 已有静态检查记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add briiirussell/cybersecurity-skills --skill ai-risk-management技能说明
name: ai-risk-management description: "Apply the NIST AI Risk Management Framework (AI RMF 1.0) and adjacent guidance to AI / ML systems — model lifecycle governance, fairness and bias evaluation, robustness, transparency, accountability, third-party model risk, monitoring for drift, and AI incident response. Broader than prompt-injection (which is the security slice). Use when the user mentions 'AI risk,' 'AI governance,' 'NIST AI RMF,' 'AI compliance,' 'ML governance,' 'model risk management,' 'AI fairness,' 'AI bias,' 'algorithmic accountability,' 'AI Bill of Rights,' 'EU AI Act,' 'AI transparency,' 'model card,' 'AI red-team program,' 'AI safety,' 'responsible AI,' 'model drift,' 'concept drift,' 'AI monitoring,' 'AI incident,' or needs to assess or govern an AI / ML system." allowed-tools: Read, Grep, Glob, WebSearch
AI Risk Management — Beyond Security, the Whole Model Lifecycle
prompt-injection covers the AI security slice — attackers manipulating LLM inputs. This skill covers everything else risk-related about deploying AI / ML systems: governance, fairness, robustness, transparency, monitoring, incident response specific to AI failures, third-party model risk, and compliance with the emerging AI regulatory landscape.
The framing is NIST AI RMF 1.0 (released 2023) — the most widely-adopted voluntary framework — plus the regulatory layer (EU AI Act, US federal policy and state laws, sector-specific guidance). Use this skill when you are deploying AI features beyond a chatbot wrapper, when a regulator asks "how do you govern your AI," or when something has gone wrong with an AI system in production.
Cross-references: prompt-injection for prompt-injection / LLM-specific security attacks; threat-modeling for design-time AI risk modeling; incident-triage and breach-patterns for AI-related incident response patterns; csf-mapping for the broader governance frame that AI RMF sits within.
Authorization
Assess AI systems the user's organization builds, deploys, or buys, using material they're entitled to share. This skill reads documentation, model cards, eval results and code. It doesn't run evaluations against production models with real user data, or against a third-party model in ways its terms forbid. Regulatory conclusions are input for counsel, not legal advice.
The NIST AI RMF — four functions
Just like the cybersecurity framework, the AI RMF organizes the work into functions. Same shape, different content.
| Function | What it covers |
|---|---|
| Govern (GOV) | Policy, accountability, roles, risk appetite, AI principles, board oversight, governance structures |
| Map (MAP) | Context — what is the AI system, what does it do, who is impacted, what could go wrong, what are the legal / ethical constraints |
| Measure (MEAS) | Evaluate the system — fairness, robustness, accuracy, explainability, privacy, security; quantitative + qualitative metrics |
| Manage (MAN) | Treat the risks — mitigations, monitoring, incident response, decommissioning, ongoing review |
The framework is voluntary but increasingly cited in contracts, RFPs, executive orders, and emerging regulations. Treat it as the lingua franca of AI risk.
Workflow
Step 1 — Inventory AI systems
Before assessment, build the inventory. Most organizations underestimate how much AI they actually deploy.
| Category | Examples |
|---|---|
| First-party trained models | Recommendation engines, fraud detection, churn prediction, internal ML pipelines |
| First-party LLM use | Customer support chat, content generation, summarization, code generation, embeddings for search |
| Third-party AI features | Stripe Radar (fraud), GitHub Copilot (code completion), Salesforce Einstein, Notion AI, Linear AI |
| Embedded AI in products you ship | Suggested responses, smart defaults, AI sorting / ranking |
| AI in HR / hiring | Resume screening, candidate matching, performance evaluation — high regulatory exposure |
| AI in customer-facing decisions | Pricing, eligibility, content moderation, ad targeting — high regulatory exposure |
| AI coding agents and extensions | Claude Code, Cursor, Codex, Gemini CLI, Copilot — and the skills, plugins, hooks, MCP servers, and CLAUDE.md / AGENTS.md files they load with developer privileges; audit with agent-supply-chain-audit |
For each, record: vendor (if any), training data source, deployment context, who it affects, the decision it informs, how decisions are reviewed.
Step 2 — MAP: assess the context per system
For each AI system in the inventory, answer:
- Purpose — what is this system's stated goal? Does the actual deployment match?
- Stakeholders — who interacts with it, who is affected by its decisions, who is in a position to challenge those decisions?
- Legal / regulatory context — is this in scope for a specific law? (EU AI Act high-risk categories, US HUD fair-housing rules, EEOC for employment AI, FTC for unfair / deceptive practices, sector laws)
- Failure modes — what does "broken" look like? (Wrong answer, biased answer, hallucinated answer, slow answer, expensive answer, refused-to-answer-something-it-should, answered-something-it-should-not)
- Reversibility — when this system makes a wrong call, can the decision be undone? (Mortgage denial: hard to undo. Spam filter: easy)
Step 3 — MEASURE: evaluate the system
The categories of evaluation, with the engineering hooks for each:
Accuracy / performance
- Test set evaluation — held-out data, not the training data
- Performance on slices of data, not just aggregate (the system that's 95% accurate overall may be 60% accurate on the demographic that's most impacted)
- Confusion matrices for classification; quantile-based error analysis for regression
- For LLMs: task-specific evals (HELM, MMLU, custom evals) — and especially custom evals on the application's actual prompts
Fairness / bias
- **Demographi