How to Run an AI Risk Assessment

An AI risk assessment is a structured review of one AI system that identifies what could go wrong, how likely and how serious each outcome is, what controls already exist, and what you will do about the gaps. It is run before a system goes live, repeated when the system or its context changes, and recorded so that an auditor, customer or regulator can see the reasoning. The method below works for a purchased tool, an AI feature inside existing software, or a system you build, and it produces the records that NIST AI RMF, ISO/IEC 42001 and the state AI laws all expect. The template is in the middle of this guide.

Risk assessment vs impact assessment vs audit

The three terms are used loosely, and mixing them up leads to assessments that answer the wrong question. They differ in whose risk they examine and who performs them.

Activity	Question it answers	Whose risk	Who performs it	When
AI risk assessment	What could go wrong with this system, how likely and how bad, and what will we do about it	The organization: data exposure, wrong outputs, security, vendor failure, legal exposure, cost, reputation	The system owner with security, privacy and the governance lead	Before launch, on material change, on a schedule by tier
AI impact assessment (also called algorithmic impact assessment)	How does this system affect the people it touches, and is that acceptable	Individuals and groups: fairness, access to services, transparency, ability to contest, privacy, safety	The system owner with legal or compliance, and input from affected groups where practical	Before deployment of any system that influences decisions about people; refreshed on material change
AI audit (including bias audit)	Does the system and its governance actually do what the assessments claim	Verification of both of the above	Someone independent of the system owner: internal audit, a third party, or an accredited body	Periodically, when required by law or contract, or before certification

Model risk management is the older cousin of all three. It comes from bank regulators’ guidance on models used in financial decisions and emphasizes independent validation, documentation and ongoing monitoring. Its vocabulary has spread to AI model risk management in other sectors, and its core idea, that a model must be validated by someone other than its builder, is worth borrowing at any size.

In practice, a high-tier system needs a risk assessment and an impact assessment, and the two are usually written as one document with two sections. Audits come later. The NIST AI RMF treats the first two as the Map and Measure functions; our NIST AI RMF guide explains the structure.

Step-by-step

1. Confirm the inventory entry and tier. The assessment starts from the AI inventory: owner, purpose, type (purchased, embedded, built), data classes, people affected and risk tier. If the entry does not exist, create it first. The tier sets the depth: a short form for moderate-tier systems, the full template for high-tier ones.
2. Describe the system in plain language. What it does, who uses it, what it takes in, what it produces, what decisions or actions follow from its output, which model or vendor sits underneath, and where a human reviews the result. If you cannot write this page, you are not ready to assess the system.
3. Identify risks by category. Walk through a fixed set of categories so nothing is skipped: validity and reliability, safety, security, accountability and transparency, explainability, privacy, fairness and bias, legal and contractual, vendor and supply chain, operational and cost. For generative AI add confabulation, information integrity, intellectual property and harmful content. For systems you build on large language models, add the application-security items from the OWASP Top 10 for LLM applications: prompt injection, sensitive information disclosure, supply chain, data and model poisoning, improper output handling and excessive agency.
4. Rate likelihood and impact. A three-point scale for each (low, medium, high) is enough. Impact is rated for the worst credible outcome, not the typical one, and separately for the organization and for affected people. Rating with the system owner, a security or privacy reviewer and someone from the business unit in the room produces better numbers than a solo exercise.
5. Evaluate against the tier and your risk appetite. Decide which rated risks are acceptable as they stand and which need treatment. Any risk rated high on impact to people needs treatment regardless of likelihood.
6. Treat the risks. For each unacceptable risk choose a treatment: avoid (do not use the system, or not for that purpose), reduce (add a control: human review, data minimization, configuration change, testing, monitoring), transfer (contract terms, insurance), or accept with a named approver. Assign an owner and a date to each control.
7. Run the impact assessment questions and the checks. For systems that influence decisions about people, answer the impact questions in the template and run bias testing where outcomes can be measured across groups. For built systems, run the security tests. Record results, including failures.
8. Document, approve, monitor. The completed assessment is approved by the governance forum for high-tier systems and by the governance lead for the rest. It states the monitoring plan and the triggers for re-assessment: model or prompt change, new data source, vendor change, new use, an incident, a new law, or the scheduled review date.

Template

Copy the structure below into a document or your governance tool. Sections 1 to 3 and 6 to 7 apply to every assessed system; section 4 applies when people are affected; section 5 applies when you build or materially configure the system.

1. System profile

• System name, inventory reference, business owner, technical owner, assessor, date, version
• Purpose and intended use; explicitly out-of-scope uses
• Type: purchased tool, AI feature in purchased software, built in-house (and the underlying model or vendor)
• Users and their training; people affected by outputs
• Decisions or actions that follow from outputs, and where a human reviews them
• Risk tier and the reason for it

2. Data

• Data entering the system (classes, sources, volume) and data it produces
• Personal or regulated data involved and the legal basis or agreement covering it
• Whether the vendor retains inputs or trains on them; retention periods; subprocessors
• For built systems: training, fine-tuning and retrieval data sources, provenance and quality checks

3. Risk register

Risk	Category	Likelihood	Impact (organization / people)	Existing controls	Treatment and new controls	Owner and date
Staff paste client contract text into the tool and the vendor retains it	Privacy, legal	Medium	High / Medium	Policy prohibits confidential data	Reduce: company account with no-retention terms; data-loss prevention rule	IT lead, date
Summaries omit a material term and a decision is made on the summary	Validity	Medium	Medium / Low	None	Reduce: review rule; source document attached to every summary	Business owner, date
Screening model ranks candidates differently by protected characteristic	Fairness	Unknown until tested	High / High	Vendor statement only	Reduce: bias testing on our applicant data before use; human review of every rejection; notice to candidates	HR lead, date

4. Impact assessment questions (systems that affect people)

1. Who is affected by this system’s outputs, directly and indirectly, and how many people?
2. What decision or outcome does the system influence, and how significant is it to the person?
3. Could the system produce different outcomes for people in protected groups, and how would we know?
4. What is the worst credible harm to an individual from a wrong, biased or unavailable output?
5. Are affected people told that AI is involved, in language they can understand?
6. Can a person ask for an explanation, and can we give one?
7. Can a person contest or appeal an outcome, and does a human make the final decision?
8. Is the data about the person accurate, current and collected for a compatible purpose?
9. What safeguards reduce the harm, and how do we know they work?
10. Have we consulted anyone who represents the affected people, or evidence of their experience?
11. Does any law or contract require a specific assessment, notice or record for this use?
12. Is the benefit to the organization and to affected people worth the residual risk, and who signs off on that?

5. Bias and security checks (built or materially configured systems)

• Accuracy against a defined test set, with the acceptance threshold and the result
• Bias testing: outcomes measured across relevant groups, the metric used, the threshold and the result
• Behavior on unusual inputs and edge cases relevant to the use
• Security testing for prompt injection, data leakage, insecure output handling and over-privileged tool access
• Logging of inputs, outputs and human overrides, and who can read the logs
• Rollback plan and kill switch

6. Decision and approvals

• Overall residual risk rating and whether it is accepted
• Conditions of approval (controls that must be in place before launch)
• Approver names, roles and dates; governance forum reference for high-tier systems

7. Monitoring and re-assessment

• Metrics to collect, who reviews them, how often, and the thresholds that trigger action
• Incident reporting path for harmful or wrong outputs
• Re-assessment triggers and the scheduled review date

Model governance basics

An assessment describes a system at a moment. AI model governance keeps it true afterward. The basics apply whether the model is yours or a vendor’s; for a vendor’s model, several items become questions you ask them.

• Model inventory. Every model and every prompt-plus-model configuration in production is listed with its version, owner, purpose and the systems that use it.
• Independent validation. Someone other than the builder reviews the design, data and test results before deployment. In a small team that can be a peer; the point is a second set of eyes.
• Documentation. A short model record (often called a model card) covering intended use, training or retrieval data, evaluation results, known limitations and the date of the last review.
• Change control. Model, prompt, retrieval-data and vendor-version changes go through a change process that asks whether re-assessment is needed. Vendor model updates count as changes even though you did not make them.
• Monitoring. Performance, drift, output quality, override rates and complaints tracked over time, with thresholds that trigger review.
• Access control. Who can change prompts, models, data sources and the tools an agent can call, with logs.
• Retirement. A defined way to switch a model off, roll back, and record that it is no longer in use.

Where AI is used in decisions about people, some state laws describe risk management programs, impact assessments and records in terms close to the above; our guide to the Colorado AI Act and US state AI laws covers what they ask for. Check the current official text for the laws that apply to you. This is general information, not legal advice. The assessment also needs a home: the structure in our guide to building an AI governance framework shows where it sits among the policy, inventory and review cadence.

Where PS3G fits

PS3G runs AI risk and impact assessments as part of its AI governance consulting service, using the template above, for organizations of every size: a single assessment of a purchased tool before a contract is signed, the assessments for every high-tier system in a six-week starter program, or an ongoing cycle under a managed-services agreement. For AI we build, our testing center of excellence runs the bias and security checks. We assess the AI features in our own HIPAA-compliant EHR and telehealth platform, DocVilla, with the same template, and we work with your counsel on the legal questions.

Related reading

• NIST AI RMF Explained: Govern, Map, Measure, Manage
• AI Governance Framework: How to Build One (Structure + Examples)
• Colorado AI Act and US State AI Laws: What Businesses Must Actually Do
• AI governance consulting services

Frequently asked questions

What is the difference between an AI risk assessment and an AI impact assessment?

A risk assessment examines what could go wrong for the organization: data exposure, wrong outputs, security, vendor failure, legal and cost exposure. An impact assessment examines the effect on the people the system touches: fairness, access to services, transparency, the ability to contest a decision. High-tier systems need both, usually written as two sections of one document.

How often should an AI risk assessment be repeated?

On any material change to the model, prompt, data, vendor or use; after an incident; when a new law applies; and otherwise on a schedule set by tier, typically annually for high-tier systems. Vendor model updates count as changes even though you did not make them, which is why monitoring and vendor release notes matter.

Who should perform the assessment?

The system owner leads it, with security and privacy reviewers, the governance lead, and legal or compliance for systems that affect people. Independence matters at the review step: someone other than the owner should approve the result, and for high-tier systems the governance forum should. External help is useful for the first few assessments and for bias and security testing.

Do we need an AI bias audit?

Run bias testing yourself for any system that influences decisions about people; it is part of the assessment. An independent bias audit is a separate verification and is required in some jurisdictions for specific uses such as automated employment decision tools, and expected by some customers. Check the current official text of the laws that apply to you. This is general information, not legal advice.

What is AI model risk management?

A discipline from financial regulation, now applied to AI generally, that requires models to be inventoried, documented, validated independently of their builders, monitored in production and subject to change control. The basics in this guide are a scaled-down version suitable for any organization that builds or materially configures AI.