Iso-31000 |

How to Actually Use ISO/IEC 23894 for AI Risk Management

Sat, 28 Mar 2026 00:00:00 +0000

Practical ISO/IEC 23894 Implementation for AI Risk Management (Without Turning It Into Shelf Decoration)

Most AI risk programs fail before the first risk is ever scored.

They fail because teams treat AI risk management as a exercise, a model review checklist, or a late-stage legal sign-off. Then the first serious issue hits. Training data rights were unclear. A model drifts in production. A vendor changes an API. An automated decision harms a customer group nobody mapped. The organization scrambles, and trust evaporates fast.

This is why ISO/IEC 23894 matters. It gives organizations a practical structure for AI risk management that fits how AI is actually built, bought, deployed, and used. In this post, I’ll show you how to turn ISO/IEC 23894 into an with governance approval gates, clear role ownership, and an implementation checklist that avoids the common failure points I keep seeing in audits, design reviews, and board briefings.

Here is why it fails: ISO/IEC 23894 is not a checklist. It is a guidance document built on top of ISO 31000, the general risk management standard, with AI-specific extensions layered in. If you treat it like a form to fill out, you will produce documentation that looks complete but protects nobody.

This post walks through the standard’s actual structure, explains what each section demands in practice, and gives you the field-tested implementation tips I have gathered from helping organizations build AI risk management programs that survive contact with real AI systems.

Why ISO/IEC 23894 Exists and What Problem It Solves

Before 2023, organizations managing AI risk had to improvise. They would borrow bits from information security frameworks, add some data governance controls, and hope the combination covered enough ground. It rarely did.

ISO/IEC 23894:2023 was created by ISO/IEC JTC 1/SC 42 to provide a structured approach for any organization that develops, deploys, or uses AI systems. The standard applies the well-established ISO 31000 risk management framework to the specific challenges AI introduces. Think of it as a translation layer. It takes proven risk management principles and shows you exactly where AI creates new wrinkles.

The standard covers three domains: principles that guide your thinking, a framework for embedding AI risk management into your organization, and processes for actually identifying, assessing, and treating AI-specific risks.

The biggest mistake organizations make is treating ISO/IEC 23894 as a standalone. It explicitly references and extends ISO 31000:2018. If your team has not read ISO 31000 first, they will misunderstand the guidance in 23894 because they will lack the foundational context. Buy both standards. Read 31000 first. Then read 23894 as the AI-specific annotation layer it was designed to be.

The Three-Part Architecture You Need to Understand

ISO/IEC 23894 mirrors the clause structure of ISO 31000 deliberately. This is not an accident. The authors wanted organizations that already use ISO 31000 to integrate AI risk management without rebuilding everything from scratch. The three parts work together as a system.

Part One: Principles (Clause 4)

The principles define the foundational values that should shape every AI risk decision your organization makes. ISO 31000 defines eight principles. ISO/IEC 23894 adds AI-specific guidance to five of them: Inclusive, Dynamic, Best Available Information, Human and Cultural Factors, and Continual Improvement.

The “inclusive” principle is where most organizations stumble first. AI systems affect a wider set of stakeholders than traditional software. The standard explicitly calls out that stakeholders can help identify risks in data collection, define fairness criteria, identify bias, and determine where human oversight is needed. This is not a suggestion. If your risk management process does not include diverse stakeholder input, you are missing risks that will surface later in the worst possible way.

The “dynamic” principle matters more for AI than for almost any other technology domain. AI systems based on machine learning can change their behavior through continuous learning. Customer expectations shift quickly. Regulatory requirements are updating constantly. Your risk management process needs to account for a system that is itself a moving target.

When I first helped a financial services firm apply the “Inclusive” principle, they interpreted “stakeholder involvement” as sending a survey to the compliance team. That is not what the standard means. You need a structured dialog with people who will be affected by the AI system’s decisions. For a credit scoring model, that means talking to loan officers, applicants from different demographic groups, and consumer advocacy organizations. Map your stakeholders before you start the risk assessment, not after.

Part Two: Framework (Clause 5)

The framework section describes how to embed AI risk management into your organizational structure. It covers leadership commitment, integration with existing management systems, organizational design, resource allocation, and communication.

Two sub-clauses deserve special attention.

Clause 5.2 on leadership and commitment adds an AI-specific requirement that many organizations overlook. Because trust and accountability are especially important for AI, the standard says top management should consider issuing public statements about their commitment to AI risk management. This is not corporate PR. It creates an accountability anchor. Once your CEO has publicly committed to responsible AI, the organization has real pressure to follow through.

Clause 5.4.3 on assigning roles is where the framework becomes . The standard requires that top management and oversight bodies allocate resources and identify specific individuals with authority to address AI risks and responsibility for monitoring AI risk processes. Not committees. Not shared inboxes. Named people with clear authority.

Part Three: Processes (Clause 6)

This is where the standard gets specific about what you actually do. The risk management process follows a sequence: define scope and context, assess risks (identify, analyze, evaluate), treat risks, then monitor and report. Each step has AI-specific extensions.

The process section is the longest part of the standard for good reason. AI risk assessment requires you to think about assets, risk sources, events. Do not try to build your AI risk management process on a blank sheet of paper. Clause 6.4.1 specifically recommends using the catalogue of AI-related risk sources in Annex B as a baseline for organizations performing risk assessment for the first time. I have seen teams spend three months trying to brainstorm AI risk sources when the standard already provides a structured catalogue. Start there. Customize from there.

Stage 1: Establishing Scope, Context, and Criteria

This stage determines what your AI risk management process covers and how it connects to your broader organizational context. Get this wrong and everything downstream is compromised.

The standard requires you to build an inventory of where AI systems are being developed or used in your organization. This sounds straightforward. It is not. In every organization I have worked with, the initial AI inventory missed at least 30% of actual AI usage. Teams embed machine learning models in spreadsheet macros, use AI-powered SaaS tools without formal procurement, or inherit AI components through acquisitions.

For external context, the standard provides Table 2 with specific considerations. You need to track relevant legal requirements for AI, ethical guidelines from government and industry groups, domain-specific AI frameworks, technology trends, and societal implications of AI deployment. For internal context, Table 3 adds considerations about how AI affects organizational culture, the availability of AI expertise, intellectual property implications, and data quality constraints.

Defining risk criteria for AI requires you to understand uncertainty across the entire AI system. The standard calls out data, software, mathematical models, physical extensions, and human-in-the-loop aspects. This is a broader scope than most organizations initially consider.

What to do: Build your AI system inventory first. Document every AI system or component, its purpose, its data sources, who built it, who operates it, and who is affected by its outputs. Then map the external and internal context factors from Tables 2 and 3. Only then define your risk criteria.

The standard warns that “AI is a fast-moving technology domain” and that measurement methods should be “consistently evaluated according to their effectiveness.” I learned this the hard way when a client’s risk criteria for a natural language processing system became obsolete within eight months because the underlying model was replaced with a fundamentally different architecture. Build a review trigger into your risk criteria. Any time the AI model architecture, training data source, or deployment context changes, the risk criteria should be re-evaluated. Do not wait for the annual review.

Stage 2: Risk Assessment, the Core of the Process

Risk assessment has three sub-stages: identification, analysis, and evaluation. The standard treats each with specific AI guidance.

Risk Identification

The standard breaks identification into five activities: identifying assets and their value, risk sources, potential events and outcomes, existing controls, and consequences. Each requires AI-specific thinking.

For assets, you need to consider three levels: organizational (data, models, the AI system itself, reputation, trust), individual (personal data, privacy, health, safety), and societal (environment, socio-cultural values, educational equity). This three-level approach is one of the most important contributions of the standard. Most organizations only think about organizational assets when they identify AI risks. The standard forces you to consider who bears the consequences.

For risk sources, Annex B provides categories including complexity of environment, lack of transparency and explainability, level of automation, machine learning specific risks, hardware issues, system life cycle issues, and technology readiness. Each category contains specific risk scenarios.

For consequences, the standard makes a critical distinction that many teams miss. It instructs you to “identify any differences between the groups who experience the benefits of the technology and the groups who experience negative consequences.” This is not theoretical. A predictive policing system might benefit a city’s police department while disproportionately harming specific communities. A hiring algorithm might benefit an HR team’s efficiency while systematically disadvantaging certain applicant groups.

What to do: For each AI system in your inventory, work through all five identification activities. Use Annex B as your starting checklist for risk sources. Document consequences at all three levels: organization, individual, and society.

The standard lists methods for identifying potential events, including published standards, scientific papers, market data, incident reports, field trials, stakeholder reports, and expert interviews. When I run risk identification workshops, I always start with incident reports on similar systems. Nothing focuses a risk identification session like showing the team a real-world failure of a system similar to theirs. Search for published incidents, regulatory enforcement actions, and academic case studies related to your specific AI application domain before the workshop begins.

Risk Analysis

Risk analysis requires you to assess both consequences and likelihood. The standard distinguishes between three types of impact assessment: business impact, individual impact, and societal impact.

For individual impact assessment, the standard specifies a detailed list of considerations: types of data used, intended impact, potential bias impact, potential impact on fundamental rights, fairness impact, safety implications, and the jurisdictional and cultural environment of the individual. This last point is easy to overlook. An AI system’s impact on an individual can vary dramatically depending on the legal and cultural context in which that individual lives.

For likelihood assessment, the standard includes a nuanced warning that many organizations miss. It states that “there can be significant technical, economic and heuristic issues with decision-making based on likelihoods, particularly when the likelihood either can’t be calculated or where the calculation has a large margin of error.” In plain language: if you cannot reliably estimate how likely an AI failure is, do not pretend you can. Focus instead on consequence severity and your ability to detect and respond to failures.

What to do: Run separate impact assessments for business, individuals, and society. Do not collapse them into a single score. For each risk, decide whether a meaningful likelihood estimate is possible. If it is not, shift your analysis to focus on consequence severity and control effectiveness.

I once watched a team assign a “low likelihood” score to a bias risk in a hiring algorithm because the model had performed well in testing. Six months after deployment, the model was producing biased outcomes because the production data distribution had drifted from the test data. The team’s likelihood estimate was based on a snapshot that was already stale. For AI systems, especially those using machine learning, likelihood estimates decay faster than for traditional systems. Reassess likelihood every time the model is retrained, the data source changes, or the deployment population shifts.

Risk Evaluation

Risk evaluation compares the analyzed risks against your established criteria to determine which risks need treatment and what priority they receive. The standard defers to ISO 31000:2018 here without adding AI-specific guidance, which tells you something important. The evaluation step is about organizational judgment, not technical analysis. You need decision-makers at the table who understand both the technology and the business context.

Stage 3: Risk Treatment and Implementation

Once risks are evaluated, you choose treatment options. The standard lists the same options as ISO 31000: avoid the risk, take or increase the risk to pursue opportunity, remove the risk source, change the likelihood, change the consequences, share the risk, or retain the risk by informed decision.

The AI-specific addition here is the concept of a risk-benefit analysis for residual risks. If you cannot reduce negative consequences to an acceptable level through treatment, the standard requires you to perform a risk-benefit analysis. This is particularly relevant for AI because some AI risks (like model opacity in deep learning) cannot be fully eliminated. You need to decide whether the benefits justify the residual risk.

What to do: For each risk that exceeds your tolerance thresholds, select a treatment option and document it in a risk treatment plan. For residual risks that remain above tolerance after treatment, conduct a formal risk-benefit analysis. Record the rationale for accepting any residual risks.

for AI systems need to be version-aware. Traditional risk treatment plans assume relatively stable systems. AI systems, especially those using continuous learning, change over time. Your treatment plan should specify which version of the model it applies to and include trigger conditions for re-evaluation. I recommend tagging each treatment plan entry with the model version, training data date, and deployment configuration it was validated against. When any of these change, the treatment plan enters a mandatory review cycle.

Stage 4: Monitoring, Recording, and Reporting

The standard’s requirements for recording and reporting are more specific than many teams expect. Clause 6.7 requires organizations to establish a system for collecting and verifying information from both implementation and post-implementation phases, and to collect publicly available information on similar systems.

This information must be assessed for relevance to the trustworthiness of the AI system. The standard specifically asks whether previously undetected risks exist or whether previously assessed risks are no longer acceptable. When either condition is true, you must perform a review of risk management activities and evaluate the effects on existing controls.

The recording requirements include: system description and identification, methodology applied, intended use description, identity of assessors, terms of reference and date, release status, and degree to which objectives have been met. This is not optional documentation. It creates the audit trail that regulators and oversight bodies will examine.

What to do: Build a risk management record template that captures all required fields. Establish a cadence for collecting and reviewing post-implementation data. Create triggers that automatically initiate risk reassessment when conditions change.

The standard says risk management records “should allow the traceability of each identified risk through all risk management processes.” In practice, this means you need a risk register with unique identifiers for each risk that persist across assessment cycles. I have seen organizations create new risk registers for each assessment, losing the historical thread. Use a single, versioned risk register where each risk has a persistent ID, and track its status changes over time. This is the only way to demonstrate to auditors that your process is continuous, not episodic.

Implementation Tips

These apply across all stages and will determine whether your AI risk management program actually works.

Tip 1: Align Risk Management with the AI System Life Cycle

Annex C of the standard maps risk management activities to AI system life cycle stages: inception, design and development, verification and validation, deployment, operation and monitoring, continuous validation, re-evaluation, and retirement or replacement. This mapping is not decorative. It tells you which risk management activities should happen at each stage.

Most organizations I work with front-load their risk management effort at the design stage and then drop attention during operation. The standard explicitly shows that risk assessment, treatment, monitoring, and recording continue through every life cycle stage, including retirement. When you decommission an AI system, you can lose decision expertise and information. Plan for that. Document what the system knew and how it made decisions before you turn it off.

Tip 2: Handle Stakeholder Identification Seriously

The standard provides a list of stakeholder categories: the organization itself, customers, partners, third parties, suppliers, end users, regulators, civil organizations, individuals, affected communities, and societies. That is nine categories. Most organizations consult two or three.

Create a stakeholder map for each AI system at the inception stage. For each stakeholder category, document what information they need, how they are affected by the system, and how you will engage them. Update this map when the system’s scope or deployment context changes. The stakeholder categories that matter most are often the ones farthest from the development team. Affected communities and end users rarely have a voice in risk assessment unless you build a specific mechanism to include them.

Tip 3: Document Your Risk Criteria Decisions Explicitly

The standard’s Table 4 on risk criteria includes a requirement that organizations “take reasonable steps to understand uncertainty in all parts of the AI system.” This includes data, software, mathematical models, physical extensions, and human-in-the-loop aspects.

When defining risk criteria, write down not just what your criteria are, but why you chose those thresholds. I worked with an organization that set a 95% accuracy threshold for an AI diagnostic tool. When a regulator asked why 95% and not 97% or 99%, nobody could answer. The threshold had been copied from a different project. Document the reasoning behind every criterion. Reference the clinical studies, industry benchmarks, or stakeholder consultations that informed your decision. This documentation is what separates a defensible risk management process from an arbitrary one.

Tip 4: Treat Transparency as a Multi-Audience Challenge

Annex B of the standard discusses transparency and explainability as risk sources. It makes a point that “the kind and level of information that is appropriate strongly depends on the stakeholders, use case, system type and legislative requirements.” One-size-fits-all transparency does not work.

Build a transparency framework that segments by audience. The standard’s Table 1 mentions tailoring transparency to “relevant personas” such as regulators, business owners, and model risk evaluators. In practice, I create three transparency tiers. Tier one is the public-facing description of what the AI system does and does not do. Tier two is the detailed technical documentation for internal reviewers and regulators. Tier three is the full model documentation, including training data provenance, architecture decisions, and test results. Each tier serves a different audience and contains different information. Trying to serve all audiences with one document produces a document that serves none of them.

What This Standard Actually Does in Practice

Part 1: The AI Risk Management Process

This is Clause 6 of the standard. It is the longest and most detailed section because it describes what you actually do.

Step 1: Define Your Scope, Context, and Criteria

Before you assess any risks, you need to answer three questions. What AI systems are we managing? What is the environment around them? And what criteria will we use to decide if a risk matters?

The standard requires you to build an inventory of where AI is being developed or used in your organization. This inventory must be documented and included in your risk management process.

Practical example: A mid-size insurance company I worked with discovered during this step that 14 different teams were using AI-powered tools, but only 3 had been formally identified by the IT governance team. Six were SaaS products with embedded ML models. Two were spreadsheet-based models built by actuaries. Three were claims processing systems. The inventory step alone changed their understanding of their AI exposure.

For context, the standard requires you to consider nine categories of stakeholders:

Your own organization
Customers, partners, and third parties
Suppliers
End users
Regulators
Civil organizations
Individuals affected by the AI system
Affected communities
Societies broadly

That last category is not abstract. If your AI system makes lending decisions, “societies” includes the economic communities shaped by those decisions over time.

The standard also requires you to consider whether your AI systems can harm human beings, deny essential services, infringe human rights through biased automated decisions, or contribute to environmental harm.

For risk criteria, the standard adds an AI-specific requirement that matters enormously in practice: you must understand uncertainty across all parts of the AI system. That includes the data, the software, the mathematical models, any physical components, and the human-in-the-loop aspects like data labeling. Most organizations define risk criteria only around the model itself and miss everything upstream and downstream.

Practical insight for risk managers: Your AI risk appetite should account for your organization’s actual AI capacity and knowledge level. The standard says this directly. If your team has limited ML expertise, your risk appetite for complex deep learning systems should be lower than an organization with a mature data science function. This sounds obvious, but I have seen organizations approve high-risk AI projects with the same risk appetite they use for rule-based automation.

Practical insight for data scientists: The standard warns that “AI is a fast-moving technology domain” and that measurement methods should be “consistently evaluated according to their effectiveness.” That means the metrics you use to evaluate model performance today might not be appropriate six months from now. Build metric review into your model monitoring cadence.

Step 2: Identify Risks

Risk identification in ISO/IEC 23894 covers five distinct activities. Each one matters.

Identify assets and their value. The standard requires you to think about assets at three levels:

Organizational assets include your data, your trained models, the AI system itself (tangible), plus your reputation and stakeholder trust (intangible).

Individual assets include personal data (tangible), plus privacy, health, and safety (intangible).

Community and societal assets include the environment (tangible), plus socio-cultural beliefs, educational access, and equity (intangible).

Practical example: When a healthcare AI startup assessed assets for their diagnostic imaging tool, they initially listed only their model and training data. The three-level framework forced them to also consider patient safety (individual intangible), the hospital’s reputation for diagnostic accuracy (organizational intangible), and equitable access to accurate diagnosis across demographic groups (societal intangible). Each of these surfaced risks that the initial asset list would have missed.

Identify risk sources. The standard provides categories: organizational factors, processes, personnel, physical environment, data, AI system configuration, deployment environment, hardware and software, and dependence on external parties. Annex B expands these into detailed scenarios (covered in Part 2 below).

Identify potential events and outcomes. The standard lists specific methods for finding these: published standards and papers, market data on similar systems, incident reports on similar systems, field trials, usability studies, stakeholder reports, expert interviews, and simulations.

Practical insight for AI security analysts: Incident reports on similar systems are the most underused source on this list. Before running any risk identification workshop, search for publicly reported failures, adversarial attacks, and regulatory actions against AI systems similar to yours. The the , and CVE databases are good starting points. Nothing sharpens a risk identification session like a real-world failure story from your domain.

Identify existing controls. Document what controls already exist and assess whether they actually work. The standard specifically calls out the importance of identifying control failures.

Identify consequences. This is where the standard adds its most valuable AI-specific guidance. It instructs you to identify differences between the groups that benefit from the technology and the groups that bear negative consequences. A resume screening tool might benefit the HR department (faster processing) while systematically disadvantaging applicants from certain educational backgrounds or geographic regions.

Consequences to organizations include investigation and repair time, lost opportunities, reputational damage, regulatory penalties, and litigation.

Consequences to individuals and societies are harder to quantify but often more severe: threats to health, violations of privacy, infringement of fundamental rights.

The standard makes a practical point that experienced practitioners already know: consequences for individuals and societies almost always affect the organization as well. A safety incident creates liability claims. A bias scandal damages the brand. Map these cascading effects explicitly.

Step 3: Analyze Risks

Risk analysis requires three separate impact assessments:

Business impact assessment. How badly does this risk affect the organization? Consider criticality, tangible versus intangible impacts, and your established criteria.

Individual impact assessment. How does this risk affect the people whose data is used or whose lives are influenced by the AI system? The standard lists specific factors: types of personal data used, potential bias impact, potential impact on fundamental rights, fairness impact, safety of the individual, existing protections against bias, and the jurisdictional and cultural environment of the individual.

That last factor is easy to overlook but critical. An AI system’s impact on an individual in the EU (where GDPR applies) differs from its impact on an individual in a jurisdiction with no data protection law. The same system creates different risk profiles in different markets.

Societal impact assessment. How broadly does the AI system reach into different populations? The standard specifically asks you to consider whether the system amplifies or reduces pre-existing patterns of harm to different social groups.

Example: A government agency deploying a predictive policing model would face very different societal impact conclusions than a private company using the same technology for retail theft prevention. The government use case reaches more broadly and carries the authority of the state, which amplifies both benefits and harms.

For likelihood assessment, the standard includes a warning that practitioners should take seriously: “There can be significant technical, economic and heuristic issues with decision-making based on likelihoods, particularly when the likelihood either can’t be calculated or where the calculation has a large margin of error.” In plain language: if you cannot meaningfully estimate how likely something is, do not force a number. Focus on consequence severity and your ability to detect and respond instead.

Practical insight for data scientists: Likelihood estimates for AI system failures decay faster than for traditional software. A model’s failure probability changes every time the data distribution shifts, the model is retrained, or the user population changes. If you assign a likelihood score, attach an expiration date to it.

Step 4: Evaluate Risks

Compare the analyzed risks against your criteria. Prioritize. Decide which risks need treatment. The standard defers to ISO 31000 here without AI-specific additions, which tells you this step is about organizational judgment, not technical analysis. Get decision-makers in the room who understand both the technology and the business.

Step 5: Treat Risks

The standard provides seven treatment options, consistent with ISO 31000:

Avoid the risk (stop or do not start the activity)
Accept increased risk to pursue an opportunity
Remove the risk source
Change the likelihood
Change the consequences
Share the risk (contracts, insurance)
Retain the risk by informed decision

The AI-specific addition: if you cannot reduce negative consequences to an acceptable level through any treatment option, you must perform a risk-benefit analysis for the residual risk. This matters because some AI risks cannot be fully eliminated. The opacity of a deep learning model, for example, is inherent to the technology. You need to decide if the benefits justify the residual risk, and you need to document that decision.

Practical insight for risk managers: Each treatment measure must be verified for effectiveness and recorded. Do not just document the plan. Document whether the treatment actually worked. I have seen organizations with detailed treatment plans and zero follow-up on whether the treatments reduced the risk as expected.

Step 6: Monitor, Record, and Report

The standard’s recording requirements are specific. Your risk management record must include:

Description and identification of the analyzed system
Methodology used
Intended use of the AI system
Who performed the risk assessment
Terms of reference and date
Release status of the assessment
Whether and to what degree objectives were met

The standard also requires that you collect information from post-implementation phases and review publicly available information about similar systems. You must assess whether previously undetected risks exist or whether previously accepted risks are no longer acceptable.

The records must allow traceability of each identified risk through all risk management processes. This means persistent risk IDs, version-controlled risk registers, and a clear audit trail from identification through treatment and monitoring.

Practical insight for all three audiences: When the standard says “collect and review publicly available information on similar systems on the market,” it means this is an ongoing obligation, not a one-time activity. Set up alerts for incident reports, regulatory actions, and published research about AI systems similar to yours. The best early warning system for your own AI risks is someone else’s AI failure.

Part 2: Risk Sources and Objectives (Your Identification Checklists)

Annexes A and B of the standard provide catalogs that serve as starting points for risk identification. The standard itself says these catalogs have “shown value” for organizations performing AI risk assessment for the first time. Use them as baselines, not as exhaustive lists.

These are the things that can go wrong or right with AI systems. For each objective, the standard provides context on why it matters for AI specifically.

Accountability. AI changes who is responsible for decisions. When a person made a lending decision, that person was accountable. When an AI system makes that decision, accountability becomes unclear. Regulators worldwide are still working out who bears responsibility. You need to know the legislation in every market where your AI system operates.

AI Expertise. Building AI systems requires interdisciplinary specialists, not just software engineers. The standard also extends this to end users: they need enough understanding of the AI system to detect and override erroneous outputs.

Practical example: A manufacturing company deployed an AI-powered quality inspection system but did not train the floor operators on how the system made decisions or what its failure modes looked like. When the system began missing defects due to a lighting change in the factory, operators trusted the system’s “pass” decisions for three weeks before someone escalated the rising customer complaint rate.

Training and Test Data Quality. Training and test data must be validated for currency, relevance, diversity, and consistency. If you source data externally, data quality is still your responsibility. The amount of data required varies with the functionality and complexity of the environment.

Practical insight for data scientists: The standard specifically calls out that training and test datasets should be independent when applicable. This is a basic ML practice, but the standard elevates it to a risk management concern. If your test set leaks into your training data, you have not just a technical problem but a governance failure.

Environmental Impact. AI can help the environment (optimizing energy use, reducing emissions) or hurt it (massive compute requirements for training). Both sides must be considered.

Fairness. Unfair outcomes can come from biased objective functions, imbalanced datasets, human biases in training data, biased product concepts, or decisions about when and where to deploy AI systems. The standard references ISO/IEC TR 24027 for deeper guidance on bias.

Maintainability. ML-based systems are trained, not programmed. Modifying them to fix defects or adapt to new requirements is fundamentally different from patching traditional software. Understand the implications before deploying.

Privacy. AI systems that depend on large datasets create privacy risks through data collection, through inference of sensitive information, and through model personalization. The standard notes that AI can infer sensitive personal data even when that data was not directly provided. A data protection impact assessment (per ISO/IEC 29134) is recommended.

Practical insight for AI security analysts: The standard highlights that protecting privacy in AI includes protecting access to models personalized for individuals or models that can be used to infer characteristics of similar individuals. This means model extraction attacks are a privacy risk, not just a security risk. If an attacker can replicate your model, they can potentially infer characteristics of your training data subjects.

. Can the system maintain performance under unexpected conditions? Neural networks are particularly challenging here because their nonlinear nature can produce unexpected behavior. Characterizing neural network robustness remains an open research problem.

Safety. AI systems in vehicles, manufacturing, robotics, and medical devices introduce safety risks that must be evaluated against domain-specific safety standards. The standard does not replace those domain standards. It adds to them.

Security. Beyond classical information security, AI introduces new attack surfaces: data poisoning (corrupting training data), adversarial attacks (crafted inputs that fool the model), and model stealing (extracting the model through query access). ISO/IEC 27005 covers general information security risk management. The AI-specific threats require additional consideration.

Practical example for security analysts: A financial institution’s fraud detection model was trained on transaction data that included a small number of poisoned records inserted by an insider. The poisoned data taught the model to classify certain fraudulent transaction patterns as legitimate. Classical information security controls (access management, encryption) did not prevent this because the insider had authorized access to the data pipeline. AI-specific controls (data provenance tracking, statistical anomaly detection on training data) would have caught it.

Transparency is about what the organization communicates. Explainability is about what the system can reveal about its own decision-making. Both matter, and they serve different purposes. Transparency builds trust with stakeholders. Explainability enables validation and verification by the organization itself.

The standard also notes a tension: excessive transparency can create privacy, security, and intellectual property risks. You need to find the right level for each stakeholder group.

These are the places where risks originate. Use this as a checklist during risk identification.

Complexity of environment. The more complex the operating environment, the harder it is to ensure your training data covers all possible situations. For autonomous driving, you cannot guarantee coverage of every scenario. For a chatbot answering questions about a fixed product catalog, coverage is more achievable. Assess how well-understood your system’s environment is, because partial understanding creates uncertainty that is itself a risk source.

Lack of transparency and explainability. If you cannot explain why your model made a specific decision, you cannot fully validate it. This affects trustworthiness, accountability, safety, security, fairness, and robustness. The standard emphasizes that explainability matters for internal validation, not just external communication.

Level of automation. Systems range from fully human-controlled to fully automated. Higher automation means less human oversight, which amplifies both the efficiency gains and the risk exposure. For systems where a human must be “ready to take over when necessary,” the handover itself is a risk source. Think about response time, operator attention, and situation awareness.

Machine learning specific risks. Data quality directly affects system behavior. Data collection processes are a risk source that is “especially hard to diagnose and detect.” Data can become unrepresentative over time. Data sourcing creates risks. Failing to secure the data pipeline opens the door to adversarial manipulation. Continuous learning systems can change their behavior in production in ways that were not anticipated at deployment.

Practical example: An e-commerce recommendation engine trained on user interaction data gradually learned to recommend increasingly sensationalized products because those generated more clicks. The continuous learning loop optimized for the engagement metric without any check on whether the recommendations were appropriate. The behavior drift was subtle enough that no one noticed for months.

System hardware issues. Hardware errors, soft errors from radiation, constraints when transferring models between different hardware platforms, and network issues for systems requiring remote processing. These are easier to overlook in AI because teams focus on model performance and forget about the physical infrastructure.

System life cycle issues. Risks exist at every stage:

Design: failing to anticipate deployment contexts
Verification and validation: inadequate testing causing regressions
Deployment: misconfigured resources
Maintenance: unsupported but still-running systems
Reuse: using a system in a context it was not designed for (the standard gives the example of a social media face detection system repurposed for criminal suspect identification, a far more demanding use case)
Decommissioning: losing the decision expertise embedded in the retired system

Technology readiness. Less mature technologies carry unknown risks. More mature technologies create complacency and technical debt. Both ends of the maturity spectrum require attention.

Part 3: The Organizational Framework

This is Clause 5. It describes how to embed AI risk management into your o .

Leadership and Commitment

Top management, supported by , must do two things the standard calls out specifically for AI:

First, consider issuing public statements about the organization’s commitment to AI risk management. This creates accountability and builds stakeholder confidence.

Second, recognize that AI risk management requires specialized resources and allocate them. An AI risk program staffed by people without AI expertise will produce documentation that misses the actual risks.

Organizational Context

The standard provides two detailed tables for understanding your external and internal context.

For external context, AI-specific considerations include:

AI-related legal requirements in your markets
Ethical guidelines from government groups, regulators, standardization bodies, civil society, academia, and industry associations
Domain-specific AI frameworks
Societal implications of your AI deployments
How continuous learning might affect your ability to meet contractual obligations
Ownership and usage rights for training data provided by third parties

For internal context, AI-specific considerations include:

How AI changes organizational culture by creating new roles and responsibilities
Deskilling risks where human decision-making is increasingly replaced by AI
The availability of AI tools that enable development without full understanding of the technology
Intellectual property implications of AI systems
Additional data quality constraints imposed by AI
The need to educate stakeholders on AI capabilities, failure modes, and failure management

Practical insight for risk managers: The external context requirement to track “ and design of AI and automated systems issued by government-related groups, regulators, standardization bodies, civil society, academia and industry associations” is broad. Create a regulatory and guidance tracker specific to AI. Assign someone to update it quarterly. The landscape is changing fast enough that annual reviews will miss significant developments.

Roles and Accountabilities

The standard requires top management and oversight bodies to identify specific individuals with:

Authority to address AI risks
Responsibility for establishing and monitoring processes to address AI risks

Notice the standard says “individuals,” not “committees.” Named accountability matters.

Practical insight: If the person accountable for AI risk management does not have authority over the AI development teams, the role is ceremonial. Verify that the accountability chain has teeth.

Communication and Consultation

The standard notes that stakeholders affected by AI systems can be “larger than initially foreseen, can include otherwise unconsidered external stakeholders and can extend to other parts of a society.” Plan your stakeholder engagement broadly from the start, because discovering a critical stakeholder group after deployment creates reactive crises.

Part 4: The Guiding Principles

Clause 4 defines eight risk management principles from ISO 31000. Five of them get AI-specific guidance.

Inclusive. AI systems affect more stakeholders than traditional systems. Engage diverse internal and external groups. Stakeholders help identify data risks, define fairness criteria, spot bias, determine where human oversight is needed, and shape transparency and explainability approaches. The standard suggests segmenting transparency frameworks by stakeholder persona (regulators, business owners, model risk evaluators) when a one-size-fits-all approach does not work.

Dynamic. AI systems change through continuous learning. Customer expectations shift rapidly. Regulations update frequently. Your risk management process must anticipate, detect, and respond to these changes in real time, not annually.

Best available information. Historical data about AI failures may be limited because the technology is relatively new. Future expectations change quickly. Track how your AI systems are used after deployment. Be aware that tracking external usage may be limited by IP, contractual, or market restrictions, and capture those limitations explicitly in your risk process.

Human and cultural factors. Monitor how your AI systems interact with pre-existing societal patterns that affect equitable outcomes, privacy, freedom of expression, fairness, safety, security, employment, the environment, and human rights.

Continual improvement. Monitor the AI ecosystem for performance successes, shortcomings, lessons learned, and new research findings. Feed previously unknown risks back into the improvement cycle.

Part 5: Risk Management Across the AI System Life Cycle

Annex C maps risk management activities to the AI system life cycle defined in ISO/IEC 22989:2022. The life cycle stages are:

Inception
Design and development
Verification and validation
Deployment
Operation and monitoring
Continuous validation
Re-evaluation
Retirement or replacement

At the organizational level, the governing body sets risk appetite, establishes general criteria, and builds catalogs of risk criteria, risk sources, mitigation measures, monitoring techniques, and reporting formats. These catalogs improve over time as feedback flows up from individual AI system risk processes.

At the project level, each AI system goes through its own risk management cycle at each life cycle stage. Risk criteria, assessments, and treatment plans are established at inception, continuously updated during design and development, verified during testing, potentially adjusted during deployment, and monitored throughout operation.

The key takeaway: risk management is not a phase. It happens at every stage. The standard explicitly shows risk assessment, treatment, monitoring, and recording activities at every life cycle stage, including retirement.

Practical insight for data scientists: The re-evaluation stage is often skipped. The standard requires that existing risk sources be examined for relevance, criteria re-evaluated against changes in scope or purpose, and regulatory updates incorporated. Build re-evaluation triggers into your model governance process. Any change in model purpose, data source, or regulatory environment should start a re-evaluation cycle.

Practical insight for AI security analysts: The retirement stage creates specific risks. When you decommission an AI system, you can lose decision expertise and institutional knowledge embedded in that system. If a replacement system is deployed, the way the organization processes information and makes decisions changes. Both of these transitions create attack surface changes and knowledge gaps that need to be assessed as security risks.

What to Do Monday Morning

If you are a risk manager: start with the AI system inventory. You cannot manage risks you do not know exist. Then map your stakeholders using the nine-category list. Then compare your existing risk criteria against the AI-specific factors in Table 4 of the standard.

If you are a data scientist: read Annex B on risk sources, particularly section B.5 on machine learning risks. Then review your current model documentation against the recording requirements in Clause 6.7. The gap between what you document today and what the standard requires is likely significant.

If you are an AI security analyst: start with Annex A sections on security (A.11), privacy (A.8), and robustness (A.9). Map your current threat model against the AI-specific attack surfaces the standard identifies: data poisoning, adversarial attacks, model stealing. Then check whether your security controls cover the full AI system life cycle or only the deployment and operation stages.

The standard gives you the structure. The work is in applying it honestly to your specific systems, your specific organization, and your specific stakeholders. That is where the real risk management happens.

ISO/IEC 23894 does not exist in isolation. It references and connects to a broader ecosystem of standards that together form a comprehensive AI governance framework.

ISO 31000:2018, Risk management, Guidelines. This is the foundational standard. ISO/IEC 23894 is built directly on top of it.

ISO/IEC 22989:2022, Artificial intelligence, Concepts and terminology. Provides the AI-specific definitions and the system life cycle model referenced throughout 23894.

ISO Guide 73:2009, Risk management, Vocabulary. Defines the risk management terms used in both 31000 and 23894.

ISO/IEC 38507:2022, Governance implications of the use of artificial intelligence by organizations. Covers the governance layer that sits above risk management.

ISO/IEC TR 24028:2020, Overview of trustworthiness in artificial intelligence. Provides background on AI trustworthiness that informs several sections of 23894.

ISO/IEC TR 24027:2021, Bias in AI systems and AI-aided decision making. Essential reading for the fairness-related risk identification and analysis sections.

ISO/IEC 29134:2017, Guidelines for privacy impact assessment. Directly relevant when AI systems process personal data.

ISO/IEC 27005:2022, Guidance on managing information security risks. Covers the security dimension of AI risk, including threats like data poisoning and adversarial attacks.

NIST AI Risk Management Framework (AI RMF 1.0). While not referenced in the standard, this U.S. framework maps closely to ISO/IEC 23894 and is increasingly expected by American regulators and enterprise customers.

EU AI Act (Regulation 2024/1689). The European regulation that makes AI risk management a legal obligation for high-risk AI systems. ISO/IEC 23894 provides a structured path toward many of its requirements.

Supporting AI ISO standards by Topic

Core AI Management & Governance Standards

ISO/IEC 42001:2023 – Information technology — Artificial intelligence — Management system (AIMS).
ISO/IEC 38507:2022 – Governance implications of the use of AI by organizations.
ISO/IEC 23894:2023 – Guidance on risk management for AI.
ISO/IEC 42005:2025 – AI system impact assessment.
ISO/IEC 42006:2025 – Requirements for auditing bodies providing AI management system certification.

Foundational Frameworks & Terminology

ISO/IEC 22989:2022 – AI concepts and terminology.
ISO/IEC 23053:2022 – Framework for AI systems using Machine Learning.
ISO/IEC 5338:2023 – AI system life cycle processes.
ISO/IEC 5339:2024 – Guidance for AI applications.

Trustworthiness, Ethics & Quality

ISO/IEC 24028:2020 – Overview of trustworthiness in artificial intelligence.
ISO/IEC 24368:2022 – Overview of ethical and societal concerns.
ISO/IEC 25059:2023 – Quality model for AI systems (SQuaRE).
ISO/IEC 12791:2024 – Treatment of unwanted bias in ML tasks.
ISO/IEC 12792:2025 – Transparency taxonomy for AI systems.
ISO/IEC 42119-2:2025 – Testing of AI systems — Part 2: Test data and results.
ISO/IEC 4213:2022 – Assessment of machine learning classification performance.

Data Quality & Analytics

ISO/IEC 24668:2022 – Process management framework for big data analytics.
ISO/IEC 5259-1:2024 – Data quality for analytics and ML — Part 1: Overview and terminology.
ISO/IEC 5259-3:2024 – Data quality management requirements and guidelines.
ISO/IEC 5259-4:2024 – Data quality process framework.

The Choice in Front of You

Organizations that treat ISO/IEC 23894 as a compliance artifact will produce binders full of risk assessments that nobody reads, risk registers that go stale within weeks, and governance structures that exist on paper but have no operational authority. When something goes wrong, and with AI systems something eventually does go wrong, they will discover that their documentation protected nobody. Not the organization, not the individuals affected by the AI system, and not the communities that bore the consequences.

Organizations that treat ISO/IEC 23894 as a living operational tool will build AI risk management into their development pipelines, their deployment decisions, and their ongoing monitoring. They will have named individuals with real authority, risk criteria grounded in evidence, stakeholder engagement that surfaces risks early, and records that tell a coherent story from inception through retirement. When something goes wrong, they will know about it faster, respond more effectively, and demonstrate to regulators that they took reasonable steps.

The standard gives you the blueprint. What you build with it depends entirely on whether you treat AI risk management as paperwork or as practice.

To maximize SEO authority and drive high-value traffic back to your core assets, this “About the Author” section is restructured to emphasize your specific expertise in the EU AI Act, Quantitative Risk, and ISO 42001. I have humanized the tone to move from a standard bio to a “Partnership Invitation,” while ensuring the links are prominent and descriptive.

About the Author: Prof. Hernan Huwyler, MBA, CPA, CAIO

The frameworks, taxonomies, and implementation toolkits shared in this article are part of the ongoing applied research and executive advisory work of Prof. Hernan Huwyler. These materials are designed for operational realism and are freely available for adaptation in your own AI Governance, Risk Management, and Compliance (GRC) programs under proper attribution.

Bridging the Gap Between AI Theory and Production Controls

As an AI GRC Strategy Director and Quantitative Risk Lead, Prof. Huwyler works with global organizations in financial services, healthcare, and the public sector to build frameworks that survive both production demands and regulatory scrutiny. His expertise is focused on tje risk-adjusted AI adoption, ensuring that innovation remains defensible through rigorous algorithmic auditing and automated compliance protocols.

Global Thought Leadership and Executive Education

Based in the Copenhagen Metropolitan Area with a professional presence in Zurich, Geneva, Madrid, and Berlin, Prof. Huwyler operates where AI development is most active. He serves as an Executive Advisor and Academic Director at IE Law School, delivering specialized corporate training on:

EU AI Act Compliance Strategy and ISO 42001 integration.
Quantitative Risk Modeling using Python and Monte Carlo simulations.
Predictive Risk Automation for Board-level decision-making.

Explore Open-Source GRC Tools and Insights

Prof. Huwyler maintains a public repository of Python-based AI governance tools, risk model templates, and automated compliance scripts to support the global practitioner community.

Technical Tools & Code: Access the for risk model templates.
Expert Commentary: Read the latest on GRC and Internal Audit at .
Professional Network: Connect on to follow real-time updates on the evolving AI regulatory landscape.

Let’s Turn AI Governance into a Competitive Advantage

If you are ready to move beyond “check-the-box” compliance and start capturing the positive ROI of AI, let’s connect. True governance isn’t about slowing down; it’s about creating the structural discipline needed to achieve massive savings through automation and to build new, resilient revenue streams that regulators and customers can trust.

Implementation Tips for Expert Calibration and AI-Augmented Risk Estimation

Thu, 12 Mar 2026 00:00:00 +0000

Why Expert Calibration Matters for GRC Professionals

Most risk assessments rely on expert judgment. When historical loss data is absent, limited, or conflicting, you ask knowledgeable people to estimate probabilities and impacts. The problem is that unstructured expert judgment is unreliable. Experts overestimate rare events, underestimate common ones, anchor to previous numbers, and conform to dominant opinions in group settings.

Expert calibration is a quantitative technique that measures and improves the accuracy of expert predictions over time. It treats expert judgment as data, subject to the same scientific principles of review, critical appraisal, and repeatability that you’d apply to any other data source in your risk assessment.

The difference between a calibrated risk assessment and an uncalibrated one is the difference between a defensible estimate and an educated guess. Regulators, auditors, and boards increasingly expect the former.

The Core Mechanism: How Expert Calibration Works

The Basic Cycle

Expert calibration follows a straightforward cycle. Ask experts to estimate potential losses or probabilities of events occurring. Compare actual outcomes to their estimates. Use multiple data points over time to determine whether an expert tends to overestimate or underestimate. Feed this information back to improve future estimates.

How to implement:

Group estimated probabilities into bands (for example, events the expert rated as 10-20% likely, 20-30% likely, and so on). Compare these bands to actual occurrence rates. A well-calibrated expert who assigns 20% probability to events should see roughly 20% of those events actually occur.

Calculate each expert’s overall accuracy by averaging multiple estimates. A perfectly calibrated expert’s estimates should, on average, match what you’d expect from a uniform distribution across probability bands.

Very low probability events present a challenge. If an expert estimates a 2% probability, you need 50 or more observations to determine whether 2% is accurate. For rare events, combine calibration data across similar event categories to build a sufficient sample.

Original implementation tip: Start building calibration histories now, even if you don’t plan to use them for six months. Every time your organization conducts a risk assessment, record each expert’s estimate alongside the question, the date, and eventually the actual outcome. Most organizations can’t calibrate their experts because they never retained the historical estimates. They have last year’s risk register but not the individual predictions that went into it. Store individual expert estimates in a structured database with fields for expert name, question, estimated probability, estimated impact range, date of estimate, and actual outcome when known. After 12 months of accumulation, you’ll have enough data points to calculate meaningful calibration scores for your most active experts. Without this history, calibration is impossible and you’re permanently stuck with uncalibrated judgment.

Two Approaches to Aggregating Expert Opinions

Behavioral Aggregation: The Workshop Method

Behavioral aggregation brings experts together in face-to-face meetings to reach shared judgment through discussion and consensus. Experts exchange and debate their knowledge, potentially producing more informed and balanced decisions.

This method is familiar. Most risk workshops use some version of it.

The problem:

Behavioral aggregation is vulnerable to well-documented biases. Group thinking causes experts to conform to the majority view even when they disagree. The halo effect allows a dominant expert’s opinion to unduly influence others. Anchoring causes experts to gravitate toward the first number mentioned. Polarization can prevent consensus even with skilled facilitation. And forced consensus, when imposed despite genuine disagreement, masks important differences in opinion and reduces the quality of the final judgment.

Original implementation tip: If you must use behavioral aggregation, implement three structural safeguards. First, collect individual written estimates before any group discussion begins. This prevents anchoring to the first number spoken aloud. Second, give equal time to every expert, actively drawing out quiet participants and managing dominant voices. Third, never force consensus. If experts genuinely disagree after discussion, document the disagreement and the range of estimates rather than artificially converging on a single number. A documented range of expert opinion is more honest and more useful than a false consensus that nobody actually believes. I’ve facilitated dozens of risk workshops where the “consensus” estimate was the number the most senior person in the room stated first. Everyone else adjusted toward it. The estimate reflected hierarchy, not expertise.

Algorithm Calibration: The Mathematical Method

Algorithm calibration limits expert interaction to training and briefing sessions. Consensus is not achieved through discussion but through mathematical aggregation of individual expert opinions.

This approach makes the aggregation process explicit and auditable. The Classical Model, developed by Roger Cooke, uses a linear combination of judgments weighted by each expert’s past performance in estimating risk impacts and probabilities. Better-calibrated experts receive higher weights. Poorly calibrated experts receive lower weights or zero weight.

The tradeoff:

Algorithm calibration can be less effective when experts strongly disagree and receive little feedback from their peers. The mathematical aggregation may miss contextual nuances that discussion would surface. But it eliminates group biases entirely, produces reproducible results, and creates an auditable record of exactly how the final estimate was derived.

Original implementation tip: Use algorithm calibration as your primary method and behavioral discussion as a supplementary input. Collect individual estimates first using the structured elicitation protocol described below. Aggregate them mathematically using calibration weights. Then, if the weighted estimates show extreme divergence among high-weight experts, convene a focused discussion limited to understanding why those experts disagree. The discussion informs whether the divergence reflects genuine uncertainty (which should be preserved in the final estimate as a wider distribution) or a misunderstanding of the scenario (which should be corrected). This sequence, individual estimation first, mathematical aggregation second, targeted discussion third, captures the benefits of both approaches while minimizing the biases of each.

Mathematical Aggregation Methods

Bayesian Updating

Use each expert’s opinion to update your existing knowledge about the risk. Start with a prior estimate based on historical data or organizational experience. Then adjust that estimate based on each expert’s input, weighted by how confident you are in both your prior and in each expert’s judgment.

How to implement:

Define your prior distribution based on available data. For each expert opinion, update the distribution using Bayes’ theorem. The result is a posterior distribution that incorporates both your historical knowledge and the experts’ collective judgment. Experts whose opinions align with strong historical evidence reinforce the estimate. Experts whose opinions diverge from historical patterns shift the estimate only if their track record or the strength of their reasoning justifies it.

Original implementation tip: The Bayesian approach works best when you have a meaningful prior, meaning real historical data to start from. If your prior is purely a guess, the Bayesian update is just averaging guesses with extra mathematical notation. Before choosing this method, honestly assess whether your prior distribution is based on data or assumption. If it’s based on data, Bayesian updating is powerful. If it’s based on assumption, opinion pooling or the Cooke method may be more appropriate because they don’t pretend you have knowledge you don’t have.

Opinion Pooling (Weighted Average)

Assign each expert a specific weight reflecting their relative expertise and trustworthiness. Combine their opinions as a weighted average. The result is a blended estimate that reflects how much you value each expert’s input.

The Cooke method is a specific form of opinion pooling where weights are determined empirically by each expert’s past accuracy, not by subjective assessment of their credentials.

How to implement:

In the Cooke method, give more weight to experts who have been more accurate in the past, measured through calibration questions with known answers. Experts who consistently predict historical outcomes correctly receive higher weights. Experts who consistently miss receive lower weights or zero weight.

Calculate weights by scoring each expert’s responses to calibration questions against known correct answers. The simplest scoring method assigns 1 for correct and 0 for incorrect, totals the scores, and converts them to percentages. More sophisticated scoring uses proper scoring rules that evaluate the full probability distribution each expert provides, not just point estimates.

Original implementation tip: The weight assignment step is where most implementations fail. Organizations resist giving zero weight to experts with impressive titles or seniority. But the entire point of calibration is that credentials don’t guarantee accuracy. An expert with 20 years of experience who consistently overestimates by 300% should receive less weight than a junior analyst who consistently hits within 20% of actual outcomes. If you can’t bring yourself to weight experts by demonstrated accuracy rather than organizational rank, don’t use the Cooke method. You’ll corrupt it by overriding the calibration data with political judgments, and the result will be worse than simple averaging because it will carry a false veneer of scientific rigor.

The Structured Elicitation Protocol

Step-by-Step Implementation

The structured elicitation protocol reduces biases and improves accuracy through a disciplined process. It treats expert judgments with the same rigor you’d apply to operational risk data.

Phase 1: Preparation

Identify relevant experts from various disciplines. Note that domain expertise doesn’t guarantee unbiased or error-free judgment. Gather relevant information about the problem, including historical data, regulatory context, and comparable cases. Prepare easy-to-understand data presentations. Share information with attendees before the meeting so they arrive informed.

How to implement:

Select experts with diverse perspectives. For a GDPR fine estimation, you might include a data protection officer, a legal privacy advisor, a compliance officer, a privacy consultant, a head of compliance, and a head of data governance. Diversity of viewpoint is more valuable than depth in a single perspective.

Prepare calibration questions with known answers related to the risk domain experts will predict. These questions test each expert’s accuracy before you ask them to estimate unknowns.

Original implementation tip: The quality of your calibration questions determines the quality of your entire process. Calibration questions must be from the same domain as the prediction you’re asking experts to make, must have objectively verifiable correct answers, must span a range of difficulty levels, and must not be so obvious that every expert gets them right (which provides no differentiation). I typically prepare five to seven calibration questions per session. Three questions is the minimum for meaningful differentiation. Fewer than three doesn’t provide enough signal to separate well-calibrated experts from lucky guessers. For the GDPR fine estimation case, calibration questions might ask about the most common fine amount, the 75th percentile fine, and the probability of exceeding a specific threshold, all based on published regulatory data that can be verified.

Phase 2: Workshop Opening

Explain the workshop objectives and outline the problem structure and key uncertainties. Emphasize that exact probability knowledge isn’t required. Highlight how distributions allow for uncertainty expression. Present prepared data and information, encouraging open dialogue about variability and uncertainty. Discuss the logical structure and potential correlations, exploring scenarios that could lead to extreme outcomes.

Original implementation tip: Spend at least 20 minutes on training experts to think in distributions rather than point estimates. Most professionals are trained to give single numbers: “the fine will be €100,000.” Calibrated estimation requires ranges: “I’m 90% confident the fine will fall between €30,000 and €400,000.” This is a skill that must be taught. Use a simple warm-up exercise: ask experts to estimate something they can verify immediately, like the distance between two cities or the population of a country, as a 90% confidence interval. Then reveal the answer. Most people’s first confidence intervals are far too narrow, capturing the true answer less than 50% of the time instead of 90%. This exercise demonstrates overconfidence viscerally and motivates experts to widen their ranges appropriately. Run this exercise at the start of every calibration session.

Phase 3: Workshop Facilitation

Encourage experts to develop their own opinions based on group discussion, giving equal prominence to quiet and dominating experts. Allow time for private consideration and explanation of parameter uncertainty. Emphasize that distributions don’t require more knowledge than point estimates.

Phase 4: Individual Estimations

Conduct one-on-one interviews with each expert using three-point estimates: minimum (best case), most likely, and maximum (worst case). Gather individual estimates without group influence.

How to implement:

The three-point estimate captures the expert’s uncertainty range. The minimum represents the lowest plausible outcome. The most likely represents the mode of their mental distribution. The maximum represents the highest plausible outcome. These three points can be fitted to a distribution (triangular, PERT, or beta) for further analysis.

Collect estimates individually to prevent anchoring and conformity bias. Even after a group discussion phase, the actual numerical estimates must be provided privately.

Original implementation tip: When collecting three-point estimates, ask for the minimum and maximum first, then the most likely value. If you ask for the most likely value first, experts anchor to it and set their minimum and maximum too close, producing artificially narrow ranges. By asking for extremes first, you force the expert to think about what could go wrong (maximum) and what the best realistic outcome looks like (minimum) before settling on their central estimate. This simple sequencing change consistently produces wider, more realistic ranges. I’ve tested both sequences with the same expert groups and the extremes-first approach produces ranges that are 30 to 50% wider, which better reflects genuine uncertainty.

Phase 5: Calibration Feedback

Compare past estimates to actual outcomes to assess biases or patterns. Identify experts who consistently estimate accurately. Identify large differences in expert opinions and reconvene if necessary to discuss discrepancies. Provide feedback on estimation performance and discuss techniques for improving future estimates.

Phase 6: Consensus Building

Facilitate a discussion to reach a shared understanding of risks and uncertainties, avoiding forced agreement on specific numbers. Summarize key points and insights. Outline next steps for using the gathered information in the risk analysis.

Phase 7: Follow-Up

Document workshop outcomes and distribute results to participants. Plan for future calibration sessions to track improvement over time. Allow for estimate revisions as new information becomes available.

Original implementation tip: The follow-up phase is where most organizations drop the ball. They conduct the workshop, produce the aggregated estimate, use it in the risk assessment, and never revisit it. Without follow-up, there’s no learning. Schedule a calibration review six months and twelve months after each session. At the review, compare the aggregated estimate to any actual outcomes that have materialized. Update expert calibration scores. Share the results with the experts. Over time, this feedback loop demonstrably improves estimation accuracy. The Good Judgment Project documented that calibration feedback improved forecasting accuracy by 10 to 15% within the first year. Without feedback, accuracy stays flat or degrades. The feedback loop is what transforms expert judgment from a static input into an improving instrument.

Step 1: Gather Historical Data for Calibration

Before asking experts to estimate anything, gather objective data to calibrate their accuracy and provide context.

For GDPR fines related to processing personal data without legal grounds (Article 6(1)) in Spain over the past two years, the data shows 87 fines ranging from €240 to €1,200,000 with a mean of €72,941, a median of €20,000, and a mode of €10,000 (appearing 8 times). The standard deviation of €154,431 indicates a wide spread. The 25th percentile is €6,000, the 75th percentile is €70,000, and the 90th percentile is €200,000. Banking sector fines tend to be higher: €1,200,000, €200,000, and €70,000.

Original implementation tip: The statistical analysis of historical data serves two purposes. First, it provides the correct answers for calibration questions. Second, it gives experts an empirical foundation for their estimates. Share the summary statistics with experts before the session. Don’t hide the data to “test” their knowledge. The goal isn’t to trick experts. It’s to produce the most accurate possible estimate of future fines. Informed experts produce better estimates than uninformed ones. However, share the summary statistics, not the raw dataset. Experts who review 87 individual fine records will anchor to memorable outliers. Experts who see percentile distributions develop more balanced mental models. Present the data as distributions and percentiles, not as a list of cases.

Step 2: Design Calibration Questions

Prepare calibration questions based on the known statistics. Each question has a correct answer derived from the historical data.

Question 1: What is the most likely (mode) fine for processing personal data without legal grounds in Spain? Options: €10,000 / €70,000 / €200,000 / €1,200,000. Correct answer: €10,000.

Question 2: What do you estimate as the 75th percentile fine for GDPR violations related to insufficient legal grounds? Options: €20,000 / €70,000 / €200,000 / €500,000. Correct answer: €70,000.

Question 3: What is the probability a fine will exceed €200,000 for violating Article 6(1) GDPR? Options: 0-10% / 11-30% / 31-50% / 51-70% / 71-90% / 91-100%. Correct answer: 0-10% (the 90th percentile is €200,000, so approximately 10% of fines exceed this level).

Original implementation tip: Design calibration questions that test different aspects of the expert’s understanding: central tendency (mode or median), distribution shape (percentiles), and tail risk (probability of exceeding a threshold). An expert who correctly identifies the most common fine but overestimates tail risk has a specific bias pattern that the calibration can address. An expert who gets the percentiles right but misidentifies the mode has a different pattern. Three well-designed questions that test different distribution characteristics provide more differentiation than ten questions that all test the same type of knowledge. Also, use multiple-choice format for calibration questions rather than open-ended responses. Open-ended responses are harder to score consistently and create ambiguity about whether a “close” answer should receive partial credit.

Step 3: Collect Expert Responses

Six experts across different roles respond to the three calibration questions. Their responses are compared to the correct answers.

The Data Processing Officer answers €10,000 (correct), €200,000 (incorrect), 0-10% (correct). The Legal Privacy Advisor answers €70,000 (incorrect), €200,000 (incorrect), 0-10% (correct). The Compliance Officer answers €10,000 (correct), €70,000 (correct), 0-10% (correct). The Privacy Consultant answers €200,000 (incorrect), €500,000 (incorrect), 11-30% (incorrect). The Head of Compliance answers €70,000 (incorrect), €70,000 (correct), 0-10% (correct). The Head of Data Governance answers €70,000 (incorrect), €200,000 (incorrect), 31-50% (incorrect).

Original implementation tip: Notice that the Compliance Officer scored 100% on calibration questions while the Privacy Consultant and Head of Data Governance scored 0%. This is a common pattern. Domain expertise and seniority don’t predict calibration accuracy. The Privacy Consultant may have deep knowledge of privacy law but poor calibration on quantitative estimates. The Head of Data Governance may understand data governance frameworks but have no feel for regulatory penalty distributions. The Cooke method handles this elegantly by assigning zero weight to experts who demonstrate poor calibration, regardless of their title. The hardest part of implementation is presenting these results to the experts themselves. Do it with transparency and respect. Frame it as “calibration accuracy for this specific question set” rather than “you don’t know what you’re talking about.” Calibration scores measure estimation skill, not domain knowledge. A poorly calibrated expert may still contribute valuable qualitative insights during the discussion phase.

Step 4: Assign Weights Based on Calibration Performance

Score each expert’s responses (1 for correct, 0 for incorrect) and calculate calibration weights.

The Data Processing Officer scores 2 out of 3 (67%), assigned weight 25%. The Legal Privacy Advisor scores 1 out of 3 (33%), assigned weight 12%. The Compliance Officer scores 3 out of 3 (100%), assigned weight 37%. The Privacy Consultant scores 0 out of 3 (0%), assigned weight 0%. The Head of Compliance scores 2 out of 3 (67%), assigned weight 25%. The Head of Data Governance scores 0 out of 3 (0%), assigned weight 0%.

Assigned weights are calculated by dividing each expert’s percentage by the total of all non-zero percentages (267%), producing the final weight distribution.

Original implementation tip: The weight calculation is simple arithmetic, but its implications are profound. Two of six experts receive zero weight. Their estimates will not influence the final aggregated prediction at all. In a traditional workshop, these two experts would have equal voice with everyone else, potentially pulling the estimate toward their incorrect mental models. The Cooke method eliminates this influence mathematically. When presenting the methodology to stakeholders, emphasize that zero weight doesn’t mean the expert’s opinion is worthless. It means their quantitative estimation accuracy, as measured by the calibration questions, doesn’t support giving their numerical estimates influence over the final aggregate. They can still contribute qualitative context during discussions. But when it comes to the number, calibrated experts drive the result.

Step 5: Aggregate the Weighted Responses

Multiply each expert’s estimate by their assigned weight and sum the results.

Using the calibration question responses for the most common fine, the aggregated estimate is €32,472. This is significantly lower than a simple average of €71,667 because the two experts who estimated high values (Privacy Consultant at €200,000 and Head of Data Governance at €70,000) received zero weight.

For a more accurate bank-specific estimate, ask experts to provide a revised estimate for the specific bank scenario. The aggregated bank-specific estimate is €106,236, driven primarily by the Compliance Officer (37% weight, €100,000 estimate) and the Head of Compliance (25% weight, €150,000 estimate).

Original implementation tip: Always collect both a general estimate and a scenario-specific estimate. The general estimate calibrated against historical data tells you how accurate each expert is at reading the base rate. The scenario-specific estimate applies their judgment to the actual case you care about, weighted by their demonstrated accuracy. The general estimate acts as a sanity check. If the scenario-specific aggregated estimate is dramatically different from the historical base rate, you need to understand why. In this case, the bank-specific estimate of €106,236 is higher than the general most-common estimate of €32,472 because experts appropriately adjusted for the banking sector’s higher fine profile. That’s a reasonable, explainable deviation. If the bank-specific estimate were €5,000,000, you’d need to investigate whether the experts are incorporating genuine sector-specific factors or simply overreacting to headline cases.

Using AI as Expert Estimators

The Method

Large language models can serve as additional “experts” in the calibration process. The approach treats each LLM as an independent estimator whose predictions are weighted by demonstrated accuracy, just like human experts.

How to implement:

Use multiple LLMs with diverse training data to estimate potential fines or impacts. Develop standardized prompts that provide consistent information about the risk scenario, relevant regulations, historical data, and the required output format. Calibrate LLM outputs using the same Cooke method applied to human experts: test them against known historical data and assign weights based on accuracy. Combine predictions from multiple LLMs using weighted averaging.

The prompt structure should specify the role the LLM should adopt (such as a Data Protection Officer at a financial institution), the specific regulation and article at issue, the three scenarios to estimate (best case, most common, worst case), the factors to consider (severity, intent, cooperation, mitigation actions), and the requirement to reference historical cases and regulatory guidelines.

Original implementation tip: The prompt design is critical. Inconsistent prompts across LLMs make comparison meaningless. Build a standardized prompt template that you use identically across all models. The template should include the exact same scenario description, the exact same historical context, and the exact same output format requirements. The only variable should be the LLM itself. I structure prompts with four sections: role definition, scenario description with specific regulatory context, action steps specifying the required outputs, and outcome expectations specifying the format and evidence requirements. Test the prompt on one model first to verify it produces the expected output structure. Then deploy it across all models simultaneously.

Calibrating AI Estimates Against Reality

In the GDPR fine case study, five LLMs produced dramatically different estimates for the most common fine.

Llama estimated €200,000. Claude estimated €400,000. Mistral estimated €3,000,000. Gemini estimated €220,000. GPT-4o estimated €60,000.

When calibrated against the actual most common fine of €10,000, GPT-4o was closest (still off by a factor of six), while Mistral was off by a factor of 300.

Using the Cooke method, each LLM’s responses were scored against known historical data (best case, most common, worst case). Claude-3.5-sonnet achieved the best calibration (50% assigned weight) because its estimates had the lowest total absolute error percentage. Mistral received 32% weight. Llama received 9%. Gemini received 8%. GPT-4o received only 3% weight despite having the most accurate most-common estimate, because its best-case and worst-case estimates were significantly off.

The aggregated AI estimate for the bank-specific scenario was €1,182,291, compared to the human expert estimate of €106,236.

Original implementation tip: The AI estimates in this case were dramatically higher than human expert estimates, with the AI aggregate more than 10x the human aggregate. This divergence itself is valuable information. It suggests either that LLMs are poorly calibrated for regulatory fine estimation in specific jurisdictions (likely, given their training data includes global cases that may skew distributions upward), or that human experts are underestimating tail risk and the LLMs are capturing something the humans miss, or that the LLMs are anchoring to the maximum possible fine under GDPR (4% of global turnover or €20 million) rather than to actual enforcement patterns in Spain. Don’t automatically prefer the human estimate or the AI estimate. Investigate the divergence. In this case, the historical data strongly supports the human estimate range: the actual 90th percentile of Spanish GDPR fines is €200,000, making an aggregate estimate above €1 million an outlier relative to enforcement history. The AI models appear to be poorly calibrated for jurisdiction-specific fine estimation. Document this finding and adjust your methodology accordingly.

When to Use AI Estimators

AI estimation is most valuable when you need rapid preliminary estimates across many scenarios before investing in human expert time, when you want to identify the range of plausible outcomes to inform your calibration question design, when you’re looking for scenarios or factors that your human experts might not have considered, and when you want to stress-test human estimates by comparing them to an independent source.

AI estimation is least reliable when jurisdiction-specific enforcement patterns differ significantly from global averages (as in the Spain case), when the scenario involves novel regulatory frameworks with limited enforcement history, when contextual factors (organizational size, cooperation level, remediation speed) heavily influence outcomes, and when you need defensible estimates for regulatory or board reporting.

Original implementation tip: Use AI estimates as one input to your calibration process, not as a replacement for it. Include LLM estimates alongside human expert estimates in your aggregation. Apply the same Cooke method to assign weights based on calibration accuracy. In the Spain GDPR case, the AI estimates would receive low aggregate weight because their calibration accuracy was poor relative to the human experts. In a domain where LLMs demonstrate better calibration, perhaps because there’s more training data or less jurisdiction-specific variation, they might receive higher weight. Let the calibration data determine the weighting, not your assumptions about whether humans or machines are “better.” The Cooke method doesn’t care whether the estimator is human or artificial. It cares whether the estimator is accurate.

Building a Repeatable Calibration Program

Institutional Calibration Infrastructure

Individual calibration sessions are valuable. A sustained calibration program is transformative.

How to implement:

Maintain a calibration database that records every expert’s estimates, every calibration question and correct answer, every weight assignment, every aggregated result, and every actual outcome when it materializes.

Track each expert’s calibration score over time. Identify experts who are improving (the feedback loop is working) and those who aren’t (they may need additional training or should receive lower weights).

Build a library of calibration questions organized by risk domain: regulatory fines, cybersecurity incidents, operational losses, project overruns, market events. As you accumulate questions with known answers, your calibration testing becomes more robust and differentiated.

Schedule calibration sessions quarterly for your most critical risk domains. Use shorter calibration exercises (three to five questions) as part of regular risk committee meetings to keep estimation skills sharp.

Original implementation tip: Measure and report your organization’s aggregate calibration improvement over time. If you’re running quarterly sessions with calibration feedback, your expert pool’s average accuracy should improve measurably within 12 months. Track two metrics. First, the average Brier score across all experts and all questions, which should decrease over time (lower is more accurate). Second, the percentage of experts whose 90% confidence intervals actually contain the true outcome 90% of the time, which should approach 90% from below as calibration training takes effect. Present these metrics to the risk committee as evidence that your risk assessment process is improving in measurable, auditable terms. This is how you move from “we think our risk estimates are reasonable” to “we can demonstrate that our estimation accuracy has improved by X% over the past four quarters.” The second statement is what boards and regulators want to hear.

Brier Scores for Ongoing Accuracy Tracking

A Brier score measures the accuracy of probabilistic predictions. It ranges from 0 (perfect accuracy) to 1 (complete inaccuracy). For each prediction, the Brier score is calculated as the squared difference between the predicted probability and the actual outcome (1 if the event occurred, 0 if it didn’t).

How to implement:

For each expert’s probability estimate, record the predicted probability and the actual outcome. Calculate the Brier score for each prediction. Average Brier scores across multiple predictions to get each expert’s overall accuracy metric.

Use Brier scores as an alternative or supplement to the simple correct/incorrect scoring used in the Cooke method. Brier scores capture nuance that binary scoring misses: an expert who assigns 80% probability to an event that occurs is more accurate than one who assigns 51%, even though both would be scored as “correct” under binary scoring.

Original implementation tip: Report Brier scores to experts individually and confidentially. Show them how their score compares to the group average without identifying other experts. Competitive benchmarking against an anonymous group average motivates improvement more effectively than abstract accuracy metrics. Frame it as a professional development tool: “Your Brier score this quarter was 0.21 versus the group average of 0.18. Here are the questions where your estimates diverged most from outcomes.” This is the same feedback mechanism that the Good Judgment Project used to develop superforecasters. It works because it provides specific, measurable, actionable feedback tied to actual outcomes, which is exactly what most professional development programs lack.

Common Implementation Failures and How to Avoid Them

Failure: Skipping calibration and going straight to estimation. Without calibration questions, you have no basis for weighting experts. Every expert gets equal weight, which means poorly calibrated experts have as much influence as accurate ones. Always include calibration questions, even if you only have three.

Failure: Using the same experts for every assessment. Expert fatigue reduces accuracy over time. Rotate experts across sessions. Bring in fresh perspectives. Maintain a pool of qualified experts for each domain rather than relying on the same three people for every risk assessment.

Failure: Not providing feedback. Calibration without feedback is just measurement. Feedback is what drives improvement. Share calibration results with experts after every session. Show them where they were accurate and where they weren’t. Discuss techniques for improving (widening confidence intervals, adjusting for known biases, considering base rates before estimating).

Failure: Treating the aggregated estimate as a point value. The Cooke method produces a weighted point estimate, but the underlying expert distributions contain information about uncertainty. Report the aggregated estimate as a distribution (using the three-point estimates from each expert, weighted by calibration scores) rather than as a single number. A single number implies false precision.

Failure: Allowing political override of calibration weights. When a senior executive receives zero weight because their calibration accuracy was poor, organizational pressure to “adjust” the weights is inevitable. Resist this. Document the calibration methodology before the session and commit to applying it without modification. If you allow political overrides, you’ve destroyed the method’s value and you’re back to hierarchy-driven estimation with extra steps.

Original implementation tip: Build the calibration methodology into a formal procedure document that your risk committee approves before the first session. The document should specify how calibration questions are selected, how scoring works, how weights are calculated, and that weights are applied mathematically without subjective adjustment. Get this approval once. Then reference it every time someone challenges the weights. The pre-approved procedure document prevents ad hoc political interventions because overriding the weights now requires overriding a committee-approved methodology, which creates its own accountability.

Integrating Calibrated Estimates Into Your Risk Framework

Connecting to Enterprise Risk Management

Calibrated expert estimates should feed directly into your quantitative risk assessment process, not sit in a separate workstream.

How to implement:

Use the three-point estimates from calibrated experts to parameterize loss distributions in your risk models. The weighted minimum, most likely, and maximum values define a PERT or triangular distribution that can be input to Monte Carlo simulations.

Report calibrated estimates alongside their uncertainty ranges. The board shouldn’t see “€106,236.” They should see “€106,236 weighted mean estimate from calibrated experts, with a 90% range of €40,000 to €300,000 based on the distribution of individual estimates.”

Track the accuracy of your calibrated estimates against actual outcomes and report the tracking results to the risk committee. This creates a continuous improvement loop that raises confidence in your risk assessment process over time.

Original implementation tip: When presenting calibrated estimates to the board, lead with the methodology’s credibility, not just the number. Explain that the estimate comes from X experts whose accuracy was tested against Y calibration questions with known answers, that experts were weighted by demonstrated accuracy, and that the method is based on the Cooke Classical Model used by regulators and international agencies for structured expert judgment. This framing differentiates your estimate from the typical “we asked some people and averaged their guesses” approach. Boards increasingly expect quantitative rigor in risk assessment. Calibrated expert judgment, properly documented, meets that expectation. Uncalibrated workshop consensus does not.

Key References

Expert Calibration Methods:

Cooke, R.M. (1991). “Experts in Uncertainty: Opinion and Subjective Probability in Science.” Oxford University Press.
Tetlock, P.E. (2015). “Superforecasting: The Art and Science of Prediction.” Crown Publishers.
Kahneman, D. (2011). “Thinking, Fast and Slow.” Farrar, Straus and Giroux.

Structured Expert Judgment:

OECD/NRC (2018). “Expert Judgement in Risk and Decision Analysis.” (Guidance on the Cooke Classical Model)
European Food Safety Authority (EFSA) guidance on expert knowledge elicitation (2014, updated 2019)

Scoring and Accuracy:

Brier, G.W. (1950). “Verification of Forecasts Expressed in Terms of Probability.” Monthly Weather Review.
Good Judgment Project documentation (goodjudgment.com)

AI Risk Estimation:

NIST AI RMF 1.0 (2023), Measure function
ISO/IEC 23894:2023 (AI Risk Management)

Regulatory Data:

AEPD (Agencia Española de Protección de Datos) enforcement decisions database
GDPR Enforcement Tracker (enforcementtracker.com) for cross-jurisdictional fine data
EU AI Act, Regulation (EU) 2024/1689, Article 99 (penalties)

The organizations that treat expert judgment as data, measure its accuracy, and improve it over time will consistently produce better risk estimates than those relying on unstructured workshops and colorful matrices.

The math isn’t complex. The discipline is. Calibration requires admitting that credentials don’t guarantee accuracy, that feedback is essential for improvement, and that mathematical aggregation produces more defensible results than consensus driven by hierarchy.

The choice between calibrated estimation and uncalibrated guessing is the choice between a risk function that can demonstrate its value quantitatively and one that relies on institutional trust to justify its existence. In an environment where regulators, auditors, and boards increasingly demand evidence, only one of those approaches survives scrutiny.

Quantitative Risk Assessment Using Monte Carlo Simulations and Convolution Methods in R

Thu, 12 Mar 2026 00:00:00 +0000

Why Probabilistic Risk Modeling Matters for GRC Professionals

Picture a risk committee meeting. Someone points at a heat map and says, “Vendor concentration risk is High.” Twenty minutes of discussion follow. Nobody asks the question that actually matters: how much money are we talking about, and how much should we set aside for it? Nobody can answer it, because a color on a grid was never built to answer it.

That’s the quiet failure at the center of most enterprise risk programs. A 3x3 or 5x5 matrix takes a likelihood rating and an impact rating, both invented on the spot, multiplies them together, and calls the result a risk score. The math doesn’t hold up. Ordinal numbers, “3” for likely, “4” for severe, aren’t real quantities. You can’t multiply them any more than you can multiply two zip codes and get a meaningful address. Risk researchers have been pointing this out for close to two decades, and the finding holds up every time someone tests it: matrices routinely rank smaller risks above bigger ones, compress genuinely different exposures into the same box, and give false confidence to numbers nobody can defend in front of a CFO.

There’s a way out, and it doesn’t require a data science degree or a six-figure software license. Monte Carlo simulation lets you describe uncertainty as a probability distribution instead of a guess, run that distribution through tens of thousands of possible futures, and read off a statistically grounded answer. Pair it with convolution, a technique that combines how often something happens with how bad it is when it does, and you get a full loss curve instead of a single number. That curve is what finance teams actually need for reserve setting, capital allocation, and insurance decisions, because it speaks their language: probability and dollars, not colors and adjectives.

The barrier used to be cost and complexity. Enterprise risk simulation platforms carry real license fees, and statistical programming isn’t a skill most GRC professionals picked up in their compliance training. That barrier is mostly gone. An runs Monte Carlo simulation with convolution in a matter of seconds for 100,000 scenarios, is free to use, and runs in a browser through Google Colab with no local installation at all.

This guide walks through how the method works, how to set it up, how to choose the right distributions, and how to turn the output into something a board will actually act on.

Key takeaway: A risk matrix gives you a color. Monte Carlo simulation with convolution gives you a probability-weighted range of dollar outcomes you can reserve against, defend to an auditor, and use to price the ROI of a new control. It runs for free, in seconds, in your browser.

The Two Building Blocks: Monte Carlo Simulation and Convolution

What Monte Carlo Simulation Actually Does

Monte Carlo simulation generates thousands of random scenarios drawn from probability distributions you define for each risk variable. Instead of handing you one “expected loss” figure, it hands you a full population of possible outcomes, showing you the range, the shape, and how likely each level of loss actually is.

In practice, you need two inputs for any risk you’re modeling:

Frequency: how many times the event is likely to happen in a given period. This is a discrete quantity (you can’t have 2.3 breaches), so it’s typically modeled with a Poisson distribution.
Severity: how much each event costs when it happens. This is a continuous quantity, and for most operational losses it’s modeled with a lognormal distribution, because losses tend to be right-skewed: plenty of small ones, a handful of very large ones.

The simulation then runs thousands of iterations. In each one, it draws a random number of events from the frequency distribution and a random loss amount from the severity distribution, then combines the two. Do that 100,000 times and you have a dataset of possible total losses you can analyze statistically instead of a single guess you have to defend on faith.

Speed is not a real obstacle here. Ten thousand iterations complete in about half a second, plenty for an exploratory pass or a workshop where you’re testing assumptions live. A hundred thousand, the standard for most assessments, finishes in a few seconds. A million, reserved for regulatory capital calculations or board-level reserve recommendations where precision earns its keep, takes well under a minute. The accuracy gain from a hundred thousand to a million runs is marginal for everyday work, so there’s no reason to sit through a longer run every time you want to test an assumption during a live session.

If you want the full quantitative framework behind everything described above, including the complete distribution taxonomy, the open-source Python Monte Carlo engine, and domain-specific applications across AI risk, cyber exposure, compliance debt, and financial risk, The Risk Management Blueprint by me, Hernan Huwyler, builds it chapter by chapter for practitioners who are ready to move past the color grid for good.

The book covers 26 chapters under one unified probabilistic methodology, with over 70 percent of its pages dedicated to applied quantitative methods rather than governance theory. You can start with the first four chapters for free and decide whether the rest is worth your time before spending a dollar. Preview the first four chapters of The Risk Management Blueprint here: , or get the full book directly on Amazon at

The Risk Management Blueprint for Quantitative and Predictive Models by Hernan Huwyler

Why Convolution Beats Simple Multiplication

The naive approach to quantifying risk is to take an expected frequency, multiply it by an expected severity, and call that the risk exposure. Four expected events times a $20,000 average loss gives you $80,000. That number is not wrong, exactly. It’s just almost useless, because it’s a single point with no sense of how much that number could vary, and variation is precisely what a reserve or a capital buffer exists to cover.

Convolution is the mathematical operation that properly combines two full probability distributions instead of two single numbers. It preserves the shape of both the frequency distribution and the severity distribution, so the output isn’t a point estimate, it’s an entire curve. Two risks with the identical expected loss can have very different tail behavior: one might cluster tightly around its average, the other might have a long, thin tail of rare catastrophic outcomes. Simple multiplication treats them as identical. Convolution tells them apart, which is exactly the distinction that matters when you’re deciding how much capital to hold against each one.

There’s a nuance worth flagging here, because it trips people up the first time they run this. If your organization has been using deterministic “worst case” scenario planning, where someone picks a single pessimistic number and treats it as the ceiling, convolution’s output at high percentiles will usually come in lower than that old worst case, because a true worst case assumes the bad outcome happens with certainty, which is almost never realistic. But if your baseline has been simple expected-value multiplication, convolution’s tail percentiles will come in noticeably higher than that single center-of-mass number, because a plain average was never designed to show you the tail in the first place; it can’t, since it’s just one number. Neither of these is a contradiction, and neither is an error in the new model. It’s the difference between measuring the middle of a distribution and measuring the whole thing. When you make this switch, document it, and tell your stakeholders plainly: the earlier numbers weren’t wrong, they were incomplete, and the shift is a gain in precision, not a change in your risk appetite.

Getting Set Up: Two Ways to Run This Today

Google Colab: Zero Installation, Zero IT Ticket

Google Colaboratory gives you a cloud-based notebook that runs R without touching your local machine, which quietly solves the single biggest adoption barrier in most companies: getting IT approval to install anything. Go to , start a new notebook, switch the runtime to R, paste in the script, and run it cell by cell. You need a Google account and an internet connection. That’s the entire prerequisite list.

One practical wrinkle: Colab sessions time out after inactivity and don’t save your data between sessions, so get in the habit of saving your customized script to Google Drive or downloading it locally when you’re done for the day. If you’re running assessments regularly, it’s worth building one template notebook per risk domain, operational, compliance, cyber, with your organization’s typical distribution types and parameter ranges already filled in. Customizing a pre-built template for a new assessment takes about five minutes. Building one from a blank notebook takes closer to half an hour. That difference compounds fast once you’re running quarterly assessments across a dozen risk categories.

The full walkthrough, with every code block laid out step by step, is published on , and the source scripts live in his , including the convolution model under PythonMinMaxConvMCS and a compliance-specific variant under PythonTComplianceImpacts.

RStudio: For Teams That Want This in Their Workflow

For regular use integrated into an organization’s existing tooling, install R locally: download R 4.3.2 or later from , install RStudio as your development environment, and add the handful of required libraries. R runs cleanly on Windows, macOS, and Linux, and every piece of it, base install and libraries alike, is free and open source.

If your organization pushes back on installing new software, the cost comparison makes the case for you. Commercial risk simulation platforms with this kind of capability typically run into five figures per user, per year, in enterprise licensing. This script produces statistically equivalent output, mean, median, percentiles, loss exceedance curves, for the specific job of Monte Carlo simulation with convolution, at zero license cost. It won’t give a non-technical user a polished GUI, and it doesn’t carry the full feature set of a commercial platform. But for the core task, quantifying a loss distribution and setting a defensible reserve, it gets you there. Bring that comparison, along with a quick note on R’s open-source licensing, to your procurement conversation.

Configuring the Model: Five Inputs That Do All the Work

The entire model runs on five parameters, and every one of them should trace back to historical loss data or a properly calibrated expert estimate. None of them should be a number someone typed in because it “seemed about right.”

Simulations — how many scenarios to run. Start at 100,000 for a standard assessment.
Events — the expected number of loss events per year, feeding the Poisson distribution. Pull this from your incident log, near-miss records, or a structured expert elicitation if you have no internal data yet.
Loss — the expected average financial loss per event, feeding the lognormal distribution.
Mean (Standard Deviation) — the spread of losses around that average, expressed as a proportion. A value of 0.2 means losses typically vary by about 20% around the mean; push it to 0.4 and you’re describing a much wider, heavier-tailed world.
Reserve — the percentile at which you want your reserve set. 0.8 covers 80% of simulated scenarios; 0.95 covers 95%. Your organization’s risk appetite statement should be the thing that sets this number, not a habit.

Simulations <- 100000
Events <- 4
Loss <- 20000
Mean <- 0.2
Reserve <- 0.8
set.seed(123)

set.seed(123) is a small line that does a lot of quiet work. It forces the random number generator to produce the same sequence every time, which means anyone re-running your script with the same seed gets identical results. That’s not a nice-to-have. It’s what makes the output defensible in an audit trail and reproducible in a peer review, two things a color-coded matrix never had to worry about.

The standard deviation parameter deserves more attention than it usually gets, because it has an outsized effect on the tail. Moving it from 0.2 to 0.4 doesn’t just widen the distribution modestly, it materially increases both the probability and the size of the worst outcomes. Before you commit to a final number, run the model five times with standard deviation values of 0.1, 0.2, 0.3, 0.4, and 0.5, holding everything else fixed, and plot the 95th percentile loss from each run. That sensitivity check takes about five minutes and tells you exactly how much your reserve calculation is riding on an assumption you may not be fully sure of. It’s remarkable how often a risk team locks in a round-number standard deviation without ever checking what happens to the output if that number is off by even 10%.

Choosing the Right Distributions

Getting the shape right matters as much as getting the numbers right. A model built on the wrong distribution will produce confident, precise-looking output that’s quietly wrong.

Frequency: The Poisson Distribution

The Poisson distribution models how many times an event occurs in a fixed period, assuming events happen independently and at a roughly constant average rate. It’s a solid default for most operational event counts: fraud incidents per year, breaches per quarter, compliance violations per period.

It works well when you have a reasonable estimate of the average rate, events don’t cluster or trigger one another, and the chance of an event in any small window is roughly steady. It stops working well when events cluster (one breach raising the odds of the next), when the rate is visibly trending up or down over time, or when the average frequency climbs above roughly 30 events per period, at which point a normal distribution often fits better.

Pull the Events parameter from at least three years of incident history if you have it. A single year can be an outlier in either direction. If you logged 2 events last year, 6 the year before, and 3 the year before that, your average is roughly 3.7, and that’s the number to use, not last year’s count in isolation. When an auditor eventually asks why you assumed 4 events a year, you want a documented, evidence-based answer on hand, not “it felt reasonable.”

Severity: The Lognormal Distribution

The lognormal distribution models positive-only values with a long right tail: most losses land in a moderate range, but a few run far larger. That pattern shows up consistently across operational, compliance, and cybersecurity losses, which is why lognormal is the default choice for financial impacts, fines, and remediation costs.

Impact <- rlnorm(n = Simulations, meanlog = log(Loss), sdlog = Mean)

meanlog = log(Loss) converts your dollar figure onto the log scale the distribution requires, and sdlog = Mean controls how wide that distribution spreads.

Before you trust the choice, check it against your actual data. Plot your historical losses as a histogram. If it’s right-skewed with a long tail, lognormal fits. If your losses cluster around two clearly separate values, say, small procedural fines in one cluster and rare, large enforcement actions in another, a single lognormal curve will flatten that pattern into something that isn’t really there. In that case, build a mixture of two lognormal distributions, one per cluster, weighted by how often each type occurs. It’s a small code change, a handful of lines, and it materially improves the fit for any risk with a genuinely bimodal loss pattern.

Beyond Poisson and Lognormal

The two defaults cover most operational risk work, but they’re not the only tools available, and swapping them in only takes changing one function call:

rnorm() for a normal distribution, when losses are genuinely symmetric around the average rather than skewed.
rgamma() for a gamma distribution, when you want more flexible control over skewness than lognormal offers.
rweibull() for a Weibull distribution, standard in reliability engineering for time-to-failure and equipment breakdown risk.
runif() for a uniform distribution, when all you genuinely know is a floor and a ceiling with nothing in between.
rbinom() for a binomial distribution, when you’re modeling a fixed number of independent trials, each with the same probability of a “bad” outcome (for example, the odds that any one of 40 vendors has a material failure this year).
rnbinom() for a negative binomial distribution, when your frequency data is more erratic than Poisson assumes, some periods clustering with several events, others with none, a pattern statisticians call overdispersion.

Don’t pick a distribution because it’s the one you remember from a textbook. Pick it because it fits your data, and prove that fit rather than assert it. R’s fitdistrplus library exists for exactly this: run fitdist(your_data, "lnorm") and fitdist(your_data, "gamma") side by side and compare their AIC (Akaike Information Criterion) scores, where a lower AIC signals a better-fitting model relative to its complexity. Write down the fit statistics along with your choice. “We selected lognormal based on goodness-of-fit testing against three years of loss history” is a sentence that survives a board meeting or a regulatory exam. “We used lognormal because that’s what people usually use for operational risk” is not.

Inside the Convolution Engine

Here’s what’s actually happening under the hood once you hit run. For each of your 100,000 iterations, the script draws one random event count from the Poisson distribution and one random loss amount from the lognormal distribution, then convolves them, mathematically combining the two so the interaction between “how many” and “how much” is preserved rather than flattened into an average.

combined_distribution <- lapply(1:Simulations, function(i) {
 conv <- numeric(length(Prob[i]) + length(Impact[i]) - 1)
 for (j in seq_along(Prob[i])) {
 for (k in seq_along(Impact[i])) {
 conv[j + k - 1] <- conv[j + k - 1] + Prob[i] * Impact[i]
 }
 }
 conv
})
x <- sapply(1:Simulations, function(i) sum(combined_distribution[[i]]))

The output, x, is a vector of 100,000 total-loss values, one per simulated scenario. That vector is your aggregate loss distribution, and it’s the raw material for every statistic and chart that follows.

Run the naive calculation alongside it and the difference becomes concrete fast. Simple multiplication of Events × Loss gives 4 × $20,000 = $80,000. In a representative run of the model, the simulated mean lands close to that, around $81,599, which is reassuring; the center of the distribution roughly agrees with the naive estimate. But the 80th percentile comes in at $115,867, about 44% above the mean, and the 95th percentile sits higher still. The simple multiplication gave you the middle of the story. The simulation gives you the whole thing, tails included, and the tails are where the actual risk decisions live. When you present results, show the full distribution, not just the average. The mean tells a committee that everything looks manageable. The 95th percentile tells them what happens on a bad year. Both matter, and leaving either one out of the room is a mistake.

Reading the Output Like a Risk Committee, Not a Statistician

summary(x) hands you the core statistics. Using the illustrative example above, four expected events, a $20,000 average loss, and a 20% standard deviation, a representative run produces something like this:

Statistic	Value
Minimum	$0 (scenarios with zero events)
25th Percentile	$49,383
Median	$75,715
Mean	$81,599
75th Percentile	$107,206
80th Percentile (Reserve)	$115,867
Maximum	$408,113

Here’s how each of those numbers translates into something a business decision can be built on:

The median is the most typical single outcome, half of all simulated scenarios land below it. The mean sitting above the median confirms the right skew: a handful of high-loss scenarios are pulling the average up above what actually happens most often, which is the standard signature of operational risk data. The interquartile range, roughly $49,000 to $107,000 here, is your “normal range,” the band your baseline planning should comfortably absorb. The reserve figure, set at your chosen percentile, tells you what you’d need to set aside to cover that share of possible outcomes, and by definition leaves the remaining share uncovered; at the 80th percentile, that’s a 20% chance actual losses exceed what you’ve reserved. The maximum is your single worst simulated draw, low-probability but not zero, and it’s the number that should be informing your insurance conversations and catastrophic-loss planning even though you’ll never hold a full reserve against it.

When you report the reserve number, always attach the coverage probability out loud. Don’t say “the reserve should be $115,867." Say: "A reserve of $115,867 covers 80% of simulated scenarios. There’s a 20% chance actual losses exceed that. Covering 95% would require $X instead.” Then let the committee choose the coverage level they’re comfortable holding capital against. Building a standing reserve table, dollar figures at the 50th, 75th, 80th, 90th, and 95th percentiles, turns this into a menu with clear risk-reward tradeoffs instead of a single number handed down from the model. Setting the reserve is a business decision. The model’s job is to lay out the honest options; leadership’s job is to pick one and own the tradeoff.

Turning Numbers Into Pictures

The Histogram

hist(x, main = "Histogram of Expected Losses", xlab = "Total Loss", ylab = "Frequency")

A histogram shows the shape of your simulated outcomes at a glance: the most common loss range, the right-tail skew stretching toward extreme values, and the overall spread. This is the single most effective way to make the point that risk isn’t a number, it’s a distribution, to an audience that’s used to thinking in single figures.

For a board deck rather than a technical committee, dress it up a little. Mark the mean and the reserve line explicitly, and color the tail beyond the reserve so the uncovered scenarios are visually obvious rather than buried in the data.

hist(x, main = "Distribution of Potential Losses", xlab = "Total Loss ($)", col = "lightblue")
abline(v = quantile(x, 0.8), col = "red", lwd = 2)
abline(v = mean(x), col = "blue", lwd = 2)

The red line marks your reserve level. The blue line marks the mean. Everything to the right of the red line is the 20% of scenarios your current reserve doesn’t cover. One chart like this communicates more about real exposure than a thirty-page qualitative risk report, because it makes the gap visible instead of describing it in adjectives.

The Loss Exceedance Curve

number_sequence <- seq(0.01, 1, by = 0.001)
y <- sapply(number_sequence, function(i) quantile(x, probs = i))
plot(number_sequence, y, type = "l", xlab = "Percentile", ylab = "Loss", main = "Loss Exceedance Curve")

A loss exceedance curve plots the probability of exceeding a given loss threshold across the full distribution, showing exactly how coverage level and required reserve trade off against each other. It’s the standard tool for insurance analysis, reserve calibration, and comparing risk tolerance across different scenarios on the same chart.

This is also where you can put a real dollar figure on the value of a control. Run the model twice, once with your current parameters, once with the parameters you’d expect after implementing a proposed control, reduced event frequency, reduced average severity, or both, and overlay the two curves. The gap between them at any percentile is the financial value of that control. That’s the calculation behind a sentence like: “Implementing this control shifts the 95th percentile loss from $X to $Y, a $Z reduction in potential exposure. The control costs $W. Net return: $Z minus $W.” No qualitative matrix produces that sentence. A pair of loss exceedance curves does, directly.

Where This Gets Used: Four Domains, Four Playbooks

Financial Risk

Model potential losses from market moves, credit defaults, or liquidity events by setting Events to the expected count of adverse events per period and Loss to the average financial impact per event. For credit risk specifically, pull historical default rates and loss-given-default figures to parameterize the model, run it separately by risk grade across your portfolio, and aggregate the results into a portfolio-level credit loss estimate. Compare that against your current loan loss provisions. If your simulated 90th percentile meaningfully exceeds what you’re currently holding, you now have a quantitative, defensible basis for recommending an increase, not just a hunch.

Compliance and Regulatory Risk

Estimate potential fines, remediation costs, and enforcement expenses by building a database of enforcement actions in your jurisdiction and industry for the specific regulation in question. Most regulators publish this data. Use it to set your Events parameter (how many enforcement actions per year hit organizations comparable to yours) and your Loss parameter (the average fine size), with the standard deviation pulled from the spread in that same dataset. A compiled set of GDPR enforcement actions against Spanish organizations, for instance, shows an average fine in the tens of thousands of euros but a standard deviation several times larger than the mean, evidence of just how lopsided regulatory penalties actually are, with a handful of large fines pulling the whole distribution far past what a “typical” fine would suggest. That kind of variability is precisely why lognormal, not a flat average, is the right shape here. Present the output to a compliance committee as: “Based on historical enforcement patterns, there’s an X% chance a fine exceeding €Y gets imposed. Recommended reserve at the 90th percentile: €Z.”

Cybersecurity Risk

Set Events to the expected number of breaches, ransomware incidents, or data loss events per year, and Loss to the average all-in cost per incident, response, remediation, notification, legal fees, and business interruption combined. Widely cited industry breach-cost research (annual reports from major cybersecurity and insurance research groups) gives you a reasonable starting point when internal data is thin, but treat those benchmarks as a starting shape, not a final answer. Adjust them for your organization’s size, data volume, regulatory footprint, and incident response maturity; a global bank’s breach profile and a regional retailer’s are not the same distribution wearing different labels. Let external data inform the shape of the curve and your own incident history calibrate its scale.

Operational and Project Risk

Apply the same model to equipment failure, supply chain disruption, process breakdowns, or project overruns wherever you can estimate a frequency and a severity. For project risk specifically, it often makes more sense to break the single Loss parameter into separate models for cost overrun, schedule delay, and quality failure, run each one, and combine the output vectors with c() into a single project-level aggregate. That gives you a picture that respects how differently those three failure modes actually behave instead of flattening them into one generic “project risk” number.

Back-Testing: Proving the Model Isn’t Just Precise-Looking Fiction

A model is only worth trusting once it’s been checked against reality. Back-testing means comparing what the model predicted against what actually happened, and using the gap to recalibrate.

After each assessment period, quarterly or annually, record the actual total loss and find where it lands in your simulated distribution. If actual outcomes keep showing up in the extreme tails, above the 95th percentile or below the 5th, the model is miscalibrated somewhere upstream. Track this over time: for a well-calibrated model, roughly 50% of actual outcomes should fall inside the interquartile range, about 90% inside the 90th percentile band, and about 95% inside the 95th. Those aren’t arbitrary benchmarks; they’re just what “calibrated” means by definition, so persistent deviation from them is your signal to go back and adjust.

Keep a running back-testing log: date, risk assessed, the parameters used (Events, Loss, standard deviation), the predicted statistics, and the actual outcome once it materializes. After eight to twelve periods of data, you can calculate real calibration metrics. If actual losses keep exceeding your 80th percentile prediction, you’re underestimating risk and need to raise your input parameters. If actuals keep landing below the 25th percentile, you’re over-reserving. Bringing back-tested accuracy to a risk committee earns a kind of credibility a brand-new, unproven model simply can’t claim yet, and it’s the same core validation logic that supervisory guidance on model risk management has long required of financial models, applied here to operational and compliance risk instead of credit models.

From Model to Boardroom: Reserves, Scenarios, and Control ROI

Reserve Setting and Capital Allocation

Build a reserve table for each material risk showing the dollar figure at the 50th, 75th, 80th, 90th, and 95th percentiles, and bring it to the risk committee with a recommended confidence level tied to your organization’s stated risk appetite, regulatory obligations, and capital position.

Connect that table directly to the risk appetite statement rather than treating them as separate documents. If the statement says reserves should cover 90% of potential scenarios, the model’s 90th percentile output is your target reserve, full stop. If your current reserve sits below that, you’ve just converted a vague concern into a specific funding gap: “Our stated appetite requires reserves covering 90% of scenarios, which this model puts at $X. Current reserve is $Y. The gap is $X minus $Y.” That’s a very different conversation from “we probably need more reserves,” and it’s the version that actually gets funded, because it names a number instead of a feeling.

For portfolio-level aggregation across several material risks, resist the temptation to just add the individual reserves together. Simple addition assumes every risk hits its worst case simultaneously, which overstates the true combined exposure. Either run a joint simulation that accounts for correlation between the risks, or apply a documented diversification factor to the summed total, and explain your reasoning for whichever approach you pick.

Scenario Analysis and the Financial Case for Controls

Run the baseline model with today’s parameters, then change one input at a time and compare the outputs. What happens to the 80th percentile if event frequency doubles? If average severity rises 50%? If a proposed control cuts frequency from 4 events a year to 2? Document each variant side by side against the baseline so the comparison is visible at a glance, not buried in separate reports.

This is the mechanism behind quantifying a control’s value in dollars rather than adjectives. Run the model once with current parameters and once with the parameters you’d expect post-control, then look at how much the reserve requirement shrinks at your chosen percentile. That shrinkage is the control’s financial value. Set it against the control’s cost and you get a return figure: a $50,000-a-year control that cuts the 90th percentile reserve requirement by $200,000 delivers a 4x return. That reframes the pitch from “we should do this because it reduces risk,” which is easy to defer, to “this delivers a 4x return on investment in reduced reserve requirements,” which tends to get approved.

Six Ways Quantitative Models Go Wrong

Even a well-built simulation fails if you fall into one of these habits:

Using assumed parameters instead of data. The model produces confident-looking output regardless of whether the inputs are grounded in evidence or invented on the spot. A simulation built on made-up numbers is just computational fiction with better production values. Document the source and evidence behind every input.
Ignoring whether the distribution actually fits. Defaulting to lognormal without checking it against your real loss history bakes in a systematic bias. Test the fit whenever you have the data to do it.
Reporting only the mean. The mean is the least useful number in the whole output for risk decisions. The tails are where decisions actually get made. Always pair the mean with percentile-based statistics.
Running it once and filing the report. Risk profiles shift as the business, its controls, and the threat landscape all evolve. Re-run the model quarterly with updated parameters and track how the results move over time.
Skipping set.seed(). Without a fixed seed, every run of the model produces slightly different numbers, which makes runs impossible to compare cleanly and creates an audit trail headache nobody needs. Set it, and record it.
Treating the output as a prophecy. The model’s output is only as good as its inputs and assumptions. Present it as “given these assumptions, the model estimates,” not “the loss will be $X.” Uncertainty in, uncertainty out, and a sensitivity analysis is how you show your audience exactly how much of that uncertainty is riding on which assumption.

One habit worth adding on top of all six: build a documentation template once and reuse it for every assessment, the risk assessed, data sources for each parameter, the distribution chosen and why, the simulation count, the seed, the software and version, the date, the author, the statistics, the sensitivity results, and the back-testing history. Treat it as a model card for your risk simulations. When an auditor asks how you got to a number, you hand them the template instead of reconstructing your reasoning from memory under pressure.

Beyond R: Python, and Where AI Actually Fits

A refactored Python version of the same methodology lives alongside the R code in the , including a full convolution build under PythonMinMaxConvMCS. If your data science team already works in Python, or you want to plug this into an existing machine learning pipeline or a web application, start there instead of forcing an R detour just to match the original methodology. The underlying math is identical regardless of language, and a tool your team already knows and will actually keep using beats a theoretically superior one that quietly falls out of use. If your team already lives in R for statistical work, there’s no reason to switch.

Layering AI and machine learning on top of this foundation is a real and growing extension, not a replacement for it. Predictive models can forecast frequency parameters from leading indicators before they show up in a loss log. Natural language processing can pull structured loss data out of unstructured incident reports to feed the severity distribution automatically. Reinforcement learning can help optimize which combination of controls to fund given a simulated loss curve. But sequence matters here. Prove the basic Monte Carlo model’s value first, produce reserve recommendations, back-test them, show they hold up, and only then layer AI capability on top. Organizations that skip straight to AI-driven risk prediction without ever validating a basic quantitative foundation end up with sophisticated-looking output built on assumptions nobody has tested. The simulation is the foundation. AI is refinement on top of it, not a substitute for it.

For a walkthrough of the same convolution logic built out in Python with a step-by-step presentation format, the covers the same operational, compliance, and cyber use cases with the Python implementation front and center.

Making the Switch: Getting Your Organization Off Red-Yellow-Green

Moving an organization from matrices to probability distributions is a change management project as much as a technical one, and it goes better in stages than as a mandate.

Start with one risk domain where your historical loss data is strongest, financial risk and cybersecurity usually have the most complete records. Run the model there, produce results, and set them side by side with the previous qualitative assessment. Let the gap speak for itself, especially in the tails and in reserve figures, rather than arguing the case in the abstract.

Don’t rip out every matrix at once. Run the quantitative model in parallel with the existing qualitative process for two or three assessment cycles and let stakeholders watch both outputs land against real outcomes. The case for the quantitative approach tends to make itself once actual losses fall neatly inside the simulated range while sitting outside whatever the old matrix predicted.

Invest in training. A two-day program covering basic R or Python, probability distributions, and how to interpret statistical output is generally enough to get a risk analyst running and customizing this model on their own. That’s a modest investment that pays off across every risk domain you touch afterward, not just the first one.

The resistance you’ll hit is rarely about technical difficulty. It’s about the loss of subjective control. A matrix lets a senior risk officer set the rating wherever judgment points. A quantitative model lets the data drive the output, with judgment applied only to the documented, testable inputs. Some people experience that as a loss of influence. It’s worth reframing out loud: this is an upgrade in credibility, not a demotion. The risk professional’s role shifts from rating things subjectively to choosing the right distribution, interpreting the output, designing the scenarios, and translating the numbers into a business decision, work that commands more respect from finance and the executive table than a colored square ever did. A CFO who has never once acted on a red-yellow-green matrix will engage immediately with a probability-weighted loss curve, because it’s the same language they already use for every other financial decision they make.

This shift also happens to be exactly what frameworks like ISO 31000 and COSO ERM have been asking for all along, quantified risk analysis tied to real decisions, rather than an ordinal scoring exercise that satisfies an audit checkbox and stops there. The method described here doesn’t compete with those frameworks. It’s how you actually execute the “risk analysis” step they’ve always called for, instead of substituting a color for it.

Frequently Asked Questions

What is convolution in risk management? Convolution is the mathematical operation that combines a frequency distribution (how often a risk event happens) with a severity distribution (how large the loss is each time) into a single, full probability distribution of total loss. It preserves the shape of both inputs instead of collapsing them into one averaged number, which is what lets it show the tail risk that simple multiplication misses entirely.

How many Monte Carlo simulations do I actually need? Ten thousand iterations are enough for a quick exploratory pass. A hundred thousand is the standard for a full assessment and typically finishes in a few seconds. A million is worth the extra runtime only for high-stakes work like regulatory capital calculations, where the marginal precision gain matters more than the extra wait.

Is Monte Carlo simulation actually better than a risk matrix? For any decision that requires a dollar figure, reserve setting, capital allocation, insurance purchasing, control ROI, yes, decisively. A matrix can rank risks relative to each other in a rough, ordinal way, but it was never built to answer “how much should we reserve,” and the math behind multiplying two ordinal scores together doesn’t produce a meaningful quantity in the first place.

Which distribution should I use for loss severity? Lognormal is the right default for most financial losses, fines, and remediation costs, because it’s right-skewed and can’t go negative, matching how real losses actually behave. Switch to a mixture of two lognormal curves if your data is genuinely bimodal, to gamma if you need more flexible control over skew, or to a normal distribution only if your losses are genuinely symmetric, which is rare for operational risk.

Can I run this without paying for software? Yes. The full methodology, in both R and Python, is published as an open-source script that runs for free in Google Colab with no local installation required.

Go Deeper

For readers who want to run this themselves or dig into the full technical detail behind the method:

— the full methodology paper, with the mathematics behind combining Poisson frequency and lognormal severity through convolution.
— every code block from setup to reserve table, explained in sequence.
— the full R and Python source, including the convolution model and a compliance-specific impact variant.
— the same framework built out in Python, covering operational, compliance, and cyber risk.

A risk matrix tells a committee that something is “High.” A Monte Carlo simulation with convolution tells them there’s a 20% chance losses exceed $115,867 next year, and that reserving at the 95th percentile instead would cost more but close most of that gap. The first statement starts a conversation. The second one ends with a decision, a dollar figure, and a documented rationale an auditor can actually follow. The tools to make that switch are free, published, and run in under a minute. The only thing left standing in the way is the habit of reaching for the familiar color chart instead.

Key References

Methodology:

Huwyler, H. (2025). “Quantitative Risk Assessment in R: An Open-Source Convolutional Framework for Modeling Uncertainty and Reserves.” Quantitative Finance and Risk Management, Volume 10.
Cox, A.L. (2008). “What’s Wrong with Risk Matrices?” Risk Analysis, 28(2), 497-512.
Krisper, M. (2021). “Problems with Risk Matrices Using Ordinal Scales.” arXiv:2103.05440.
Thomas, P., Bratvold, R., Bickel, E. (2014). “The Risk of Using Risk Matrices.” SPE Economics & Management, 6(2), 56-66.

Monte Carlo Methods:

Ferrero, A. et al. (2023). “General Monte-Carlo Approach to Consider a Maximum Admissible Risk in Decision-Making Procedures.” Acta IMEKO, 12(4).
Burtescu, E. (2012). “Decision Assistance in Risk Assessment: Monte Carlo Simulations.” Informatica Economică, 16(4), 86-92.
Young, H.K., Ingall, L. (2009). “Exploring Monte Carlo Simulation Applications for Project Management.” IEEE Engineering Management Review, 37(2).

Convolution in Risk Management:

Yam, W.S. (2022). “Convolution Approach for Value at Risk Estimation.” Review of Pacific Basin Financial Markets and Policies.
Giuseppina Bruno, M., Tomassetti, A. (2006). “On the Calculation of Convolution in Actuarial Applications.” ACM.

Code Repository:

GitHub: github.com/hwyler/Paper2024/blob/main/RBaseModel
Published under open-source license for free use

Software:

R: cran.rstudio.com (free, open source)
Google Colaboratory: colab.research.google.com (free, cloud-based)

The gap between qualitative risk assessment and quantitative risk assessment is not a matter of sophistication. It’s a matter of utility. A risk matrix tells you a risk is “high.” A Monte Carlo simulation tells you there’s a 15% probability that losses will exceed $250,000 in the next 12 months and that reserving $180,000 covers 90% of scenarios. The first statement informs a discussion. The second statement informs a decision.

The tools to make this transition are free, the methodology is published, and the code runs in under five seconds. The only remaining barrier is the willingness to replace familiar but flawed methods with unfamiliar but accurate ones. The organizations that make this transition build risk functions that speak the language of finance, earn board-level credibility, and produce assessments that survive regulatory scrutiny. The ones that don’t will continue filling out colorful matrices and wondering why nobody uses them for actual decisions.

The 45 AI Threat Vectors That Your Security Team Probably Isn't Tracking

Thu, 12 Mar 2026 00:00:00 +0000

A Practitioner’s Field Guide

Most AI threat models are incomplete. Not slightly incomplete. Fundamentally incomplete.

Last year I reviewed the threat model for a financial services company deploying a credit decisioning AI. Their security team had identified seven threat vectors. Seven. They covered the obvious ones: data breaches, unauthorized access, denial of service. They missed 38 others, including 15 that were specific to AI systems and had no equivalent in their traditional IT threat catalog.

Three months after deployment, the model started producing subtly biased outputs. Not because of an external attack. Because a data scientist on the team had inadvertently introduced a feature engineering flaw that created a proxy variable for a protected characteristic. It was a negligence threat, an internal one, and it was not on anyone’s radar because the threat model only considered adversarial external actors.

This pattern repeats across almost every organization I work with. Security teams build threat models based on their experience with traditional systems. They focus on external attackers, malicious intent, and system-level exploits. They miss the internal negligence threats that cause the majority of real-world AI failures. They miss model-specific attack vectors that have no parallel in conventional cybersecurity. They miss the human and organizational threats that create the conditions for technical failures.

This post catalogs 45 distinct AI threat vectors organized across a two-dimensional taxonomy: intent (adversarial versus negligent) and target category (data, model, system, and human). Each vector includes a clear explanation, practical context, and implementation guidance. Use this as a working reference to audit the completeness of your own AI threat models.

Why Traditional Threat Models Fail for AI

Traditional threat modeling frameworks like STRIDE, PASTA, and even MITRE ATT&CK were designed for conventional information systems. They handle network attacks, authentication bypasses, privilege escalation, and data exfiltration well. They were not designed for systems where the “logic” is learned from data rather than written in code, where the attack surface includes the training pipeline itself, and where some of the most damaging threats come from well-intentioned internal teams making honest mistakes.

AI systems introduce three categories of threat that traditional frameworks handle poorly.

First, the model itself is an attack surface. Traditional systems have deterministic logic. If you protect the infrastructure and the data, the system behaves as designed. AI models are different. An attacker can manipulate the model’s behavior by carefully crafting inputs, without ever breaching the perimeter or accessing the infrastructure. They can extract sensitive training data by querying the model’s API. They can clone the model’s functionality through systematic probing. None of these attacks require the kind of infrastructure compromise that traditional threat models focus on.

Second, the training pipeline is a persistent vulnerability. Traditional systems are vulnerable during operation. AI systems are vulnerable during development. Poisoned training data, biased labels, flawed feature engineering, and compromised pre-trained models all introduce vulnerabilities before the system ever reaches production. By the time the model is deployed, the damage is already embedded in its weights.

Third, negligence threats cause more cumulative damage than adversarial threats. In traditional cybersecurity, the adversary is the primary concern. In AI, the internal team building and operating the system creates more risk through oversight, insufficient testing, poor documentation, and inadequate monitoring than external attackers do through deliberate exploitation. A threat model that only considers malicious actors misses the majority of the threat landscape.

Original implementation tip: When I first expanded an organization’s AI threat model beyond traditional categories, the security team pushed back hard. “We already cover insider threats,” they said. They did, but their insider threat model focused on malicious insiders who steal data or sabotage systems. It did not cover the data scientist who chooses features without considering proxy discrimination, the ML engineer who skips robustness testing under deadline pressure, or the architect who fails to build monitoring into the deployment pipeline. These are not insider “threats” in the traditional security sense. They are negligence risks that require fundamentally different controls. Treat them as separate categories in your threat model, not as subcategories of “insider threat.”

The Taxonomy Structure

This threat taxonomy organizes 45 vectors across two dimensions.

The first dimension is intent. Adversarial threats involve deliberate, intentional actions designed to compromise the AI system. Negligent threats involve unintentional actions or omissions that create vulnerabilities or cause harm. This distinction matters because adversarial and negligent threats require different controls. You defend against adversaries with detection, deterrence, and response. You defend against negligence with process, training, governance, and automation.

The second dimension is agent. External agents operate outside the organization’s boundary, including hackers, competitors, nation-state actors, and third-party vendors. Internal agents operate within the organization, including developers, data scientists, architects, operators, and end users.

Within each combination of intent and agent, threats target one of four categories: data (the information the system processes and learns from), model (the learned representations, algorithms, and parameters), system (the infrastructure, APIs, and operational environment), and human (the people who build, operate, and interact with the system).

The result is a comprehensive matrix that surfaces threats most organizations overlook because they fall outside the traditional “external attacker targeting our infrastructure” frame.

Adversarial External Threats: The Ones You Expect

This quadrant contains the threats most security teams already think about, plus several AI-specific vectors they likely do not.

Data Threats from External Adversaries

Two vectors target the data layer from outside the organization.

Data exfiltration occurs when attackers extract sensitive data from the AI system, either during training or inference. This differs from traditional data theft because the AI model itself can leak data. An attacker who gains access to model outputs, gradients, or confidence scores can sometimes reconstruct training data without ever accessing the database directly. The model becomes an unintentional data disclosure channel.

Data poisoning occurs when attackers introduce malicious or corrupted data into the training dataset. This is uniquely dangerous because the corruption happens before the model is deployed. If an attacker can influence any data source that feeds into the training pipeline, whether through compromised public datasets, manipulated web scraping sources, or tampered third-party data feeds, they can embed biases or backdoors that persist through every subsequent version of the model until the poisoned data is identified and removed.

Data poisoning is the adversarial data threat I worry about most because it is the hardest to detect and the longest-lasting in impact. Traditional data validation checks look for obvious anomalies: missing values, out-of-range entries, format errors. Poisoned data is designed to look normal. The individual data points are plausible. The corruption is statistical, not syntactic. Defending against it requires comparing training data distributions across time windows to detect subtle shifts, validating training data provenance to verify that sources have not been compromised, and testing model behavior on held-out validation sets from trusted sources. If your training data comes from any source you do not fully control, you have exposure to this vector.

Model Threats from External Adversaries

Six vectors target the model itself, and these are the threats most traditional security teams have the least experience with.

Deception involves crafting inputs designed to make the AI model produce incorrect predictions or classifications. Think of carefully modified images that cause a computer vision system to misclassify objects, or subtly altered text inputs that cause a natural language model to produce wrong outputs. The modifications are often imperceptible to humans but effective against the model.

Evasion is deception’s cousin, focused specifically on bypassing detection or classification mechanisms. An attacker modifies malicious inputs to evade an AI-based security system, such as altering malware signatures to bypass AI-powered threat detection or modifying fraudulent transactions to avoid AI-based fraud scoring.

Exploitation targets weaknesses in the AI system’s implementation rather than the model’s learned behavior. Insecure APIs, poor input validation, missing authentication, and misconfigured endpoints are the entry points. This vector bridges traditional cybersecurity and AI-specific risk because the vulnerabilities are conventional but the assets being targeted (models, training pipelines, inference endpoints) are AI-specific.

Inversion attacks reverse-engineer the model to infer sensitive information about training data. By carefully analyzing the model’s outputs across many queries, an attacker can deduce characteristics of the data the model was trained on. For models trained on medical records, financial data, or personal information, this vector directly threatens privacy even if the underlying database is perfectly secured.

Membership inference is related but distinct. Instead of reconstructing training data, the attacker determines whether a specific known data point was part of the training set. “Was this person’s medical record used to train your diagnostic model?” is a question that, if answerable through the model’s API, creates privacy and compliance exposure regardless of whether the attacker can reconstruct the full record.

Oracle attacks (also called model extraction or model stealing) involve an attacker querying the model extensively to understand its decision boundaries, then building a replica model that reproduces the original’s behavior without access to the training data. The attacker essentially steals the intellectual property embedded in the model through its public-facing API.

Transfer learning attacks exploit vulnerabilities in pre-trained models or the transfer learning process itself. Many organizations build their AI systems on top of pre-trained foundation models. If the foundation model contains embedded vulnerabilities, biases, or backdoors, every downstream model inherits them. This vector is growing in importance as more organizations build on third-party foundation models they did not train and cannot fully audit.

Of these six model-level threats, oracle attacks and transfer learning attacks are the two most underestimated. Oracle attacks are underestimated because organizations assume their model’s logic is protected by keeping the code proprietary. It is not. If the model is accessible through an API, its behavior can be replicated through systematic querying. Rate limiting helps but does not eliminate the risk. Transfer learning attacks are underestimated because organizations treat pre-trained foundation models as trusted components without verifying what is in them. I worked with a team that fine-tuned a publicly available language model for customer service automation. Nobody audited the base model for embedded biases or vulnerabilities. The assumption was “it’s from a reputable provider, so it’s safe.” That assumption is not supportable. If you use pre-trained models, document your trust assumptions about those models explicitly, test for bias and adversarial vulnerability in the fine-tuned model, and include the base model in your threat surface.

System Threats from External Adversaries

Eight vectors target the infrastructure and operational environment.

Advanced persistent threats involve state actors or organized crime groups using sophisticated tools to compromise the AI system over an extended period. These are the most resource-intensive attacks and typically target high-value AI systems in critical infrastructure, financial services, or defense applications.

API-based attacks exploit vulnerabilities in the APIs used to interact with the AI system. This includes injecting malicious data through API endpoints, exploiting authentication weaknesses, or abusing API rate limits to conduct model extraction.

Denial of service overwhelms the AI system with excessive traffic or resource demands. AI systems can be particularly vulnerable because inference workloads on complex models consume significant computational resources, making resource exhaustion attacks more efficient than against simpler services.

Model freezing attacks target the AI system’s update mechanism, preventing the model from receiving updates. A model that cannot be updated remains vulnerable to known threats and cannot adapt to data drift, effectively turning a dynamic system into a static one.

Model parameter poisoning introduces malicious updates or perturbations directly into the model’s parameters (weights, biases) rather than through training data. This vector is particularly relevant for federated learning systems where multiple parties contribute model updates.

Poorly designed APIs represent a threat vector that straddles adversarial and negligent categories. While the design flaw is internal, external attackers exploit it. Insecure API design that exposes model internals, lacks input validation, or provides excessive information in error messages creates the conditions for multiple other attacks.

Side-channel attacks exploit non-functional characteristics of the AI system, such as timing differences in inference responses, power consumption patterns during computation, or electromagnetic emissions, to extract information about the model or its data. These attacks do not target the model’s logic directly but extract information through observable physical or computational characteristics.

Supply chain compromise involves attackers tampering with hardware, software components, or third-party services in the AI system’s supply chain. Compromised ML libraries, poisoned pre-trained models distributed through public repositories, or tampered GPU firmware all fall into this category.

Supply chain compromise is the system-level external threat I spent the most time helping organizations address last year. The AI supply chain is broader and less controlled than most organizations realize. A typical ML pipeline might include open-source libraries (PyTorch, TensorFlow, scikit-learn), pre-trained models from public repositories (Hugging Face, GitHub), data from third-party providers, cloud services for training and inference, and container images from public registries. Each component is a potential entry point. The most practical defense is maintaining a software bill of materials (SBOM) for your AI systems that includes not just code dependencies but also model provenance, training data sources, and infrastructure components. When a vulnerability is discovered in any component, the SBOM tells you immediately which AI systems are affected. Without it, you are guessing.

Human Threats from External Adversaries

One vector targets the human element from outside.

Social engineering involves attackers manipulating developers, data scientists, or users into revealing sensitive information or interacting with the AI system in ways that compromise security. AI teams are often targeted because they have access to valuable IP (models, training data, feature engineering pipelines) and may not have received the same security awareness training as traditional IT staff. A data scientist who shares a model architecture diagram on a conference poster may not realize they have disclosed information useful for a model extraction attack.

Adversarial Internal Threats: The Ones You Underestimate

This quadrant contains only two vectors, but both are high-impact.

Human Threats from Internal Adversaries

Data sabotage occurs when insiders intentionally alter or destroy data used for training or inference. Unlike external data poisoning, an insider has legitimate access to data systems and can make changes that appear routine. A disgruntled data engineer who subtly modifies preprocessing scripts or alters label distributions can compromise model performance in ways that are extremely difficult to trace.

Subversion involves authorized developers or contractors intentionally sabotaging, exfiltrating, or manipulating the AI system. This goes beyond data sabotage to include embedding backdoors in model code, exfiltrating trained model weights for competitors, or introducing vulnerabilities that can later be exploited. The insider’s authorized access makes traditional perimeter defenses irrelevant.

Internal adversarial threats against AI systems are harder to detect than their equivalents in traditional IT for one specific reason: the normal behavior of an AI developer already includes activities that would be flagged as suspicious in other contexts. A data scientist routinely downloads large datasets, modifies data processing logic, changes model parameters, and deploys updated models. These are their job functions. Distinguishing between a legitimate model update and a sabotage event requires understanding what the model should be doing, not just what the developer is doing. The most effective control I have found is mandatory peer review for all changes to training data, feature engineering code, and model parameters before they reach production. Not automated testing, though that helps too. Human review by a second qualified person who can assess whether the change makes sense in context. This catches both intentional sabotage and unintentional errors.

Negligent External Threats: Your Vendors and Dependencies

This quadrant covers unintentional risks introduced by parties outside your organization.

Human Threats from External Negligence

Supply chain negligence occurs when third-party vendors introduce vulnerabilities through insecure libraries, dependencies, or tools used during AI system development, deployment, or maintenance. Unlike supply chain compromise (which is intentional), this vector reflects genuine negligence: a vendor fails to patch a library, releases an update with a security flaw, or provides tooling that does not meet security standards. The impact on your AI system is the same whether the vulnerability was introduced deliberately or carelessly.

Third-party data risk arises when organizations rely on external data sources that may be of poor quality, biased, or inadvertently altered. The third party is not acting maliciously. They simply do not maintain the data quality standards your model requires. Training on degraded external data produces degraded model performance, and the organization consuming the data may not detect the quality decline until outputs start failing.

System Threats from External Negligence

Outdated dependencies represent the use of unsupported software libraries in AI systems that introduce known, exploitable vulnerabilities. This is technically a negligence issue, an external provider stops maintaining a library, but the security impact is the same as an adversarial exploit because attackers actively scan for systems using deprecated dependencies.

Third-party data risk is the negligent external threat I encounter most frequently in practice. Organizations build models on external data feeds and assume the data quality will remain stable. It does not. I worked with a company whose fraud detection model degraded over four months because a third-party transaction data provider changed their data formatting without notification. The change was minor, a modification to how categorical fields were encoded, but it silently corrupted the feature engineering pipeline. The model’s accuracy dropped from 91% to 78% before anyone noticed. The fix was straightforward but the damage was done. For every external data dependency, establish a data quality SLA with the provider that specifies format, completeness, timeliness, and quality metrics. Monitor incoming data against those SLAs automatically. When a deviation occurs, alert before the data enters your training pipeline, not after your model degrades.

Negligent Internal Threats: Where Most AI Failures Actually Originate

This is the largest quadrant in the taxonomy, containing 24 of the 45 vectors. That distribution is not an accident. It reflects reality. The majority of AI failures in production stem from internal negligence, not external attacks.

Data Threats from Internal Negligence

Two vectors target the data layer through internal oversight.

Inaccurate data labeling occurs when developers fail to properly label training data during preparation. Labels are the ground truth the model learns from. If a medical imaging dataset contains mislabeled scans, the model learns to associate the wrong visual patterns with the wrong diagnoses. Unlike external data poisoning, this is not malicious. It is the predictable result of insufficient quality assurance in the annotation process, often driven by time pressure, undertrained annotators, or ambiguous labeling guidelines.

Bias in data occurs when data scientists or developers incorporate biased data during training, producing discriminatory or unfair outputs. This can result from historical bias embedded in the data itself (past lending decisions that reflected discriminatory practices), selection bias in how data was collected (underrepresenting certain populations), or measurement bias in how variables were recorded. The developers are not trying to create discriminatory outcomes. They are training on data that reflects existing inequities.

Bias in data is the negligent data threat with the highest regulatory and reputational impact. It is also the one where I see the most dangerous misconception. Teams believe that removing protected characteristics like race or gender from the training data eliminates bias. It does not. Other variables in the dataset frequently serve as proxies for protected characteristics. Zip code correlates with race. Job title correlates with gender. Part-time employment status correlates with caregiving responsibilities. Removing the protected characteristic while leaving the proxy variables in the feature set gives the appearance of fairness while producing the same discriminatory outcomes. The effective control is to test model outputs for disparate impact across protected groups, regardless of whether protected characteristics appear in the input features. Test the outputs, not the inputs.

Model Threats from Internal Negligence

Five vectors target the model through internal oversight.

Data and model drift occurs when developers fail to account for changes in data distribution or underlying concepts over time. A model trained on data from 2022 may not perform well on data from 2025 if customer behavior, market conditions, or the relationships between variables have shifted. This is not a one-time risk. It is an ongoing degradation that accelerates the longer a model operates without retraining or recalibration.

Feature engineering flaws result from unintentionally introducing vulnerabilities through poor feature selection. Selecting features that are easily manipulated by adversaries, including features that leak future information (data leakage), or failing to consider how feature distributions might shift in production all fall into this category.

Overfitting occurs when developers create models that fit too closely to the training data, learning noise and idiosyncrasies rather than genuine patterns. An overfit model shows excellent performance in testing and poor performance in production. From a security perspective, an overfit model is also more predictable to an adversary who understands its training data, making it easier to craft adversarial inputs.

Overfitting to noise is a specific variant where the model learns from irrelevant data rather than meaningful signal. The model performs well on training metrics but produces unreliable results in real-world deployment because it has memorized artifacts in the training data rather than learning the underlying relationship.

Unexplainability results when developers create AI systems too complex and opaque to understand or explain. This is a threat vector because opacity prevents detection of errors, biases, and vulnerabilities. A model whose decisions cannot be explained cannot be audited, cannot be debugged when it fails, and cannot satisfy regulatory requirements for explainability. Opacity does not cause harm directly, but it creates the conditions under which every other threat vector becomes harder to detect and address.

Data and model drift is the negligent model threat that causes the most cumulative financial damage because it is slow, silent, and continuous. I have never worked with an organization that detected drift proactively on their first AI deployment. They always detected it reactively, after business outcomes degraded enough for someone to notice. The detection lag ranged from three weeks to nine months depending on how closely business stakeholders monitored the model’s downstream effects. The fix is statistical monitoring of input feature distributions and output prediction distributions, compared against baseline distributions from the training period. When statistical tests detect a significant shift, trigger an alert. Do not wait for business outcome metrics to degrade, because by then the model has been making suboptimal decisions for weeks or months. Population Stability Index (PSI) is a good starting metric. Monitor it weekly at minimum for production models.

Human Threats from Internal Negligence

This is the richest subcategory in the entire taxonomy, containing 12 vectors. Each represents a different way that well-intentioned people create AI risk through oversight, insufficient skill, or organizational failure.

Inadequate documentation occurs when teams provide insufficient documentation for AI models, data sources, and decision-making processes. Without documentation, models cannot be maintained by anyone other than their original developer, security reviews cannot assess the system’s design assumptions, and regulatory compliance cannot be demonstrated.

Inadequate monitoring results from failing to build proper monitoring mechanisms to detect anomalies, adversarial activity, or model degradation in real time. An unmonitored model is a model whose failures go undetected until they manifest as business losses, customer complaints, or regulatory actions.

Inadequate maintenance occurs when teams fail to regularly update and maintain AI models, leaving them vulnerable to known attacks, data drift, and exploits as the system ages. Models, like all software, require ongoing maintenance. Unlike traditional software, model maintenance includes retraining, recalibration, and feature re-evaluation in addition to patching.

Inadequate testing results from failing to sufficiently test and validate models before deployment. This includes insufficient unit testing of data pipelines, absence of adversarial robustness testing, incomplete validation against held-out datasets, and failure to test for bias and fairness.

Inadequate training affects end-users and operators who receive insufficient instruction on how to interact with or manage AI systems. A model that is technically sound can still produce harmful outcomes if the humans using it do not understand its limitations, do not know when to override its recommendations, or cannot recognize when its outputs are unreliable.

Insecure design results from architects or developers failing to build secure AI systems from the start. This includes objective functions susceptible to manipulation, model architectures that leak information through their outputs, and deployment configurations that expose internal model details.

Insider threat (unintentional) covers authorized team members who inadvertently cause harm. A developer who accidentally pushes a model trained on test data to production. A data engineer who modifies a preprocessing script that breaks feature normalization. An operations team member who changes a configuration parameter without understanding its downstream effects.

Insufficient access control results from poorly managed access permissions that allow unauthorized access to models, data, or systems. This is a process failure, not a technology failure. The tools to enforce access control exist. The organization simply has not implemented them with sufficient rigor for AI-specific assets.

Lack of governance occurs when organizations fail to establish proper governance frameworks for AI development and deployment. Without governance, teams operate independently, security practices are inconsistent, accountability is undefined, and risk accumulates without visibility.

Over-reliance on AI results from decision-makers depending on AI outputs without sufficient human oversight or fail-safes. When users treat AI predictions as infallible, they stop applying the human judgment that catches model errors. This vector is particularly dangerous in high-stakes domains like healthcare, criminal justice, and financial services.

Unclear AI accountability occurs when nobody is clearly defined as accountable for AI-related decisions, model performance, or security. When accountability is unclear, risks go unmanaged because everyone assumes someone else is responsible.

Of these 12 human negligence vectors, inadequate monitoring and unclear accountability are the two that create the most cascading damage. They amplify every other threat in the taxonomy. An adversarial attack against an inadequately monitored system succeeds for longer. Data drift in a system with no accountable owner goes unaddressed for months. Bias in a model that nobody monitors against fairness metrics persists indefinitely. If you can only address two human negligence vectors immediately, address these two. Assign a named individual, not a team, as accountable for each production AI system. Then build monitoring that runs at the same speed as the model’s decision-making. Everything else becomes more manageable once you have visibility and ownership in place.

System Threats from Internal Negligence

Five vectors target the system infrastructure through internal oversight.

Inadequate incident response results from failing to develop or implement effective response plans for AI-specific incidents. AI incidents differ from traditional IT incidents. A model producing biased outputs is not a “system down” event. It does not trigger the same alerts. It requires different diagnostic procedures and different remediation steps. If your incident response playbook does not include AI-specific scenarios, your response will be improvised when it matters most.

Inadequate logging results from insufficient logging and monitoring infrastructure that makes it difficult to detect, investigate, or respond to incidents. If you cannot see what the model received as input, what it produced as output, and how its behavior has changed over time, you cannot diagnose problems or provide evidence for regulatory investigations.

Insecure data storage results from failing to implement secure storage for AI-specific data assets. Training data, model weights, feature engineering code, and hyperparameter configurations all contain sensitive intellectual property and potentially personal data. Storing them with the same (or lesser) security controls as general-purpose data creates exposure.

Insufficient redundancy results from failing to build AI systems with adequate fail-safes. A single point of failure in the model serving infrastructure, the data pipeline, or the monitoring system can take down the entire AI capability. AI systems often have complex dependency chains that create hidden single points of failure.

Misconfiguration results from incorrectly configuring security settings, environments, or system components. Cloud environment misconfigurations, exposed model endpoints, overly permissive IAM roles, and unencrypted data storage are the most common variants.

Inadequate incident response for AI systems is the system-level negligence threat I have spent the most time remediating. Traditional incident response plans categorize incidents by severity and system type, but they almost never include AI-specific incident categories. What do you do when a model starts producing outputs that are statistically different from its validation period behavior? What do you do when a fairness audit reveals disparate impact? What do you do when you discover that training data was contaminated three months ago and every model version since then is potentially compromised? These are AI incidents that require AI-specific response procedures. Build an AI incident response appendix for your existing plan. Include at minimum: model rollback procedures, retraining triggers, bias investigation protocols, adversarial attack containment steps, and stakeholder notification procedures. Then tabletop exercise these scenarios. The first time you run through an AI incident scenario, you will discover gaps in your response capability that are fixable before a real incident occurs.

Cross-Cutting Implementation Tips

These apply across all four quadrants of the threat taxonomy.

Assess your taxonomy coverage quarterly. AI threat vectors evolve as attack research advances, new model architectures emerge, and regulatory requirements expand. A taxonomy that was comprehensive six months ago may have gaps today. Assign someone to monitor AI security research (MITRE ATLAS updates, conference proceedings from NeurIPS and USENIX Security, regulatory guidance from NIST and the EU AI Office) and flag new vectors that should be added.

Original implementation tip: When reviewing your taxonomy coverage, do not just ask “have any new threat vectors emerged?” Also ask “have any of our existing vectors changed in severity or likelihood?” The relative importance of threat vectors shifts as your AI systems mature. Early in deployment, development negligence threats (inadequate testing, insecure design) are most relevant because the system is new and untested. Six months into production, operational negligence threats (inadequate monitoring, data drift, inadequate maintenance) become dominant. Twelve months in, adversarial threats increase as your AI system becomes a known, valuable target. Reassess priority rankings at each quarterly review.

Balance adversarial and negligence controls in your budget. Security teams naturally gravitate toward adversarial controls because they are more dramatic and more familiar. Adversarial robustness testing, penetration testing, red-teaming: these feel like “real” security work. Process controls, governance frameworks, training programs, and documentation standards feel like bureaucracy. In practice, the negligence controls prevent more incidents.

Original implementation tip: I track a simple metric with every organization I advise: the ratio of AI incidents caused by adversarial action versus negligence. Across 14 organizations over three years, the ratio has consistently been approximately 15% adversarial and 85% negligence. Yet budget allocation for adversarial controls versus negligence controls is typically inverted: 60-70% on adversarial defenses and 30-40% on process and governance. Reallocate to match the actual threat distribution. This does not mean reducing adversarial defenses. It means increasing investment in monitoring, documentation, testing processes, training, and governance until the budget reflects where incidents actually originate.

Map each vector to specific assets and controls. A threat vector without a corresponding asset mapping tells you what could happen but not where it could happen to you. A threat vector without a corresponding control tells you what to worry about but not what to do. For each vector in this taxonomy that applies to your AI systems, document three things: which specific assets are exposed, which controls currently mitigate the vector, and what residual exposure remains.

The most effective way I have found to operationalize a threat taxonomy is to create a traceability matrix with four columns: threat vector, exposed assets, current controls, and residual risk rating. Populate this matrix for every production AI system. When a new asset is deployed, add rows. When a new threat vector is identified, add rows. When a control is implemented or modified, update the current controls column and reassess residual risk. This matrix becomes the working document for your AI security program. It tells you at any point what threats you have addressed and what gaps remain. Without it, the taxonomy is an intellectual exercise. With it, the taxonomy drives action.

Key References and Standards

This threat taxonomy draws from and aligns with the following authoritative frameworks.

MITRE ATLAS (Adversarial Threat Landscape for Artificial Intelligence Systems) provides the primary reference taxonomy for AI-specific adversarial threats.

NIST AI RMF (AI 100-1) provides the risk management framework for identifying and addressing AI threats across the lifecycle.

ISO/IEC 27005:2022 provides the information security risk management process for integrating AI threats into enterprise risk assessment.

ISO/IEC 23894:2023 provides AI-specific risk management guidance including threat identification.

ISO/IEC 42001:2023 provides AI management system requirements for governance of the threat landscape.

OWASP Machine Learning Security Top 10 provides a practitioner-focused list of common ML security threats.

EU AI Act (Regulation 2024/1689) establishes regulatory requirements that make several of these threat vectors compliance-relevant.

NIST SP 800-30 Rev. 1 provides guidance on threat identification and risk assessment methodology.

ENISA Threat Landscape for AI provides European regulatory perspective on AI-specific threats.

Using This Taxonomy to Find Your Gaps

Organizations that treat this taxonomy as a reference list will read it, nod, and return to their existing threat models unchanged. They will continue to overweight adversarial external threats because those threats are familiar and dramatic. They will continue to underweight internal negligence threats because those threats feel mundane and uncomfortable to discuss. When an AI failure occurs, and it will, they will discover that the vector was sitting in a taxonomy they read but never operationalized.

Organizations that treat this taxonomy as an audit tool will do something different. They will take each of their production AI systems and map every applicable threat vector to the specific assets, existing controls, and residual risk for that system. They will discover gaps, mostly in the negligent internal quadrant, and they will prioritize closing them. They will update their incident response plans to include AI-specific scenarios. They will build monitoring that catches negligence-driven failures before they reach customers. They will assign accountability for each production model to a named individual who cannot hide behind a team name.

The threat your AI system faces tomorrow is almost certainly already in this taxonomy. The question is whether you have mapped it to your assets, built controls for it, and assigned someone to watch for it.

Which quadrant of this taxonomy has the least coverage in your current AI threat model? For most organizations, the answer is negligent internal. That is where your biggest gap probably lives.

About the Author

The frameworks, tools, and implementation guidance described in this article are part of the applied research and consulting work of Prof. Hernan Huwyler, MBA, CPA, CAIO. These materials are freely available for use, adaptation, and redistribution in your own AI governance, risk management, and compliance programs. If you find them valuable, the only ask is proper attribution. If you like the content, please like the article and share it.

Prof. Huwyler serves as AI GRC Consultancy Director, AI Risk Manager, and Quantitative Risk Lead, working with organizations across financial services, technology, healthcare, and public sector to build practical AI governance frameworks that survive contact with production systems and regulatory scrutiny. His work bridges the gap between academic AI risk theory and the operational controls that organizations actually need to deploy AI responsibly.

As a Speaker, Corporate Trainer, and Executive Advisor, he delivers programs on AI compliance, quantitative risk modeling, predictive risk automation, and AI audit readiness for executive leadership teams, boards, and technical practitioners. His teaching and advisory work spans IE Law School Executive Education and corporate engagements across Europe and globally.

Based in the Copenhagen Metropolitan Area, Denmark, with professional presence in Zurich and Geneva, Switzerland, Madrid, Spain, and Berlin, Germany, Prof. Huwyler works across jurisdictions where AI regulation is most active and where organizations face the most complex compliance landscapes.

His code repositories, risk model templates, and Python-based tools for AI governance are publicly available at . His ongoing writing on Governance, Risk Management and Compliance appears on his blogger website at (more than 500k views).

Connect with Prof. Huwyler on LinkedIn at to follow his latest work on AI risk assessment frameworks, compliance automation, model validation practices, and the evolving regulatory landscape for artificial intelligence.

If you’re building an AI governance program, standing up an AI risk function, preparing for EU AI Act compliance, or looking for practical implementation guidance that goes beyond policy documents, reach out. The best conversations start with a shared problem and a willingness to solve it with rigor.

The 49 AI Quality Characteristics That Define Whether Your System Actually Works

Thu, 12 Mar 2026 00:00:00 +0000

A Practitioner’s Guide to ISO/IEC 25059

A model with 95% accuracy that nobody can explain, nobody can maintain, and nobody trusts is not a quality AI system. It is a liability waiting to surface.

I learned this the hard way. Three years ago, I helped deploy a classification model for a European insurer. The accuracy metrics were excellent. The data science team celebrated. Six weeks later, the project was in crisis. The model could not be updated without breaking downstream integrations (maintainability failure). Users did not understand why it made specific recommendations and stopped trusting it (transparency failure). The system consumed three times the expected cloud resources during peak periods (performance efficiency failure). And when a regulator asked how the model made decisions about claims, nobody could provide an adequate explanation (accountability failure).

The model worked. The system did not.

That distinction, between a model that produces correct outputs and a system that delivers quality across its full operational lifecycle, is exactly what ISO/IEC 25059:2023 addresses. This standard defines 49 quality characteristics for AI systems across 11 requirement domains. It builds on the established software quality model of ISO/IEC 25010 but adapts it for the specific challenges of AI: opacity, learned behavior, data dependency, drift, fairness, and the unique ways AI systems interact with human judgment.

This post walks through all 49 characteristics with practical implementation guidance for each. Use it as a checklist for AI system design, a framework for quality assurance, and a reference for identifying which characteristics create risk when they are absent.

Why Software Quality Models Are Not Enough for AI

ISO/IEC 25010 is the standard quality model for software products and systems. It has served the industry well for conventional software. It defines characteristics like reliability, security, maintainability, and usability that apply to any software system.

AI systems need more.

A traditional software system does what its code tells it to do. If the code is correct, the system is correct. An AI system does what its training data and learned parameters tell it to do. Correctness is probabilistic, not deterministic. The system can be “correct” on average while failing catastrophically for specific populations or edge cases.

Three gaps in traditional software quality models become critical for AI.

First, transparency and explainability are not optional quality attributes for AI. They are functional requirements. A user who cannot understand why an AI system made a particular decision cannot verify it, trust it, or correct it. Traditional software quality models treat transparency as a nice-to-have. For AI, it is a prerequisite for accountability and regulatory compliance.

Second, AI systems degrade in ways traditional software does not. Data drift, concept drift, and model decay cause AI system quality to deteriorate over time even without any code changes. A quality model that only evaluates the system at deployment misses the ongoing quality challenges that define AI operations.

Third, AI systems create societal risks that traditional software rarely produces. Bias, discrimination, loss of autonomy, environmental impact, and ethical harms are quality concerns specific to AI that require explicit quality characteristics and measurement approaches.

ISO/IEC 25059 fills these gaps by extending the traditional quality model with AI-specific characteristics across every domain. The result is a comprehensive framework for evaluating whether an AI system is genuinely fit for purpose, not just whether it produces accurate outputs.

Original implementation tip: When I introduce ISO 25059 to organizations, the most common initial reaction is overwhelm. Forty-nine characteristics feels like an impossibly large quality surface to manage. The practical approach is to prioritize. Not every characteristic is equally relevant for every AI system. A customer-facing recommendation engine needs strong transparency, user controllability, and fairness characteristics. An internal process automation system needs strong reliability, maintainability, and robustness characteristics. Map the 49 characteristics to your specific system’s risk profile and context of use. Identify the 10 to 15 that are most critical. Focus your quality assurance resources there. Then expand coverage over time.

Domain 1: Functional Suitability

Four characteristics define whether the AI system does what it is supposed to do.

Functional Completeness

The degree to which the system’s functions cover all specified tasks and user objectives. The AI system provides all necessary functions for its intended purpose and fulfills all explicitly stated and implied user needs.

This sounds basic. It is the characteristic most often violated in AI projects because teams focus on the model’s prediction function and neglect the surrounding functions that make the prediction useful: data preprocessing, output formatting, error handling, user feedback mechanisms, and integration with downstream workflows.

To get this right, conduct rigorous requirements gathering that maps all user tasks to system functions. Validate coverage through user acceptance testing and traceability matrices that link requirements to implemented functions.

Original implementation tip: The functional completeness gap I find most often is the absence of a “decline to predict” function. Most AI systems are built to produce an output for every input. But there are inputs where the system should not produce a prediction because it does not have sufficient confidence, because the input falls outside its training distribution, or because the decision requires human judgment. Build the ability to abstain into your system. A credit scoring model that says “I cannot score this application with sufficient confidence, route to a human underwriter” is more functionally complete than one that produces a low-confidence score that a loan officer treats as definitive.

Functional Correctness

The degree to which the AI system provides correct results with the needed degree of precision. Outputs and effects are accurate and yield the right result.

For AI systems, “correct” is probabilistic. A model with 90% accuracy is wrong 10% of the time. The question is not whether the system is perfect but whether its error rate falls within acceptable bounds for its application context, and whether errors are distributed fairly across populations.

Establish ground truth datasets and validate continuously against predefined accuracy metrics such as F1-score, precision, and recall. Use adversarial testing to challenge model outputs and expose weaknesses.

Functional Appropriateness

The degree to which functions facilitate the accomplishment of specified tasks and objectives. Functions are suitable for the user’s stated goals and context of use.

An AI system can be functionally complete and correct while still being inappropriate. A sentiment analysis model that classifies customer feedback into three categories (positive, negative, neutral) is functionally correct but functionally inappropriate if the customer service team needs to distinguish between 12 specific complaint types to route tickets effectively.

Conduct task analysis and user studies to ensure functions align with actual user goals and workflows. Prioritize features based on user value, not technical feasibility.

Functional Adaptability

The degree to which the AI system can be adapted for different specified tasks and environments. The system can be modified or configured for new purposes or contexts.

AI systems that cannot adapt become obsolete quickly. Business requirements change, data distributions shift, and new use cases emerge. A system designed for a single, fixed purpose delivers diminishing value over time.

Design systems with configurable parameters and hooks for retraining. Use feature flags and modular architecture to enable adaptation to new tasks without full redevelopment.

Original implementation tip: Functional adaptability is the suitability characteristic that determines long-term ROI, and it is almost always underinvested in during initial development because the pressure is on delivering the first use case. I worked with a logistics company that built a demand forecasting model tightly coupled to a single product category. When they wanted to extend it to two additional categories, they discovered the data pipeline, feature engineering, and model architecture were all hard-coded for the original category. Extending took nearly as long as building from scratch. Build adaptability into the architecture from day one, even if you are deploying for a single use case. Parameterize data sources, feature definitions, and model configurations. The marginal cost during initial development is small. The cost of retrofitting adaptability later is enormous.

Domain 2: Performance Efficiency

Three characteristics define whether the system uses resources appropriately.

Time Behaviour

The degree to which response and processing times meet requirements. The system delivers results within required time constraints.

For AI systems, time behavior is more variable and harder to predict than for traditional software. Inference latency depends on model complexity, input size, hardware availability, and concurrent load. Training time depends on dataset size, model architecture, and compute resources.

Profile system components to identify bottlenecks. Set Service Level Objectives for latency and throughput. Optimize models through quantization, pruning, or distillation for target deployment environments.

Resource Utilisation

The degree to which resource usage meets requirements. The system uses appropriate amounts of processing capacity, memory, and network bandwidth.

AI workloads consume significantly more resources than traditional applications. GPU costs for training, memory requirements for large models, and storage demands for training data can all exceed initial estimates.

Monitor compute, memory, and network usage during both inference and training. Right-size infrastructure and use auto-scaling. Prefer efficient model architectures for resource-constrained deployment environments.

Capacity

The degree to which maximum limits of system parameters meet requirements. The system handles the specified maximum number of items, users, or data volume.

Perform load and stress testing to determine system limits across users, transactions, and data volume. Design architecture to scale horizontally. Build in rate limiting and graceful degradation so that exceeding capacity reduces performance rather than causing failure.

Original implementation tip: The performance efficiency characteristic that catches organizations off guard is resource utilization during retraining, not during inference. Teams size their infrastructure for inference workloads and then discover that monthly retraining jobs require 10 times the compute resources. The retraining job competes with inference for GPU capacity, degrading production performance during the retraining window. Separate training and inference infrastructure, or schedule retraining during off-peak periods with dedicated resource allocation. Monitor resource utilization during both operational modes separately.

Domain 3: Compatibility

Two characteristics define how the system coexists with its environment.

Co-existence

The degree to which the AI system performs its functions while sharing a common environment and resources with other products. The system operates without negatively impacting other systems.

Test the AI system in a staging environment that mirrors production, including all other applications that share resources. Ensure the AI system does not monopolize shared CPU, memory, or network bandwidth during peak inference or training periods.

Interoperability

The degree to which systems can exchange and use information. The AI system effectively communicates with other specified systems.

Adopt standard data formats like ONNX and PMML for model exchange, and standard API protocols like REST and gRPC for communication. Implement rigorous schema validation for all data exchanges. Use API gateways for consistent management of interfaces.

Original implementation tip: Interoperability failures are among the most common reasons AI projects fail during the transition from development to production. The model works perfectly in the data science team’s environment but cannot consume data from the production pipeline because formats, schemas, or encoding conventions differ. Test interoperability between the development environment and the production environment early, ideally within the first two weeks of development. Discovering format mismatches at deployment is expensive. Discovering them during initial development is cheap.

Domain 4: Usability

Seven characteristics define the human experience of interacting with the AI system. This is the largest traditional usability domain and includes two AI-specific additions: user controllability and transparency.

Appropriateness Recognisability

The degree to which users can recognize whether the system is appropriate for their needs. The system’s capabilities and limitations are clear to potential users.

Provide clear documentation of capabilities, limitations, and intended use cases. Create a Model Card or similar fact sheet that communicates what the system does, what it does not do, what data it was trained on, and where it performs well or poorly.

Learnability

The degree to which the system enables users to learn how to use it effectively. The system supports users in acquiring operational knowledge.

Develop intuitive interfaces, comprehensive documentation, and interactive tutorials. Incorporate contextual help. Conduct usability testing to measure the learning curve across different user populations.

Operability

The degree to which the system is easy to operate and control. User effort for operation is minimized.

Design clear and consistent interfaces and APIs. Provide effective error messages and status indicators. Automate complex operational tasks where possible.

User Error Protection

The degree to which the system protects users against making errors. The system prevents, detects, and helps users recover from mistakes.

Implement input validation, confirmation dialogs for critical actions, and undo functionality. Use constraints to prevent invalid inputs. Guide users through complex tasks with clear step-by-step workflows.

User Interface Aesthetics

The degree to which the interface enables pleasing interaction. The design is visually and interactively appealing.

Apply established design systems for visual consistency. Ensure a clean, uncluttered interface. Conduct user research on aesthetic perception to ensure the design supports rather than hinders the user’s task.

Accessibility

The degree to which the system can be used by people with the widest range of characteristics and capabilities. The system accommodates diverse user needs including disabilities.

Follow WCAG 2.1 guidelines. Test with screen readers, ensure keyboard navigation, provide alt text for images, and support high contrast modes. Accessibility is not optional. It is a quality requirement and increasingly a legal one.

User Controllability (AI-Specific)

The degree to which users can control the AI system’s behavior. Users can initiate, adjust, or stop the system’s operations.

Provide settings to adjust system behavior such as confidence thresholds and filters. Allow users to start, stop, and correct operations. Ensure humans can always override AI decisions. This characteristic is directly tied to the EU AI Act’s requirements for human oversight of high-risk AI systems.

Transparency (AI-Specific)

The degree to which the system’s functions, decisions, and outputs are understandable to the user. The system provides explanations for its behavior and results.

Implement Explainable AI techniques like LIME and SHAP to provide output explanations. Document the model’s purpose, training data, and algorithms. Be explicit about the system’s AI nature. Transparency is not a single feature. It is a quality that must be designed into every interaction between the system and its users.

Original implementation tip: Of the seven usability characteristics, user controllability is the one most often missing from AI system designs, and it is the one regulators are asking about most frequently. I reviewed an AI system for a healthcare provider that provided diagnostic recommendations to physicians. The system had no mechanism for a physician to adjust the confidence threshold, no way to request an alternative recommendation, and no clear process for overriding the system’s output when clinical judgment disagreed. The system treated every recommendation as a final answer rather than an input to human decision-making. When the EU AI Act’s human oversight requirements were mapped against the system’s capabilities, the gap was significant. Build controllability from the start. Provide clear controls for adjusting, overriding, and stopping AI behavior. Document how these controls work and verify that users know how to use them.

\[Suggested image placement: A visual showing all 11 quality domains arranged in a wheel or grid, with the number of characteristics per domain indicated, highlighting the AI-specific additions in a distinct color\]

Domain 5: Reliability

Five characteristics define whether the system performs consistently and recovers from failures. This domain includes one critical AI-specific addition: robustness.

Maturity

The degree to which the system meets reliability needs under normal operation. The system is stable with a low failure rate in its standard operating environment.

Establish a robust CI/CD pipeline with automated testing. Track mean time between failures. Use canary deployments to gradually roll out updates and catch stability issues before full deployment.

Availability

The degree to which the system is operational and accessible when required. The system has minimal downtime.

Design for redundancy with failover mechanisms across availability zones. Monitor uptime and establish Service Level Agreements. Implement health checks and graceful degradation so that partial failures do not cause total outages.

Fault Tolerance

The degree to which the system operates as intended despite hardware or software faults. The system continues functioning during component failures.

Build systems that handle component failures without total collapse. Use retries with exponential backoff, circuit breakers, and fallback mechanisms. Design stateless services where possible to simplify recovery.

Recoverability

The degree to which the system can recover data and re-establish desired state after failure. The system restores service and data quickly.

Implement automated backup and restore procedures for models and data. Define and test a Disaster Recovery plan. Track mean time to recovery. Ensure recovery points are consistent, meaning the model, its configuration, and its data are all restored to the same point in time.

Robustness (AI-Specific)

The degree to which the system functions correctly despite invalid inputs, stressful conditions, or adversarial attacks. The system maintains performance under perturbation.

This is the reliability characteristic most specific to AI and most critical for security. Test with noisy, out-of-distribution, and adversarial inputs. Use data augmentation, adversarial training, and defensive distillation to improve resilience. Monitor for data drift that degrades robustness over time.

Original implementation tip: Robustness is the reliability characteristic that creates the most direct link between quality and security. A model that is not robust against adversarial inputs is both a quality failure and a security vulnerability. Yet robustness testing is consistently treated as a security activity performed by the security team rather than a quality activity performed by the development team. This separation creates gaps. The security team tests for adversarial attacks. The development team tests for accuracy. Nobody tests for the space in between: inputs that are not adversarial but are unexpected, noisy, or from a different distribution than the training data. These “natural” robustness failures are more common than adversarial attacks and cause more cumulative damage. Include robustness testing in your development quality assurance process, not just in your security testing program.

Domain 6: Security

Six characteristics define the system’s security posture. This domain includes one AI-specific addition: intervenability.

Confidentiality

The degree to which data are accessible only to those authorized. The system protects data from unauthorized disclosure.

Encrypt data at rest and in transit. Implement strict role-based access controls and the principle of least privilege. Anonymize or pseudonymize training data. Consider secure multi-party computation for sensitive applications.

Integrity

The degree to which the system prevents unauthorized modification of data or functions. The system ensures data and system accuracy and completeness.

Use hashing and digital signatures to verify data and model artifacts have not been tampered with. Maintain an immutable audit trail. Validate inputs to prevent injection attacks, including adversarial inputs designed to manipulate model behavior.

Non-repudiation

The degree to which actions can be proven to have taken place. The system provides evidence for transactions that cannot be denied later.

Implement secure logging and auditing for all significant actions and decisions. Use digital signatures to ensure actions can be attributed to a specific entity or user. For AI systems making consequential decisions, non-repudiation is essential for regulatory compliance and dispute resolution.

Accountability

The degree to which actions can be traced to the entity that bears responsibility. The system enables assignment of responsibility.

Maintain clear ownership of models and system components. Establish audit trails that log system decisions, data sources, and user interactions. Define clear lines of responsibility for every component and every decision the system produces.

Authenticity

The degree to which the identity of a subject or resource can be proved. The system verifies that entities are genuine.

Implement strong authentication mechanisms including multi-factor authentication for system access. Verify the provenance of training data and model packages to prevent tampering or supply chain attacks. For AI systems, authenticity extends beyond user identity to include data authenticity and model authenticity.

Intervenability (AI-Specific)

The degree to which the system allows human intervention in its operation. Authorized humans can oversee and interrupt the system’s functions.

Design human-in-the-loop processes for critical decisions. Provide clear interfaces for oversight, intervention, and manual override. Ensure the system can be paused or stopped safely at any point without data loss or inconsistent state. This characteristic is a direct requirement of the EU AI Act for high-risk AI systems.

Original implementation tip: Accountability and intervenability work together and fail together. An AI system that logs every decision (accountability) but provides no mechanism for a human to intervene when they see a problematic pattern in those logs (intervenability) creates awareness without agency. A system that allows human override (intervenability) but does not log who overrode which decision and why (accountability) creates agency without traceability. Design these two characteristics as a pair. The logging system should inform the intervention interface, and every intervention should be logged with the rationale for the override.

Domain 7: Maintainability

Five characteristics define whether the system can be changed, fixed, and improved over time.

Modularity

The degree to which the system is composed of discrete components where changes to one have minimal impact on others.

Architect as loosely coupled components: separate data processing, training, and inference services. Use well-defined interfaces between components. This enables updating one component without risking the stability of others.

Reusability

The degree to which components can be used in more than one system or context.

Develop and package model components, feature pipelines, and datasets as reusable assets. Create shared libraries with clear documentation. Use containerization to make components portable across environments.

Analyzability

The degree to which the impact of an intended change can be assessed. The system can be diagnosed for deficiencies or failure causes.

Implement comprehensive logging and monitoring for all components. Use distributed tracing to follow requests through the system. Maintain detailed documentation of architecture and data lineage so that when something fails, the cause can be traced efficiently.

Modifiability

The degree to which the system can be changed without introducing defects or degrading quality.

Write clean, well-documented code. Avoid tight coupling. Use version control for all artifacts including code, data, and models. Implement feature toggles for controlled rollout of changes.

Testability

The degree to which test criteria can be established and tests can be performed effectively.

Design systems with testing in mind from the start. Create isolated test environments. Automate unit, integration, and regression tests for both models and code. Monitor test coverage and maintain it as the system evolves.

Original implementation tip: Analyzability is the maintainability characteristic that determines how quickly you can respond to AI incidents, and it is the one most organizations invest in only after their first major incident. When a model starts producing unexpected outputs in production, the first question is always “what changed?” Without comprehensive data lineage, model versioning, and input/output logging, answering that question can take days. With them, it takes minutes. Build analyzability into your system before you need it. The cost of implementing logging, tracing, and lineage tracking during initial development is a fraction of the cost of retrofitting them during a production incident investigation.

Domain 8: Portability

Three characteristics define how the system moves between environments.

Installability

The degree to which the system can be successfully deployed and removed in a specified environment.

Package using standard tools like Docker containers, Helm charts, or pip packages. Automate deployment scripts. Provide clear installation documentation and dependency lists. AI systems often have complex dependency chains that make installation significantly harder than traditional software.

Replaceability

The degree to which the system can substitute for another product in the same environment.

Adopt standard interfaces and protocols to avoid vendor lock-in. Ensure data and models are exportable in standard formats. Document APIs and dependencies thoroughly so that replacement is feasible when needed.

Adaptability

The degree to which the system can be adapted for different environments without custom modification.

Use configuration files to manage environment-specific parameters. Avoid hard-coding values. Design the system to be environment-agnostic, sourcing configuration externally. Follow the Twelve-Factor App methodology for environment-independent design.

Original implementation tip: Portability characteristics collectively determine your vendor lock-in risk. I worked with an organization that deployed an AI system on a single cloud provider’s proprietary ML platform, using provider-specific data formats, training APIs, and deployment tools. When they needed to move to a multi-cloud architecture for resilience, the migration cost exceeded the original development cost. The system scored zero on all three portability characteristics. Before committing to a platform, evaluate your system against these three characteristics. If you score poorly on all three, you have accepted significant lock-in risk. Make that acceptance explicit and documented rather than accidental and discovered later.

Domain 9: Quality in Use

This is the most expansive domain, containing 14 characteristics that evaluate the system’s quality from the perspective of actual use by real users in real contexts. Several of these characteristics are specific to AI and address societal and ethical dimensions that have no equivalent in traditional software quality models.

Effectiveness

The degree to which accurate and complete results are achieved. The system helps users achieve specified goals with precision and comprehensiveness.

Define clear metrics for accuracy and completeness aligned to user goals, not just model performance metrics. Implement robust validation and user acceptance testing. Track task success rates to measure real-world effectiveness.

Efficiency

The degree to which results are achieved with appropriate resources. The system minimizes user time, effort, and resource expenditure.

Measure time-on-task and steps to completion for key user journeys. Optimize workflows and system performance to reduce user effort. Efficiency in the quality-in-use sense is about the user’s experience, not the system’s computational efficiency.

Usefulness

The degree to which the system is capable of achieving specified goals. The system serves a practical purpose and delivers tangible benefits.

Conduct task analysis and user research to ensure the system solves a real problem. Prioritize features that deliver the highest value. Continuously validate usefulness through feedback and usage metrics.

Trust

The degree to which users have confidence that the system will behave as intended. The system is reliable, dependable, and predictable.

Design for reliability, transparency, and fairness. Provide explanations for outputs and allow human oversight. Be clear about system limitations to build appropriate trust, not excessive trust that leads to overreliance.

Pleasure

The degree to which users obtain satisfaction from using the system. The experience is positive and enjoyable.

Apply user-centered design principles. Conduct usability testing to identify and eliminate frustration points. Reward user actions positively through clear feedback and smooth interactions.

Comfort

The degree to which users are satisfied with physical comfort during interaction. The system minimizes physical strain such as eye fatigue or repetitive stress.

Design interfaces that adhere to ergonomic principles. Ensure readable text, comfortable interaction patterns, and support for assistive technologies.

Transparency in Use

The degree to which users can understand the system’s functions, decisions, and outputs in practice. The system provides clarity on operations and reasoning.

This extends the usability transparency characteristic into actual use contexts. Implement Explainable AI techniques suitable for end-users, such as natural language explanations rather than technical feature importance scores. Ensure explanations are actionable and understandable by non-technical users.

Economic Risk Mitigation

The degree to which the system mitigates potential economic risks. The system protects users and stakeholders from financial loss and wasted investment.

Conduct cost-benefit and ROI analyses. Implement safeguards against errors that could lead to significant financial loss. Ensure transparency in automated financial decisions. This characteristic is particularly relevant for AI systems that make or influence financial decisions at scale.

Health and Safety Risk Mitigation

The degree to which the system mitigates health and safety risks. The system prioritizes human well-being above all else.

Perform rigorous risk assessments such as Failure Mode and Effects Analysis for safety-critical applications. Implement fail-safes, human-in-the-loop controls, and continuous monitoring for hazardous situations. Comply with relevant safety standards such as IEC 61508 for functional safety.

Environment Risk Mitigation

The degree to which the system mitigates environmental risks. The system minimizes negative environmental impacts.

Monitor and optimize computational efficiency and energy footprint. Prefer cloud regions powered by renewable energy. Consider the full lifecycle environmental impact including training, inference, and data storage.

Societal and Ethical Risk Mitigation

The degree to which the system mitigates societal and ethical risks. The system avoids causing harm, promotes fairness, and upholds ethical principles.

Establish an AI Ethics board and guidelines. Proactively test for and mitigate biases. Ensure fairness, accountability, and transparency throughout the AI lifecycle. Conduct impact assessments for high-risk applications.

Context Completeness

The degree to which the system can achieve goals in all specified contexts of use. The system functions effectively across all intended situations, environments, and user profiles.

Identify and test all specified contexts during development. Use diverse datasets that represent all intended environments and user groups. Monitor for context drift in production where the system encounters situations outside its training distribution.

Flexibility

The degree to which the system can achieve goals in contexts beyond those initially specified. The system adapts to unanticipated situations.

Design with modular and adaptable architectures. Allow configuration and customization. Use techniques like transfer learning to enable adaptation to new contexts without full redevelopment.

Original implementation tip: The quality-in-use characteristics that organizations most consistently neglect are the four risk mitigation characteristics: economic, health and safety, environmental, and societal/ethical. These characteristics feel like “someone else’s job.” The AI development team focuses on effectiveness and efficiency. The compliance team handles economic risk. The safety team handles health and safety. Nobody owns environmental or societal risk mitigation as a quality characteristic of the AI system itself. But ISO 25059 places these squarely within the quality model. They are quality attributes of the system, not external governance requirements. Treat them as you would any other quality characteristic: define metrics, set targets, test against them, and monitor in production. The system’s quality is incomplete without them.

Using This Framework for Risk Identification

The 49 characteristics in ISO/IEC 25059 serve a dual purpose. They define what quality looks like for an AI system, and they identify where risk lives when quality is absent.

Every characteristic that scores poorly represents a risk. Low functional correctness means the system produces errors. Low robustness means the system is vulnerable to adversarial inputs. Low transparency means decisions cannot be explained to regulators. Low intervenability means humans cannot stop the system when it malfunctions.

Map this directly to your AI risk assessment. For each characteristic, ask three questions. How does our system perform against this characteristic? What is the consequence if this characteristic fails? What controls do we have in place to maintain this characteristic over time?

The answers populate your risk register with specific, measurable, and controllable risks rather than generic categories like “model risk” or “AI quality issues.”

Original implementation tip: I use the 49 characteristics as a structured interview guide during AI risk assessments. For each production AI system, I walk through every characteristic with the development team, operations team, and business owner. Each conversation takes about two hours. The output is a quality profile for the system with a red/amber/green rating for each characteristic. Red-rated characteristics map directly to risks in the risk register. Amber-rated characteristics map to watch items with monitoring requirements. This approach produces a more comprehensive risk identification than any brainstorming-based approach I have used, because the characteristics serve as prompts that surface risks the team would not think of on their own. “How does your system handle invalid inputs?” (robustness) and “Can a user override the system’s decision?” (intervenability) consistently uncover risks that open-ended risk identification sessions miss.

Key References and Standards

This quality framework draws from and aligns with the following authoritative sources.

ISO/IEC 25059:2023 for the primary AI system quality model that defines the 49 characteristics described in this post.

ISO/IEC 25010:2023 for the foundational software product and system quality model that ISO 25059 extends.

ISO/IEC/IEEE 29148 for requirements engineering practices that support functional suitability assessment.

ISO 9241-210 for human-centered design principles that support usability assessment.

NIST AI RMF (AI 100-1) for the AI risk management framework that connects quality characteristics to risk management.

NIST AI 100-2 for adversarial machine learning guidance that supports robustness assessment.

EU AI Act (Regulation 2024/1689) for regulatory requirements that make several quality characteristics legally mandatory for high-risk AI systems.

W3C WCAG 2.1 for web accessibility guidelines that support the accessibility characteristic.

ISO/IEC 27001 for information security management standards that support security characteristics.

IEC 61508 for functional safety standards relevant to health and safety risk mitigation.

The Difference Between Accurate and Good

Organizations that evaluate their AI systems only on accuracy metrics will continue deploying systems that work in testing and fail in production, that produce correct outputs nobody trusts, that cannot be maintained by anyone other than their original developer, and that create regulatory exposure because they cannot explain their decisions. High accuracy on a test set is one characteristic out of 49. Treating it as the only one that matters is how quality failures happen.

Organizations that evaluate their AI systems across the full quality model will build systems that are not only accurate but explainable, robust, maintainable, fair, controllable, and recoverable. They will catch quality gaps during development rather than discovering them through production incidents. They will satisfy regulatory requirements because the quality characteristics regulators care about, transparency, accountability, intervenability, fairness, were designed in from the start.

An AI system that scores well on one quality characteristic and poorly on 48 others is not a quality system. It is a model with infrastructure around it. The infrastructure is where quality lives or dies.

Which of the 49 characteristics is weakest in your most important AI system? If you cannot answer that question, start with the structured interview approach described above. Two hours will reveal gaps that months of operation have hidden.

The AI Loss Taxonomy Your Risk Assessments Are Missing

Thu, 12 Mar 2026 00:00:00 +0000

Incident Types and Direct Loss Categories That Define Real Exposure for AI Projects

Here is a question that reveals whether your AI risk program is mature or performative: Can you name the specific types of losses your AI systems could produce?

Not vague categories like “financial impact” or “reputational damage.” Specific, measurable loss types with clear boundaries between them. The difference between a regulatory fine and a legal compensation payment. The difference between algorithm remediation costs and data regeneration costs. The difference between customer churn and business disruption.

I asked this question to the risk committee of a healthcare AI company two years ago. The room went quiet. They had a risk register with 20 AI risks, each rated on a five-point scale for likelihood and impact. But when I asked “what kind of impact?” nobody could decompose their generic “high impact” ratings into the specific loss types that would actually appear on a financial statement or in a regulatory action.

That gap matters. You cannot quantify what you cannot classify. And you cannot prioritize controls, calculate return on investment, or purchase appropriate insurance if you cannot distinguish between the types of losses your AI systems might generate.

This post provides two complementary taxonomies. The first catalogs 37 distinct AI-related incident types across eight categories, each classified by whether it creates internal losses (relevant to risk assessments) or external losses (relevant to impact assessments) or both. The second catalogs 15 direct loss types across five domains that map to specific financial line items. Together, they give you the vocabulary and structure to make your AI risk assessments financially precise.

Why Generic Loss Categories Fail

Most AI risk assessments use three to five impact categories: financial, operational, reputational, regulatory, and strategic. These categories are so broad that they obscure more than they reveal.

When a risk assessment says an AI system has “high financial impact,” does that mean the organization will pay regulatory fines? Lose customers? Write off a failed project? Pay for emergency model remediation? All of these are “financial impact,” but they involve different stakeholders, different timescales, different control strategies, and different insurance coverage. Lumping them together makes the risk assessment useless for decision-making.

The same problem applies to incident classification. “AI bias” is not a single incident type. It manifests as biased outputs, unequal performance across groups, unfair discrimination, and lack of diversity in development teams. Each manifestation has different causes, different controls, and different loss profiles. Treating them as one incident type produces controls that are too generic to be effective.

The solution is granularity. Not complexity for its own sake, but sufficient decomposition to enable specific, actionable analysis. The taxonomies in this post provide that granularity.

Original implementation tip: When I first introduced a granular loss taxonomy to a financial services client, their initial reaction was that it added unnecessary complexity. They were managing 15 AI risks with five impact categories and felt that was sufficient. I asked them to take their highest-rated risk, “model produces biased outputs,” and trace it to specific financial consequences. They identified regulatory fines quickly. Then I asked about legal compensation payments to affected customers, algorithm remediation costs for retraining the model, control remediation costs for fixing governance gaps found during investigation, customer churn from affected populations, and reputation damage from media coverage. The total potential exposure across these six loss types was four times their original “high impact” estimate. Granularity did not add complexity. It revealed exposure they had been underestimating.

The incident taxonomy organizes 37 distinct incident types across eight categories. Each incident is classified as producing internal losses (considered in risk assessments), external losses (considered in impact assessments), or both.

This distinction matters for assessment methodology. Internal losses affect the organization directly through operational disruption, remediation costs, and control failures. External losses affect individuals, communities, or society through harm, discrimination, or rights violations. Many incidents produce both, requiring assessment from both perspectives.

Category 1: Cognitive Degradation

Three incident types address AI’s impact on human cognitive and decisional capacity.

Addiction and digital wellness (external only) occurs when AI systems contribute to addictive behaviors and negative impacts on digital wellness. Recommendation algorithms that maximize engagement metrics can create patterns of compulsive use. AI-driven content curation that prioritizes emotional arousal over informational value degrades the quality of users’ information environment. This is an external loss because the harm falls on users, not the organization, but regulatory attention to digital wellness is increasing, which creates secondary compliance exposure.

Loss of autonomy (internal and external) occurs when AI systems make decisions that diminish user control. This happens when automated decision-making replaces human judgment in contexts where individuals should retain meaningful choice. Internally, this manifests when employees lose the ability to exercise professional judgment because AI systems override their input. Externally, customers or citizens experience reduced agency in decisions affecting their lives, such as credit, employment, or healthcare.

Overreliance on AI (internal and external) occurs when users anthropomorphize, trust, or depend on AI systems beyond what the system’s capabilities warrant. Internally, decision-makers who treat model outputs as infallible stop applying critical judgment. Externally, users develop inappropriate emotional or material dependencies on AI systems, or form expectations the system cannot meet.

Original implementation tip: Overreliance on AI is the cognitive degradation incident type that creates the most immediate organizational risk, and it is almost never included in AI risk assessments. I worked with a lending organization where loan officers had become so accustomed to following the AI’s credit recommendations that they stopped reviewing the underlying data. When the model began producing anomalous scores due to a data pipeline issue, officers approved loans they would have questioned under manual review. The model was technically malfunctioning, but the actual failure was human. The loan officers had ceded their judgment to the system. The control is not technical. It is procedural: require documented human rationale for a sample of AI-supported decisions, and audit whether the rationale demonstrates independent judgment or simply restates the AI’s recommendation.

Category 2: Discrimination

Five incident types address unfair or unequal treatment produced by AI systems.

Bias in AI outputs (internal and external) occurs when models produce systematically biased predictions or recommendations. This is the broadest discrimination incident type and encompasses statistical bias embedded in model outputs that disadvantages specific groups.

Exposure to toxic content (external only) occurs when AI systems expose users to harmful, abusive, unsafe, or inappropriate content. Content recommendation systems, generative AI outputs, and AI-moderated platforms all carry this risk. The loss is borne by the affected users, but regulatory and reputational consequences flow back to the organization.

Lack of diversity in AI development (internal and external) occurs when homogeneous development teams build systems that reflect their own perspectives and blind spots. This is a root cause incident type. It does not produce harm directly but creates the conditions for bias, unfair discrimination, and unequal performance across groups.

Unequal performance across groups (internal and external) occurs when AI systems deliver different levels of accuracy, reliability, or quality for different user populations. A facial recognition system that works well for some skin tones and poorly for others. A speech recognition system that understands some accents and fails on others. The performance disparity itself is the incident, regardless of whether it results from intentional design or data limitations.

Unfair discrimination (internal and external) occurs when AI systems treat individuals or groups unfairly in consequential decisions. This goes beyond statistical bias in outputs to encompass the downstream effects: denied loans, rejected applications, misclassified individuals, or misrepresented groups.

Original implementation tip: The discrimination incident type that is hardest to detect is unequal performance across groups, because standard accuracy metrics can mask it completely. A model with 92% overall accuracy might have 97% accuracy for the majority population and 74% accuracy for a minority group. The aggregate metric looks fine. The disaggregated metrics reveal a serious problem. When I audit AI systems for discrimination risk, I require performance metrics disaggregated by every protected characteristic available in the data. If protected characteristics are not in the data, which is common, I require proxy analysis using correlated variables. The first time you disaggregate your model’s performance metrics, you will almost certainly find disparities you did not know existed.

Category 3: Disinformation Warfare

Three incident types address AI’s role in the information environment.

Disinformation and influence at scale (internal and external) occurs when AI systems enable large-scale manipulation of public opinion. This includes using AI to generate convincing fake content, automate social media manipulation, or conduct targeted influence campaigns. Internally, organizations face risk when their AI tools are misused for this purpose. Externally, society bears the cost of degraded public discourse.

False or misleading information (internal and external) occurs when AI systems generate or spread incorrect or deceptive information. This includes hallucination in large language models, inaccurate summaries, fabricated citations, and confidently stated falsehoods. Unlike deliberate disinformation, this often results from model limitations rather than malicious intent, but the impact on users who rely on the information is the same.

Pollution of information ecosystem (external only) occurs when AI-generated misinformation accumulates at sufficient scale to undermine shared reality. Filter bubbles, echo chambers, and the displacement of human-created content by AI-generated content of unknown reliability all contribute to this systemic effect.

Original implementation tip: False or misleading information is the disinformation incident type with the most immediate organizational liability, particularly for companies deploying generative AI in customer-facing applications. I advised a professional services firm that deployed a generative AI assistant to help clients navigate regulatory requirements. Within the first month, the assistant fabricated a regulation that did not exist and cited it confidently to a client. The client made a business decision based on the fabricated guidance. The firm’s liability exposure from that single incident exceeded the entire annual budget for their AI program. The control that would have prevented this is output verification: for any generative AI system providing factual information to external users, implement a verification layer that checks generated claims against an authoritative source before presenting them. This adds latency and cost. It also prevents lawsuits.

Category 4: Economic Displacement

Nine incident types address AI’s macroeconomic and organizational effects, making this the largest incident category.

Changes in employment patterns (internal and external) covers reduced quality of employment and increased exploitation of workers as AI reshapes job roles. Competitive dynamics (internal only) addresses the organizational risk from racing to deploy AI systems before they are safe, a pattern that increases the probability of releasing error-prone systems. Disruption of traditional industries (internal and external) covers economic instability when AI displaces established business models.

Economic and cultural devaluation of human effort (internal and external) occurs when AI-generated output reduces the perceived or actual value of human-created work. This affects pricing, employment, and professional identity across creative, analytical, and service industries.

Environmental harm (external only) covers the energy consumption, water usage, and carbon emissions from training and operating large AI systems. Governance failure (internal and external) occurs when regulatory frameworks cannot keep pace with AI development, creating gaps in oversight.

Increased inequality and decline in employment quality (internal and external) addresses the broader societal pattern of AI benefits accruing to capital owners while labor bears displacement costs. Job displacement and economic disruption (internal and external) covers direct job losses and industry disruption. Power centralization and unfair distribution of benefits (external only) addresses the concentration of AI capabilities and their economic benefits among a small number of organizations.

Original implementation tip: Of the nine economic displacement incident types, governance failure is the one that creates the most direct and immediate organizational risk, because it applies to every organization deploying AI, regardless of industry or scale. Governance failure is not just about regulators failing to keep pace with technology. It is also about your organization failing to build internal governance that compensates for regulatory gaps. I worked with a technology company that was deploying AI across 14 use cases with no centralized governance body, no standardized risk assessment process, and no consistent documentation requirements. Each team made independent decisions about model deployment, monitoring, and retirement. When the EU AI Act requirements became concrete, the company had no way to determine which of their systems qualified as high-risk, what documentation existed for each system, or who was accountable for compliance. They spent 11 months and significant resources building governance retroactively that would have cost a fraction to build proactively. If your organization deploys AI and does not have a governance framework, this is your highest-priority incident type to address. Not because governance failure is the most dramatic risk, but because its absence makes every other risk harder to manage.

Category 5: Exploitation

Two incident types address deliberate misuse of AI for harm.

AI weaponization (external only) covers the use of AI systems to develop cyber weapons or tools capable of mass harm. This is primarily a societal risk but creates organizational exposure when an organization’s AI tools or models are repurposed for weaponization by third parties.

Fraud, scams, and targeted manipulation (external only) covers the use of AI to conduct fraud, run scams, or manipulate individuals through personalized deception. AI-generated deepfake voices used in CEO fraud, AI-crafted phishing messages personalized from scraped data, and AI-assisted identity theft all fall here.

Category 6: Malicious Actors and Misinformation

Three incident types address AI-enabled attacks and synthetic media.

AI-powered phishing and social engineering (external only) covers the use of AI to create sophisticated, personalized phishing attacks and social engineering campaigns. AI enables attackers to generate convincing communications at scale, personalized to each target using publicly available information.

Use of AI for social engineering (external only) is a related but broader category covering all uses of AI to manipulate human behavior for unauthorized access or information disclosure.

Deepfakes and AI-generated content (external only) covers AI-generated synthetic media used to spread misinformation, impersonate individuals, or manipulate public opinion. This includes fake video, audio, images, and text that are increasingly difficult to distinguish from authentic content.

Category 7: Privacy Infringement

Five incident types address AI’s impact on personal data and privacy.

AI system security vulnerabilities and attacks (external only) covers exploitation of vulnerabilities in AI systems leading to unauthorized access, data breaches, or system manipulation causing unsafe outputs.

Collection of personal data (external only) covers AI systems that collect personal data without adequate consent. This includes passive data collection through AI-powered sensors, inference of personal characteristics from behavioral data, and collection that exceeds stated purposes.

Compromise of privacy (external only) occurs when AI systems memorize and leak sensitive personal data, or infer private information about individuals without consent. This is distinct from data breaches because the privacy compromise occurs through the model’s normal operation, not through a security failure.

Data breaches and unauthorized access (external only) covers traditional security incidents applied to AI contexts, including unauthorized access to training data, model weights, or inference logs containing personal information.

Surveillance and monitoring (external only) covers AI-powered surveillance that erodes trust and creates unease among individuals and communities. Facial recognition in public spaces, behavioral monitoring in workplaces, and predictive policing systems all carry this risk.

Original implementation tip: Compromise of privacy is the privacy incident type that is most specific to AI and least covered by traditional privacy controls. A large language model can memorize and reproduce fragments of its training data, including personal information, in its outputs. This is not a data breach in the traditional sense. No attacker exploited a vulnerability. The model simply learned its training data too well and reproduces it when prompted in certain ways. Traditional privacy controls focus on securing data at rest and in transit. They do not address data that is encoded in model weights. The control for this risk is differential privacy during training (adding noise to prevent memorization of individual data points) combined with output filtering that detects and blocks personal information in model responses. If your AI system was trained on data containing personal information, this incident type applies to you.

Category 8: Value Misalignment

Seven incident types address fundamental alignment between AI systems and human values.

AI possessing dangerous capabilities (external only) covers AI systems that develop or access capabilities increasing their potential for mass harm. This is an emerging and contested risk category, but it is increasingly relevant as AI systems become more capable.

AI pursuing its own goals in conflict with human goals (external only) covers AI systems acting contrary to the intentions of their designers or users. This ranges from reward hacking in reinforcement learning systems (achieving the stated objective through unintended means) to more speculative scenarios of advanced AI systems developing emergent goals.

AI system reliability and maintainability (internal and external) covers systems that are not reliable or maintainable, leading to errors and failures with significant consequences. This is particularly critical in applications requiring moral reasoning or operating in safety-critical environments.

Lack of accountability (internal and external) occurs when AI decision-making processes have no clear accountable party, leading to situations where harmful outcomes cannot be attributed, corrected, or prevented from recurring.

Lack of capability or robustness (internal and external) covers AI systems that fail under varying conditions. A model that works in testing but fails in production, a system that degrades when input distributions shift, or an application that produces errors under edge cases all represent this incident type.

Lack of explainability (internal and external) occurs when AI systems cannot explain their decisions to stakeholders who need to understand them, whether those stakeholders are regulators, affected individuals, or internal decision-makers.

Lack of transparency or interpretability (internal and external) covers broader challenges in understanding AI decision-making processes, leading to difficulty enforcing compliance, holding actors accountable, and identifying errors.

Original implementation tip: The value misalignment incident type I find most practically relevant for organizations today, the one that is neither speculative nor distant, is lack of accountability. Every AI failure I have investigated has had an accountability gap at its root. Not the absence of a responsible person in an organizational chart, but the absence of a person who knew they were responsible, had the authority to act, and had the information needed to act in time. The control is deceptively simple: for every production AI system, publish an accountability card that names the individual accountable for model performance, the individual accountable for data quality, the individual accountable for compliance, and the individual accountable for incident response. Post these accountability cards where the operations team can see them. Update them when people change roles. Test them by calling the named individuals during a tabletop exercise and verifying they know they are accountable and know what to do.

Part 2: Direct Loss Types

The incident taxonomy tells you what can happen. The direct loss taxonomy tells you what it costs. These 15 loss types map to specific financial line items that appear in budgets, financial statements, and insurance claims. They give your risk quantification the precision needed for credible Monte Carlo simulation and ROI analysis.

Five domains organize the 15 loss types.

Domain 1: Compliance Losses

Four loss types address the financial consequences of regulatory and legal exposure.

Regulatory fines cover penalties for violating AI regulations like the EU AI Act, privacy laws like GDPR, or sector-specific requirements. They also cover sanctions for data breaches, discriminatory outcomes, or copyright infringements produced by AI systems. These are typically the most visible AI losses because they are public, quantifiable, and reported.

Legal compensations cover settlement payments to affected parties for harm caused by AI malfunctions or decisions. This includes attorney fees and court costs for defending lawsuits from individuals or groups. Unlike regulatory fines, which are imposed by authorities, legal compensations arise from private litigation. They can be larger than fines and take longer to resolve.

Contractual credits cover service credits issued to customers when AI performance falls below guaranteed levels. Refunds and discounts applied for missed availability or accuracy commitments. These losses are often overlooked in risk assessments because they are managed by commercial teams, not risk teams, but they can be significant for organizations selling AI-powered services.

Legal response costs cover external legal counsel fees for investigating and responding to AI-related claims, as well as internal legal team costs for compliance reviews and regulatory correspondence. These costs are incurred regardless of whether the organization is ultimately found liable.

Control remediation covers costs to fix governance gaps identified in failed AI audits. This includes documentation, implementation, and certification expenses for new compliance controls and frameworks. This loss type often surprises organizations because it represents the cost of building governance they should have built proactively.

Original implementation tip: When estimating compliance losses for risk quantification, the most common error is using historical fine amounts as the basis for estimates. Historical data underestimates future exposure for two reasons. First, AI-specific regulations like the EU AI Act establish fine structures that far exceed previous penalties: up to 35 million euros or 7% of global annual turnover for certain violations. Second, regulatory enforcement of AI is in its early stages. The fines imposed in 2025 and 2026 will set precedents that do not yet exist in historical data. For AI compliance loss estimation, use the maximum penalty structures defined in applicable regulations as the upper bound of your range, not historical fine amounts. Your calibrated experts should estimate the probability of enforcement action and the likely penalty within the regulatory range, but the range itself should reflect the legal maximum, not past experience.

Domain 2: IT/Technical Losses

Three loss types address the costs of technical remediation and infrastructure.

Data regeneration covers costs to rebuild training datasets when data becomes corrupted, poisoned, or drifted beyond usability. This includes expenses for new data collection, labeling, cleaning, and validation. Data regeneration is expensive because high-quality training data is the most time-consuming and labor-intensive component of AI development. Rebuilding a corrupted training dataset can take months and cost more than the original data preparation.

Algorithm remediation covers engineering costs to retrain models that produce biased or inaccurate predictions. This includes compute resources for retraining, testing expenses for validation, and the data science team time required to diagnose the root cause, design the fix, and verify the corrected model’s performance. For complex models, remediation can require multiple retraining cycles.

Infrastructure overruns cover unexpected cloud computing and storage costs from inefficient AI resource usage. Emergency scaling expenses when systems face performance bottlenecks or capacity issues. AI workloads are computationally intensive and unpredictable. A model retraining job that runs longer than expected, a sudden spike in inference requests, or an unoptimized training pipeline can generate infrastructure costs that significantly exceed budget.

Original implementation tip: Algorithm remediation is the technical loss type most consistently underestimated in risk assessments. Teams estimate the compute cost of retraining but forget the human costs: the data science team time to diagnose the root cause (which can take weeks for complex model failures), the opportunity cost of pulling those data scientists off other projects, the testing and validation time for the remediated model, and the business cost of operating with a degraded model during the remediation period. When I help organizations estimate algorithm remediation costs, I use a formula that includes compute costs (typically the smallest component), data science team labor at fully loaded cost for the estimated remediation duration, lost productivity for the business processes that depend on the model during remediation, and any expedited procurement costs for additional compute resources or external expertise. The total is typically three to five times the compute cost alone.

Domain 3: Operational Losses

Five loss types address the business impact of AI failures on operations.

Decision errors cover financial losses from incorrect AI-driven business decisions made at scale. This includes costs of resource misallocation in operations, investments, or strategic planning based on flawed AI recommendations. The defining characteristic of decision error losses is scale. An AI system making thousands of decisions per day can accumulate significant losses before the error pattern is detected.

Operational inefficiency covers manual intervention costs when staff must correct or override AI outputs. Lost productivity from rework and staff time diverted to address AI failures. This loss type captures the ongoing drag on organizational performance that occurs when an AI system works poorly but not badly enough to take offline.

Development waste covers write-offs of failed AI projects that never reach production deployment. Sunk costs in licenses, development efforts, and procurement that yield no value. Industry estimates suggest that between 60% and 85% of AI projects fail to reach production. Each failed project represents development waste that should be included in the organization’s AI loss profile.

Business disruption covers revenue loss during downtime when AI-dependent processes stop functioning. Emergency replacement costs and lost transactions from service interruptions. This loss type is particularly relevant for organizations where AI systems sit in the critical path of revenue-generating processes.

Provider switching covers contract termination fees and cancellation penalties with current AI vendors. Migration costs, integration expenses, and negotiation time for new provider onboarding. This loss type is often triggered by other incidents, such as a vendor’s quality declining, a security breach at the vendor, or a strategic decision to reduce vendor dependency, but the switching costs themselves represent a distinct financial impact.

Original implementation tip: Development waste is the operational loss type with the highest aggregate financial impact across most organizations I work with, and it is almost never included in AI risk assessments because it is treated as a project management issue rather than a risk management issue. When I aggregate the fully loaded costs of failed AI projects across an organization, including salaries, compute resources, license fees, and opportunity costs, the total frequently exceeds the organization’s estimated exposure from all other AI risk scenarios combined. Include development waste in your loss taxonomy. Estimate it by multiplying the average fully loaded cost of an AI project by the historical failure rate. If you do not track your AI project failure rate, start. That number alone will change how your organization evaluates AI investments.

Domain 4: Revenue Losses

Two loss types address top-line financial impact.

Customer churn covers lost revenue from customers leaving after negative AI experiences or failures. Acquisition costs for replacing churned clients and margin erosion from retention efforts. This loss type has a compounding effect because the cost of acquiring a new customer is typically several times the cost of retaining an existing one.

Reputation damage covers brand value decline and crisis management costs following publicized AI incidents. Lost business opportunities and reduced market position from negative media coverage. This is the loss type most organizations acknowledge but least effectively quantify.

Original implementation tip: Reputation damage is the loss type I spent the most time helping organizations quantify, because it is the one where calibrated estimation is most valuable and most difficult. The approach that works is decomposition. Do not try to estimate “reputation damage” directly. Instead, estimate its measurable downstream effects. How many deals in the pipeline would be delayed or lost? (Estimate the pipeline value at risk.) How much would customer acquisition costs increase, and for how long? (Estimate the increment times the acquisition volume times the duration.) How much additional spending on PR and crisis management would be required? (Get a range from your communications team.) What revenue from existing contracts would be at risk of non-renewal? (Estimate the percentage of contracts with reputation-sensitive renewal decisions.) Add these components together. The total is more defensible than any direct estimate of “reputation damage” and more useful for risk quantification.

Connecting Incidents to Losses: The Traceability Requirement

The two taxonomies in this post are designed to work together. Each incident type produces one or more direct loss types. Mapping these connections creates the traceability needed for effective risk quantification.

Take a concrete example. The incident type “bias in AI outputs” (Discrimination category, internal and external) can produce the following direct losses: regulatory fines (if the bias violates the EU AI Act or fair lending laws), legal compensations (if affected individuals or groups file lawsuits), algorithm remediation (costs to diagnose and fix the biased model), control remediation (costs to build governance controls that should have prevented the bias), customer churn (if the affected population includes customers who leave), and reputation damage (if the bias becomes public).

Each of these loss types has a different magnitude, different timing, and different probability. Regulatory fines are large but require a regulatory investigation, which may take months. Legal compensations can exceed fines but require plaintiffs to organize and file. Algorithm remediation costs are incurred immediately but are typically the smallest component. Reputation damage may or may not materialize depending on media attention.

Without this incident-to-loss mapping, your risk quantification combines everything into a single “impact” number that is neither precise enough for Monte Carlo simulation nor useful enough for control investment decisions.

Original implementation tip: Build an incident-to-loss mapping matrix for every AI system in your portfolio. Down the left side, list every applicable incident type from this taxonomy. Across the top, list every applicable direct loss type. In each cell, indicate whether the incident could produce that loss type, and if so, provide a rough magnitude range. This matrix becomes the foundation for your FAIR-based risk quantification. When you estimate the impact component of a risk scenario, you are not estimating a single number. You are estimating the aggregate of all applicable loss types for the specific incident. This granularity dramatically improves the quality of Monte Carlo simulation inputs and the credibility of the outputs.

Internal Versus External: Why the Distinction Matters

The taxonomy classifies each incident type as producing internal losses, external losses, or both. This classification is not academic. It determines which assessment methodology applies.

Internal losses are costs borne by the organization. They are addressed through risk assessments that quantify exposure to the organization and inform control investment decisions. When you run a Monte Carlo simulation to calculate annualized loss exposure, you are modeling internal losses.

External losses are harms borne by individuals, communities, or society. They are addressed through impact assessments that evaluate potential harm to affected parties and inform responsible AI decisions. External losses may or may not create financial exposure for the organization (through fines, lawsuits, or reputation damage), but they matter independently because they represent real harm to real people.

Some incident types produce only internal losses. Competitive dynamics, for example, creates risk for the organization through unsafe AI deployment but does not directly harm external parties. Some produce only external losses. Surveillance and monitoring, for example, harms individuals and communities but may not create direct financial losses for the organization until it triggers regulatory action or public backlash.

Most incident types produce both. Bias in AI outputs, for example, creates internal losses through remediation costs and external losses through discriminatory harm to affected individuals.

Mature AI risk programs assess both dimensions for every applicable incident type. Immature programs assess only internal losses and are surprised when external harms generate regulatory, legal, or reputational consequences they did not anticipate.

Original implementation tip: The practical implication of the internal/external distinction is that you need two different assessment processes, and they should involve different people. Internal loss assessment is a financial exercise led by risk managers, using techniques like FAIR quantification and Monte Carlo simulation. External impact assessment is an ethical and societal exercise that should involve ethicists, affected community representatives, legal experts, and domain specialists, not just risk managers. I have seen organizations try to combine both assessments into a single process run by the risk team. The financial analysis crowds out the impact analysis every time. When a risk manager and an ethicist are in the same room, the conversation gravitates toward quantifiable financial exposure because that is what the risk manager knows how to discuss. Keep the assessments separate. Conduct them with different teams. Then combine the findings in a governance review where both perspectives inform the decision.

Cross-Cutting Implementation Tips

Four principles apply across both taxonomies.

Use the incident taxonomy to audit your risk register. Take every risk in your current AI risk register and map it to the incident types in this taxonomy. If a risk in your register maps to multiple incident types, decompose it. If incident types in this taxonomy have no corresponding risk in your register, you have a gap. This audit typically reveals that existing risk registers are too coarse and miss 40% to 60% of applicable incident types.

Original implementation tip: When I conduct this audit with organizations, the most common gaps are in the cognitive degradation and value misalignment categories. Risk teams are comfortable identifying bias, security, and privacy risks. They are much less comfortable identifying risks related to overreliance on AI, loss of autonomy, lack of explainability, or accountability gaps. These “softer” incident types are not soft in their consequences. Lack of accountability contributed to more AI incidents I have investigated than any specific technical failure. Include the full taxonomy in your audit, not just the categories that feel comfortable.

Use the direct loss taxonomy to improve your quantification. For every risk scenario you quantify, decompose the impact into the specific direct loss types that apply. Estimate each loss type separately using calibrated ranges. Then aggregate them for the total impact distribution. This produces more accurate estimates than a single “impact” range because subject matter experts can estimate specific loss types more credibly than they can estimate total impact.

Original implementation tip: When conducting estimation workshops, present loss types one at a time, not all at once. Ask experts to estimate regulatory fine exposure, then legal compensation exposure, then algorithm remediation costs, then customer churn impact, and so on. This prevents anchoring, where the first estimate influences all subsequent estimates, and produces wider, more honest ranges. The first time I tried this approach, the aggregate impact estimate was 2.3 times higher than the single “total impact” estimate the same experts had provided before decomposition. Decomposition reveals exposure that aggregation hides.

Update both taxonomies as the AI landscape evolves. New incident types emerge as AI capabilities expand. Generative AI created incident types like hallucination and prompt injection that did not exist five years ago. Autonomous agents will create new incident types that do not exist today. Review and update your taxonomies at least annually, and whenever a significant new AI capability is deployed within your organization.

Align your taxonomy with regulatory requirements. The EU AI Act, NIST AI RMF, ISO 42001, and ISO 23894 each reference specific types of AI-related harms and losses. Map your taxonomy to the categories used by the regulations that apply to your organization. This ensures that your risk assessments address every category a regulator will ask about and that your documentation uses consistent terminology.

Key References and Standards

This loss taxonomy draws from and aligns with the following authoritative frameworks.

ISO/IEC 42001:2023 for AI management system requirements covering governance and accountability for AI-related incidents and losses.

ISO/IEC 23894:2023 for AI risk management guidance, including classification of AI-specific risks and impacts.

ISO/IEC 27005:2022 for the information security risk management process, including loss event classification.

EU AI Act (Regulation 2024/1689) for the regulatory framework defining prohibited practices, high-risk requirements, and penalty structures for AI systems.

NIST AI RMF (AI 100-1) for the AI risk management lifecycle including harm categorization.

FAIR (Factor Analysis of Information Risk) for quantitative loss modeling taxonomy and methodology.

OECD AI Principles for the international framework addressing AI-related societal impacts.

UNESCO Recommendation on the Ethics of Artificial Intelligence for the broader ethical framework covering cognitive, social, and economic impacts.

MIT AI Risk Repository for the comprehensive academic catalog of AI risk incident types that informed several categories in this taxonomy.

Making These Taxonomies Operational

Organizations that file these taxonomies as reference documents will continue making the same mistakes. Their risk assessments will use generic impact categories that obscure actual exposure. Their incident response plans will not cover incident types they have not named. Their loss estimates will undercount by factors of two to five because they have not decomposed generic “impact” into specific loss types. When an AI incident occurs, they will discover that they cannot quantify their exposure because they never built the vocabulary to describe it precisely.

Organizations that operationalize these taxonomies will build risk assessments that distinguish between 37 distinct incident types and 15 direct loss categories. They will estimate exposure with the granularity needed for credible Monte Carlo simulation. They will map incidents to losses to controls, creating traceability that survives regulatory scrutiny. Their boards will understand AI risk in specific financial terms because the risk team can articulate exactly what kinds of costs would appear and on which financial lines.

The precision of your AI risk management cannot exceed the precision of your loss taxonomy. Name the losses specifically, or accept that your risk numbers are wrong.

Which loss types in this taxonomy are missing from your current AI risk assessments? Start with the ones you have never estimated. Those are where your biggest quantification gaps live.

The AI Risk Taxonomy Most Organizations Never Build

Thu, 12 Mar 2026 00:00:00 +0000

Top Risk Scenarios and Controls That Actually Protect Your AI Project

A risk register with 15 vaguely worded AI risks and a color-coded heat map is not a taxonomy. It is a liability.

I reviewed an organization’s AI risk assessment last year that listed “AI bias” as a single risk with a “medium-high” rating. That was it. No decomposition into the dozen distinct ways bias manifests. No distinction between bias in training data, bias from proxy variables, bias from temporal misalignment, or bias from feedback loops. No specific controls mapped to specific failure modes. When their credit model produced discriminatory outcomes six months later, nobody could trace the failure to a gap in their controls because their taxonomy was too shallow to reveal where the gaps were.

The difference between organizations that manage AI risk effectively and those that just talk about it comes down to granularity. You need a taxonomy that decomposes AI risk into specific, actionable scenarios, each linked to a named control with concrete activities. This post provides exactly that: a structured taxonomy of 100 AI risk scenarios across 14 domains, with recommended controls mapped to COBIT 2019 governance objectives. Every scenario follows a consistent structure: what can go wrong, why it matters, and what to do about it.

This is a long reference piece. Use it as a working document, not a single-sitting read.

Why Most AI Risk Taxonomies Fail

The typical AI risk taxonomy fails for three reasons.

First, it operates at the wrong altitude. “Model risk” is not a scenario. It is a category that contains dozens of scenarios, each with different causes, different impacts, and different controls. When you treat a category as a scenario, your controls become generic and your residual risk unmeasurable.

Second, it ignores organizational and process risks. Most AI taxonomies obsess over technical risks like adversarial attacks and data poisoning while overlooking the governance, people, and operational risks that cause the majority of real-world AI failures. A model that degrades because nobody owns monitoring in production is not a technical failure. It is a governance failure.

Third, it lacks traceability from risk to control. Identifying a risk without mapping it to a specific, implementable control activity is an academic exercise. The taxonomy must create a direct line from “what could go wrong” to “what are we doing about it” to “how do we verify it is working.”

The taxonomy presented here addresses all three failures. It spans 14 domains from strategy through business continuity, covers 100 distinct scenarios, and links each one to a named control with specific activities. I have organized it to follow the natural lifecycle of AI in an enterprise, from strategic planning through development, deployment, operations, and ongoing governance.

Original implementation tip: When I first built an AI risk taxonomy for a European bank, I started with the technical risks because that is where the AI team’s attention naturally went. We ended up with 40 technical scenarios and 5 organizational ones. After the first major incident, which was caused by unclear model ownership between data science and IT operations, we realized our taxonomy was inverted. The organizational and governance risks caused more actual damage than the technical ones. Start your taxonomy with strategy, governance, and people risks. Then layer in the technical domains. This sequencing forces the right conversations early.

Domain 1: Business Value Risks

Strategy risks sit at the top of the taxonomy because every other risk domain inherits from them. If your AI strategy is flawed, your technical controls cannot compensate.

Two scenarios define this domain.

The first is strategy deficiency. Wasted resources and reputational damage may occur when an organization lacks a clear enterprise-wide AI strategy, leading to inefficient investments and potential misuse of AI. This is a Priority 1 risk.

The recommended control is an enterprise AI strategy. Develop and put in place a comprehensive AI strategy aligned with overall business objectives. Create clear guidelines for AI adoption and integration across departments. Establish governance structures with defined roles, responsibilities, performance metrics, and risk management protocols. Document policies and maintain evidence of governance through reports and records. Review and update the strategy regularly to reflect changes in technology, business needs, and regulatory requirements. Communicate the strategy across the organization to achieve alignment and stakeholder buy-in.

The second scenario is misaligned strategy. Missed opportunities may occur when insufficient stakeholder engagement leads to AI systems that do not support business goals or expose the organization to unacceptable risks. Also Priority 1.

The recommended control is stakeholder alignment. Establish stakeholder engagement processes to ensure AI systems align with business goals. Develop communication protocols and document policies for continuous alignment. Collect and maintain meeting records, stakeholder feedback, and communication logs as evidence.

Original implementation tip: The strategy risk I see most often is not the absence of a strategy. It is the presence of multiple competing strategies. The data science team has a roadmap. The IT department has an automation strategy. The business units each have their own AI wishlists. These strategies contradict each other in ways nobody notices until budget conflicts or architectural incompatibilities surface months later. Before you write a strategy document, conduct a strategy reconciliation exercise. Collect every existing AI-related plan, roadmap, and initiative list across the organization. Map them on a single page. The conflicts will be immediately visible. Resolve those conflicts first. Then write the unified strategy.

Domain 2: Governance Risks

Governance is where principles become operational. Eight scenarios span this domain, and most organizations have gaps in at least half of them.

Misaligned ethics is a Priority 1 scenario. Reputational damage may occur when AI decisions conflict with organizational cultural and ethical values, leading to poor decisions, negative public perception, and legal repercussions. The control is responsible AI principles: develop AI ethics guidelines, establish an ethics review board, and integrate ethical considerations into the design, development, and deployment of AI systems.

Overconfidence in automation is equally critical. Wasted resources result from unrealistic expectations about AI capabilities, leading to disappointment, wasted investments, and erosion of trust. The control is capability and limitations communication: communicate the limitations and potential risks of AI technologies and avoid overstating capabilities to ensure realistic expectations and informed decision-making.

Governance erosion occurs when AI negatively impacts existing governance mechanisms, reducing control over data processing and increasing breach risk. The control is control integration: update existing governance frameworks to incorporate AI-specific considerations and ensure alignment with established policies and risk management protocols.

Compliance failure carries the most immediate financial consequences. Regulatory penalties and reputational damage result from non-compliance with internal or external AI requirements. The control is compliance audit: regularly test, audit, monitor, and assess AI system compliance with internal policies, external regulations, and ethical guidelines, and report findings to relevant stakeholders.

Vendor lock-in limits flexibility when exit strategies for AI systems are absent. The control is exit planning: include exit strategy considerations in the design and procurement of AI systems, ensuring the ability to migrate to alternative providers.

Three Priority 2 governance scenarios round out this domain. Trust deficit limits innovation when organizations lack trust in AI technologies. The control is an AI framework that documents and communicates limitations and capabilities, provides clear explanations of AI decisions, and establishes processes for independent verification. Communication breakdown results from a lack of common language for AI concepts. The control is AI glossary management: develop and maintain a glossary of AI terms and ensure consistent terminology across the organization. Ownership vacuum leads to unauthorized AI development and security breaches when ownership and operating models are undefined. The control is operating model definition: establish clear roles, responsibilities, and accountabilities for AI initiatives, including appropriate segregation of duties.

Original implementation tip: The governance risk that causes the most silent damage is the ownership vacuum. I worked with a technology company where three separate teams claimed partial ownership of a production ML model. Data science owned the algorithm. Platform engineering owned the infrastructure. The business unit owned the use case. Nobody owned the model in production. When performance degraded, each team assumed another team was monitoring it. The model served degraded predictions for 11 weeks before a customer complaint triggered investigation. Fix this by creating a RACI matrix for every production AI system that names one individual, not a team, as the accountable party for model performance in production. One name. Not a distribution list.

Domain 3: Decision-Making Risks

Two Priority 1 scenarios address how AI risk integrates into enterprise decision-making.

Risk integration gap occurs when organizations fail to integrate risk assessment and controls into the AI framework. Unidentified or unmitigated risks result, along with potential compliance violations and reputational damage. The control is mandatory risk and impact assessments: create and enforce a comprehensive policy mandating systematic identification, evaluation, and management of AI-related risks. Include quantitative model impact assessments with statistical analyses of threat prevalence and potential losses. Integrate these processes with existing framework protocols covering confidentiality, integrity, availability, compliance, contracts, and responsible AI principles.

AI model risk exposure results from outdated quantitative model risk management practices. The control is a quantitative risk model: develop and maintain up-to-date practices for AI model risk management, conduct impact assessments and statistical analyses to evaluate accuracy and reliability, address model metrics and acceptance criteria, and incorporate regular reviews of data, operational practices, and cybersecurity controls.

Original implementation tip: Most organizations attempt to integrate AI risk into their existing risk framework by adding a few AI scenarios to their enterprise risk register. This approach fails because the existing register was designed for risks that behave differently. AI risks are dynamic. A model that was within tolerance last quarter may be outside tolerance this quarter because the underlying data distribution shifted. Instead of simply adding AI rows to your existing register, create a parallel cadence of AI-specific risk reviews that feed into the enterprise register. Monthly AI risk reviews that update quarterly enterprise risk reports. This gives AI risks the attention frequency they require while maintaining integration with enterprise governance.

Domain 4: People Risks

Three scenarios cover the human element of AI risk.

Resource misalignment (Priority 1) occurs when unclear resourcing requirements in the AI strategy lead to staffing inefficiencies. The control is staff planning: define and document human resource requirements, including recruitment, role profiles, training, retention strategy, and third-party involvement, in alignment with the AI strategy and roadmap.

Talent flight (Priority 1) results when poor development and retention of human talent produces AI solutions misaligned with organizational values. The control is talent alignment: establish HR processes to recruit, develop, and retain talent aligned with the AI strategy, including continuous professional development and performance evaluations.

Knowledge deficit (Priority 1) creates ineffective AI operations and poor incident response when IT knowledge is not retained and developed. The control is knowledge continuity: assign and document specific individuals to fulfill business-as-usual roles and sustainment functions. Ensure ongoing knowledge retention through formal knowledge management practices, continuous training, and documentation of key processes and incidents.

Original implementation tip: The knowledge deficit risk is particularly dangerous with AI systems because the knowledge required is specialized and often held by a single individual. I have seen organizations where one data scientist understood the feature engineering pipeline, and when that person left, nobody could retrain the model. The entire production system became fragile overnight. For every critical AI system, maintain a “bus factor” register. For each key knowledge area, list how many people can perform the function. If the number is one, you have a Priority 1 risk that requires immediate cross-training or documentation. Yeah, this sounds obvious. But count how many of your production AI systems depend on a single person’s knowledge. The number will concern you.

Domain 5: Architecture Risks

Architecture risks span seven scenarios across three priority levels.

Unexplainability (Priority 1) is the inability to understand or explain AI decisions due to missing functionality. The control is explainability by design: integrate explainability as a functional requirement in design, build, and testing phases. Ensure explanations are clear and accessible, with documentation of explainability features and traceability of decision-making processes.

Incompatibilities (Priority 1) cause operational issues from integration, scalability, and compatibility problems. The control is compatibility testing: develop testing procedures ensuring the AI model is compatible with the production environment, scalable to meet business needs, and integrated with other systems. Perform thorough compatibility testing across software, hardware, and network environments. Conduct scalability assessments including stress testing. Develop standardized integration protocols covering data formats, API usage, and security requirements.

Misaligned architecture (Priority 2) prevents unified automation when AI architecture is undefined. The control is architecture alignment: define and document an enterprise AI architecture including preferred technologies, design concepts, logging protocols, security controls, and monitoring requirements.

Segregation deficiency (Priority 2) creates security and data integrity losses in cloud or multi-tenant environments. The control is architectural segregation: define IT architecture principles enforcing segregation of AI system components and data from other infrastructure.

Unavailability (Priority 2) disrupts business operations due to insufficient AI system resilience. The control is high availability: define and monitor availability metrics, establish redundancy plans, and test for system reliability.

Ineffective security (Priority 3) results from failing to embed security by design. The control is security by design: incorporate security principles into the development methodology, ensuring all components adhere to established security standards.

License noncompliance (Priority 3) creates legal and financial exposure. The control is license management: establish a license management system ensuring appropriate licenses and timely renewals for all AI system components.

Original implementation tip: Explainability by design is the architecture control most frequently treated as an afterthought. Teams build complex ensemble models or deploy large language models, and only when a regulator or auditor asks “how does this model make decisions” do they realize explainability was never a requirement. Retrofitting explainability onto a deployed model is expensive and sometimes impossible. Add explainability to your definition of done for model development. If the development team cannot demonstrate how the model produces its outputs before deployment, the model does not deploy. This one requirement, enforced consistently, prevents an entire category of compliance and trust risks.

Domain 6: Lifecycle Risks

This is the largest domain, spanning 10 scenarios, because the AI lifecycle from data to deployment contains the most failure points.

Poor hypothesis (Priority 1) produces unreliable outcomes from inadequate governance around hypothesis development. The control is hypothesis testing: establish governance controls requiring approval of hypotheses based on predefined criteria, with regular reviews for ongoing relevance.

Poor algorithms (Priority 1) leads to ineffective performance. The control is algorithmic controls: develop governance policies for algorithm development and maintenance, including validation and testing procedures with regular reviews.

Flawed logics (Priority 1) produces unreliable outputs from inaccurate model parameters. The control is logic validation: establish rigorous logic validation and testing procedures, with approval requirements for any changes to model logic.

Data accuracy failures (Priority 1) cause erroneous outputs. The control is data accuracy verification: develop and enforce data accuracy verification standards including validation, error detection, and correction before and during model use, with continuous monitoring and automated alerts.

Six Priority 2 scenarios complete this domain. Unallocated roles create regulatory and operational failures through unclear data governance responsibilities. The control is data ownership: establish roles including data owners and stewards with regular audits. Data corruption results from unintended interactions between AI and other systems. The control is data integrity monitoring: implement controls to monitor data interactions with automated alerts for corruption incidents. Integration failure occurs from corrupted data inputs or outputs between systems. The control is integration testing: develop strict testing procedures with continuous monitoring for data anomalies. Poor data results from inadequate governance over learning and production data. The control is data governance: enforce quality checks with documented standards and periodic audits. Incomplete inputs lead to incorrect outcomes. The control is data completeness control: implement validation processes with protocols for handling incomplete datasets.

At Priority 3, inaccurate results from models not reflecting underlying parameters are addressed by model validation: comprehensive validation protocols including sensitivity analysis and performance benchmarks.

Original implementation tip: The lifecycle risk that consistently surprises organizations is data accuracy failure during the transition from development to production. The training data has been cleaned, validated, and verified. The model performs beautifully in testing. Then in production, the live data feed introduces formats, edge cases, and quality issues that never appeared in the training set. I watched a fraud detection model go from 94% accuracy in testing to 71% in the first week of production because the live transaction data contained encoding inconsistencies that the training data had been cleaned of. Build a data reconciliation step between your training pipeline and your production pipeline. Compare distributions, formats, and quality metrics between the data the model was trained on and the data it receives in production. Do this before go-live and continuously afterward.

Domain 7: Development Risks

Sixteen scenarios cover the development phase, making it the second largest domain.

At Priority 1, four scenarios demand immediate attention. Design flaw occurs when poor methodology is not consistently applied. The control is development standards: establish and maintain AI development standards integrated with broader development standards. Inaccurate model results from undefined model universe definition. The control is model universe: define and document the AI model universe including data sources, quality, transformations, and assumptions, updated regularly. Variable misalignment causes incorrect results from mistaking correlation for causality. The control is relationship modeling: establish quality controls ensuring relationships between variables are defined correctly, including interdependencies. Overfitting causes loss of reliability when models perform well on training data but poorly on new data. The control is overfitting mitigation: design algorithms for flexibility with documented testing and validation.

Learning bias (Priority 1) deserves special attention. Loss of accuracy and reliability occurs from data bias producing discriminatory outcomes. The control is bias mitigation: implement controls considering sensitivities across ethical, political, ethnic, racial, gender, and cultural groups, with documented evaluation processes and evidence of bias checks.

At Priority 2, five scenarios address operational development risks. To-be inaccuracy results from poor knowledge of desired processes. The control is to-be analysis: maintain documentation of user stories and end-to-end process flows with program sponsor approval. Insufficient segregation occurs when testing environments do not match production. The control is environment segregation: maintain separate development, QA/test, and production environments. Temporal misalignment causes accuracy loss when data time scales conflict. The control is synchronization verification: establish controls ensuring data source alignment with the AI system’s time scale. Data duplication produces inflated insights from processing duplicate data. The control is duplication mitigation: implement file and data validation checks with documentation. AutoML issues create complexity and lack of transparency. The control is AutoML guides: develop guidelines for automated machine learning use with regular complexity assessments and explainability tool integration.

At Priority 3, six scenarios cover remaining development risks. As-is ignorance results from poor knowledge of current processes. The control is as-is analysis: document pre-automation process narratives during the design phase. Undefined controls create vulnerabilities. The control is a control matrix covering all key areas. Control gap occurs when controls are not implemented in the developed solution. The control is control testing to verify processes align with design. Weak traceability compromises logging effectiveness. The control is bot identification with unique identifiers. Improper testing results from insufficient go-live strategy. The control is testing execution with comprehensive documentation. User acceptance deficiency results from inadequate business input. The control is test approvals with documented feedback and sign-off.

Original implementation tip: Variable misalignment, the correlation-versus-causation problem, is the development risk I find most often in production AI systems. And it is rarely caught by automated testing because the model’s statistical metrics look fine. A model might achieve high accuracy by using a variable that correlates with the target in training data but has no causal relationship. When the correlation breaks, which it eventually does, the model fails silently. The most effective countermeasure I have found is a mandatory “causal review” step in the development process where a domain expert, not a data scientist, reviews the feature set and challenges each variable’s causal relationship to the outcome. Data scientists are trained to find patterns. Domain experts are trained to question whether those patterns make sense. You need both perspectives before deployment.

Domain 8: Project Risks

Five scenarios cover project-level risks.

Operational misalignment (Priority 2) results from lacking strategic alignment between AI initiatives and organizational strategy. The control is a business case: establish a strategic alignment framework with formal approval and periodic review by stakeholders.

Problem mismatch (Priority 2) occurs when model design does not match the business problem. The control is iterative development: adopt approaches like Agile for continuous testing and refinement, with prototyping to identify mismatches early.

Management gap (Priority 2) results from poor project management methodology. The control is program management: implement project timelines, resource allocation, stakeholder engagement plans, and continuous alignment with business requirements.

Poor benefits (Priority 3) occurs when benefits management fails to track ROI. The control is impact value: develop a benefits management framework with metrics for short, medium, and long-term tracking.

Assurance deficit (Priority 3) results from lacking independent assurance. The control is assurance: engage an independent function to evaluate AI program setup, with regular reports on quality, costs, benefits, compliance, and internal control.

Original implementation tip: Problem mismatch is the project risk that wastes the most money. I worked with a retail organization that spent eight months building a demand forecasting model to solve what turned out to be a supply chain visibility problem. The model was technically excellent but solved the wrong problem. Their forecast accuracy improved by 15%, but the real issue was that they could not see inventory positions across warehouses in real time. The fix required a dashboard, not a model. Before approving any AI project, require the project team to answer one question in writing: “Why does this problem require machine learning, and what would the non-ML alternative look like?” If they cannot articulate why ML is necessary, there is a good chance a simpler solution would be more effective.

Domain 9: Operations Risks

Nine scenarios cover the operational phase where most AI failures actually manifest.

Performance drift (Priority 2) is the operational risk with the highest real-world impact. Model accuracy degradation from data drift and concept drift occurs when stability checks are insufficient. The control is stability monitoring: implement model stability checks requiring ongoing validation, benchmarking, and performance evaluation to detect drift.

Resource laxity (Priority 2) results from inadequate control over IT resource usage given AI’s unpredictable demands. The control is project monitoring: implement controls to monitor IT resource demands more closely than other systems.

Error oversight (Priority 2) leads to unauthorized changes and incidents from undetected errors. The control is incident management: establish a consistent approach with clear procedures, timely resolution, and integration with regular incident management.

Undetected error (Priority 2) causes delayed resolution from lacking procedures. The control is error resolution: perform timely exception processing with issue and performance monitoring.

Unsupported jobs (Priority 2) results from insufficient job monitoring. The control is job monitoring: monitor system jobs and interfaces ensuring completeness and timeliness.

Capacity issues (Priority 2) arise when availability and capacity management cannot meet evolving demand. The control is capacity management: implement availability and capacity management with scalability embedded in design.

At Priority 3, shadow AI is the scenario most organizations underestimate. Inability to ensure AI aligns with strategy and risk appetite occurs when the organization lacks an inventory of all AI solutions. The control is AI inventory: maintain a complete, up-to-date inventory of all AI platforms, solutions, and use cases, including dependencies and ownership.

IP loss (Priority 3) occurs when AI system intellectual property held by third parties is at risk. The control is IP protection: establish a repository of relevant IP, accessible in-house, secured with regular backups.

AI component blindness (Priority 3) results from lacking understanding of IT components and relationships. The control is configuration management: establish a configuration management database fed through change management.

Original implementation tip: Shadow AI is growing faster than most governance teams realize. Every time an employee uses ChatGPT to draft a customer response, builds a quick predictive model in a Jupyter notebook, or connects a third-party AI tool to company data through a browser extension, they create shadow AI. I conducted a shadow AI audit at a financial services firm last year. The governance team believed they had 12 AI systems in production. We found 47 AI tools and models being used across the organization, most without any risk assessment, data governance, or access controls. The 35 unknown systems included four that processed customer PII. Start your shadow AI inventory not by asking teams to self-report, which underestimates the problem, but by auditing network traffic, SaaS subscriptions, cloud resource usage, and API calls for AI-related activity.

Domain 10: Monitoring Risks

Four scenarios address the monitoring function that keeps deployed AI systems safe.

Outcome blindness (Priority 1) occurs when AI system behavior is not monitored against business and ethical requirements. The control is outcome monitoring: implement regular review of AI system outcomes using data analytics to ensure performance aligns with requirements. Maintain audit trails and ensure controls operate at the same pace as monitored activities.

Monitoring ineffectiveness (Priority 1) reduces operational effectiveness from inadequate monitoring. The control is operational monitoring: develop a real-time monitoring and alerting framework to detect anomalies, establish KPIs and KRIs as the basis for effective monitoring, and trigger alerts followed by documented follow-ups.

Undetected issues (Priority 1) cause financial losses and compliance fines from lacking post-deployment monitoring. The control is post-deployment monitoring: develop a monitoring framework defining specific metrics, thresholds, and alerts. Implement automated tools for continuous real-time tracking. Define key performance indicators and review them regularly against business objectives.

Control override (Priority 2) leads to financial loss when automated stop/loss controls fail. The control is automated stop/loss: design controls to halt unintended AI behavior with an override process for exceptions, assessing exceptions against risk appetite and business impact.

Original implementation tip: The monitoring risk that catches organizations off guard is the gap between monitoring cadence and AI decision speed. I worked with an organization that monitored their AI system’s output quality weekly. The system made 50,000 decisions per day. By the time they detected a quality degradation in their weekly review, the system had already made 350,000 decisions at reduced quality. Match your monitoring frequency to your decision frequency. If your model makes real-time decisions, you need real-time monitoring. If your model runs daily batch predictions, daily monitoring may suffice. But weekly monitoring for a real-time system is a control that exists on paper but provides no actual protection.

Domain 11: Security Risks

Seven scenarios span security from Priority 1 through Priority 3.

Lack of auditability (Priority 1) prevents validation of AI outcomes. The control is auditability: securely store and ensure timely retrieval of data and algorithms, comply with data privacy regulations, prevent data context loss, and apply the vault principle.

Unauthorized access (Priority 2) leads to inappropriate changes to AI learning and processing data. The control is data access: securely configure AI input datasets to prevent unauthorized changes with completeness and accuracy checks.

At Priority 3, five scenarios address specific security concerns. Security breach results from inconsistent security management. The control is cyber security: apply a consistent approach integrated with regular security processes, aligned with ISO 27001. Malware attack affects AI environment integrity. The control is malware protection: implement protection systems and monitor patches, protecting self-learning components against malicious attacks. Data breach occurs from insecure handling of temporary files. The control is encryption: encrypt code, data storage, and network communications. Vulnerability blindness results from undetected security weaknesses. The control is vulnerability testing: conduct periodic penetration tests and red-team reviews.

Original implementation tip: The security risk unique to AI that most security teams miss is the attack surface created by the model itself. Traditional security teams protect the infrastructure around the model, the servers, networks, APIs, and databases. But the model is an attack surface too. An adversary who can query a production model thousands of times can extract information about the training data through model inversion attacks. They can find decision boundaries through systematic probing. They can manipulate outputs through carefully crafted inputs. Your security testing must include model-specific attack scenarios, not just infrastructure penetration testing. If your red team does not include someone who understands adversarial machine learning, your testing has a blind spot.

Domain 12: Access Control Risks

Twelve scenarios cover access management for both human users and automated bots. All are Priority 3, but their aggregate effect is significant.

These scenarios cover compromised bot accounts, compromised user accounts, excessive bot access, excessive user access, inadequate account provisioning, inadequate access revocation, undetected bot access, undetected user access, excessive privileged access, segregation of duties conflicts, weak authentication, and unauthorized third-party access.

The controls follow a consistent pattern: bot control and user control for accountability, bot access authorization and user access authorization for least-privilege enforcement, account provisioning for formal approval processes, access revocation for timely deprovisioning, bot access review and user access review for periodic validation, privileged access authorization for restricting powerful accounts, access segregation for preventing conflicts, authentication for strong credential management, and third-party control for extending security standards to external users.

Original implementation tip: The access control risk specific to AI that most organizations handle poorly is bot account management. When a bot, an automated process, accesses systems, it typically uses a service account. These service accounts often accumulate privileges over time as the bot’s functions expand, but nobody conducts the same periodic access reviews for bot accounts that they do for human accounts. I audited one organization where a bot account for a data preprocessing pipeline had accumulated database administrator privileges, access to the production model repository, and write access to the training data store. Nobody had reviewed the bot’s access in 18 months. Treat bot accounts with the same access governance rigor as human accounts. Include them in quarterly access reviews. Apply least-privilege principles. Document and approve every privilege.

Domain 13: Change Management Risks

Seven scenarios address how changes to AI systems introduce risk.

IT impact assessment (Priority 1) is the most critical. Disruptions to other IT services may occur from AI system changes with insufficient impact analysis. The control is IT impact assessment: mandate thorough impact analysis for all AI changes, focusing on effects on related IT services, requiring integration testing with documented results.

Inadequate ongoing testing (Priority 1) causes missed defects. The control is testing protocol: establish comprehensive testing protocols for ongoing AI validation with pre- and post-implementation tests executed by independent teams.

At Priority 2, undetected errors result from inadequate automated monitoring. The control is automated error monitoring: deploy tools that continuously validate the AI system after changes, detecting anomalies in real time.

Five Priority 3 scenarios cover unauthorized changes, untracked modifications, poor change control, and insufficient validations. Controls include formal change management processes, modification logging, change control procedures, and validation procedures requiring pre-deployment tests across functional, security, and performance criteria.

Original implementation tip: The change management risk specific to AI that organizations consistently underestimate is the cascading impact of retraining. When a model is retrained on new data, the outputs change. Sometimes subtly, sometimes dramatically. If downstream systems or business processes depend on the model’s output characteristics, for example expected score ranges, output distributions, or decision thresholds, retraining can break those dependencies without triggering any traditional change management alerts. Treat model retraining as a change that requires the same impact assessment, testing, and approval as a code deployment. Because functionally, it is one.

Domain 14: Third-Party and Business Continuity Risks

The final domain covers four third-party scenarios and five business continuity scenarios.

For third parties, black box solution (Priority 2) creates business disruption when the organization cannot understand the AI system’s logic. The control is contract review: define intellectual property ownership, include escrow agreements, ensure right to audit, and outline roles and responsibilities. Third-party default (Priority 3) exposes the organization to lower control maturity. The control is due diligence: subject third parties to at least the same level of control as internal operations. Third-party dependency (Priority 3) creates concentration risk. The control is third-party segmentation: identify and categorize suppliers by criticality with contingency plans. Shadow third-party (Priority 3) results from lacking an updated vendor inventory. The control is third-party management: develop a comprehensive inventory integrated into risk and continuity planning.

For business continuity, inability to recover (Priority 1) causes prolonged disruptions when rollback mechanisms are absent. The control is roll-back: establish mechanisms to identify and recover the last known good AI state, with processes, algorithms, and cleansed data available for rapid retraining. Ineffective backups (Priority 1) results from inability to restore AI services. The control is backup restoration: implement appropriate backup and snapshot procedures, including frequent snapshots of learning data, with ability to roll back completely.

Ineffective fallback (Priority 2) results from lacking alternative processing facilities. The control is fallback facility: establish alternative processing capabilities with regular risk assessments. Fragility (Priority 3) and ineffective response (Priority 3) address business continuity planning and testing, with controls for continuity planning aligned with ISO 22301 and continuity testing through regular BCP simulations.

Original implementation tip: The business continuity risk most specific to AI is the inability to recover the model’s learned state. Traditional systems can be restored from backups because their logic is deterministic. An AI model’s “logic” is its trained weights, which are the product of specific training data processed in a specific sequence with specific hyperparameters. If you lose the trained model and do not have the exact training data, preprocessing pipeline, and training configuration documented and backed up, you cannot recreate it. I have seen an organization lose a production model to a storage failure and spend six weeks recreating it because they had backed up the model artifacts but not the training pipeline configuration. Back up everything: the model, the training data, the preprocessing code, the feature engineering pipeline, the hyperparameter configuration, and the training environment specification. Test restoration by actually rebuilding the model from backups at least annually.

Implementation Tips

These four principles apply across all 14 domains and 100 scenarios.

First, prioritize by actual exposure, not by perceived sophistication. The scenarios rated Priority 1 in this taxonomy are not necessarily the most technically interesting. They are the ones that cause the most organizational damage when they materialize. Strategy deficiency, ownership vacuum, and compliance failure cause more real-world harm than adversarial machine learning attacks. Fund controls for Priority 1 scenarios before you invest in exotic defenses against lower-probability technical attacks.

Original implementation tip: When presenting this taxonomy to leadership, resist the temptation to lead with the technically impressive scenarios like adversarial attacks or model inversion. Lead with the governance and strategy scenarios that connect to business outcomes leadership already cares about. “We lack a defined owner for our production AI models” resonates more with a board than “we are vulnerable to model extraction attacks.” Start with the risks they can feel, then build toward the ones they need to understand.

Second, map controls to your existing control framework. This taxonomy aligns to COBIT 2019 objectives across four domains: Evaluate, Direct and Monitor (EDM), Align, Plan and Organize (APO), Build, Acquire and Implement (BAI), and Deliver, Service and Support (DSS), plus Monitor, Evaluate and Assess (MEA). If your organization uses a different framework, map these controls to your existing structure. The worst outcome is creating a parallel AI control framework that nobody integrates into operational governance.

Original implementation tip: When mapping these controls to your existing framework, do not create 100 new control activities. Many of these AI controls are extensions of controls you already have. Data access controls for AI systems should be managed through the same access management processes you use for other systems. Change management for AI should follow the same change management framework with AI-specific additions. Identify which controls are genuinely new (explainability by design, bias mitigation, stability monitoring) and which are extensions of existing controls. New controls need new processes. Extensions need updated procedures. The distinction matters for implementation cost and adoption speed.

Third, document decisions and rationale, not just outcomes. For every control in this taxonomy, maintain evidence that demonstrates not just that the control exists, but why specific decisions were made. When an auditor or regulator asks why you accepted a particular residual risk, “because we assessed it and decided it was within tolerance” is insufficient. They need to see the assessment, the alternatives considered, and the governance approval.

Original implementation tip: Create a standard decision record template with five fields: the decision, the alternatives considered, the rationale for selection, the assumptions that must remain valid, and the conditions that would trigger reassessment. Use this template for every Priority 1 and Priority 2 control decision. It takes five minutes per decision and saves hours of reconstruction during audits. I have seen organizations that adopted this practice clear regulatory examinations in half the time of those that relied on informal documentation.

Fourth, reassess at a cadence that matches your risk velocity. AI risks change faster than traditional IT risks. Models degrade, data drifts, new attack techniques emerge, and regulations evolve. A taxonomy that is reviewed annually is a taxonomy that is wrong for 11 months of the year. Review Priority 1 controls quarterly, Priority 2 semi-annually, and Priority 3 annually at minimum. Update the taxonomy itself whenever a new risk scenario materializes that is not covered.

References and Standards

This taxonomy draws from and aligns with the following authoritative frameworks.

ISO/IEC 27005:2022 for the information security risk management process structure.

ISO/IEC 23894:2023 for AI-specific risk management guidance.

ISO/IEC 42001:2023 for AI management system requirements covering governance, ethics, and accountability.

COBIT 2019 for IT governance and management objectives, providing the control mapping framework used throughout this taxonomy.

NIST AI RMF (AI 100-1) for the AI risk management lifecycle framework.

EU AI Act (Regulation 2024/1689) for risk-based regulatory requirements governing AI systems in EU markets.

ISO 22301 for business continuity management systems referenced in the continuity domain.

ISO/IEC 27001 for information security management systems referenced in the security domain.

MITRE ATLAS for the adversarial threat landscape specific to AI and machine learning systems.

FAIR (Factor Analysis of Information Risk) for quantitative risk analysis methodology when assessing the scenarios in this taxonomy.

Making This Taxonomy Work

Organizations that treat this taxonomy as a reference document to satisfy an audit requirement will miss its value entirely. They will have a comprehensive list of 100 scenarios that nobody operationalizes, controls that exist in policy but not in practice, and a false sense of security that evaporates at the first real incident. The taxonomy becomes shelfware, and the organization remains exposed to the same risks it cataloged so carefully.

Organizations that treat this taxonomy as a living operational tool will use it differently. They will map their existing AI systems against these 100 scenarios to identify gaps. They will prioritize control implementation based on the priority ratings and their own risk appetite. They will assign named owners to each applicable control. They will review and update the taxonomy as new AI capabilities are deployed, new threats emerge, and new regulations take effect. Their risk conversations will be specific, traceable, and grounded in concrete scenarios rather than abstract categories.

A taxonomy that names 100 things that can go wrong is only useful if it drives 100 decisions about what to do right.

Which of these 14 domains has the biggest gaps in your organization right now? If you are honest with yourself, I suspect the answer is not the technical domains. It is strategy, governance, or people. Start there.

Iso-31000 |

How to Actually Use ISO/IEC 23894 for AI Risk Management

Practical ISO/IEC 23894 Implementation for AI Risk Management (Without Turning It Into Shelf Decoration)

Why ISO/IEC 23894 Exists and What Problem It Solves

The Three-Part Architecture You Need to Understand

Part One: Principles (Clause 4)

Part Two: Framework (Clause 5)

Part Three: Processes (Clause 6)

Stage 1: Establishing Scope, Context, and Criteria

Stage 2: Risk Assessment, the Core of the Process

Risk Identification

Risk Analysis

Risk Evaluation

Stage 3: Risk Treatment and Implementation

Stage 4: Monitoring, Recording, and Reporting

Implementation Tips

Tip 1: Align Risk Management with the AI System Life Cycle

Tip 2: Handle Stakeholder Identification Seriously

Tip 3: Document Your Risk Criteria Decisions Explicitly

Tip 4: Treat Transparency as a Multi-Audience Challenge

What This Standard Actually Does in Practice

Part 1: The AI Risk Management Process

Step 1: Define Your Scope, Context, and Criteria

Step 2: Identify Risks

Step 3: Analyze Risks

Step 4: Evaluate Risks

Step 5: Treat Risks

Step 6: Monitor, Record, and Report

Part 2: Risk Sources and Objectives (Your Identification Checklists)

AI-Related Objectives to Protect (Annex A)

AI-Related Risk Sources (Annex B)

Part 3: The Organizational Framework

Leadership and Commitment

Organizational Context

Roles and Accountabilities

Communication and Consultation

Part 4: The Guiding Principles

Part 5: Risk Management Across the AI System Life Cycle

What to Do Monday Morning

Key References and Related Standards

Supporting AI ISO standards by Topic

Core AI Management & Governance Standards

Foundational Frameworks & Terminology

Trustworthiness, Ethics & Quality

Data Quality & Analytics

The Choice in Front of You

About the Author: Prof. Hernan Huwyler, MBA, CPA, CAIO

Bridging the Gap Between AI Theory and Production Controls

Global Thought Leadership and Executive Education

Explore Open-Source GRC Tools and Insights

Let’s Turn AI Governance into a Competitive Advantage

Implementation Tips for Expert Calibration and AI-Augmented Risk Estimation

Why Expert Calibration Matters for GRC Professionals

The Core Mechanism: How Expert Calibration Works

The Basic Cycle

Two Approaches to Aggregating Expert Opinions

Behavioral Aggregation: The Workshop Method

Algorithm Calibration: The Mathematical Method

Mathematical Aggregation Methods

Bayesian Updating

Opinion Pooling (Weighted Average)

The Structured Elicitation Protocol

Step-by-Step Implementation

Case Study: Estimating GDPR Fines for a Spanish Bank

Step 1: Gather Historical Data for Calibration

Step 2: Design Calibration Questions

Step 3: Collect Expert Responses

Step 4: Assign Weights Based on Calibration Performance

Step 5: Aggregate the Weighted Responses

Using AI as Expert Estimators

The Method

Calibrating AI Estimates Against Reality

When to Use AI Estimators

Building a Repeatable Calibration Program

Institutional Calibration Infrastructure

Brier Scores for Ongoing Accuracy Tracking

Common Implementation Failures and How to Avoid Them

Integrating Calibrated Estimates Into Your Risk Framework

Connecting to Enterprise Risk Management

Key References