A bank's credit scoring model gets validated, back-tested, documented, and reviewed on a fixed schedule before it ever touches a customer application. A large language model summarizing a loan file, drafting a suspicious activity report, or deciding which collections account to escalate next often gets none of that. It ships as a "tool," not a "model," and slides past the governance process built specifically to catch things like it.
That gap is not hypothetical. It is the live regulatory seam that examiners, risk officers, and vendors are all currently arguing about: does generative and agentic AI count as a "model" under existing model risk management (MRM) rules, and if it doesn't cleanly fit, who is accountable when it fails?
What Model Risk Management Actually Is
Model risk management is the discipline banks use to make sure the quantitative tools driving decisions — credit underwriting, fraud detection, capital allocation, stress testing — behave as intended, are understood by the people relying on them, and degrade gracefully or get caught when they don't. In the U.S., the framework nearly every bank of consequence still builds on is the Federal Reserve and OCC's joint guidance, first issued in 2011, which defines a model broadly as any method that processes input data through a quantitative or statistical technique to produce an output used for decisions.
Under that framework, a model's life cycle typically runs through a consistent set of stages:
- Development and documentation — why the model exists, what data trains it, what assumptions it embeds.
- Independent validation — a team separate from the developers checks conceptual soundness, tests outcomes against benchmarks, and probes for weaknesses.
- Ongoing monitoring — performance is tracked against expected ranges, with defined triggers for re-validation.
- Governance and reporting — a model inventory, risk tiering, and escalation path so senior management and the board know what's running and how risky it is.
- Change control — any material update to logic, data, or scope re-enters the pipeline rather than shipping quietly.
This structure was designed for models with a relatively narrow, stable purpose: a logistic regression scoring default risk, a Monte Carlo simulation for value-at-risk, a rules engine flagging transactions. Inputs and outputs are bounded. Behavior is reproducible. You can define "wrong" with a confusion matrix.
Why Generative and Agentic AI Breaks the Template
Large language models and the agents built on top of them violate almost every assumption that made the traditional MRM playbook tractable:
- Non-deterministic output. The same prompt can produce different phrasing, different reasoning paths, or different conclusions across runs, which makes standard back-testing awkward.
- Open-ended scope. A single deployed model might draft emails, summarize documents, answer customer questions, and assist with compliance research — one system, many use cases, each with different risk profiles.
- Emergent behavior from tool use. Agentic systems that can call APIs, query databases, or initiate transactions introduce risk that lives in the orchestration layer, not just the underlying model weights.
- Vendor opacity. Many banks consume foundation models through an API rather than training them, which means the institution using the model may have no visibility into its training data, fine-tuning process, or update cadence.
- Fast iteration cycles. Foundation model providers ship updates on a timeline measured in weeks, not the annual or multi-year cadence traditional validation cycles were built around.
None of this means these systems are ungovernable. It means the governance has to be adapted rather than assumed to transfer automatically, and right now, in many institutions, that adaptation hasn't happened.
Why This Matters Right Now
The specific trigger is a regulatory gap that examiners and compliance teams are actively working through: the Fed's SR 26-2 guidance and the OCC's Bulletin 2026-13 both updated supervisory expectations around model risk, but generative AI was largely left outside their formal model-risk scope. That's not an oversight so much as a genuinely hard classification problem — regulators are still working out whether a general-purpose language model deployed across a dozen internal use cases should be treated as one model, many models, or something outside the existing taxonomy entirely.
The practical effect inside banks is a live gap between what's deployed and what's formally governed. A general-purpose chatbot assisting loan officers, a document-summarization tool feeding into underwriting notes, or an agent triaging fraud alerts can plausibly avoid the same validation rigor applied to a traditional statistical model, simply because it doesn't cleanly match the definitions examiners have historically used. Compliance and risk teams are left interpreting how far existing guidance stretches, and in the interim, a lot of generative AI is going into production under lighter internal review than the models it's starting to sit next to — or replace.
This is precisely the kind of gap that tends to get closed retroactively, after an incident, rather than proactively. Banks that wait for the rules to catch up before building internal governance are making a bet that nothing goes wrong in the meantime.
What "Auditable" Actually Requires for AI Systems
Auditability isn't a single checkbox — it's the ability to reconstruct, after the fact, what the system did, why, on what basis, and who is accountable for that outcome. For traditional models, this was largely satisfied by documentation and back-testing. For generative and agentic AI, it requires several additional layers.
Decision Provenance
If an AI agent recommends denying a loan modification, flags a transaction as fraudulent, or drafts language in a compliance filing, the institution needs to answer: what inputs produced this output, what version of the model generated it, and what instructions (system prompts, retrieved documents, tool outputs) shaped the response. Without logging at this granularity, "the AI did it" isn't an explanation — it's a black box with a timestamp.
Human Accountability Mapping
Every AI-assisted decision needs an owner who isn't the AI. Model risk frameworks have always required a named accountable party for each model in the inventory; agentic systems need the same, but extended to cover the tools an agent can invoke and the actions it can take autonomously versus the ones that require human sign-off.
Behavioral Monitoring, Not Just Accuracy Monitoring
Traditional monitoring asks "is this model still accurate?" Generative AI monitoring has to ask a broader question: is the system still behaving within its intended scope? That includes tracking for prompt injection attempts, unauthorized tool calls, drift in tone or policy adherence, and cases where the system exceeds the boundaries it was deployed for — a customer service bot suddenly offering financial advice it wasn't approved to give, for instance.
Version and Vendor Change Control
When a bank doesn't control the underlying model — because it's consumed via API from a third-party provider — change control becomes a vendor management problem as much as an internal one. A silent model update from the provider can change behavior overnight in ways the bank never tested. Contracts, monitoring, and re-validation triggers all need to account for this.
Practical Implications for Banks and Fintechs
For institutions building or buying AI systems that touch credit decisions, fraud detection, customer communications, or regulatory filings, the gap between deployed capability and formal governance translates into concrete near-term work.
| Traditional MRM Component | AI-Era Adaptation Needed |
|---|---|
| Model inventory | Extend to cover LLM-based tools, agents, and the specific use cases each is deployed for, not just the underlying model |
| Independent validation | Add scenario-based and adversarial testing (prompt injection, jailbreak attempts, edge-case inputs) alongside accuracy checks |
| Ongoing monitoring | Add behavioral and scope-drift monitoring, not just statistical performance tracking |
| Change control | Extend to vendor-side model updates the bank doesn't directly control |
| Documentation | Capture prompts, retrieval sources, and tool permissions, not just training data and methodology |
| Risk tiering | Reassess tiering per use case — a summarization tool and an autonomous transaction agent are not the same risk class even if built on the same base model |
A few practical steps institutions are taking, or should be considering, to close the gap before regulators formalize it:
- Inventory every AI touchpoint, including shadow deployments — tools individual teams adopted without going through a formal procurement or risk review.
- Tier by decision impact, not by technology type. A chatbot that answers FAQ questions is a different risk class than one that drafts language used in a regulatory filing, even if the underlying model is identical.
- Require logging at the point of decision, not just at the point of model output — capture what a human did with the AI's recommendation, since accountability ultimately sits with the human in most current regulatory interpretations.
- Build kill switches and scope limits into agentic systems before deployment, not as an afterthought — the ability to immediately constrain what an agent can access or do is a governance control, not just an engineering nicety.
- Treat vendor foundation models as third-party risk, with contractual visibility into update schedules and the right to test before a new model version goes live in production.
Where the Line Gets Blurry
Not every AI use case needs the full weight of traditional model validation, and treating a low-stakes internal tool with the same rigor as a credit underwriting model wastes resources without reducing real risk. The harder question is where that line sits, and right now different institutions are drawing it in different places, which is itself a symptom of the regulatory gap. A tool that only drafts internal notes for human review carries different risk than one whose output flows directly into a customer-facing decision without a human checkpoint — but "flows directly into" is doing a lot of work in that sentence, and agentic systems make the boundary between "assists a decision" and "makes a decision" increasingly porous.
Real Limitations and Open Questions
It's worth being honest about what current approaches to AI governance in banking don't yet solve well.
- Explainability remains partial. Techniques exist to probe why a large language model produced a given output, but they fall short of the clean, auditable logic traces that regulators are used to from simpler statistical models. "We can show you the inputs and a plausible reasoning trace" is not the same as "we can prove the model weighted these factors in this order."
- Aggregate risk across many small deployments is hard to see. A dozen teams each deploying a modest AI tool for a narrow task can, in aggregate, create meaningful institutional exposure that no single risk review catches, because each deployment looks low-risk in isolation.
- Validation methodology for generative outputs is still maturing. There isn't yet a broadly agreed-upon equivalent of a confusion matrix for judging whether an LLM's summary of a loan file is "correct" — evaluation often relies on human review samples, which doesn't scale the way statistical back-testing does.
- Vendor transparency is inconsistent. Some foundation model providers offer detailed model cards and change logs; others treat training data and update details as proprietary, leaving bank risk teams to build governance around a system they can only partially inspect.
- Regulatory guidance itself is unsettled. Examiners are still developing consistent expectations, which means institutions building governance now are, to some extent, building ahead of a target that hasn't fully stabilized.
None of these are reasons to avoid AI adoption in banking — they're reasons to build internal governance that assumes the regulatory framework will eventually formalize, rather than waiting for it to arrive before doing the work.
What to Watch Next
A few developments will likely shape how quickly this gap closes:
- Formal rulemaking or supplemental guidance that explicitly brings generative AI into scope under existing model risk frameworks, rather than leaving institutions to interpret by analogy.
- Examiner findings and enforcement actions that establish, in practice rather than in guidance documents, where the bar actually sits — these tend to move faster than formal rulemaking and often set the real operating standard.
- Industry-standard evaluation methodologies for generative AI outputs in regulated contexts, which would give validation teams something closer to the back-testing rigor they have for statistical models.
- Vendor-side transparency commitments, particularly around change management for foundation models used in regulated industries, as banks push back on being unable to test before a silent update ships.
- Convergence (or divergence) across international regulators — the EU's approach to AI risk classification and U.S. banking regulators aren't using identical frameworks, and multinational institutions will need governance that satisfies both.
FAQ
What is model risk management in banking?
Model risk management is the set of practices banks use to develop, validate, monitor, and govern the quantitative models that drive decisions like credit underwriting, fraud detection, and capital planning. In the U.S. it's built primarily on Federal Reserve and OCC guidance that requires independent validation, ongoing monitoring, and a documented inventory of every model in use.
Does generative AI count as a "model" under existing MRM rules?
It depends on interpretation, and that ambiguity is currently the core issue. Traditional guidance defines a model broadly enough that generative AI could plausibly fall under it, but recent supervisory updates have left generative AI largely outside formal model-risk scope, creating a gap that institutions are interpreting differently in the absence of explicit rulemaking.
Why can't banks just apply existing model validation processes to AI systems?
Traditional validation assumes deterministic, narrowly scoped outputs that can be back-tested against clear benchmarks. Generative AI produces variable outputs, often serves multiple use cases from a single deployment, and frequently runs on infrastructure the bank doesn't fully control, all of which strain a process designed for statistical models with fixed inputs and outputs.
What happens if a bank deploys AI without adequate model risk oversight?
At minimum, it creates blind spots — decisions influenced by AI that no one is formally accountable for reviewing, monitoring, or explaining. As regulatory guidance catches up, institutions with weak internal governance are more exposed to examiner findings, remediation costs, and reputational damage if an ungoverned AI system produces a harmful or discriminatory outcome.
How is agentic AI different from generative AI for risk purposes?
Generative AI produces content — text, summaries, recommendations — that a human typically reviews before acting on it. Agentic AI can take actions directly, such as calling APIs or initiating transactions, which shifts risk from "did the output mislead a human decision-maker" to "did the system itself do something it shouldn't have," a category traditional model risk frameworks weren't built to address.
What should a bank do now, before regulators formalize generative AI guidance?
Build an internal governance framework that assumes formalization is coming: inventory every AI deployment, tier by decision impact rather than technology type, require logging sufficient to reconstruct any AI-assisted decision, and treat vendor foundation models as a third-party risk requiring contractual visibility into change management.
Are smaller banks and fintechs subject to the same expectations as large banks?
Formal supervisory guidance historically scales with institution size and complexity, so smaller banks and fintechs face less prescriptive direct examination pressure. That said, fintechs partnering with banks or operating under a bank charter are increasingly expected to meet comparable governance standards as part of that partnership, and the gap in generative AI oversight affects institutions of every size that are deploying these tools into real decisions.
Building AI systems that hold up under audit and examination is a design problem as much as a compliance one, and teams that need help architecting that from the ground up can talk to Woyce Technologies.