ML governance and model risk management is the set of roles, processes and controls that make sure every machine learning model is known, owned, built to standard, independently validated, approved for its intended use, monitored in production and retired properly. It treats models as assets that can cause financial, legal or reputational harm if they are wrong or misused. A practical programme keeps a model inventory, tiers models by risk, requires documentation such as model cards, separates development from validation for higher-risk models, monitors performance, drift and fairness, and controls every change. These principles, long established in banking, now apply to any organisation using models for important decisions.
Key takeaways
- You cannot govern models you do not know about: start with a complete inventory.
- Tier models by risk so effort goes where impact is highest.
- Separate building from validating for medium- and high-risk models.
- Monitor performance, drift and fairness continuously, with clear thresholds and owners.
- Route every significant change back through validation and approval.
Why model risk management matters
Models can be wrong because of poor data, flawed assumptions, changes in the world or misuse outside their intended purpose. In banking, supervisors have long required formal model risk management, for example through the US Federal Reserve’s SR 11-7 and the Bank of England’s model risk management principles (SS1/23). Broader AI regulation, such as the EU AI Act, brings similar expectations of documentation, oversight and monitoring to high-risk uses in other sectors.
The governed model lifecycle
| Stage | Key controls |
|---|---|
| Inventory | Every model registered with owner, purpose, data, users and status |
| Risk tiering | Tier by impact, complexity and reliance; sets depth of controls |
| Development | Documented data, assumptions, methods, limitations and test results |
| Validation | Independent review proportionate to tier |
| Approval | Sign-off for intended use, with any conditions |
| Monitoring | Performance, drift, fairness and usage against thresholds |
| Change and retirement | Changes revalidated; retired models decommissioned and recorded |
Model cards are a widely used format for documenting a model’s purpose, data, performance and limitations.
Roles and accountability
Smaller organisations can combine roles, but should still ensure someone other than the model’s builder reviews higher-risk models. For a lightweight starting point, see AI governance for small businesses.
What model validation covers
- Conceptual soundness: is the approach suitable for the problem and data?
- Data quality and relevance: is the training data representative and correctly prepared?
- Performance: results on held-out and out-of-time data against a baseline.
- Robustness: sensitivity to input changes and edge cases.
- Fairness: outcomes across relevant groups.
- Explainability: can decisions be explained to users and regulators?
- Implementation: does the deployed model match what was validated?
Monitoring models in production
- Track input data drift and prediction drift.
- Measure outcome performance once ground truth arrives.
- Monitor fairness metrics and overrides by users.
- Set thresholds that trigger investigation, retraining or rollback.
- Automate monitoring in your MLOps platform; see MLOps vs DevOps.
Governing generative AI models
Large language models and agents fit the same lifecycle, with extra attention to evaluation sets, hallucination and safety testing, prompt and tool versioning, and vendor model changes. See LLMOps architecture, the cost of LLM hallucinations and the agent development lifecycle. In financial crime use cases such as AML monitoring, model risk management is a regulatory expectation.
Common questions
What is model risk management?
The framework for identifying, validating, approving, monitoring and controlling models so that errors or misuse do not cause financial, legal or reputational harm.
Who should validate machine learning models?
For medium- and high-risk models, someone independent of the development team, with the skills to challenge data, methods and results.
What should a model inventory include?
Owner, purpose, users, data sources, risk tier, validation status, approval conditions, monitoring metrics and version history.
How often should models be revalidated?
On significant changes and at a frequency set by risk tier, often annually for high-risk models, plus whenever monitoring shows material drift.
Does model risk management apply outside banking?
Yes. Any organisation relying on models for important decisions, such as pricing, hiring, credit or healthcare, benefits from the same discipline, and regulations increasingly expect it.
Put your models under proportionate governance
Share your model portfolio and regulatory context. Our machine learning team can set up the inventory, validation approach and monitoring for your models.