Clifford Chance

In December 2024, the Monetary Authority of Singapore (MAS) issued an Information Paper on Artificial Intelligence Model Risk Management (AI MRM) following a thematic review of selected banks' AI MRM practices in mid-2024.

The Information Paper outlines good practices by the MAS observed during the review, focusing on: 

• governance and oversight
• key risk management systems and processes
• development and deployment
• generative AI
• third-party AI.

Although the good practices were derived from a review of bank practices, the MAS has indicated that they should generally apply to other financial institutions (FIs) when developing and deploying AI.

Governance and Oversight

Establishing Cross-Functional AI Oversight Forums

MAS observed that in addition to existing risk governance frameworks and structures, several banks have established cross-functional AI oversight forums. These serve as key platforms for coordinating AI governance and oversight of AI usage across various functions, and play an important role in addressing emerging challenges and potential gaps in risk management. Data and analytics, risk management, legal and compliance, technology, audit, and other relevant business and corporate functions are typically represented at such forums.

Updating Policies and Procedures

MAS noted that most banks have reviewed and updated policies and procedures to keep pace with AI use and development. Several banks have also compiled AI-related policies and procedures into a central guide to ensure consistent application of AI standards across the bank.

Developing Principles for Responsible Use

Most banks have central statements and principles on the responsible use of AI, including guidelines to ensure fair, ethical, accountable, and transparent use of AI. Additionally, some banks operationalise such central statements and principles by mapping them to key controls and the relevant functions.

Developing AI Capabilities

Banks have developed training programs to upskill staff and senior executives on using AI effectively and responsibly. Some banks have also set up AI Centres of Excellence to drive innovation, promote best practices and build AI capabilities.

Key Risk Management Systems and Processes

Identifying AI Usage and Risks

Identifying where AI is used is important in order to apply the relevant governance and risk management controls. MAS noted that most banks leveraged definitions in existing MRM policies and procedures as a foundation for identifying AI models, and extended or adapted these definitions to account for AI-specific characteristics. Some banks developed tools or portals to facilitate the process of identifying and classifying AI across the bank in a consistent manner.

Comprehensive AI Inventory

Most banks maintained a formal AI inventory with a comprehensive record of where AI is used in the bank. Some banks utilised their AI inventory to track the use of AI through their lifecycle, and to establish checkpoints for different risk management processes at various stages of the AI lifecycle, while other banks used the AI inventory to support the identification and monitoring of aggregate AI risks and interdependencies across different AI models and systems. The AI inventory may also serve as a central repository for AI artifacts for model maintenance, validation and incident or issue management. Most banks have clear policies on the scope of the AI assets to be inventoried, the roles responsible for maintaining the inventory, and the process for updating it.

Risk Materiality Assessments

Risk materiality assessments are critical for banks to calibrate their approach to risk management of AI. Quantitative and qualitative risk dimensions covering three broad areas are typically considered in assessing risk materialities. The three areas are:

• impact on the bank, its customers and stakeholders;
• complexity of the nature of AI model, or novelty of the AI use case; and
• reliance on AI or autonomy granted to the AI.

Development and Deployment

Updating MRM Standards

Most banks updated existing MRM standards for development, validation, deployment, monitoring and change management to address risks posed by AI. Key standards that banks generally regard as relevant to AI include: data management, model selection, performance evaluation, documentation, validation, mitigating model limitations, and monitoring and change management.   

Most banks established baseline standards that applied to all AI across the bank, regardless of risk materiality. For AI that were of greater risk materiality, enhanced standards apply.

Robust Data Management

General bank-wide data governance and management standards apply to data used for AI. Additional data management standards are established to ensure that data used for AI development and deployment is fit for purpose. Most banks generally focused on the appropriateness of data for AI use cases, the representativeness of data for development, robust data engineering during development, robust data pipelines for development, and documentation of data-related aspects for reproducibility and auditability.

Some banks have established additional data management standards, such as approvals for high-risk data use cases, building feature marts for training models, improving metadata management and tagging of unstructured data.

Strict Model Selection

Most banks have a strict model selection process, requiring developers to justify their selection of a more complex AI model, or go beyond qualitative justifications and develop challenger models to demonstrate performance uplifts as part of the justification process.

Ensuring Robustness and Stability

To ensure the robustness and stability of AI models, most banks collected testing datasets that allowed predictions or outputs from AI models to be evaluated in the bank's context.

Most banks aligned the choice of testing or evaluation approaches with the intended outcomes that the AI models were meant to support. Such tests include: sensitivity analysis, stability analysis, sub-population analysis, error analysis and stress testing. Most banks have criteria or thresholds for performance measures. These thresholds are clearly defined and documented, and mutually agreed by developers and validators.

To address overfitting risks, banks employed certain mitigants including favouring models of lower complexity, constraining the complexity of models, applying explainability methods for feature selection, and performing additional testing on unseen data (such as out-of-sample data) for model evaluation.

Applying Explainability Methods

Development standards across all banks generally required developers to apply global and/or local explainability methods. Some banks have a list of global and local explainability methods that could be applied to explain the outputs of AI models, while other banks further defined the minimum level of global and/or local explainability required for different use cases. Certain banks required explainability to be explored across all AI, but allowed users and owners to decide on the acceptable level of explainability, justified based on the use case.     

Fairness Assessments

Generally, banks take the following approach for assessing fairness:

• defining the list of protected features which would require additional analysis and justification;
• determining whether such features were used in training AI models;
• where necessary, determining the extent to which AI-driven decisions systematically disadvantaged against at-risk groups; and
• where necessary, providing adequate justifications on the use of protected features.

Reproducibility and Auditability

To facilitate reproducibility and auditability of AI, most banks expanded existing documentation requirements to incorporate the relevant AI development processes, such as key data management steps, how the AI model was trained or fit to the training dataset, evaluation of performance of the AI model, how the final model was selected, explainability methods used, feature selection process and fairness assessments. Most banks also have documentation templates, typically designed by the MRM function, that developers are required to follow.    

Independent Validation

The banks required independent validation albeit there is a range of approaches in establishing independent validation requirements. MAS noted most banks required independent validation only for AI of higher risk materiality, with other AI subject to peer review. The independent validation function would be equipped with the requisite skills and incentives needed to conduct independent reviews of AI.

Deployment, Monitoring and Change Management

Pre-deployment, banks placed significant focus on implementing controls to ensure that the AI functions as intended in the production environment, and these controls are typically based on existing technology risk management guidelines. Some banks conducted additional tests such as forward testing and live edge case testing. Non-AI specific pre-deployment checks remain relevant.

Post-deployment, all banks monitored their AI on an ongoing basis to ensure their AI operates as intended. Key measures monitored include robustness, stability, data quality and fairness measures. These measures were tracked against predefined thresholds or tiered thresholds.

Most banks have a process for reporting, tracking and resolving issues or incidents arising from the monitoring.

Contingency plans are implemented particularly for AI supporting high-risk functions. These plans typically outline fallback options, such as alternative systems or manual processes, and would be subject to regular reviews and testing. For mission-critical AI applications (such as for trading), certain banks had kill-switches in place.

Periodic reviews of portfolio of AI are conducted. Banks also revalidate AI in production on an ongoing basis, with AI deemed critical to risk management, regulatory compliance, business operations, or customer outcomes revalidated more frequently and intensely.

Significant or material changes to AI in production are subject to approval by control functions. Version controls are also in place to track changes, including changes from third-party AI providers.

"Dynamic AI" that is automatically updated is subject to enhanced controls for change management, risk management, and performance monitoring requirements.   

Generative AI

The following aspects of Generative AI also require further consideration: risks of hallucinations and unexpected behaviours, testing difficulties, lack of transparency from Generative AI providers which lead to challenges in explainability, and fairness with Generative AI.

Strategies and approaches

Some banks built key enabling services for Generative AI across multiple use-cases to enable scalability.

To manage potential risks, most banks started with a limited use-case, rather than deploying Generative AI in direct customer-facing applications without a human-in-the-loop. They have clear policies and procedures for Generative AI pilots, which are typically bound by time and user limits.

Process controls

Some banks instituted cross-functional risk control checks at key stages of the AI lifecycle. MAS observed that one bank established developmental and validation guidelines specific to different Generative AI task archetypes. Extensive user education and training on the limitations of Generative AI tools was another area of focus.

Technical controls

Banks typically engaged in significant research to assess suitability of a Generative AI model. Some banks undertook a range of assessments, from standalone, functional to end-to-end assessments. They also established models for assessing different aspects of Generative AI model performance. Some banks additionally curated testing datasets that were specific to the use cases that the Generative AI models were used for. Other key tests adopted include model vulnerability testing, and stability and sensitivity testing. Most banks also have input and output guardrails that utilise filters to manage risks. Some banks developed guardrails that were reusable.

Data security risks were mitigated using private cloud solutions or open-source models on-premise. Legal agreements with solution providers, data loss prevention tools and limits on data classification were also used to mitigate data security risks. Banks were also exploring the use of "grounding methods" to address risks.    

Third-party AI

Use of third-party AI poses additional risks such as unknown biases from pre-training data, data protection concerns and concentration risks from interdependencies. Various approaches were used to mitigate these risks:

• rigorous compensatory testing to verify robustness and stability and detect potential biases
• robust contingency plans including having backup systems or manual processes for business continuity
•  incorporating contractual clauses on performance guarantees, data protection, audit rights, notification/approval requirements when AI is introduced in existing third party providers' solutions
• staff training on AI literacy and risk awareness.

Conclusion

FIs should take reference from the good practices listed in this Information Paper when developing and deploying AI. Their AI MRM frameworks should also be regularly updated to keep pace with the evolving AI landscape. Non-AI specific areas, such as general data governance and management, and technology and cyber risk management, should also be reviewed to account for AI developments.

FIs should keep a close watch for further regulatory guidance, as the MAS has indicated that it plans to build on the focus areas above and issue supervisory guidance for all FIs in 2025.