Model Risk Controls - Key controls

Model Development Controls:

• Documentation: Ensure that all models are fully documented, including assumptions, limitations, methodologies, and design. This provides transparency and clarity in model structure and use.
• Conceptual Soundness: Validate that the model’s design and assumptions align with the intended purpose and that the methodologies are appropriate.
• Data Quality and Integrity: Ensure that the data used to build the model is accurate, consistent, relevant, and free from bias.
• Peer Review: Implement independent peer review during model development to assess conceptual soundness, data quality, and methodology.

Model Validation Controls:

A Validation Control ensures the accuracy, completeness, and reliability of data, models, systems, or processes. It involves verifying that inputs, outputs, and performance meet predefined criteria or standards. Key components include:

1. Input Validation: Ensures data is valid, complete, and properly formatted before processing.
2. Model Validation: Confirms that models perform as expected, producing accurate and reliable results.
3. Output Validation: Verifies that outputs meet expectations and are free from errors or inconsistencies.
4. Data Validation: Ensures the quality and integrity of data, including range and format checks.
5. Benchmarking: Compares results against historical or expected values to identify discrepancies.
6. Stress Testing: Evaluates performance under extreme conditions.

Validation controls are vital for ensuring system integrity, managing risks, and maintaining compliance with regulatory standards.

Model Monitoring Controls

Annual Model Review (AMR):

• Definition: The Annual Model Review (AMR) is a formalized process where all models used by an organization are reviewed at least once a year to ensure their continued accuracy, relevance, and compliance with internal and regulatory standards. Model Risk Rating is updated if necessary. 
• Purpose: The goal of the AMR process is to identify any deficiencies in models, validate their assumptions, update inputs, and ensure they remain fit for their intended purpose. It is a key component of effective Model Risk Management (MRM).
• Control Description:
• Review Frequency: All models are required to undergo a formal review annually, irrespective of whether they have undergone significant changes.
• Documentation Review: The review ensures that all model documentation (such as methodology, data sources, assumptions, and limitations) is current and accurately reflects the model in its present state.
• Performance Evaluation: The model's performance over the past year is evaluated by comparing predictions against actual results, highlighting any significant deviations that might indicate performance degradation.
• Compliance and Governance Check: Models are checked for compliance with internal governance standards, external regulations, and any applicable industry standards (e.g., Basel, IFRS 9).

Ongoing Performance Assessments (OPA): Ongoing performance assessments are critical for ensuring that predictive models (e.g., credit risk, fraud detection, forecasting models) continue to perform accurately and reliably.

• Control Objective: Ensure that predictive models continue to produce accurate, reliable, and unbiased outputs over time and that any deviations from expected performance are identified and addressed promptly.
• Continuous Monitoring: Key performance indicators such as model accuracy (e.g., ROC-AUC, precision, recall), drift in input data, and changes in predictive power are monitored in real-time.
• Performance Thresholds: A model is considered to be performing optimally if its prediction accuracy remains above 90%, and input data characteristics (e.g., distribution) remain stable.
• Periodic Assessments: Formal performance assessments are conducted on a quarterly basis to evaluate the model’s ongoing effectiveness, identify any drift in data or performance, and assess the need for retraining or recalibration.