Although not all SEC registrants are required to comply with the various model risk management guidelines, implementing model controls and oversight are critical in this evolving landscape. Investment management companies are utilizing models at an ever-increasing rate to improve performance and this need is emphasized by the SEC’s proposed guidance, Conflicts of Interest Associated with the Use of Predictive Data Analytics by Broker Dealers and Investment Advisers.
The guidance demonstrates the agency-continued focus on conflict of interest, but also follows suit of other agencies’ concerns related to the use of artificial intelligence and predictive models. Therefore, although consumer protection continues to be a focus point for regulators, there are several reasons why effective model risk management practices are critical.
What is Model Risk?
Without the proper controls over your models, unintended errors can have a substantial impact on your business and profitability. Below, we break down the three main sources of model risk:
- Errors within the model: These errors occur when various inputs are utilized within the predictive model, and they can be the result of:
-
- Assumptions being applied within the model that are not appropriate for the projected scenarios or the utilized data.
- Utilizing inaccurate or incomplete sourced data.
- Blind spots: Models are usually constructed internally by a small number of employees or a third-party to project the desired results, which can often lead to knowledge gaps. Specifically, this can result in a misunderstanding on how the models operate, how assumptions affect inputs, or if the model is conducted correctly for your use case.
- Inappropriate use: Companies are often asked to do more with less. When this concept is applied to models, it can result in stretching a model to produce projections that are not within its capabilities. Although the model will still produce outputs, these may not provide a clear and accurate picture.
How Can Model Risk be Reduced?
The first step in reducing model risk is to create the foundational elements, such as proper oversight and procedures to ensure models are accurately understood, and consistently utilized. These foundational elements should include:
Clear policies or procedures that (at a minimum) outline:
- The defined roles and responsibilities.
- The step-by-step process for new model creation.
- Ongoing oversight, including input review, assumption approval, and change management processes.
A training program to ensure the end-users understand what the model is designed to do and the system’s limitations. Additionally, there should always be sufficient back-ups trained on how to utilize and update the model as necessary.
An effective challenge program must be established from those that are receiving the results. For instance, it is vital to question whether the assumptions within the model and the outcomes are appropriate given the purpose, and use of the model. In addition, detailed testing independent from the model owner must be implemented to ensure there are not errors within the model or how the model is applied.
The Components For Effective Model Risk Management
Specific to the effective challenge, the following elements of the model should be tested within sound model risk management programs:
- Review the assumption creation process to verify that the assumptions are properly supported and applied within the model.
- Verify that data integration is complete, accurate, and safeguards nonpublic personal information.
- Verify that there is a standard process followed to customize the model for different scenarios or customers.
Although this proposed guidance mainly focuses on conflict of interest, it aligns with the SEC’s 2023 exam priorities related to emerging technology and the requirement for sound risk management practices. Updating risk management practices to properly monitor and identify evolving risks will continue to be a critical focus for investment advisors and regulators as this landscape continues to change.
If you have any concerns about how your policies or practices stack up against examiner expectations, please reach out to a member of Wolf’s seasoned Investment Management team.
