CSSF THEMATIC REVIEW ON VALIDATION OF VALUE AT RISK MODELS USED BY UCITS FOR GLOBAL EXPOSURE CALCULATION
Written by Régis Deymié, Principal Consultant
Introduction
The CSSF recently conducted a thematic review evaluating the way the VaR models used by UCITS Management Companies are validated.
This review draws on paragraphs 3 and 4 of box 22 of the CESR’s guidelines to evaluate the processes in place within UCITS Management Companies for the initial validation (paragraph 3) and the ongoing validation (paragraph 4) of the VaR Models used.
CESR Guidelines are already well established, as they were issued in 2010 by the CESR (which is the predecessor of ESMA). These rules further inspired the Alternative Investment Funds Directive (AIFMD) a few years later. Today, they are still binding to UCITS funds and should also be seen as reference for AIFMs, which gives a specific importance to this CSSF review.
As a result of this review the CSSF requests Investment Fund Management Companies (IFMs) to:
- perform a comprehensive assessment of their existing VaR model validation framework against these observations before the end of 2023; and
- implement, on that basis, in accordance with a given timeline, the necessary corrective measures (if applicable).
CSSF points of attention
In its review the CSSF provides observations on the practice found within some Luxembourgish IFMs. The CSSF also takes this opportunity to restate its vision of good practices.
The observations can be summarized into the following topics:
Independence of the VaR model validation
The CSSF emphasizes the importance that the validation of the VaR model is carried out by a unit independent from the unit in charge of the building process of the model. This means that IFMs should be able to demonstrate that the validation of their model does not involve conflicts of interests.
UCITS Coverage
The CSSF instructs that a validation should explicitly and adequately cover all the UCITS managed by the IFM for which the VaR model is used for global exposure calculation, thereby considering the specific investment strategies, portfolio positions and related risks.
IFMs should make sure that the validation shows that the model used is compatible and consistent with the type of strategy (long / short, fund of funds, specific market targeted), the type of instruments used (linear or non-linear pay offs, structured products), specific asset classes used (equity, interest rates, credit, foreign exchange, commodities), specific risk factor targeted (basis, spread, volatility).
This prevents from using a generic documentation of the model.
VaR Validation report
Where a third-party entity carried out the VaR model validation, the complete validation report must be available at the premises of the IFM in Luxembourg and be available to the CSSF upon request.
Mathematical assumptions and foundations underlying the model
The CSSF declares that the validation should not be limited to a high-level review of the mathematical assumptions/foundations of the model.
This request is consistent with the above requirement to provide a dedicated validation suitable to the particular situation of the IFM.
Review of the specific assumptions and approximations of the model
Parameters like confidence interval, horizon, decay factor, number of observations for the Montecarlo method etc. should be considered and stated in the validation.
Completeness of the VaR calculation
The CSSF insists on the obligation to cover all types of instruments in the model. If that is not possible, then the proxying process should be well structured.
Operational implementation of the VaR model by the IFM and related aspects
The CSSF expects the VaR model validation to take into account operational aspects that could impact the performance of the model.
Operational implementation is key and the validation of the model should consider all aspects of it. To our mind, the most important aspects are: data management that includes the collection of clean and reliable historical prices, integrity of data control, frequency of the update of the database, proxying process, and the review of the integrity of the portfolios.
Operational implementation should also take into account other aspects that are, for certain, specifically stated by the CSSF like back testing and stress testing.
Back-testing
The CSSF expects the following from IFMs:
- Analysis of any overshoot
- Specification of the conditions under which a review of the model could be triggered
- Global analysis of the overshoots: number of observation, concentration, amplitude …
Stress-testing
The CSSF expects IFMs to set up a stress test framework that is complementary to the VaR.
IFMs’ Challenges
Depending on their risk management organization, the CSSF points of attention present different challenges to IFMs, like costs, staff availability and capacity, access to the information, independence and conflicts of interests, systems etc …
We consider three main types of organization, within IFMs in regards of VaR computation:
IFMs having its own model and runs the VaR independently.
These entities bear the burden of maintaining the computing process and complying with the guidelines requirements as reviewed by the CSSF. In this case the Management Company is entirely independent and if the process or the documentation presents some gaps, it can decide to upgrade its system by allowing internal investment.
Potential ongoing adaptation of the tool (new pricer to be added, new instrument mapping process, new source of market data etc…) as well as the related documentation that needs to be constantly updated represents additional costs that are not always considered by IFMs. On the other hand, the knowledge is in-house.
When talking about model validation, the main aspect that the IFM will have to consider apart from cost, is conflict of interest as the IFM should make sure that the validation process is performed by a team that is independent from the building team. The CSSF uses the term “unit” which provide some flexibility to the IFM in the choice of the validation team. An IFM being part of a group could assign the validation process to a team from the same group as long as it can ensure its independence.
IFMs using a model made available by an external provider.
In this case, the Management Company takes care of running the VaR computation on its own, relying on an external model. The CSSF in its review, describes some instances, where the validation is not specific enough. The review of the CSSF emphasises the importance for an IFM to rely on a model that adequately evaluates the risk of all its portfolios, and that covers each strategy, and all portfolios’ positions.
Indeed, it may be challenging for an IFM to validate that all instruments are well mapped, or all risk factors are considered, when the IFM depends on a model that it does not manage and that sometimes may appear as opaque. The IFM could even be in a situation where it would be unable to comply with CESR Guidelines.
For example, to enable a structured product in a portfolio, the IFM needs to be able to ensure that all underlying risk factors are considered accounting also for potential embedded non-linear risks. This implies for the IFM a detailed knowledge of the system and the model or the support of the external provider to fulfil these requirements.
The CSSF mentions in its review that what it considers a “good practice”, when the provider validation document is not detailed enough, is to “complete the validation exercise performed by a third-party entity on behalf of an external risk solution provider by an additional independent validation covering specifically the above-mentioned points.”
In an extreme case, if the IFM is unable to enhance its validation, it may have to renounce some investments as long as it is unable to demonstrate its capacity to ensure a proper market risk management.
IFMs having outsourced VaR computation.
In this case the IFM relies on the provider for the production of the risk report and the maintenance of the model. The provider is then mandated by the IFM to provide the necessary transparency to the IFM.
However, as mentioned by the CSSF, the IFM retains the responsibility over its risk management process, and therefore should make sure the provider provides the necessary information, and that:
- The model is adapted.
- All positions are covered.
- The documentation is complete and contains at least (see CSSF review P8):
- the risks covered by the model;
- the model’s methodology;
- the mathematical assumptions and foundations;
- the data used;
- the accuracy and completeness of the risk assessment;
- the methods used to validate the model;
- the back-testing process;
- the stress testing process;
- the validity range of the model; and
- the operational implementation.
It is therefore important for the IFM to challenge the provider and to have a precise understanding of the model and operational process used (which can be demonstrated by written evidence). That is why the IFM should conduct a thorough due diligence at inception and on an ongoing basis on the provider.
Therefore, outsourcing of the VaR computation activity does not mean that the IFM should blindly rely on the provider.
The main services that the external provider should propose (in addition to VaR computation itself), in regards of CESR Guidelines are:
- Precise and complete documentation to be shared with the IFM.
- Transparency vis à vis the IFM.
- Awareness to UCITS and AIFMD requirements.
- Support in case of Audit or CSSF request.
Conclusion
The CSSF review reminds the funds industry (The principles are believed to be applicable to wider audience than UCITS), that the market risk measure used by IFMs to measure the global exposure of their funds is a highly sensitive and strategic decision.
Indeed, when choosing between commitment / leverage and VaR, the IFM should in the first place evaluate if the indicator is meaningful to accurately evaluate the market risk aspect of the strategy. But it should also consider its capacity to put in place and maintain a compliant process at all time. This decision should be considered also accounting for all implications in terms of capacity and costs for the Company.
By choosing VaR for all or part of its portfolios the IFM chooses a useful metric applicable to all types of assets and strategies, that is easily understandable and universally accepted. It allows the IFM to fulfil its duty of market risk management and to provide to its clients and controllers confidence in its risk management process in place.
But the IFM should consider all the implication of CESR guidelines as reviewed and commented by the CSSF, on the process to be put in place.
Computing VaR implies different costs that may sometimes be omitted in the decision to choose a solution.
Indeed, in addition to the team in charge of running the computation, the following costs are to be considered among others:
- Data management
- Maintenance of the tool
- Back testing / adaptation
- Documentation
In its Managed Services Offering, Finalyse proposes a fully outsourced market risk reporting to IFMs Including VaR computation. Thanks to its expert team, those requirements are fully complied with, allowing peace of mind to our clients.
The principle for IFMs should be to take time analysing reports instead of producing them!