Reverse Stress Testing

By François-Xavier Duqué, Managing Consultant

and Kristian Lajkep, Regulatory Compliance Officer

Stress test is an important tool at banks’ disposal to identify and quantify some of the risks that a bank faces and (perhaps even more importantly) to assure the stakeholders and the regulators of the soundness of its business plans. In particular, stress testing can make stakeholders secure in the knowledge that, to the extent that there are risks for the current business plans, they will materialise only in a limited way, even in a relatively adverse scenario, and to the extent that the impact of these risks will not be limited in such scenarios, the adequate steps can be taken to remedy them as the risks are already identified.

The reverse stress test flips the premise of the stress test on its head: the reverse stress test supplements the results of aconventional stress test by a conditional analysis of the state of the affairs, given a pre-defined scenario where some adverse outcome has already occurred. The pre-defined adverse outcomes selected for the purposes of reverse stress testing may be numerous: the institution failing or likely to fail, its business plan, in part or in full, is no longer viable, etc. Rather than starting with a scenario and looking for the outcome, the backward stress test starts with the outcome and looks for severe, but plausible, scenarios and circumstances that might cause this outcome to occur.

The EBA guidelines on stress testing identify reverse stress testing as a stress testing that meets one or more of the following characteristics:

• it is used as a risk management tool aimed at increasing the institution’s awareness of its vulnerabilities by means of the institution explicitly identifying and assessing the scenarios (or a combination of scenarios) that result in a pre-defined outcome;  
• the institution decides on the kind and timing (triggering events) of management or other actions necessary for both (a) rectifying business failures or other problems; and (b) aligning its risk appetite with the actual risks revealed by the reverse stress testing;  
• specific reverse stress testing can also be applied in the context of recovery planning (e.g. reverse stress tests applied in a wider context can be used to inform a recovery plan stress test by identifying the conditions under which the recovery might need to be planned). 
   

The simplest reason for conducting the reverse stress tests is because it is required by law. Namely, it is required by CRR implicitly in the article 177, “where appropriate” in 290(8) and most explicitly in 368(1)(g):

“The institution shall frequently conduct a rigorous programme of stress testing, including reverse stress tests, which encompasses any internal model used for purposes of this Chapter (calculating own funds requirements) and the results of these stress tests shall be reviewed by senior management and reflected in the policies and limits it sets. (…).”

A question mark looms over the operative term “where appropriate.” Whilst no regulatory paper gives an exhaustive list of all conditions that would make the reverse stress testing “appropriate” and therefore necessary, it is quite clear that the institutions that make use of the internal models would, by this very virtue, be required to conduct a reverse stress test.

It is also worth noting that not only is it required that the reverse stress tests are conducted, but also that the company makes use of its results.

As with many things for which the simplest reason is the legal requirement, there are better reasons than the regulators’ whim to take part in them. There is a good intrinsic reason to do reverse stress tests and to do them well: as hinted above, a well-performed reverse stress test is a good study of tail risk. As such, reverse stress testing may supplement the information from the normal stress tests and provide the knowledge of the risk factors that may otherwise remain hidden, but at some contingencies will be the first ones to pave the way to the institution’s demise. As such, it shifts the attention of management in an interesting way – away from typical scenarios against which the institutions are typically best protected and on to identifying weaknesses that may otherwise come up from nowhere to hurt the institutions.

Investment firms in particular may find reverse stress testing very useful to find their vulnerabilities in extreme events, particularly to receive information on the risks that are not captured by traditional stress tests based on macroeconomic shocks only. It is also to be noted that reverse stress tests may be useful (and therefore should be used) to gather insights into scenarios that involve combinations of stresses coming from both solvency and liquidity, where traditional modelling may be insufficient, or unsuited, to capture complex aspects from real situations.

As stated above, the requirement for the institutions to conduct reverse stress testing has been enshrined in the EU law as of January 2014 by Capital Requirements Directive (CRR), particularly articles 177, 290 and 368.

Apart from that, guidelines on institutions’ stress testing released in July 2018 feature reverse stress testing heavily (particularly section 4.6.5). It is worth noting that EBA guidelines are very principle-based; they do not go into details of how reverse stress tests should look like, what should be the pre-defined outcomes or what scenarios or specific factors should be considered. However, the guidelines do provide a good description of the overall principles that should govern a good reverse stress testing framework as well as the ways in which the results of these stress tests should (and could) be used.

Lastly, in October 2018, the BIS published its Stress testing principles, which replace the Principles for sound stress testing practices and supervision published in May 2009. This short document is even higher level and even more principle-based than the EBA paper, to which it does not really add any substance. Reverse stress testing is only very briefly touched in this document.

The institutions are required to not only conduct the reverse stress tests, but also to use them as a regular risk management tool. The use of the results of reverse stress tests by management body and senior management in formulating and challenging the viability of their business plan is also requested.  The EBA identifies four important steps in building and using reverse stress tests:

• Identifying the pre-defined outcome to be tested;
• Identifying possible adverse circumstances that would cause the pre-defined outcome;
• Assessing (depending on the institution’s size, nature, scale, complexity and risk) the likelihood of events included in the scenarios leading to the pre-defined outcome; and
• If the reverse stress testing indicates the risk of a business model failure as too high for its risk appetite, the institution should adopt effective arrangements, processes, systems or other measures to prevent or mitigate the identified risks and vulnerabilities. If necessary, a change of business plan may also be required.

The measures taken as a response to reverse stress testing, including (where necessary) changes to the business plan, should be documented in detail in the institution’s ICAAP documentation.

Reverse stress testing is also very useful in recovery plans. In particular, it is used to develop ‘near default’ scenarios (the point where institutions’ business models become non-viable unless planned recovery actions are implemented), and therefore enables them to test the viability and efficiency of their recovery plans. The scenarios should also take into account all possible recovery plans in all different viable scenarios as well as a possible estimation of the cross-effects of executing different recovery plan options in the same scenario. Reverse stress testing should contribute to the recovery plan scenarios by using a dynamic and quantitative scenario narrative, which should cover: 

• the recovery triggers;
• the recovery actions required and their expected effectiveness, including the method of assessing that effectiveness;
• the appropriate timing and process required for those recovery actions; and
• in the case of further stress, plans for the potential additional recovery actions required to address residual risks.

Just as with normal stress tests, it is highly advisable to create a unified reverse stress test framework for all types of risks. Thus the EBA demands the reverse stress tests to be “part of the stress testing programme, sharing the same governance, an effective infrastructure and quality standards, (with normal stress tests) (...).”and “at the same level of application as ICAAP and ILAAP.” 

The importance of this cannot be understated: it is often tempting for institutions to quickly patch holes within their regulatory compliance before the regulators catch wind of the inadequacies. Institutions often contemplate developing a reverse stress testing framework in separation of (already implemented) conventional stress tests or, even more problematically, reverse stress tests for one type of the risks (the one that their current framework addresses insufficiently). In this type of approach, it is extremely important to always keep in mind that the end result should be a united framework that is more than a sum of its parts, that risk factors affect institutions in concert and that different types of tests complement themselves. Where necessary, we have achieved some good results using this approach, but building the entire framework from the ground-up is always better.

This methodology should reflect the nature, size, scale and complexity of the business model of the institution and its portfolio composition. It should address “(…) illiquidity of markets in stressed market conditions, concentration risk, one-way markets, event and jump-to-default risks, non-linearity of products, deep out-of-the-money positions, positions subject to the gapping of prices and other risks that may not be captured appropriately in the internal models. (Furthermore, the projected) shocks should reflect the time it could take to hedge out or manage risks under severe market conditions.” (CRR 368(1)(g)).
The EBA advises the smaller institutions to focus on the more qualitative aspects of the stress testing whilst the larger institutions will need to opt for more sophisticated reverse stress testing techniques.

As a starting point, the EBA recommends that institutions begin, where appropriate, with sensitivity analyses, looking for the consequences of shifting one of the parameters to some extreme to reach predefined outcomes. Various reverse sensitivity analyses for each type of risks as well as a combination analysis where all risks are covered simultaneously should be conducted. However, whilst a good starting point, the sensitivity analysis is not a fitting tool for finding the reverse stress test scenarios and ought to be used only for intuition. The EBA seems to be convinced that the quantitative analysis, performed in concert with qualitative analysis, incorporating opinions of experts in various fields, is the best way to reach the scenarios: this may also be the reason for a relative lack of guidance on the methodology from the supervisory sources. Joint stressing of all sources of risk simultaneously, using statistical methods, is the next, more quantitative, step that also needs to be carefully developed. The plausibility of the parameter shifts should be calculated by historical (multivariate) probability distributions in order to get the first intuition of vulnerabilities.

The institutions that make use of the IRB models are also advised that, in severe stress scenarios, even though not necessarily an indication of model inadequacies, model risk will increase and may lead to a breakdown in the model’s predictability.

The reverse stress tests scenarios should be compared to those of the normal stress tests (as well as a multitude of other historical and supervisory scenarios) to assess their severity with the goal of finding a severe but plausible scenario. Reverse stress testing should also be useful for assessing the severity of scenarios for ICAAP and ILAAP stress tests. When dealing with scenarios, the EBA advises to bear in mind that the pre-defined outcome of reverse stress tests may be reached by multiple different scenarios, including the ones that are not necessarily produced by the test results.
 

It has been stated that the best way to create the reverse stress test for certain types of risks is as a part of the whole stress testing framework. As such, it is implied that the stress testing framework is a singular entity and should be treated accordingly. However, more often than not, the clients approach us with demands to improve their current stress test framework via adding/modifying reverse stress tests for one or more specific types of risk. The approach Finalyse uses to do this is fairly general, so that other companies may find a general insight in our approach too.

Requirements Analysis

This step obviously entails familiarising ourselves with what exactly it is that we are being asked to do. Being crystal clear about this is very important – even companies working on their own stress tests internally should not underestimate this step.  It also, and perhaps most importantly, consists of the analysis of available input: stress testing framework documentation, stress testing process, market data (historical series), AS-IS calculations and results, related IT infrastructure as well as recommendations from the auditors. As these will be the key components, it is very important that they are stated explicitly.

Enhancing Current risk framework

The second step is to assess, verify and to the extent possible, improve the existing stress test framework, which can encompass the following:

• Checking the relevance of the scenarios currently in use
• Time horizon of the stress
• Checking for further evidence explaining the validity of stress scenarios
• Analysis of the relevance of the stress tests in light of the structure and composition of the portfolio
• Review of the interactions between the different risk factors
• Review of the stress testing policy

For this end, Finalyse has designed and built a pragmatic tool using a historical time series of market data to compute statistical KPIs that are relevant for the stress testing purposes, and to evaluate the plausibility of the scenarios currently in use.

Develop Reverse Stress Test Framework

The third step is to develop a reverse stress testing framework for the scope of the particular risk factor in question.

The building blocks of our reverse stress testing framework would be the following ones:

Bank failure target

In the case of reverse stress testing, we start with the definition of the level of bank failure needed to assess the stress test scenarios. This does not specifically mean insolvency, but would rather relate to a level of P/L loss or capital loss that cannot be easily corrected by prospective management actions. In practice, we will look at the same key output metrics used in the usual stress testing framework and define the appropriate level of magnitude in joint collaboration with Risk and Business teams.

Tail Risk Analysis

A pragmatic way to perform this analysis is to look at the loss distribution simulated for VaR and to isolate the main risk factors responsible for losses in the neighbourhood of the level previously identified. In a historical VaR approach, the space of risk factor realisation may be too narrow because only risk factors combinations effectively observed in the past will show up. As a complementary approach, P/L loss could be simulated in a sensitivity-based Monte-Carlo setting, assuming a relatively simple fat-tail multivariate distribution that replicates the volatility and correlation structure displayed by historical series of risk factor returns. Once most influential risk factors are identified in the tail region under focus, the dimension of the problem can be further narrowed by grouping similar simulations using clustering methods, each cluster leading to the definition of a possible scenario.

The technological choices for developing the tool supporting these calculations depend on the outcome of the requirements analysis phase.

Identifying and running the unviable scenarios

Scenarios derived from the tail risk analysis should then be calibrated more precisely – using a sensitivity approach - to produce a sufficient level of loss. We would typically analyse the relevance of those scenarios through discussions with risk or business experts, and compare them with historical data and similar shocks used in other stress testing exercises (e.g. EBA stress testing). The likelihood of the scenarios would also be assessed.

Full revaluation and quantification of impacts

The following step would be to add these scenarios in the stress test framework and validate the losses calculated under full-revaluation.

Documentation

The last but not necessarily least step is to provide a consistent documentation of the stress testing framework, in line with market best practices. For many of our clients, the existing documentation of previous models tends to be fragmented and aged. In most cases, it is necessary to build up a structured document outlining the stress testing approach of the company as such, consistently with market best practices and highlighting the links to regulatory requirements. The updated stress test features and reverse market stress tests approach would fall logically in that structured document.

This approach, we believe, has the best chance to add a new reverse stress test for a specific type (or types – or indeed all of the reverse stress testing) to the existing stress testing. Note that it must be done with a full understanding of the current stress test framework, which will need, to some extent, to be reworked or reassessed.

The financial institutions at large possess a very superior knowledge in terms of their own business model and the conditions and scenarios that may cause it to underperform. They also possess a very good knowledge of the market environment that they operate in and what aspects of that environment are crucial for their businesses. As a result, institutions typically do not need much help with the qualitative analysis and pre-defining scenarios. Similarly, institutions are usually already apt to draft their recovery plans, and in the process of doing so, make use of all sorts of information (including that coming from the reverse stress tests). It is within the interest of the companies to have a governance framework that allows all available information to be duly considered by senior management. 

On the quantitative side, however, institutions may find it necessary to procure themselves a specialist to help them particularly with identifying the circumstances that would cause the pre-defined outcome, and assessing the likelihood of these conditions to occur. Building the joint stressing statistical models and making sure they fit into the overall stress test framework may require an expert hand. As an added benefit to their quantitative expertise, consultants may also double-check the qualitative aspects; whilst it is not primarily their expertise, they may work as a sanity check to the qualitative analysis and, if helpful, (owing to the experience of performing a similar task in a number of institutions) making suggestions that could lead to the improvement of the quantitative analysis.