IFRS 9 Expected Loss Model Validation

By Jan Nuyt, Country Manager and Hubert Fonteijn, Senior Consultant

The introduction of the IFRS 9 standard has led to significant efforts by financial institutions to develop the credit risk models necessary to compute (lifetime) expected losses. In these efforts, many have built on existing models, e.g. the Internal Rating Based (IRB) models, while introducing additional components to make the transition from Through-the-Cycle (TTC) models to Point-in-Time (PiT) ones. Nevertheless, in many cases, model development and validation have been completed under considerable time pressure. As a result, a new validation and review phase is a great opportunity for:

1. An all-encompassing review of all IFRS credit risk models: Probability of Default (PD), Loss Given Default (LGD) and Exposure at Default (EAD); their underlying relations and the relations with other models might provide some additional insights.
2. A substantial portion of new data to be used to review the performance of the IFRS 9 credit risk models, given the fact that model development has, in most cases, used data that only runs until 2016.
3. The overall effectiveness and efficiency of the staging procedures to be reviewed.

In this article, we are providing you with an overview of Finalyse’s stance regarding what such a review should focus on. Given our extensive IFRS 9 development and implementation experience, we find ourselves in a great position to identify potential areas of improvement and make suggestions on possible methods. In our validation framework, we focus on the following areas:

1. Review of data quality, specifically of the modelling data set and new data
2. Methodological review of all IFRS 9 credit risk models (PD/LGD/EAD)
3. Review of a selection of variables (obligor characteristics, transaction characteristics, macroeconomic factors)
4. Evaluation of performance on the modelling data set and new data, including a sensitivity analysis
5. Review of the macroeconomic scenarios
6. Evaluation of the risk stage determination and the robustness of staging thresholds

The following sections will discuss some of the elements that a validation exercise could focus on for each of these areas.

The development of the IFRS 9 credit risk models provides a unique opportunity to review the consistency of the data sets that underpin the PD, LGD and EAD models.

In financial institutions, the introduction of IRB models was in many cases the first instance in which a consistent historical data set of defaulted and performing clients and facilities was set forth. However, the subsequent redevelopment phases of the IRB credit risk models have often been implemented for each credit risk model independently, which could lead to potential discrepancies between the data sets used in these models. Therefore, the introduction of the IFRS 9 models is an excellent opportunity to take note of the quality of the data sets and their consistency: since all models have been developed at more or less the same time, we would expect, for instance, the list of defaulted clients to be aligned between the PD and LGD modelling data set of a particular portfolio. In this context, it is also important to investigate the handling of redefaults, missing data and outliers, which again would be expected to be relatively uniform across credit risk models.

If a financial institution is an IRB institution, it might be interesting to evaluate the comparability between the IRB and IFRS 9 credit risk models. What is more, any consistency with other models that may exist in the said institution, such as prepayment models, should be inspected.

The conceptual review of the IFRS 9 models will furthermore focus on the affiliation between the IFRS 9 models, the functional form of the models, the validity of the statistical techniques used in estimating these models, the use of model selection procedures (and their consistency across models) and the role of expert judgment in determining the model parameters.

Furthermore, it is important to evaluate the implementation of the ECL calculations, since the concepts introduced in IFRS 9, such as the 12-month and lifetime ECL and the conditional PD, are surprisingly subtle. The institution is required to calculate the loss allowance based on either a 12-month or a lifetime ECL. Both have their specific pitfalls: the 12-month ECL is defined as "the portion of lifetime ECLs that represent the ECLs that result from default events that are possible within the 12-month period after the reporting date". As a consequence, for the PD, it is natural to take the 1-year PD in this case, but it is still necessary to incorporate the LGD based on all recoveries and losses resulting over the full work-out period of each exposure. For the lifetime ECL, on the other hand, the complexity lies in compounding the yearly PDs into a lifetime PD.

Traditionally speaking, IRB models focus on obligor and transaction characteristics, since these models are TTC and average over the business cycle (exceptions are Best Estimate Expected Loss models). In addition to this, the IFRS 9 credit risk models could be made sensitive to macroeconomic factors, since they are required to be PiT. Consequently, a review of the variables selected for IFRS 9 models should answer two concise questions:

1. To what extent are risk drivers related to obligor and transaction characteristics in line with expectations? Are they similar to the ones in IRB models, if present?
2. Are these risk drivers related to the selected macroeconomic factors, reasonable, and do they maintain significance in the presence of newly acquired data?

The latter point especially deserves special attention. In our experience, financial institutions have felt compelled and even obliged to include macroeconomic factors in their IFRS 9 models, since these factors embody their PiT nature. In practice, however, it is possible that no robust relationship between the business cycle, on the one hand, and default and loss rates, on the other hand, exists. For instance, in the case of LGD models, financial institutions often postpone selling collateral until the economic conditions have improved, which often causes economic losses occurring during the depth of a crisis to be relatively mild. Even if a correlation between business cycle and default or loss rate exists, it is difficult to quantify this reliably, given the relatively short time series (often spanning only one full business cycle) that are available for modelling. In this light, it is important to challenge the rigorousness of the statistical procedures with which this relationship is tested. Moreover, in the presence of new data, it is important to monitor the efficiency of macroeconomic factors as meaningful risk drivers.

IFRS 9 presents a unique opportunity to compare the outcome of the full suite of credit risk models (PD/LGD/EAD) against the observed losses, in addition to evaluating the performance of each individual model. Performance testing is commonly subdivided into the evaluation of calibration quality, discriminatory power and stability. The validation will focus on a (re-)evaluation of these components on the modelling data set and on the new unseen data that have been accumulating since the development phase of the IFRS 9 models.

After a model is proven to be sound from a conceptual point of view, the model can be further reviewed by means of a sensitivity analysis, to investigate how the models behave under extreme scenarios. This check is often omitted due to time pressure.

The soundness of the procedures used to generate the macroeconomic scenarios and to integrate their outcomes is of crucial importance for robust ECL calculations. A review of these procedures would include:

1. A check on whether correlations between macroeconomic variables are properly taken into account in the scenario generation procedure and the subsequent statistical model.
2. The number of scenarios used, covering both upward and downward situations, and the probabilities that are assigned to these scenarios.
3. The consistency of the base/neutral scenario with other ‘most probable’ scenarios used and produced in the bank.

Staging entails monitoring significant increases in credit risk for each facility. This is generally achieved by considering qualitative elements and monitoring each facility’s lifetime PD. The thresholds that determine the stage migrations between stage 1 and stage 2 are important determinants of the overall provisioning levels, since they set the number of clients for which lifetime ECL has to be recognized. Although financial institutions have, understandably, invested most efforts into the development of sound credit risk models, these staging thresholds are equally important elements of the IFRS 9 framework. It is therefore of considerable interest to perform sensitivity analyses on the effects of modifications in the staging thresholds on the overall provisioning levels.

Furthermore, it is relevant to investigate whether the thresholds lead to additional facilities with increased default risk in stage 2 compared to the facilities recognized with the qualitative triggers.

The introduction of the IFRS 9 standard has come with a unique set of challenges and opportunities alike for model development and validation. The previous sections have summarized a variety of critical elements that an IFRS 9 validation assignment should focus on. It goes without saying that this is still an active area of discussion within Finalyse. Furthermore, a crucial ingredient of such discussions is the input from our clients about their specific views on and needs within the IFRS 9 domain. We therefore look forward to engaging with our clients and starting the next phase of IFRS 9 credit risk model validation and development.