Machine Learning Model Validation Framework
Finalyse offers a classic approach holding true for Machine Learning model validation. Our model validation scope can be tailored to accommodate your requirements: A purely technical and qualitative exercise focusing on whether a certain methodological choice is fit for the intended purpose. A larger scope covering the implementation of a certain methodology. And ultimately, a validation of the whole process through which the model is deployed within the organisation.
How does Finalyse address your challenges?
Bring in the experience of a multidisciplinary team of experts with extensive exposure to regulatory and non-regulatory models.
Unleash the full power of your ML models: ascertain that you can justify their performance, interpret and explain their outputs and find the right balance in case of multi-objective trade-offs.
Ensure optimal communication between your business lines, subject matter experts and the data scientists creating your ML models.
Ease the compliance with the regulatory expectations of having a proper knowledge of - and widespread awareness around the models in use (TRIM, BCBS 2005, CRD Section II, CRR2 Art. 189 and 293).
Bring in our framework and governance set-up capabilities to ensure the best traction for your models.
Ensure independence of your validation from all possible agency pressure.
How does it work in practice?
Business Case
You are using an ML model to screen credit applications. You are not aiming at minimizing the level of defaults, but are in fact trying to optimise your risk-return payoff. And you want to do it in a way compliant with the European Regulatory Framework, implying e.g.:
- No implicit or explicit discrimination on certain demographics
- Privacy respect / right to be forgotten
- Actionability of the model and overall integration in the risk management framework and governance
Working with Finalyse on your Model Validation Framework will allow you to leverage on the broad experience we have accumulated in the area of risk measurement and management, as well as on varied expert profiles brought together under a team.
Key Features
- A healthy mix of quantitative, business and regulatory profiles for all your ML validation needs.
- We can bring clarity and explainability when complexity blurs the underlying messages, or go looking for that elusive optimisation in your place – just as long as we don’t validate it as well.
- Keeping tracks of the European regulatory progression on ML models as well as more local initiatives to ensure we have the best practices in place.
Augustin de Maere is a Principal Consultant based in Finalyse Brussels, leading the Market Risk & ALM practice with François-Xavier Duqué, with a specific focus on Interest Rate Risk and Economic Capital modelling. He has been involved in the development or validation of several interest rate models, covering all the aspects of the model chain, from interest rate scenario generators to the calibration of behavioural models (non-maturity deposits, …) and the building of portfolio revaluation engine.
Nemanja Djajić is a Senior Consultant with more than 9 years of experience in credit risk modeling and data science area. He gained his experience through multiple roles in banking industry including the position of data science department director in one of the biggest banks in Eastern Europe. Nemanja’s main area of expertise lies within the development and validation of risk and business related models, using traditional or machine learning methodologies.