Finance supervisors are increasingly focusing on model risk and model validation as an area of keen interest. The FCA says that:
Model validation and model risk management processes are important elements of any valuation control framework.
More philosophically, George Box said:
All models are wrong; the practical question is how wrong do they have to be to be not useful.
In addition, supervisors are suggesting regular reviews, annually at a minimum and more frequently for “high-risk” models. It brings to mind an exchange Albert Einstein had with a student of his:
Student: The questions in this year’s exam are the same as last years!
Einstein: True, but this year the answers are different.
MQS can provide independent model validation. Typically the process is split into four stages:
1. Suitability: an assessment of the suitability and sophistication level of the model theory, taking into account the context of the use of the valuations and other models which could be used. Assumptions, pros and cons will be highlighted.
2. Benchmarking: a review of the implementation of the model – does it execute the theory correctly? This may be performed by replication and/or case-testing and/or testing against market data. Stress testing will be incorporated where appropriate.
3. Configuration: a discussion and recommendation of the configuration of the models which should be used in different circumstances, including an examination of the calibration choices available.
4. Implementation: an assessment of the use of the model within the client’s environment, in particular reviewing calibration and an examination of parameters and settings.
MQS can bring real-world experience to the model validation process. Rather than a rubber-stamping exercise, we have written, traded off and risk managed using models and so can provide far more expertise to the process.