Validation of Risk Management Models for Financial Institutions: Theory and Practice

Description

Financial models are an inescapable feature of modern financial markets. Yet it was over reliance on these models and the failure to test them properly that is now widely recognized as one of the main causes of the financial crisis of 2007–2011. Since this crisis, there has been an increase in the amount of scrutiny and testing applied to such models, and validation has become an essential part of model risk management at financial institutions. The book covers all of the major risk areas that a financial institution is exposed to and uses models for, including market risk, interest rate risk, retail credit risk, wholesale credit risk, compliance risk, and investment management. The book discusses current practices and pitfalls that model risk users need to be aware of and identifies areas where validation can be advanced in the future. This provides the first unified framework for validating risk management models. Cover Half-title Title Copyright Contents List of Figures List of Tables List of Contributors Foreword Acknowledgments 1 Common Elements in Validation of Risk Models Used in Financial Institutions 1.1 Mincer-Zarnowitz Regressions References 2 Validating Bank Holding Companies' Value-at-Risk Models for Market Risk 2.1 Introduction 2.2 VaR Models 2.3 Conceptual Soundness 2.4 Sensitivity Analysis 2.5 Confidence Intervals for VaR 2.6 Backtesting 2.7 Results of the Backtests 2.8 Benchmarking 2.9 Conclusion References 3 A Conditional Testing Approach for Value-at-Risk Model Performance Evaluation 3.1 Introduction 3.2 The General Framework 3.2.1 Conditional Backtesting 3.2.2 Conditional Volatility Test 3.3 Test Design 3.3.1 Specific Risk 3.3.2 Historical Price Variation 3.3.3 Concentration 3.3.4 Market Stress/Adverse Environment 3.3.5 Events 3.4 Summary 4 Beyond Exceedance-Based Backtesting of Value-at-Risk Models: Methods for Backtesting the Entire Forecasting Distribution Using Probability Integral Transform 4.1 Introduction 4.2 Data 4.3 Graphics of the Exceedance Count and Distribution of PITs 4.4 Quantifying Deviations from Uniformity of Distribution of PITs 4.5 Misspecification Tests Based on Exceptions 4.6 Misspecification Tests Based on the Distribution of PITs 4.7 Conclusion References 5 Evaluation of Value-at-Risk Models: An Empirical Likelihood Approach 5.1 Introduction 5.2 PIT-Based Backtesting 5.2.1 Tests Based on the Distribution of PITs 5.3 Empirical Study 5.3.1 Tests Based on the Probabilities Implied by the PITs 5.4 Conclusions and Final Remarks References 6 Evaluating Banks' Value-at-Risk Models during the COVID-19 Crisis 6.1 Introduction 6.2 Data and Summary Statistics 6.3 Were VaR models Missing Relevant Factors? 6.4 Which Factors Were Associated with Contemporaneous Backtesting Exceptions? 6.5 Comparing Linear and Logistic Regressions References 7 Performance Monitoring for Supervisory Stress-Testing Models 7.1 Introduction 7.2 Literature Review 7.3 Performance Monitoring 7.4 Performance Monitoring Tools 7.4.1 Output Sensitivity Analysis 7.4.1.1 Scenario Sensitivity Analysis 7.4.1.2 Portfolio Sensitivity Analysis 7.4.1.3 Parameter Sensitivity Analysis 7.4.1.4 Date Sensitivity Analysis 7.4.2 Output Benchmarking 7.4.2.1 Output Backtesting 7.5 Extant Performance Monitoring of DFAST Stress-Testing Output 7.6 Conclusion References 8 Counterparty Credit Risk 8.1 Introduction 8.2 Definitions and Terminology 8.2.1 Expected Credit Loss Credit Valuation Adjustment (CVA) 8.3 Measurement, Pricing and Stress Testing 8.3.1 Calculation of CVA 8.3.2 Stress Testing 8.4 The Experiences of 1998 and 2008 8.5 The Capitalization of CCR 8.6 Validation of CCR Models 8.6.1 Generator of Future Market Scenarios 8.6.2 Pricing Models 8.6.3 Credit Exposure Calculator 8.6.4 CVA Calculator 8.6.5 Economic and Regulatory Capital Calculators 8.7 The Cost of Hedging the CVA 8.8 A Few Words on Backtesting and Stress Testing 8.9 Summary and Conclusions References 9 Validation of Retail Credit Risk Models 9.1 Introduction 9.2 Importance of Retail Credit and Retail Credit Risk 9.3 Evolution of the Retail Credit Risk Model Framework 9.3.1 Static Credit and Behavioral Scoring Model 9.3.2 Multi-Period Loss Forecasting Models 9.3.2.1 Aggregate or Segmented Pool-Level Modelling Approaches Net Charge-Off Model Static Roll-Rate Model Vintage Loss Forecasting Model 9.3.2.2 Loan-Level Model PD Model 1. Definition of Default 2. Hazard/Survival Model 3. Cox Proportional Hazard Model 4. Panel Multinomial Logistic Model 5. Landmarking Approach 6. Status Transition Model 7. Exposure at Default (EAD) Model 8 Loss Given Default (LGD) Model 9.4 Issues in Retail Credit Risk Model Validation 9.4.1 Model Development and Role of Independent Validation 9.4.2 Models' Purpose and Use 9.4.2 Evaluation of Conceptual Soundness A. Statistical Modeling Framework B. Data and Sampling C. Variable Selection and Segmentation 9.4.3 Outcome Analysis and BackTesting 9.4.4 Sensitivity Analysis and Benchmarking 9.4.5 Ongoing Monitoring 9.4.6 Future Challenges: Machine Learning and Validation 9.5 Conclusions References 10 Issues in the Validation of Wholesale Credit Risk Models 10.1 Introduction 10.2 Wholesale Credit Risk Models 10.2.1 Wholesale Lending 10.2.2 Internal Risk Rating Systems 10.2.3 Wholesale Loss Modeling Overview 10.2.3.1 Accrual Loans 10.2.3.2 FVO Loans 10.2.3.3 Other Wholesale Loss Modeling Approaches 10.2.4 C&I Loss Forecasting Models for Stress Tests 10.2.4.1 Stressed PD Modeling Approaches 10.2.4.2 Stressed LGD Modeling Approaches 10.2.5 CRE Loss Forecasting Models for Stress Tests 10.2.6 FVO Portfolio Loss Modeling 10.2.6.1 Fair Value Loss 10.2.6.2 Computing Fair Value of a Loan 10.2.7 The Core Components of an Effective Validation Framework 10.3 Conclusions References 11 Case Studies in Wholesale Risk Model Validation 11.1 Introduction 11.2 Validation of Use 11.2.1 Use Validation: AIRB Regulatory Capital Models 11.2.2 Use Validation: CCAR/DFAST Models 11.2.3 Use Validation: Summary and Conclusions 11.3 Validation of Data (Internal and External) 11.3.1 Data Validation: AIRB Regulatory Capital Models 11.3.2 Data Validation: CCAR/DFAST Models 11.3.3 Data Validation: Summary and Conclusions 11.4 Validation of Assumptions and Methodologies 11.4.1 Validation of Assumptions and Methodologies: AIRB Regulatory Capital Models 11.4.2 Validation of Assumptions and Methodologies: CCAR/DFAST Models 11.4.3 Validation of Assumptions and Methodologies: Summary and Conclusions 11.5 Validation of Model Performance 11.5.1 Validation of Model Performance: AIRB Regulatory Capital Models 11.5.2 Validation of Model Performance: CCAR/DFAST Models 11.5.2.1 Federal Reserve SR 15-18 Guidance on Assessing Model Performance 11.5.3 Model Performance Validation: Summary and Conclusions 11.5.4 Outcomes Analysis 11.5.4.1 Outcomes Analysis: AIRB Regulatory Capital Models 11.5.4.2 Outcomes Analysis: CCAR/DFAST Models 11.6 Model Validation Report 11.6.1 Model Validation Report: AIRB Regulatory Capital Models 11.6.2 Validation Report: CCAR/DFAST Models 11.6.3 Model Validation Report: Summary and Conclusions 11.7 Vendor Model Validation and Partial Model Validation 11.7.1 Partial Model Validation References 12 Validation of Models Used by Banks to Estimate Their Allowance for Loan and Lease Losses 12.1 Introduction 12.2 Pre-2020 Accounting for ALLL 12.2.1 Reserves for Non-impaired Loans 12.2.2 Reserves for Impaired Loans 12.2.3 Reserves for Purchased Credit-Impaired Loans 12.3 The Financial Crisis and Criticisms of the Incurred Loss Methodology 12.4 The New Current Expected Credit Loss (CECL) Methodology 12.5 Potential Modeling and Validation Concerns surrounding CECL 12.5.1 Issues from Extending the Loss Measurement Window to Contractual Life 12.5.2 Issues from Incorporating Reasonable and Supportable Forecasts of the Future 12.5.3 Other Issues from Changes Brought in by CECL 12.6 General Model Validation Concerns of ALLL Models 12.6.1 Data Issues 12.6.2 Modeling Issues 12.6.3 Documentation Issues 12.6.4 Performance Testing Issues 12.6.5 Other Issues 12.7 Conclusions Appendix A Description of HUD data and Analysis Appendix B An Example on Maturation Effect and CECL Loss Computations Appendix C An Example on Discounting of Cash Flows and Losses References 13 Operational Risk 13.1 Introduction 13.2 Loss Distribution Approach (LDA) 13.2.1 LDA and the 99.9th Quantile 13.2.2 Using the LDA Appropriately 13.3 Regression Modeling 13.3.1 Dates 13.3.2 Large Loss Events 13.3.3 Small Sample Size 13.3.4 Using Regression Analysis Appropriately 13.4 Model Risk 13.4.1 Backtesting 13.4.2 Sensitivity Analysis 13.4.3 Benchmarking 13.5 Conclusion References 14 Statistical Decisioning Tools for Model Risk Management 14.1 Introduction 14.2 Risk Modeling 14.3 Utility Analysis 14.4 Empirical Application 14.4.1 Home Mortgage Data 14.4.2 Disparity Analysis 14.4.3 Model Estimation Results 14.5 Model Evaluation 14.5.1 Comparison Metrics 14.5.2 Quadratic Reward Specification 14.5.3 Utility Comparisons 14.6 Discussion References 15 Validation of Risk Aggregation in Economic Capital Models 15.1 Introduction 15.1.1 Literature Review 15.1.2 Validation of Economic Capital Models 15.2 Data and Descriptive Statistics 15.2.1 Variables for Risk Types and Hypothetical Bank Construction 15.2.2 Graphical Analysis 15.3 Empirical Methodology and Results 15.3.1 Benchmarking: Alternative Copula Models 15.3.1.1 Statistical Assessment Criteria 15.3.2 VaR Estimation and Backtesting Analysis 15.3.2.1 VaR Estimation 15.3.2.2 Backtesting Analysis 15.3.3 VaR Stability 15.3.4 Stress Testing 15.4 Conclusion Appendix A: Mapping between Y9.C and Bloomberg variables Appendix B: Mergers and Acquisition list References 16 Model Validation of Interest Rate Risk (Banking Book) Models 16.1 Introduction 16.2 Earnings at Risk 16.3 Economic Value of Equity 16.4 Duration of Equity 16.5 Governance of ALM 16.6 Residential Mortgages 16.7 Commercial Loans 16.8 Credit Cards 16.9 Other Retail Loans 16.10 Wholesale Liabilities 16.11 Certificates of Deposit 16.12 Non-maturity Deposits 16.13 Investment Portfolio 16.14 Term Structure Modeling 16.15 Summary of Model Validation for ALM 17 Validation of Risk Management Models in Investment Management 17.1 Introduction 17.2 What Makes Validation of Investment Management Models Different? 17.3 Asset Management Models That May Be Validated Using Methodologies for Similar Models Used for the Bank's Own Assets 17.4 Conclusion Index