Benchmarking, Back-Testing, & Sensitivity Analysis

Introduction

CECL is a financial accounting standard that requires all companies to estimate and record an allowance for expected credit losses on their financial asset portfolios, including loans and investments. In preparing a CECL estimate, companies develop models to help predict the estimated amounts. Benchmarking, back-testing, and sensitivity analysis are analysis techniques used in the context of modeling to evaluate the accuracy and robustness of the models used. These techniques are part of a model-risk management policy and are not part of the CECL standard specifically.

Benchmarking

Benchmarking under CECL is the process of comparing the CECL allowance estimate of a financial institution with the estimates of its peers or industry standards. Benchmarking can help identify potential variances between the institution and their peers, enabling management to identify why their estimate differs from the benchmark. For example, if a financial institution’s CECL allowance estimate is significantly higher than its peers, it may indicate that the institution is using higher loss rates or different models to estimate expected loss. Management would then determine if their benchmark institutions have significantly different loss rates than their institution. Management can then conclude as to whether the variance is reasonable or expected. If not, management may determine if their model needs adjustment.

Back-Testing

Back-testing under CECL is a method of evaluating the accuracy of the CECL allowance estimates by comparing them to the actual credit losses that occurred after the estimate was developed. Back-testing involves comparing the estimated losses, both forecasted and reversion, with the actual losses incurred after the estimate was developed. Back-testing can help identify if the CECL allowance model is overestimating or underestimating the expected credit losses and provide insights into any needed model refinements.

Sensitivity Analysis

Sensitivity analysis under CECL is a method of assessing the robustness of the CECL allowance estimates to changes in key assumptions or inputs, such as changes in economic forecasts and qualitative factors. Sensitivity analysis involves testing the impact of variations in specific assumptions, such as changes in economic scenarios, loan loss histories, or other key factors, on the CECL allowance estimate. Sensitivity analysis can help identify the assumptions that have the most significant impact on the CECL allowance estimate and provide insights into how to adjust the model to make it more accurate and robust.

In summary, benchmarking, back-testing, and sensitivity analysis are important tools used in the context of model-risk management to evaluate the accuracy, comparability, and robustness of the models. By employing these techniques, companies can better understand the limitations of their models and make informed decisions about their modifications necessary for their models.

How ARCSys Consulting Can Help:

• ARCSys Consulting can help your institution determine if you have similar loss rates to similar peer institutions and create a benchmarking report
• ARCSys Consulting can create a back-testing report annually for your institution to support your annual model review and validation
• ARCSys Consulting can perform sensitivity testing on your institution’s model, based on stressing the portfolio, and provide a separate report of the results

Contact ARCSys Today!