The recent global financial crisis revealed that financial institutions – and even whole economies – are vulnerable and ill-prepared for severe systemic shocks. An extensive effort is underway to strengthen the financial sector and make banks and other institutions more resilient in the face of unexpected stress. The hope is that future crises will not lead to governments again being forced to invest billions of taxpayers' money to save the banking system. A regulatory requirement in the form of mandated stress testing is introduced to help facilitate this overhaul. Stress testing is an important tool in a holistic risk management regime.
Regulatory bodies have mandated stress testing as an important part of a risk management regime. Since the first supervisor-run stress tests for very large US banks in 2009, regulators have developed something of a dual strategy on stress testing. They have sought to examine the largest banks’ capital adequacy through regulator- and bank-run stress tests using standardized scenarios, while also pushing banks to improve their internal bank-specific programs of stress testing and capital planning. The aim is to fill a gap in bank risk and capital management regarding the exceptional losses that can be generated in severe scenarios.
However, the need to support regulatory stress scenarios and upgrade internal stress testing and capital planning has left some large banks running complex parallel processes. Furthermore, even though banks generally prefer developing internal stress testing programs rather than running their data through ‘one size fits all’ supervisory stress testing models, many banks have been shocked by the effort it takes to gather the risk data for modeling idiosyncratic risks at a more ‘bottom up’, granular level.
It’s really only now that many large banks are finding time to take stock, look at the bigger picture, and address some of the broader organizational and cross-risk challenges as well as the key methodological challenge – making sure stress testing is granular and robust enough to reflect the bank’s true, idiosyncratic risk profile, particularly in its credit portfolios.
For the slightly smaller US banks – those with $10-50 billion dollar assets plus, in effect, those hoping to attain that size in the next few years – the annual stress test race has just started. By Fall 2013, these banks must be prepared to run stress scenarios defined by the regulator using a series of macroeconomic variables, including any additional variables necessary to estimate losses and revenues in the bank’s unique set of portfolios. Regulators will examine the results of the Dodd-Frank tests, but they will also use the program as a way of probing the quality of each bank’s comprehensive internal stress testing and capital planning program.
Firms will be keen to apply relevant lessons from the big bank experience of developing stress test programs, but they will need to make special efforts to tailor their stress testing programs to their own unique risk profile – smaller banks tend to react to macroeconomic stresses in a particularly heterogeneous way. Table 2 sets out some of the capabilities required for credit stress testing across the bank’s balance sheet.
We think that the fall of 2013 will therefore prove to be a pivotal moment in terms of both improving and designing stress testing programs. It’s time to get prepared.
Strategic choices
The first step is to make sure the bank has a comprehensive stress testing framework in place – a task that’s easy to forget as banks rush to stress test specific portfolios. The framework and governance should cover such topics as the firm’s key goals when stress testing, how stress tests support and inform the bank’s risk appetite, the risk coverage and focus (e.g., in terms of types of risk and key portfolios), and key controls and reporting lines.
Figure 2 illustrates our perspective on stress testing showing how the mechanics of stress testing should flow from a clearly articulated stress testing framework.
As they improve their approach to stress testing, banks have to make informed trade-offs between three key factors:
Key Goals: For example, what balance is the bank aiming for between achieving a minimum level of compliance with regulatory rules and leveraging the effort to improve bank decision making? Which risks deserve the most attention?
Resources: We think stress test models must be constructed by teams that combine modeling expertise with fundamental insights into the risk dynamics of each business and risk area. In addition, the more sophisticated the bank’s program, the more risk data the bank is likely to require.
Approach: The bank will need to review each portfolio and risk type to put in place the right mix of approaches for each material risk, in terms of the level of sophistication of the modeling methodology and the wider governance and validation framework.
The choices in each of these areas have big implications for how the bank implements stress testing, especially in terms of scenario selection, modeling methodology, and validation.
Selecting and adapting scenarios
So far, banks have often taken their cue from the regulator in terms of the degree of economic adversity and range of macroeconomic indicators used to build a scenario. However, there is an increasing trend for banks to adapt scenarios to capture how risk exposure is shaped by geographic concentrations, product focus, and exposure to specific risk factors.
For example, a bank with a large portfolio of auto-loans in the Mid-West region might take the regulator’s most adverse scenario for larger banks (e.g., a rise in the national unemployment rate from 8% to 14%), and try to translate this first into implications for, say, Mid-West unemployment rates and other local macroeconomic factors. If the bank has significant auto loan portfolios it might look at how this ‘local’ scenario would affect new auto sales, and therefore auto loan origination; this would help the bank see how the scenario drives PPNR modeling. Then it may look at how the increase in unemployment is likely to translate into rises in unemployment in relevant localities and into slowdowns in regional industries, and how this will increase default and loss given default – the two key risk factors – after taking into account recovery rates, recovery costs, legal costs etc.
One tactical question here concerns the degree of idiosyncrasy the bank should factor in, e.g., should our example bank take account of the fact that its auto-loans are largely secured against Japanese, German and luxury imported cars, which tend to exhibit high recovery rates even during economic downturns? This may need some preliminary discussion with regulators, and the bank will need to be sure it has in place the data and analysis necessary to justify such a bank-specific assumption.
We’ve focused on retail portfolio credit risk in one portfolio here. However, banks increasingly need to develop firm-specific scenarios for other risks (e.g., income volatility, liquidity risk) across their enterprise. With some modification, however, many of the ideas discussed here are equally applicable to other portfolios in banks.
Moving from macro to micro
Regulators increasingly favor granular bottom up risk-factor driven approaches, compared to broader brush ‘top down’ approaches. Bottom up approaches capture the risk nuances of a portfolio because they track how macroeconomic factors affect the micro risk factors that, in turn, drive loss rates in the bank’s risk models.
However, forging a robust link between a shift in a macroeconomic factor and an increase in bank losses is a significant modeling challenge on a number of counts, three of which are particularly tricky.
First, the bank needs to work out which are the most relevant relationships between the macroeconomic variable and the underlying risk factors. For example, in a credit card portfolio, should the bank model the relationship between unemployment and default, or unemployment and delinquency? Or look at an intermediate variable such as the Bankruptcy Predictor.
Second, the bank needs to look at the observable relationship between the selected risk factors and macroeconomic risk factors across a long times series to pick up the effect of stresses, while also using judgment to factor in any structural changes in the industry – the analysis must be forward looking if it is to be useful.
Third, the bank needs to explore the relationships between the risk factors during an adverse scenario. These relationships may well not be simple linear extensions of the relationships seen in normal markets. It’s now well understood that PD and LGD correlations can drive up losses in a stressed environment, but establishing the extent of this correlation in particular portfolios remains work in progress.
Furthermore, there are many other risk factor relationships to build in and things can easily go wrong unless business knowledge is built into the project. For example, 
Figure 3 shows an illustrative modeling of the performance of U.S. residential mortgage-backed securities under various scenarios defined in terms of macroeconomic variables. The model was built using data from millions of first and second lien mortgages tracked over a complete housing cycle, including boom and bust – allowing us to link economic scenarios to default and prepayment projections.
We can see that the default rate varies hugely from the most favorable case of strong price appreciation and low unemployment, to the most adverse case of a 45% drop in the house price index and 15% unemployment. However, there is some danger that the analyst will simply apply the PD rates to a base case projection of prepayment rates, whereas the lower figure illustrates the fall off in prepayment under a strongly adverse scenario. It is this kind of correlated movement in risk factors that raises portfolio losses to exceptional levels in the real world.
Again, the challenge here is to explore the effect of adverse scenarios across the bank’s full set of business lines and portfolios, taking account of risk factor correlations. Firms should focus on ‘holistic’ stress testing that takes account of the various interplays between risk factors and macroeconomic factors.
This can help identify additional explanatory variables. For example, in addition to the unemployment rate, consumer price indices (CPI) may help to explain the historical behavior of loans used to fund income-producing real estate. A drop in price is often related to a significant fall in consumer and commercial spending and can act as a financial panic indicator in stressed times. Understanding the inter-relationships between macro-economic variables in various scenarios and how these drive loan-level risk factors (e.g., PD and recovery rates) is crucial for successfully modeling stress scenarios.
Modeling methodology: Benchmarking and backtesting rating transition approaches
So far, we’ve been discussing quite granular, bottom up approaches. However, for some ‘low default’ portfolios, e.g., C&I loans, regulators might be happy for banks to apply rating or Probability of Default (PD) transition approaches that have a broader brush, top down flavor.
For example, the bank might choose to look at how ratings and PDs have behaved in the past, including times when macroeconomic conditions have changed, and extrapolate from this to predict the behavior of ratings over a given period. 
Figure 4 shows a generic, illustrative credit transition matrix based on percentage PD rates.
But there is a fundamental challenge. How can the bank build a rating transition matrix that can be shown to accurately capture the stress risk in its unique portfolio of exposures?
The first thing the bank can do is to use internal risk models and risk and loss data from its historical records to begin capturing how internal ratings (or PDs) behave during a stressful period. But there are likely to be gaps in the bank’s internal data that need to be augmented, and the bank will need to benchmark any internal analysis using a transition matrix built up from a suitable, richer set of external data.
This is not as easy as it sounds because we can’t simply assume that the bank’s portfolio behaves like the wider universe of pubic ratings during a period of stress. Instead, the bank really needs to build a benchmark matrix that is based on obligors of a similar character to its own, e.g., energy industry obligors, or CRE obligors in the same lending sector and geographical region. Ideally, the bank should also compare the output of its rating models using stressed inputs to the output of appropriate external models or scorecards.
With this kind of robust benchmark to hand, the bank can check for differences between its own matrix and the external benchmark. At a minimum, the bank should be able to provide a good reason for any differences based on the characteristics of the portfolio, e.g., that the bank focuses on a particularly creditworthy kind of counterparty.
Banks can use similar approaches to improve their stress scenario estimates for other risk factors, for example, they can use carefully selected external data to augment gaps in their Loss Given Default data when calculating LGD for stressed periods (and to provide relevant benchmarks).
Validating your approach and making it transparent
The validation of risk models is often taken to be a regulatory-driven exercise, but inspiring confidence in the models is also vital if stress test results are going to be used to drive important bank business decisions. 
As Figure 5 shows, validating the bank’s approach to stress testing really falls into two main parts: qualitative validation and quantitative validation.
Qualitative validation is extremely important for stress testing and it includes key factors such as making sure the bank has a best-practice stress testing framework and that proper governance is in place. For example, is the board involved in both questioning and applying stress test results and how has the bank managed the challenge of building inter-disciplinary teams?
Qualitative validation should also look at the conceptual soundness of a stress test model – is it the right approach given the nature and materiality of the risk – as well as the bank’s tactics for model design, data collection, risk factor selection and so on. The bank will have to explain why it has chosen a given route, for example, why it has approached one risk using a broad, top-down approach and another using a particular kind of bottom-up risk analysis.
Finally, while stress testing is as much an art as a science, quantitative validation can be extremely valuable. For example, the bank can use carefully chosen external data to benchmark and justify its choice of recovery rate for a particular portfolio during a stressed period, or to explain its assumptions concerning the downturn correlations and inter-relationships between risk factors that it has applied in the stress testing projections.
Likewise, while it can be difficult to back test the output of stress testing models directly (e.g., comparing projections for an adverse scenario against the realization of the scenario), the bank can test how well the model performs in projecting baseline or mildly adverse scenarios (e.g., how well it has projected trends in loss rates, revenue).
An overriding issue here is transparency. In order to validate its approach to stress testing to regulators and to internal voices, the bank will need to have documented its approach, laid out all the most important assumptions, and backed these up with benchmarks and external opinions.
Conclusion
Stress testing has become the new frontier of bank risk management. Banks are expected to run comprehensive, holistic, and coherent enterprise-wide stress tests based on transparent and robust methodologies. This is a major challenge, but it can be met if banks set out their goals and stress testing frameworks clearly, and make sure that they approach the task with the right mix of business and modeling expertise – supported by independent benchmarking models and relevant data.
S&P-StressTesting-AM-June13-Bro-s_1.pdfABOUT THE AUTHOR
