Over the last few weeks – both before and after they were officially released – a great deal of critical attention has been turned to the “stress tests†that were organized by the Treasury Department to determine the health of major U.S. banks. A major criticism of these tests has been that the “adverse scenario†used in them is actually not that adverse. It makes assumptions about economic growth, employment, rates of mortgage default, and other factors that hardly constitute a “worst case scenario.†Consequently, argue these critics, the “stress tests†do not really “stress†the banks at all.Â
In light of this criticism, a discussion has emerged about what the point of these stress tests really are, and I think that this secondary discussion is quite interesting for us in understanding how these stress tests relate to some of the phenomena that we have been thinking and writing about. A major argument that has been made by defenders of these stress tests – and it is an argument that I find quite convincing – is that in fact the “adverse scenario†is in no way shape or form intended to present a “worst case†scenario. We know what would happen in a worst case scenario: the government would take these banks over and guarantee their deposits. Instead, the stress tests are supposed to present a “moderately bad†scenario, one in which it is possible that a system of regulation based on mandated reserve ratios or capital requirements would allow banks to weather the storm without further government intervention.
The point I would make here is this: In some important ways, the stress tests resemble catastrophe modeling on a technical level. That is, they collect a tremendous amount of data about the banking system, and then run simulations to understand how it would respond to certain adverse conditions. But on the level of political technology it works differently. In areas such as civil defense, pandemic preparedness, or catastrophe insurance, the point was really to understand how a system would respond if the worst imaginable thing actually came to pass. In the case of the stress tests, the question is whether the system could make it through a reasonably adverse economic scenario without external intervention.
Taking this point one step further, it seems that what is emerging is perhaps a two-tiered system for financial system security. On the one hand, there is continued acknowledgement that in the absolutely worst cases – the “unthinkable†economic emergencies – the remedy will be government intervention to save the banking system. On the other hand, mechanisms like the stress test are intended to ensure that the banking system could survive on its own under moderately adverse economic conditions. In the future, the stress test might, of course, serve another role as well, as was suggested yesterday in the New York Times Breaking Views blog: It could be institutionalized as the basis for a more flexible kind of financial regulation, one that did not set hard and fast rules about capital requirements or reserve ratios, but that used much more intensive exchange of data to assess banks financial position, and to try to prevent the kinds of risk exposures that led to the current crisis.
How does this map onto our thinking about emerging concepts like systemic risk regulation as an example of vital systems security?
Thanks for this post Stephen.
I think the contrast you draw between “stress testing” in the case of financial system security and other areas of preparedness ARC has been involved with is particularly interesting and fruitful. I would like to make two sets of comments, one on the two tiered structure of financial system security and the other on what what kind of a phenomenon and problem “stress testing” is a response to.
First, the controversy over what a “worst case scenario” is is particularly interesting (especially in the light of the latest unemployment figures that are well below the assumptions of scenarios used) and revealing as to the degree to which financial system has become a vital and indispensable infrastructure for the well being of the US economy since the mid-1970s. As Greenspan and his heirs have pointed out in the 1990s and most recently in defense of the financial system, the struggle over how much capital ratios of financial institutions, whose adequacy is at stake in the case of stress testing, should be is also a struggle over how rapid an economy should grow as higher ratios mean lower leveraging and hence less liquidity and credit in the overall system. In short, the controversy is also a controversy within politics of growth in which one has to optimize within the axioms of an entirely safe banking system with very low credit availability and excessive risk bearing system in which there are great vulnerabilities to shocks along with high levels of liquidity and credit. Resilience, a term often used in discussion about the well being of the system overall, seems to be the key category that describes the state of the system in which one has an optimum balance between these two poles.
In this respect, it is only natural that we see this two tier understanding of what a worst case scenario is: As long as, we have a macro-economic model based on a circular relationship between population and economic growth, one cannot afford to do a full-scale stress testing based on “real” worst-case scenarios as such an exercise would probably mean that one should not have a financial system that can supply the level of liquidity and credit necessary for high levels of growth modern economies have been producing in the last two decades. (This is also the case for stress tests being run within financial institutions, since again if they were to run these tests with much worse conditions they would probably stop being financial institutions to begin with.) In other words, the government has to insure for the worst case scenario through the tacit promise of a bail out rather than through autonomous mechanisms of security that are supposed to make government intervention in the system unnecessary.
This however does not really answer what kind of a phenomenon we are faced with, but rather explains the consequences of a new truth game upon the politics of growth and the Economy. In my mind, the right question to ask with regard to stress testing is what type of an assemblage does this practice belong as an element. In the light of this question, I would argue that the assemblage we are facing constitutes the third axis of vital system security which I would call “vulnerability reduction” that is distinct both from emergency preparedness and readiness.
In the case of emergency preparedness, as both Andy and you point out, the goal is to increase the response capacity of an apparatus and the infrastructural capacity that would supplement such a response if an imagined worst case scenario were to take place. And in the case of readiness, as Andy’s piece shows, the object of a simulation or preparedness exercise is the emergency response experts; so the object of intervention is the very subject of emergency response expertise. In both cases, the mode in which planning is enacted is preparedness.
Whereas in the case of stress testing, as we have discussed before the mode seems to be a hybrid between preparedness and prevention. On the one hand, one is doing an anticipatory form of planning by projecting into the future and its potential effects which means one is doing a preparatory mode of security. On the other hand, however, one is interested in the prevention of the break down of an infrastructure which would in turn constitute an event of emergency and crisis. In this respect, as you also point out the norm of this political technology is distinct from that of emergency preparedness, since the very point of vulnerability reduction is to decrease the likelihood of emergencies in the first place. It almost seems to be the case that one is interested in engineering (infra)structures that can absorb shocks of certain size through reforming the constitutent elements of the interiority of a substance (which has a curious familiarity to the case of “structural adjustment” as we know from your work on Russia).
The question you pose as to what its relation to systemic risk regulation is also helps us to analytically distinguish simulations in stress testing from other uses of simulations, since in this case what is simulated is not really an entire system, but only the certain nodes that are constitutive of the system of (nominal) flows overall. And I think these nodes are represented through their balance sheets that are supposed to reveal the market positions of these companies. Obviously not all nodes are stress tested, but rather only those that are “systemically important”. Then the real question is what a node that is systemically important is and how we know if one is so. At this point, it seems that the criteria of “too-big-to-fail” is used in a rather haphazard manner. But some of the actors that I am in touch with are already doing work on changing this criteria with a notion of “too-interconnected-to-fail” based on the techniques of payment flow topographies. However, for such a map to be produced one first needs a payments and settlements infrastructure for forms of payments other than those that are “large payments between banks” from which the Federal Reserve can collect data on the flow structure of the financial system…
Onur — I agree very much with the question you pose concerning capital adequacy. This is precisely the issue that emerged around financial innovation. Basically, a whole lot of people became convinced that all these new derivatives and other hedging techniques made it possible to massively increase leverage without simultaneously increasing the vulnerability of the system. I think you and I agree that this is why people who think that there is something to “explain” in terms of a new willingness to behave in a risky or speculative way are missing the point. The issue is that they convinced themselves that they were precisely *not* behaving in a risky or speculative way.
Let me push you a little harder on this issue of vulnerability reduction. I wonder how this relates, for example, to the case of catastrophe insurance that I have written about. In that case the purpose of new financial instruments — like cat bonds — was to spread risk over enormous capital pools, such that even a catastrophic event like a major hurricane might not be a real event in the global financial system as a whole. I think that this is, indeed, an example of vulnerability reduction in the sense that you discuss it here. It is “prevention” only in the sense that you are stopping a perturbation (a natural disaster, for example) from threatening the vital system in question, which is the insurance industry in its entirety, and in its relationship to that event (bankrupt insurance companies can’t make payments). Now I would argue that what is different about the issues we have been discussing in the financial system is not that it is an axis of vital systems security that we have not addressed before. I think we very much have. Rather, I think it is that this is a system in which, at th end of the day, there has to be a “hard” backstop, simply because there is no way to spread the risks of the financial system over some broader capital pool. This is what they thought they were doing; but actually they were in part increasing vulnerability by creating a mass of interdependent risks.
I guess what interested me in the NYT blog was the idea that the stress tests are significant because they have created the beginnings of an infrastructure for massively more intensive surveillance of risks in the system. That is definitely anticipating a key aspect of systemic risk regulation.
Yes, we definitely agree that the issue is not a “speculative ethic” per se as they, along with the regulators, thought that they were engaged in *safe* investment practices that not only allowed them to increase their leverage without increasing the vulnerability of the system, but also as the regulators kept emphasizing again and again increased *resilience* of the system and the overall economy. Along with the monetary policy, financial system was the core technology that allowed what Bernanke had called “great moderation” just a few years ago. (Apart from the derivatives, there is also a tread with regard to futures contracts that I have been trying to trace that were designed as mechanisms to smooth out fluctuations in raw commodity prices that one should include in this picture…)
What I meant with regard to vulnerability reduction was not necessarily specific to the financial system security, but rather a more amorphous assemblage form that we can find in other domains such as (physical) infrastructure construction such as dams or bridges thanks to civil engineering and their ‘stress and materials tests’ and the kind of economic planning expertise we saw at OEP. So, my contention was not necessarily to point that it is a “new” phenomenon to the extent that it is an issue not discussed before, but it is “new” in the sense that it is a new analytical distinction one can draw in the analysis of VSS.
I think the translation and hence the constitution of an event as an event from one domain into the other is a great way of posing the question of how to do vulnerability reduction. (The history of the history of stockpiling as a technique of creating buffers against (price) shocks and fluctuations, for instance, definitely work through such a logic.) After all capital adequacy levels are nothing but stockpiles of assets with distinct qualities and properties in terms of their degrees of liquidity.
It is definitely true that the problem is that one does not have a broader capital pool (other than the taxpayer money) by which one can insure the system. However, I am not sure to what degree they were by spreading risk increasing the vulnerability of the system per se, since such a statement in a subtle way implies that the increase in vulnerability was unilinear and in continuity with the vulnerability of the system prior to the insertion of these new techniques and instruments. I think the irony of the situation is that they were indeed reducing certain types of vulnerabilities in the banking system, but simultaneously introducing a new ontology of risk and vulnerability that was distinct from its prior forms that most of financial regulation started in the 1930s was a response to. In this vein, I think the interesting question, and one that will come back through neo-liberal reflections most likely, is who to blame? The investors who were trying to increase the safety of their life activities or the regulators who were doing a mode of regulation and security focusing solely on investor safety and not the system security per se?
Stephen and Onur – this is a really interesting exchange, and I agree that we see, in the current discussion of “stress tests,” the attempt to operationalize VSS norms such as resilience and vulnerability reduction. I have a fairly simple question: there is much discussion of “the financial system.” Is it obvious to everyone what exactly the elements of this system are? ‘Healthy’ banks, available credit, investment capital, some regulatory structures…. Or are its contours still being specified: for example, how does insurance fit in? What are the differences, within the system, between banks and other financial institutions? How does one know what the “critical nodes” of the system are?
Thanks for these questions Andy; they are very insightful.
I would say the answer to the question of what constitutes “the financial system” depends on the paradigm of regulation one ascribes to. That is to say from the perspective of regulatory framework that takes legally defined institutions, such as commercial and investment banks and insurance companies as in case of the Glass-Steagell, I think it is quite clear what “the system” is. And yet one can argue from our perspective that under Glass-Steagell one is not really concerned with a “system” per se, but rather with a particular sector of the economy which happens to be based on an exchange regime whose primary object itself is money and constituted by sub-sectors mentioned above. In this scheme, to be considered as an element of “the system” one has to be recognized by law, literally, as a firm engaged in specific activities as described in law. Thus, a company such as General Motors which is supposed to belong to the car industry or a consumer of credit such as you and me would not be considered as part of “the financial system”.
However, as it turns out at an ontological level “the system” not only overflowed the nominal boundaries of law even before the Glass-Steagell was revoked in the late 90s, but also it has become so extensive and central to the functioning of the economy that it has made the consumers and the producers (of material commodities and non-financial services) in a way part of “the system”. In a way, it would not be wrong to say the crisis is a regulatory crisis as much as if not more than a mere financial and economic crisis.
From the perspective of “systemic risk regulation”, however, “the financial system” is described not based on nominal categories provided by the legislature, but based on an ontology of nominal flows. The financial system is literally the space in which credit flows through exchanges between different entities. Hence, the question is not anymore regulating different legal entities, but securing the uninterrupted flow of such an ontology of credit. So from this perspective, there is no reason why “the consumer” or excessively financialized firms such as the GM should not be seen as elements of “the system”. However, the question whether they are recognized as such yet is an empirical one which is not so clear to me yet. On the one hand, they are since for both the Fed and the Congress consumer default and debt is obviously an important issue (as we can see in the current efforts to regulate credit card fees and interests of commercial banks). On the other hand, those who are thinking on systemic risk regulation “financial system” is still a space that seems to exclude the commercial consumption side.
As far as the “critical nodes” go, currently size is the prime criteria. The experts that I have been working on lately, however, offer a more nuanced way of determining the vitality of entities over the entire system through network optimization methods that to my understanding take an entire data set containing the all empirical possible flows in a space and reducing it through optimization methods to a deeper “structure” that is supposed to be the core of the financial system. Just to give a sense of the necessity of this process, in the case of large payments and settlements between all commercial banks in the US, one has a network topography of 70,000 interactions and 8,000 nodes in the form of “raw data”. Once this topology is optimized to its core structure, one gets a much more neat system consisting of only 60 or so nodes and only 180 interactions in which only 25 nodes are “critical”.
Onur — I guess this raises the question of what “size” means? It seems like “risk exposure” would be something like the relevant figure? One question that interests me is whether this would be a simple dollar amount or also take into account “likelihood” or “frequency.” If it does, one presumably needs a scenario — or a model — that is different from the network topology itself. (By the way: topology or topography?)
Again I think the answer to what “size” means depends on who we are thinking of. My understanding is for regulators who are at the top of the bureaucratic hierarchy “size” simply means the dollar amount of a bank as represented on their balance sheets. Because they were not prepared to do VSS, i think they simply had to rely on crude indicators as such (remember they did not know how much derrivatives a company was holding in its portfolio).
But when it comes to the systems engineers and more technically oriented economists, they have a more nuanced understanding that is not merely the monetary value of a company’s portfolio. And you are right about the need for a scenario; that is what “simulation” means anyways (though i think it is in a subtle way distinct from “scenario planning”). That is why they use “power law” distributions in their simulations I believe.
Oh and the right term is “topology”… i am not exactly sure what the distinction is and how important it is though.
This was a very interesting set of comments, and I’m sorry I am responding to it so very late.
1) Stephen, your point that the scenarios can only be adverse, not worst-case, as worst-case scenario is called nationalization, is very interesting, especially the difference you pick out between stress testing here and in other vital system. But I do not think this can count as any sort of validation or justification of the stress tests, insofar as the adverse scenario wasn’t even as bad as the actual scenario, the data points we already had, at the time the stress tests where announced. In addition, complex exposures were modeled more generously than normal commercoal loans (favoring the multinational zombies over the more prosaic regionals); Citi was allowed to count the value of asset sales it had not yet made; the number of personnel assigned to accomplish the tests was, obviously, pitifully, far from adequate; and after the results were in, the banks negotiated the amounts down behind closed doors over the course of a week. Whatever the logic of financial stress testing might be, in light of all of the above, I’m prepared to call the current example nothing but a farce, nothing but political gamesmanship. However Foucauldian we might be, at some point can’t we call ideology ideology?
2) For your reference, this is a very interesting passage from Geithner’s remarks at the Bond Market Association’s annual meeting in NYC in 2005:
“In this regard, an important question to ask is how well the current capital frameworks capture the possibility of extreme events, those far in the “tail” of the distribution. Stress testing can play an important role in addressing these concerns. Stress tests should allow institutions to assess likely losses under extreme market events, those that happen too rarely to be captured under traditional value-at-risk measures, but that could cause very significant losses to the institution should they occur. Institutions have long engaged in this kind of analysis for internal management purposes. Now, however, those at the forefront of risk management are assessing the adequacy of their value-at-risk results against stress losses and finding ways of integrating the results of stress testing into their capital frameworks. Indeed, an important aspect of our supervisory process includes critically assessing stress-testing regimes.
“More rigorous and comprehensive stress testing of large shocks across multiple markets, geographic regions and business lines is vital, particularly for systemically important institutions. In this context, we need to see more attention paid to risks to market liquidity, and the effects on market liquidity that could result from the exit of a major dealer. Stress regimes need to capture market risk and credit risk across the firm, incorporating exposures in priced credit products, such as credit default swaps and structured credit products, and the strong linkages between these priced credits and traditional credit instruments such as bank loans. Stress regimes need to take into account the effects of a firm’s own actions and trading strategies on market prices during times of stress, and the constraints on their room for maneuver imposed by size.”
3) Geithner continues further, and brings out another aspect security of the financial system, operational infrastructure, which I think must go under your heading of ‘resilience’? There are risks in the financial system that are not credit risks, but rather vulnerabilities in the infrastructure by which credit risk flows… and these too are vulnerable to exceptional events.
“While critically important, capital alone does not define an institution’s strength. It is vitally important for firms to continue to invest in strong internal controls and to make sure that advances in the operational infrastructure keep pace with rapid growth, particularly in complex transactions.
“Strong operational controls can help ensure smooth market functioning in times of stress, and when they are weak they can exacerbate adverse market dynamics. An important element of a sound operational control environment is timely and accurate information for senior managers, especially when it is most critical for them to have a clear picture of the firm’s exposure. As such, strong operational controls can help reduce some dimensions of uncertainty and therefore help markets function better in conditions of stress.”
4) Re: Andrew Lakoff’s excellent question on what constitutes the financial system, and Onur’s elaboration about the position of consumption, Obama’s announcement of the Consumer Financial Protection Agency, stripping some powers from the Fed in order to vest them there, seems to me to be a recognition that the consumer is in fact part of the financial system. It also, interestingly, purposefully institutionalizes an opposition between the systemic risk regulator (Fed) and this new agency. I wonder whether this is bad risk management infrastructure, because it splits the authority over a singular interconnected system, or excellent, because it might prevent the distribution of protection vs. risk from becoming too concentrated at the top (megabanks) vs. at the bottom (consumers).