Reflexive modeling: profiting from financial models without being trapped in them

The credit crisis has imposed on Americans a crash course on the risks of financial models. If derivatives, as Warren Buffet famously put it, are “financial weapons of mass destruction,” models are now seen as the nuclear physics that gave rise to the bomb — powerful, incomprehensible and potentially lethal. Given their dangers, what should Wall Street do with its models?

At one extreme, skeptics have attacked models for their unrealism, lack of transparency, and limited accountability. Models, they charge, are black boxes that even expert users fail to understand. Models become dangerously inaccurate when the world changes. And whenever a bad model fails, it is all too easy for traders to conjure up the “perfect storm” excuse. Wall Street, the skeptics conclude, needs to curtail its addiction to models.

At the other extreme, academics in finance and Wall Street practitioners dismiss the backlash as barking up the wrong tree. Models certainly produce the wrong results when fed the wrong assumptions. But the real culprit in this case is not the model, but the over optimistic trader in his greedy quest for the bonus. Paraphrasing the National Rifle Association (“guns don’t kill people, people kill people”), defenders of models place the blame with bad incentives: “models don’t kill banks,” we hear them saying; “bankers kill banks.” To the proponents of modeling, then, the crisis underscores the need for yet more calculations. That is, for bigger and better models.

Does Wall Street need more models or less models? We see this as a false choice. The debate, in our view, needs to shift from the models themselves to the organization of modeling. We have identified a set of organizational procedures, which we call “reflexive modeling,” that lead to superior financial models.

Consider, first, what a financial model ultimately is. Whether as an equation, an algorithm or a fancy Excel spreadsheet, a financial model is no more than a perspective, a point of view about the value of a security. Models are powerful: they reveal profit opportunities that are invisible to mom-and-pop investors. But there’s a catch: they do not always work. Because stock prices are the outcome of human decisions, financial models do not actually work like the iron law of Newtonian gravity.

Models, then, pose a paradox. They hold the key to extraordinary profits, but can inflict destructive losses on a bank. Because a model entails a complex perspective on issues that are typically fuzzy and ambiguous, they can lock traders into a mistaken view of the world, leading to billionaire losses. Can banks reap the benefits of models while avoiding their accompanying dangers?

Our research suggests how. We conducted a sociological study of a derivatives trading room at a large bank on Wall Street. The bank, which remained anonymous in our study, reaped extraordinary profits from its models, but emerged unscathed from the credit crisis. For three years, we were the proverbial fly on the wall, observing Wall Street traders with the same ethnographic techniques that anthropologists used to understand tribesmen in the South Pacific (The study can be downloaded at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1285054).

The key to outstanding trades, we found, lies outside the models. Instead, it is a matter of culture, organizational design and leadership.

The bank that we observed introduced reflexivity in every aspect of its organization. From junior traders to their supervisors, everyone at the bank was ready to question their own assumptions, listen for dissonant cues, and respect diverse opinions.

How? As many have already suggested, individuals certainly matter. The bank hired people with a healthy dose of humility and an appreciation for the limits of their smarts. This often meant older traders rather than younger hotshots.

But the key to the bank’s reflexiveness did not just lie in individuals. By reflexiveness we don’t mean super-intelligent traders engaged in some heroic mental feat – splitting and twisting their minds back on themselves like some intellectual variant of a contortionist. Reflexivity is a property of organizations.

The architecture of the bank, for instance, was crucial. The open-plan trading room grouped different trading strategies in the same shared space. Each desk focused on a single model, developing a specialized expertise in certain aspect of the stocks.

To see why this was useful, think of a stock as a round pie. Investors on Main Street often eat the pie whole, with predictably dire consequences. The professionals that we saw, by contrast, sliced stocks into different properties. Each desk was in charge of a different property, and the different desks then shared their insights with each other. This could happen in a one-minute chat between senior traders across desks, or in an overheard conversation from the desk nearby. This communication allowed traders to understand those aspects of the stock that lay outside their own models — the unexpected “black swans” that can derail a trade.

Sharing, of course, is easier said than done. The bank made it possible with a culture that prized collaboration. For instance, it used objective bonuses rather than subjective ones to ensure that envy did not poison teamwork. It moved teams around the room to build the automatic trust that physical proximity engenders. It promoted from within, avoiding sharp layoffs during downturns.

Most importantly, the leadership of the trading room had the courage to punish uncooperative behavior. Bill, the manger of the room, made it abundantly clear that he would not tolerate the view, prominent among some, that if you’re great at Excel, “it’s OK to be an asshole.” And he conveyed the message with decisive clarity by firing anti-social traders on the spot — including some top producers.

In other words, the culture at the bank was nothing like the consecration of greed that outsiders attribute to Wall Street. We refer to it as “organized dissonance.”

The bank went so far as to use its own models to be reflective upon modeling. The traders translated stock prices into the model estimates developed by their competitors. This information often planted healthy doubts on the traders’ own estimates, sending them back to the drawing board when necessary. Interestingly, this form of “reverse engineering” was accomplished by using the traders’ own models in reverse, much as one can flip a telescope to make something close-up look like it is far away.

Our study suggests that a lack of reflexivity –that is, the lack of doubt on the part of banks– may be behind the current credit crisis. We are reminded of infantry officers who instructed their drummers to disrupt cadence while crossing bridges. The disruption prevents the uniformity of marching feet from producing resonance that might bring down the bridge. As we see it, the troubles of contemporary banks may well be a consequence of resonant structures that banished doubt, thereby engendering disaster.

This blog post was coauthored with David Stark. David Stark is chair of the Department of Sociology at Columbia and is the author of The Sense of Dissonance (Princeton University Press, 2009).