Emanuel Derman on the accuracy of models

September 9, 2008

What better fun than a good old academic debate? By now, the readers of SocFinance all know of the “Black-Scholes accuracy” controversy that erupted this summer at the Academy of Management.

To recap, Yuval’s presentation in a session I that co-organized made the provocative claim that Black-Scholes succeeded despite being inaccurate. One of the two scheduled discussants, chosen for his stated affinity to the social studies of finance, attacked Yuval’s claim with an energy rarely seen at the Academy. It was, so to speak, as if the best man in a wedding suddenly refused to sign on, only to then launch into a litany of the reasons why the wedding should be stopped. The surprise was, to say the least, widespread.

Rapidly recovered from shell-shock, Yuval has been blogging on the issue with characteristic inspiration and drawing an inordinate share of good comments. So, rather than add more gasoline to the fire, I thought I would enlist Emanuel Derman, one of the most respected academic-practitioners in quantitative finance. Derman, co-author with Fisher Black of the Black-Derman-Toy interest rate model, chair of Columbia’s Financial Engineering program and author of “My Life as A Quant,” writes the following:

“Creating a successful financial model is not just a battle for finding the truth, but also a battle for the hearts and minds of the people who use it. The right model and the right concept, when they make thinking about value easier, can stick and take over the world. A firm whose clients start to rely on the results that its model generates can dominate the market. This is what happened with Salomon’s concept of option-adjusted spread.” (p. 198)

“When I returned to Goldman in that first month of 1990 … I soon met Dan O’Rourke … Dan and I saw the world similarly. First, we both understood that models were inadequate, no matter how good they were.” (p. 212)

“The prices of out-of-the-money puts were unexpectedly larger than those of other options… Everyone referred to this asymmetry as “the smile”… the existence of the smile was completely at odds with Black-Scholes’ twenty-year-old foundation of options theory. And, if the Black-Scholes formula was wrong, then so was its predicted sensitivity of an option’s price movements in its underlying index, it’s so-called delta. In this case, all traders were using the Black-Scholes model’s delta were hedging their books incorrectly.” (p. 225).

What to make of Derman’s claims? For one, his views are not inconsistent with those of another renown intellectual in the world of quantitative finance, Nassim Taleb. Taleb, author of a fascinating treatise on Knightian Uncertainty titled The Black Swan, was recently quoted in the Financial Times on his views on Black-Scholes:

At the heart of his objections is the notion that the model does not give you a prediction of an option’s value in practice, as the market and many academics say it does. Rather, it tells you how much you would have to pay for a purely theoretical position.

But Mr Taleb is unbowed. “I am simply questioning and my questions are valid. I am not doing this for the money. The fact is other models are not subject to the same unrealistic assumptions behind Black-Scholes, even though it is Black- Scholes that essentially launched the quant revolution.”

What to make of all this resistance to the accuracy of models? Are Derman and Taleb playing enfant terrible? Are the purveyors of models selling us lemons? At the core of the controversy, I believe, is a fundamental question that orthodox finance has not, to my knowledge, yet addressed: what is the practical accuracy of a theoretically correct model?

4 Responses to “Emanuel Derman on the accuracy of models”

  1. […] one any longer. Why? Mainly, because the discussion and the evidence around it (see the quotes from Derman and others) show us that many market participants were fully aware that the Black-Scholes-Merton model was not […]

  2. tom brakke Says:

    Here’s a couple of things I wrote on models and how they are used by investment firms, preceded in each case by the posting summary:

    Investment organizations rely on models of all kinds to do their business, and need to properly vet those models at their creation and throughout their useful lives: http://researchpuzzle.com/blog/2008/09/04/finding-a-valid-approach/

    Debates on global climate change tend to devolve into the predictable points of view of the opposite camps, while an analyst needs to take a more disciplined approach to the question at hand: http://researchpuzzle.com/blog/2008/07/03/warming-up-the-models/

  3. In answer to the question” what is the practical accuracy of a theoretically correct model?” I ask “What do you mean by ‘correct'”?

  4. […] SocFinance just reminded me there is life outside the Wall Street bailout. The post, titled “Emanuel Derman on the Accuracy of Models” engaged the criticismsl of Emanuel Derman and Nassim Taleb of financial models, which are […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: