Emanuel Derman on the accuracy of models

What better fun than a good old academic debate? By now, the readers of SocFinance all know of the “Black-Scholes accuracy” controversy that erupted this summer at the Academy of Management.

To recap, Yuval’s presentation in a session I that co-organized made the provocative claim that Black-Scholes succeeded despite being inaccurate. One of the two scheduled discussants, chosen for his stated affinity to the social studies of finance, attacked Yuval’s claim with an energy rarely seen at the Academy. It was, so to speak, as if the best man in a wedding suddenly refused to sign on, only to then launch into a litany of the reasons why the wedding should be stopped. The surprise was, to say the least, widespread.

Rapidly recovered from shell-shock, Yuval has been blogging on the issue with characteristic inspiration and drawing an inordinate share of good comments. So, rather than add more gasoline to the fire, I thought I would enlist Emanuel Derman, one of the most respected academic-practitioners in quantitative finance. Derman, co-author with Fisher Black of the Black-Derman-Toy interest rate model, chair of Columbia’s Financial Engineering program and author of “My Life as A Quant,” writes the following:

“Creating a successful financial model is not just a battle for finding the truth, but also a battle for the hearts and minds of the people who use it. The right model and the right concept, when they make thinking about value easier, can stick and take over the world. A firm whose clients start to rely on the results that its model generates can dominate the market. This is what happened with Salomon’s concept of option-adjusted spread.” (p. 198)

“When I returned to Goldman in that first month of 1990 … I soon met Dan O’Rourke … Dan and I saw the world similarly. First, we both understood that models were inadequate, no matter how good they were.” (p. 212)

“The prices of out-of-the-money puts were unexpectedly larger than those of other options… Everyone referred to this asymmetry as “the smile”… the existence of the smile was completely at odds with Black-Scholes’ twenty-year-old foundation of options theory. And, if the Black-Scholes formula was wrong, then so was its predicted sensitivity of an option’s price movements in its underlying index, it’s so-called delta. In this case, all traders were using the Black-Scholes model’s delta were hedging their books incorrectly.” (p. 225).

What to make of Derman’s claims? For one, his views are not inconsistent with those of another renown intellectual in the world of quantitative finance, Nassim Taleb. Taleb, author of a fascinating treatise on Knightian Uncertainty titled The Black Swan, was recently quoted in the Financial Times on his views on Black-Scholes:

At the heart of his objections is the notion that the model does not give you a prediction of an option’s value in practice, as the market and many academics say it does. Rather, it tells you how much you would have to pay for a purely theoretical position.

But Mr Taleb is unbowed. “I am simply questioning and my questions are valid. I am not doing this for the money. The fact is other models are not subject to the same unrealistic assumptions behind Black-Scholes, even though it is Black- Scholes that essentially launched the quant revolution.”

What to make of all this resistance to the accuracy of models? Are Derman and Taleb playing enfant terrible? Are the purveyors of models selling us lemons? At the core of the controversy, I believe, is a fundamental question that orthodox finance has not, to my knowledge, yet addressed: what is the practical accuracy of a theoretically correct model?