On financial models: practice vs. theory

October 2, 2008

A comment on an previous post on SocFinance just reminded me there is life outside the Wall Street bailout. The post, titled “Emanuel Derman on the Accuracy of Models,” engaged some fascinating criticisms by Emanuel Derman and Nassim Taleb of financial models, which are very much consistent with Yuval’s.

My original post ended with an open question: what is the practical usefulness of theoretically correct models? To this question, Derman just offered his own answer:

In answer to the question” what is the practical accuracy of a theoretically correct model?” I ask “What do you mean by ‘correct’”?

Rather than reply with a difficult-to-see comment, I took it as an opportunity to write new post.

What do I mean by “correct”? What I have in mind is the distinction between functional relationships and specific values. For instance, Black-Scholes can be though of a model that, essentially says the following:


where y equals option value, and x equals the future volatility of the underlying stock. Let’s assume this is true (and the readers of MacKenzie and Millo know it’s not, but bear with me).

How useful is this? Very useful, one would think. If you want to know y, all you need is to find out x, and you’re done. But this overlooks the problem of estimating x. Now, a rational expectations model would argue that market actors will, over time and through trial and error manage to develop unbiased estimates of x. In practice, however, things are not so easy. Market actors may make mistakes, and x itself is probably a function of other parameters… the model of which could be unknown, or unstable over time.

My point, then, is that even if the original model, f(x), is correct, putting it to work still is problematic.

How do you get around that problem? Following my own research, what I’ve seen is that arbitrageurs on Wall Street use models in a different way — to reflect on their own estimates by looking at the implied estimates that their competitors developed. This is called “backing out” x.

But — wait! This itself creates a new problem. If you use the model to bet on y as well as to back out x from the prices of y, the same model is being used for two different things… to represent x, and to intervene in y. And this, in turn poses new and fascinating epistemic challenges. Stay tuned.

6 Responses to “On financial models: practice vs. theory”

  1. Mike Says:

    Interesting conversation about BS modeling. You are correct that the only input into the equation is the variance of the underlying. There is some research that, in line with existing research about cognitive and ‘behavioral’ bias in economic decisions and asset pricing that is a popular research topic in economics/finance, the estimation of that variance is subject to the same cognitive bias.

    This paper kicked off that literature:


  2. danielbeunza Says:

    Mike – thanks for the reference. I checked it out. More broadly, I confess i find behavioral finance, and especially cognitive biases, often uninteresting. My fieldwork on arbitrage has revealed that these people do not simply make decisions on their head. Rather, they use models, spreadsheets, databases. Studying these people as if they were “naked” decision-makers amounts to studying astronauts as if they were boyscouts.

  3. tmsiva Says:


    First, thank you for your talk yesterday at Columbia. I wish we had gotten to talk a little more about Parsons’ Pact in relation to the rethinking of the division between politics and economics that has been made possible recently. (Hint: I think the passive tense I just used in that last sentence is a mistake!)

    Second, I’m not sure what you mean by the following being a problem: “If you use the model to bet on y as well as to back out x from the prices of y, the same model is being used for two different things… to represent x, and to intervene in y.”

    Surely Ian Hacking’s old argument fits perfectly here. The BSM model represents in order to intervene. That’s what made it popular. And what I have been taking MacKenzie to be arguing all along.

    The point about backing out is, I think, that traders back out to implied vols in order to, in fact, trade. Witness price quotes (and from that settlements) in BSM vols in some markets.

    Am I misunderstanding the problem you’re addressing?

  4. danielbeunza Says:

    Tmsiva — clearly you are well versed with how derivatives trading works in practice. The point of differenciating between intervening and representing is that a model can be a good representation but a bad tool for intervention.

    Because a model (the representation) does not give you a numerical valuation of the independent variables, its use in trading (intervention) is only as good as the estimates of those variables you have. People try to refine their estimates by comparing them with the backed-out estimates from their rivals. As I see it, using the model to back out volatility is part of the intervention strategy (but I could be wrong here).

    The problem in intervention arises using implied volatility. When the implied volatility is different from your own estimate, is that an opportunity or a sign that you´re using the wrong model?

    Well, you can never know from the data, because implied volatility is itself the outcome of the model in the first place.

    That is, I think, the key problem that traders contend with in using models.

  5. tmsiva Says:

    Thanks for the clarification, Daniel.

    I think there are two related distinctions worth making regarding the BSM model. The first is chronological: roughly, pre-1987 and post-1987. The second is between traders, risk managers, quants, etc., who are continuing to try to model options (to find better and better models), and traders, etc., who in a post-’87 world use the BSM model to help them trade implied volatility itself.

    “The problem in intervention arises using implied volatility. When the implied volatility is different from your own estimate, is that an opportunity or a sign that you´re using the wrong model?”

    I think rather that the problem is of knowing when to shift from trading based on one’s estimations (e.g. having a view on implied volatility) to finding better models and back again. In other words, I don’t think anyone really believes, post-’87, that their model is “right” or even that that’s in principle possible. The relationship is much more subtle than that, hence Derman’s comment to the previous post.


  6. danielbeunza Says:

    I absolutely agree.

    The question is, what do you mean by “more subtle”. I will be posting my own version of “much more subtle” in a full paper in a few days… hopefully we can continue the discussion from there.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: