Follow up to the Market Devices session in AoM

August 31, 2008

As promised, here are some notes following the Market Devices session that Daniel Beunza, Dan Gruber and Klaus Webber arranged (thanks again!). I refer here mostly to the comments our discussant, Bruce Kogut, made. He made some excellent points there. In fact, they made me think critically about the core elements of the performativity approach and, as a result, sharpen the argument. Actually, having read some of the comments to this post in orgtheory, especially Ezra Zuckerman’s, I think that this follow up corresponds with that discussion too.

Bruce refered to the empirical point in my point that says that Black-Scholes-Merton was not accurate and he asked something along the lines of ‘how can one say that model X was not accurate if there was no alternative (or that there were and model X was the least inaccurate one). Here Bruce touched one of the core points of performativity. On one hand, the historical data shows that Black-Scholes-Merton was never very accurate and, as you rightly pointed out, the actors were (or became) fully aware of that fact. So, do we have here a case whereby Black-Scholes-Merton was simply the least inaccurate model in existence?

This question is penetrating because it drills to a core concepts that the performativity approach has been presenting (but, until now, was not explicit enough about it) of ‘scientific’ and ‘organisational’ accuracies. Originating in the sociology of science, performativity ‘plays against’ the scientific concept of accuracy and validity. That is, the concept according to which predictions are valid if, and to the degree to which, they correspond to a universal set of assumptions. Taking this concept to an extreme, predictions can never ‘become accurate’ as a result of interactions between the prediction and the world around it. Hence, theories or models can become better predictors of reality only as a result of improvements in the theories themselves.

The sociology of scientific knowledge claims that ‘scientific’ accuracy is created and maintained by an ‘organisational’ element. Predictive models are typically subjected to a continuous set of public experiments and persuasion trials where their predictive powers are challenged. Hence, to have scientific accuracy, a stable set of ties has to emerge and exist between different actors. Such a network of ties represents an organisational form that, if structurally stable, makes the model ‘organisationally accurate’. That is, enough actors (and ones in influential positions) share opinions regarding the usefulness, efficacy of the practices and/or technologies that use the model.

So, was the Black-Scholes-Merton model simply the best prediction method available, in spite of the fact it that was not accurate scientifically? The interdependency between scientific accuracy and organisational accuracy tells us that we cannot judge the scientific accuracy of a predictive model while separating it from its organisational accuracy (especially in a ‘public experiment’ environment such as financial markets). In fact, the important element here, as you rightly pointed out in your comments, is that market participants decided to develop applications based on the Black-Scholes-Merton model (e.g. margin calculations, calculation of required capital) and, crucially, to develop interdependencies based on the model. This structural dynamic is what made Black-Scholes-Merton ‘organisationally accurate’: an inter-organisational space, composed of options exchanges, the clearinghouse and the SEC, emerged where the model, its results and their validity were accepted (and acted upon). Note that this is not an ‘anything goes’ approach; it does not predict that any model could be accepted and made into an industry standard. It does suggest, however, that inter-subjectivity among organisational actors is crucial for the acceptance of risk management models and that we should examine the dynamics of that process when analysing such inter-organisational environments, such as modern financial markets.

4 Responses to “Follow up to the Market Devices session in AoM”

  1. danielbeunza Says:

    I fully agree. But I notice that your drifting away from the view that Black-Scholes was inaccurate, and to challenge the notion that accuracy exists at all, given the interdependence between the model and the world. While I agree, I also somehow wish for a sharper challenge to the essentialist notion that better models are the more accurate ones.

  2. typewritten Says:

    Well, “The Usefulness of Inaccurate Models” is a hell of a title in the line of a “sharper challenge”!

  3. marthapoon Says:

    I’m with Daniel on this one. Instead of asking about accuracy, I wonder if a useful question might be to ask: According to what ‘test’ is a model made to be accurate and/or useful? That is, what are the means / methods by which it is shown that BSM is inaccurate but less inaccurate than other models?

    Posing the question in this way dissolves the distinction between ‘scientific’ and ‘organizational’ accuracy drawn by Yuval in that both kinds of accuracy become types of arrangements that must be achieved through ongoing activity. The distinction between these types of accuracy would then be a product of specific arrangements (i.e. the rise of new methods of testing the model that produce different results as to its qualities).

    My question to Yuval is, when did the ‘inaccuracy’ of the model become apparent? Was this a retrospective analysis, or where users accutely aware of this while they were adopting the tool for its practical virtues?

  4. […] Feasibility of PLoS ONE’s Peer Review Model? Saved by HatakeKakashiSama on Sun 30-11-2008 Follow up to the Market Devices session in AoM Saved by LarsKasper on Sun 30-11-2008 Remembering the Model T Saved by newball on Mon 10-11-2008 […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: