High Frequency Trading and market sociality: “it’s really machines all the way down”?

October 21, 2010

Following up on Daniel’s point on HFT missing from mythical accounts of Wall St, and continuing thoughts from an earlier post on the May 2010 stock market plunge: here is an interesting commentary from Ars Technica on High Frequency Trading and the crash. Bringing a Computer Science perspective, the article considers HFT as a case of large system:

the market as described in the SEC report looks like an awful lot like a giant, multithreaded software application. And on May 6, the market did what every piece of multithreaded software eventually does in response to just the wrong mix of execution conditions and inputs: it crashed.

The author then narrates the mechanism of the crash, based on the SEC report, as one where thousands of algorithms are interpreting each other’s response. The idea here is not just that machine trading might generate its own panics, but we also get a picture of how derivative markets and equities markets are linked–via the activity of HFTs and other algorithms–and how humans fit into this operation. For example,

The algorithms that buy and sell stocks in the equities market were using the previous few minutes’ action in the derivatives market as inputs to guide their trading, and when they registered the giant sell-off described above, some of them had safety controls that told them to stop trading so that the humans could take a look to see what was wrong. And as these algorithms pulled out of the market, the market got more illiquid and prices dropped faster.

A CS perspective can provide us with a new description which entails new questions:

to be a single multithreaded app, as opposed to an unrelated collection of multithreaded apps, the different threads must somehow interact with one another. In other words, the threads must share and jointly modify some kind of state. What state do the various apps and algorithms that run on Wall Street’s machines share? At the very least, every part of the market shares the quote feed.

The price of, say, AAPL at any given moment is a numerical value that represents the output of one set of concurrently running processes, and it also acts as the input for another set of processes. AAPL, then, is one of many hundreds of thousands of global variables that the market-as-software uses for message-passing among its billions of simultaneously running threads. Does it really matter that those threads are running on separate machines at different institutions?

It is worth reading the Comments section where people consider the proposition and debate whether HFT is really a multithreaded application or something else–a “complex adaptive system”, an “ecosystem”, a “capacitor”, a case of “control systems theory”, and so on.

Even if it’s not “machines all the way down”, as the author at one point suggests, these are discussions to which sociologists of finance should pay attention because they actually deal with the substance of current financial markets, as social spaces with interactions. The intricacies of how machines deal with each other should really become part of the fundamental understanding of market behavior (for a succinct argument see e.g. Callon and Muniesa 2005).

9 Responses to “High Frequency Trading and market sociality: “it’s really machines all the way down”?”

  1. danielbeunza Says:

    Zsuzsi, this is very intriguing. Your post is similar to a remark I once heard form a colleague at the information systems department at the LSE: is the entire credit crisis just a case of computer glitch?

    In relation to the post, I have a naive question and a comment. The naive question is short: what is a multi-threaded application?

    As for the comment, it is interesting that according to this report the problem came in the transition from automated to manual trading. This is common to other technological systems with distributed cognition. Reportedly, some early models of fly-by-wire Airbus planes crashed because the automatic pilot did things that the pilot did not see being done (as he would, if they were done by a co-pilot).

    Studying distributed cognition, however, takes several skills at the same time. One needs to understand the machine, and the socio-technical use that the users are making of the machine. In this sense, it is a very welcome development that authors of the report on the flash crash have hired an anthropologist of finance, Melissa Fisher, who is a good friend and a terrific scholar. Well done, members of the commission.

  2. multithread Says:

    (Multithreading as a widespread programming and execution model allows multiple threads to exist within the context of a single process. These threads share the process’ resources but are able to execute independently. The threaded programming model provides developers with a useful abstraction of concurrent execution.

    However, perhaps the most interesting application of the technology is when it is applied to a single process to enable parallel execution on a multiprocessor system.

    This advantage of a multithreaded program allows it to operate faster on computer systems that have multiple CPUs, CPUs with multiple cores, or across a cluster of machines — because the threads of the program naturally lend themselves to truly concurrent execution. In such a case, the programmer needs to be careful to avoid race conditions, and other non-intuitive behaviors. In order for data to be correctly manipulated, threads will often need to rendezvous in time in order to process the data in the correct order. Threads may also require mutually-exclusive operations (often implemented using semaphores) in order to prevent common data from being simultaneously modified, or read while in the process of being modified. Careless use of such primitives can lead to deadlocks.)

  3. zsuzsannavargha Says:

    multithread, thanks for the detailed explanation. A shorter definition is in the article itself.

    Daniel, the Airbus case is really interesting and incidentally, supports your point about the need to not separate the users of the machines from the machines. I completely agree with that. It just seems that even in the sociology of finance there could be more attention to what the trading machines are doing live, and that’s where I thought the engineers’ view is so valuable. For example, the recognition that the quote feed is the key state that all of the threads modify and which creates the problem of access to the current quote and techniques to solve access such as described above (queuing, semaphores), can be used to think about how machines are designed to achieve social order and how they do it in practice. We can bring in Alex Preda’s and Fabian Muniesa’s work on stock tickers to understand HFT.

  4. Danielbeunza Says:

    Thanks! I can see how the analogy can apply to the flash crash, but i shpuld add that in this case there were more than one exchane involved, each with its own multi-threaded system.

    So what are we talking about here? That exchanges are like threads, and the system is mukti-thread, or that one specific exchange (arca?) crashed?

  5. Danielbeunza Says:

    Zsuzsi — agreed. The trafic light metaphor of social order in an electronic world is excellent. Reg-NMS, from that standpoint, can be thought of changing the circulation code in a city, with natural unintended consequences.


  6. Things are so bad in this economy that any help is great. I hope things turn around soon enough.


  7. Thanks! I can see how the analogy can apply to the flash crash, but i shpuld add that in this case there were more than one exchane involved, each with its own multi-threaded system.

    So what are we talking about here? That exchanges are like threads, and the system is mukti-thread, or that one specific exchange (arca?) crashed?

  8. Brian Sullivan Says:

    This article may be of interest

    http://www.debtgeneration.org/excerpts1vol.html


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: