Ten Years after the Flash Crash, We Still Need to Make Algorithmic Trading Less Risky. Can Culture Save The Day?

May 6, 2020

From Christian Borch

Ten years ago this day, US markets opened to a palpable sense of unease. Investors were concerned about the ongoing Greek debt crisis, which grew from the global financial meltdown in 2008.

At 2:32pm Eastern Daylight Time, what started as nervousness and hesitation collapsed into pure terror as US markets nosedived. In 36 minutes, the Dow Jones Industrial Average plummeted 998.5 points – more than nine percent – in one of the largest and fastest same-day declines in the history of US markets. The greater part of this drop saw market values in the range of $1 trillion evaporate within five minutes.

As markets continued their free fall, trading was suspended in panic. When trading activity resumed, an equally remarkable and opposite phenomenon happened. Prices rebounded, and just half an hour later, US markets had recovered most of their losses.

Since all this happened in a blink, commentators have dubbed this astonishing event the “Flash Crash.” No solid estimates exist of specific investors’ gains and losses from the Flash Crash, but prices recovered quickly enough that the event had no demonstrable economic effects in the long term – unlike the 2008 global financial crisis and the ongoing Covid-19 pandemic.

It might surprise some, however, that the Flash Crash had little to do with the European sovereign debt crisis itself. In fact, it was precipitated by a group of smaller, underestimated agents: fully automated, high-frequency trading algorithms. That afternoon in May, these algorithms entered an escalating feedback loop, their cascading orders triggering a brief but precipitous market crash.

The Flash Crash is a defining moment in the modern history of financial markets. It reveals the Janus face of present-day trading algorithms – algorithms can unleash great instability when tossed together, despite their reputation as impartial, rational actors. The Flash Crash also laid bare how little we know about the inner machinations of trading algorithms, as the confusion of US regulators investigating the phenomenon later showed.

In 2010, the Commodity Futures Trading Commission (CFTC) and the Securities and Exchange Commission (SEC) issued a joint report arguing that a large sell order from Waddell & Reed Financial, a Kansas City-based asset management firm, had triggered unanticipated activity among high-frequency trading algorithms. The algorithms then engaged in a game of “hot potato,” swiftly buying and selling from one another in a downward spiral that caused the Flash Crash.

The CFTC–SEC explanation of the Flash Crash sparked controversy when it was released, and its findings have since been contested by academics and market observers. Years later, US regulators unexpectedly changed their tune when they identified the British trader Navinder Singh Sarao as the Flash Crash’s main culprit. Sarao had purportedly designed – out of his parents’ home in Hounslow in the outskirts of London – a set of high-frequency trading algorithms aimed at manipulating market prices, which ultimately caused the markets to crash. Sarao eventually pleaded guilty to charges of market manipulation, but experts remain divided on whether any individual’s manipulative algorithms can bring US markets to their knees. Are financial algorithms truly that powerful – or are markets penetrated by them inherently that fragile?

After the Flash Crash, regulatory authorities implemented “circuit breakers,” which halt trading if prices move too much too quickly, as their main line of defence. When the Covid-19 pandemic struck global markets in March this year, these circuit breakers kicked in to cushion the outbreak’s ripples in the market. But even circuit breakers – a triage rather than vaccination – have not deterred the several smaller flash crashes that have occurred since 2010.

Even though the Flash Crash occurred ten years ago, we are still in the dark about how algorithmic markets work and how to prevent their ills. One thing is clear, however: the algorithmic nature of contemporary financial markets renders them more vulnerable to crashes, which begs the question of whether we can prevent such sudden crashes when financial algorithms continue to operate in our markets.

In the past six years, colleagues and I have interviewed executives, traders, programmers, exchange officials, and regulators in the US and Europe to better understand how algorithms shape today’s financial markets. Through two research projects with high-frequency trading firms of all sizes and sorts, we have observed algorithmic traders at work and studied their daily operations, concerns, tasks, and overall cultures.

We have realized that individual firms with lax practices for development, testing, and monitoring algorithms face devastating risks. Take the example of Knight Capital, a major US trading firm that suffered a shattering algorithmic mishap in August 2012. The firm’s new software unexpectedly triggered dormant code, which generated orders that lost the company more than $460 million in just 45 minutes. Unable to recover from this mishap, Knight Capital was soon acquired by a competitor.

Though Knight Capital may seem an unlucky exception, a financial culture that does not support the rigorous use of financial algorithms risks markets themselves. Without a thorough understanding of their own algorithms, can firms avoid triggering devastating feedback loops once algorithmic interaction sets in? The cultures of algorithmic trading firms matter in designing more robust markets, and we need to better understand how these firms are organized, how they operate, and how their staff think and work. Crucially, we need to grasp their procedures for developing, testing, monitoring, and understanding trading algorithms long before their cascading effects begin.

There is good news, however – a small group of firms has developed organizational cultures that stress rigor at all levels of algorithmic design and trading. These firms strive to eliminate any negative effects their algorithms may have on markets, and they have developed an ethos built on ensuring market integrity in every respect.

This ethos involves establishing procedures to make sure financial algorithms cannot engage in manipulative behaviour. These firms obsess over bug detection as well as continuous monitoring and testing. They learn from the algorithmic incidents that inevitably occur, whether triggered internally or from outside their operations. Throughout, these firms expend massive, ongoing efforts to comprehend how and why their algorithms behave the way they do, alone and together with other algorithms.

Today, European regulators require some minimum testing and monitoring of firms’ trading algorithms. This is an important first step, but hardly a sufficient one if we wish to avoid a consistent stream of flash crashes. We need a more fundamental attention to the cultures of trading firms, and perhaps even a paradigm shift. After the 2008 financial crisis, many called for banks to implement a new corporate culture with fewer detrimental risk incentives, though little progress has since been made in that arena. Even so, the potential consequences of algorithmic spirals running amok suggest that we should not shy away from driving such change while we still can.

Firms whose cultures, business models, and algorithmic designs emphasize market integrity offer a way forward in ensuring the health and stability of financial markets. Avoiding future flash crashes and mastering the collective power of financial algorithms will require an industry-wide commitment to cultures of accountability, transparency, and integrity.

Our algorithms will settle for nothing less.

Christian Borch is Professor of Economic Sociology and Social Theory at the Copenhagen Business School and the PI of the ERC-funded AlgoFinance research project. His latest book is Social Avalanche: Crowds, Cities and Financial Markets (Cambridge UP, 2020).



3 Responses to “Ten Years after the Flash Crash, We Still Need to Make Algorithmic Trading Less Risky. Can Culture Save The Day?”

  1. This is a terrific piece, Christian! Thanks for celebrating this anniversary.

    For what it’s worth, I think that your piece is particularly great because it underscores the continuities between face to face and algorithmic markets: rigorous organizational practices are important both for ‘analog’ and ‘digital’ organizations, and while speed of transactions, etc, certainly matter, organizations that have lax development, testing and surveillance practices are bound to face greater risks, all things considered.

    There are two other minor things I guess matter for discussions about the flash crash, ten years on.

    The first is about how exceptional the event actually was (and whether it had any long term repercussions on market stability). This was not the first flash crash (the 1962 ‘Kennedy slide’ illustrates this to a degree–it was an equally rapid market movement connected to market makers withdrawing from their obligations). Markets are prone to fat tailed events more than we would like, but the issue is whether these have dramatic effects on broader economic outcomes (like capital raising, confidence in markets, degrees of financialization, portfolio performance, etc.) I would guess (though this requires an empirical answer) that despite multiple flash crashes, the effects of these are not as stark as we would expect. The bull market of the last decade happened, despite numerous mini-flash crashes.

    The second is the degree to which regulation lags changes in market structures. I find the idea of cultural change within firms appealing, but having a stronger regulatory directive (or, indeed, having the regulator take a stronger command of the market’s infrastructures) would also be a plausible way for fostering greater levels of stability. The problem with the cultural argument, in my view, is that it does not question the location of power within finance: those in the system are nudged or incentivized to adopt better procedures in their own interest and for the public good, but ultimately remain in control of their actions. This is, I guess, the crux of the issue: our response so far to the algorithms that drive markets is one that doesn’t question the centrality of finance in contemporary societies and allows the incumbent actors to define, in large degree, what we ought to understand for accountability, transparency, and integrity.

  2. Christian Borch Says:

    Thanks a lot for these comments, JP.

    Your point about the economic effects of these flash crashes is very important: these types of crashes do not seem to shake the economy in any way similar to, say, the 1987 crash, the 2008 financial crisis, etc. So, from a comparative point of view, the economic effects of those flash crashes that have taken place so far seem miniscule. Why then do we care about them? Why has the 2010 Flash Crash attracted so much attention? I guess one reason is that these events draw attention to how little we (still) know about algorithmic markets. I have great respect for US regulators, but their accounts about what happened during that heated half hour in May 2010 have been pretty contested and this demonstrates all too well the complexity at stake here.

    This takes me to your point about regulation. In many ways, I agree with you. Regulation would probably be a more powerful tool. However, as for other algorithmic domains, the regulation of algorithmic trading appears somewhat hesitant. While regulation is probably always a few steps behind, I think this shows rather clearly in this particular field. This doesn’t mean that regulation is without effects (Nathan Coombs has demonstrated this nicely for HFT); I just think it needs cultural backing, as it were. In fact, the two might well go hand in hand, with regulation pushing for greater organizational sensitivity to integrity and explainability in the algorithmic domain in order to enhance market stability. There are steps in that direction already. I think they could be stronger. Would this change the power dynamics of present-day markets? Most likely not.

  3. […] or telling the truth, for example – should transfer into the world of algorithms. In a recent blog, the sociologist Christian Borch has argued that culture is needed to prevent further flash crashes […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: