Ten Years after the Flash Crash, We Still Need to Make Algorithmic Trading Less Risky. Can Culture Save The Day?

From Christian Borch

Ten years ago this day, US markets opened to a palpable sense of unease. Investors were concerned about the ongoing Greek debt crisis, which grew from the global financial meltdown in 2008.

At 2:32pm Eastern Daylight Time, what started as nervousness and hesitation collapsed into pure terror as US markets nosedived. In 36 minutes, the Dow Jones Industrial Average plummeted 998.5 points – more than nine percent – in one of the largest and fastest same-day declines in the history of US markets. The greater part of this drop saw market values in the range of $1 trillion evaporate within five minutes.

As markets continued their free fall, trading was suspended in panic. When trading activity resumed, an equally remarkable and opposite phenomenon happened. Prices rebounded, and just half an hour later, US markets had recovered most of their losses.

Since all this happened in a blink, commentators have dubbed this astonishing event the “Flash Crash.” No solid estimates exist of specific investors’ gains and losses from the Flash Crash, but prices recovered quickly enough that the event had no demonstrable economic effects in the long term – unlike the 2008 global financial crisis and the ongoing Covid-19 pandemic.

It might surprise some, however, that the Flash Crash had little to do with the European sovereign debt crisis itself. In fact, it was precipitated by a group of smaller, underestimated agents: fully automated, high-frequency trading algorithms. That afternoon in May, these algorithms entered an escalating feedback loop, their cascading orders triggering a brief but precipitous market crash.

The Flash Crash is a defining moment in the modern history of financial markets. It reveals the Janus face of present-day trading algorithms – algorithms can unleash great instability when tossed together, despite their reputation as impartial, rational actors. The Flash Crash also laid bare how little we know about the inner machinations of trading algorithms, as the confusion of US regulators investigating the phenomenon later showed.

In 2010, the Commodity Futures Trading Commission (CFTC) and the Securities and Exchange Commission (SEC) issued a joint report arguing that a large sell order from Waddell & Reed Financial, a Kansas City-based asset management firm, had triggered unanticipated activity among high-frequency trading algorithms. The algorithms then engaged in a game of “hot potato,” swiftly buying and selling from one another in a downward spiral that caused the Flash Crash.

The CFTC–SEC explanation of the Flash Crash sparked controversy when it was released, and its findings have since been contested by academics and market observers. Years later, US regulators unexpectedly changed their tune when they identified the British trader Navinder Singh Sarao as the Flash Crash’s main culprit. Sarao had purportedly designed – out of his parents’ home in Hounslow in the outskirts of London – a set of high-frequency trading algorithms aimed at manipulating market prices, which ultimately caused the markets to crash. Sarao eventually pleaded guilty to charges of market manipulation, but experts remain divided on whether any individual’s manipulative algorithms can bring US markets to their knees. Are financial algorithms truly that powerful – or are markets penetrated by them inherently that fragile?

After the Flash Crash, regulatory authorities implemented “circuit breakers,” which halt trading if prices move too much too quickly, as their main line of defence. When the Covid-19 pandemic struck global markets in March this year, these circuit breakers kicked in to cushion the outbreak’s ripples in the market. But even circuit breakers – a triage rather than vaccination – have not deterred the several smaller flash crashes that have occurred since 2010.

Even though the Flash Crash occurred ten years ago, we are still in the dark about how algorithmic markets work and how to prevent their ills. One thing is clear, however: the algorithmic nature of contemporary financial markets renders them more vulnerable to crashes, which begs the question of whether we can prevent such sudden crashes when financial algorithms continue to operate in our markets.

In the past six years, colleagues and I have interviewed executives, traders, programmers, exchange officials, and regulators in the US and Europe to better understand how algorithms shape today’s financial markets. Through two research projects with high-frequency trading firms of all sizes and sorts, we have observed algorithmic traders at work and studied their daily operations, concerns, tasks, and overall cultures.

We have realized that individual firms with lax practices for development, testing, and monitoring algorithms face devastating risks. Take the example of Knight Capital, a major US trading firm that suffered a shattering algorithmic mishap in August 2012. The firm’s new software unexpectedly triggered dormant code, which generated orders that lost the company more than $460 million in just 45 minutes. Unable to recover from this mishap, Knight Capital was soon acquired by a competitor.

Though Knight Capital may seem an unlucky exception, a financial culture that does not support the rigorous use of financial algorithms risks markets themselves. Without a thorough understanding of their own algorithms, can firms avoid triggering devastating feedback loops once algorithmic interaction sets in? The cultures of algorithmic trading firms matter in designing more robust markets, and we need to better understand how these firms are organized, how they operate, and how their staff think and work. Crucially, we need to grasp their procedures for developing, testing, monitoring, and understanding trading algorithms long before their cascading effects begin.

There is good news, however – a small group of firms has developed organizational cultures that stress rigor at all levels of algorithmic design and trading. These firms strive to eliminate any negative effects their algorithms may have on markets, and they have developed an ethos built on ensuring market integrity in every respect.

This ethos involves establishing procedures to make sure financial algorithms cannot engage in manipulative behaviour. These firms obsess over bug detection as well as continuous monitoring and testing. They learn from the algorithmic incidents that inevitably occur, whether triggered internally or from outside their operations. Throughout, these firms expend massive, ongoing efforts to comprehend how and why their algorithms behave the way they do, alone and together with other algorithms.

Today, European regulators require some minimum testing and monitoring of firms’ trading algorithms. This is an important first step, but hardly a sufficient one if we wish to avoid a consistent stream of flash crashes. We need a more fundamental attention to the cultures of trading firms, and perhaps even a paradigm shift. After the 2008 financial crisis, many called for banks to implement a new corporate culture with fewer detrimental risk incentives, though little progress has since been made in that arena. Even so, the potential consequences of algorithmic spirals running amok suggest that we should not shy away from driving such change while we still can.

Firms whose cultures, business models, and algorithmic designs emphasize market integrity offer a way forward in ensuring the health and stability of financial markets. Avoiding future flash crashes and mastering the collective power of financial algorithms will require an industry-wide commitment to cultures of accountability, transparency, and integrity.

Our algorithms will settle for nothing less.

Christian Borch is Professor of Economic Sociology and Social Theory at the Copenhagen Business School and the PI of the ERC-funded AlgoFinance research project. His latest book is Social Avalanche: Crowds, Cities and Financial Markets (Cambridge UP, 2020).