Checklists: a market device that guides calculation and affect

January 20, 2010

Today I read a book excerpt that persuaded me. Atul Gawande, professor at Harvard Medical School and staff writer at The New Yorker, has written a book about the many benefits of checklists. Generalizing from his own experience with surgeons, Gawande argues that checklists are useful in finance or airline cockpits.

That checklists can be critical should be of no surprise to sociologists of finance. It is, after all, another material device that provides what Andy Clark calls “scaffolding” to complement our limited cognitive skills. And indeed, I myself came across checklists in my own research on derivatives traders. In my latest paper with David Stark, we describe how the head of the merger arbitrage desk is very clear about the importance of checklists and rigorous procedure:

Taking a position, then, involves a successive winnowing of the possible contingencies involved in the merger as the arbitrageurs think through the deal. The traders search through a form of mental decision tree in which each specific merger is considered in relation to similar deals that they encountered in the past. Max explains, “it’s almost like you’ve been in this road before and [the past incidences] direct you.” The advantage of this system, which Max describes as a “process-driven arbitrage,” is that numerous issues need not be taken into account.

In other words: process, process, process; our head trader even mentioned the word check-list. At first I was surprised to see that the traders were using such a mundane tool. Now I see that they are not an exception. Gawande explains that many of the best value investors and venture capitalists also rely on lists.

But Gawande goes beyond giving examples. Crucially, he explains why checklists complement judgment. As it turns out, a checklist is very different from a shopping list. While a shopping list is just a form of distributed memory (make sure I don’t forget…), a professional checklist entails a systematic self-examination of one’s own past errors of judgement — blindspots, if you will — and the creation of a tool that avoids them. Here’s how he talks about it:

Looking back, Pabrai noticed that he had repeatedly erred in determining how leveraged companies were. The information was available; he just hadn’t looked for it carefully enough. In large part, he believes, the mistakes happened because he wasn’t able to damp down the cocaine brain.

The anonymous investor I spoke to – I’ll call him Cook – made a checklist. But he was even more methodical. He enumerated the errors known to occur at any point – during the research phase, during decision-making, during execution of the decision and even in the period after making an investment. He then designed detailed checklists to avoid the errors, complete with clearly identified pause points at which he and his team would stop and run through the items together.

What the checklist does, in other words, is help in the calculation process. If calculation, as Callon reminds us, entails judgement and affect, managing these two becomes crucial. The list is a disciplining device that allows the actor to channel the affect.

5 Responses to “Checklists: a market device that guides calculation and affect”

  1. Tom Hickey Says:

    Helpful. I learned the value of checklists the hard way as a kid, when leaving home an essential piece of equipment could turn an otherwise fun hike or camping trip into a real bummer. And it’s amazing how many pilots run out of gas. Checklists are a lifesaver in many areas.

    I have a friend who is a bond trader and his pride, which he credits with this success, is his proprietary checklist that he developed over many years of experience.

  2. danielbeunza Says:

    Tom — you wet our appetite. Can you tell us more about that bond trader friend? How does he think about checklists? What does he write in them? Etc.

    • Tom Hickey Says:

      Hi Daniel,

      It’s his experience that developing a comprehensive checklist is key to success. As I said, his list is proprietary because it contains his fundamental trading principles. He has spent years thinking this through and testing it in order to hone it down to what is essential in every trade. When trade didn’t work as expected, he always thought it through to see what he had might missed, and in this highly disciplined way he constantly improved the list.

      The thing that strikes me about him is that he was as much interested in figuring out the dynamics of the bond market as making in making money. It was an intellectual and professional challenge for him. As he put it to me, I was searching for the holy grail of the market. This was the inner rationale that could be expressed clearly and relatively simply in a set of operational trading principles that determine the outcome of every trade, making it possible to enter at the right point and exit at the right point, and knowing why.

      He also emphasized that a checklist has to be personal, too. Everyone has particular biases that they have to be aware of and figure out how to avoid. He also said that without a checklist, this is very difficult to accomplish in trading because these are the things that one is not likely to see in particular situations, even though one is generally aware that they are one’s weaknesses.

      He had studied all the indicators of technical analysis and the various systems of applying them and found that they didn’t work well enough to make money and avoid losses. This was brought home early in his career in a big wipe-out.

      As a result of this painful experience, he realized that he would have to conduct his own disciplined investigation based on his own experience, critiquing everything thing he did and trying to distill the basic principles involved that could be applied universally to trading. It’s taken years but now he feels that he’s got it boiled down to a checklist that works really well for him, eliminating the costly guessing.

      Hope this helps,

  3. danielbeunza Says:

    This is fascinating. It confirms what Gawande says… that the lists have to be about things where one’s own decision-making process is biased. What I find particularly interesting is the thought that the list, like an equation, is a way to distill and codify the tacit experience acquired over the years… a tool for “knowledge management”, even though the expression has been so overused.

    Now, here’s a question that I’ve been asking myself: can a list be thought of as model? The question is relevant in that i’m wondering whether to include them in the statement “the models that arbitrageurs use”. Do lists cause similar problems as models? It strikes me that lists, like models, are about codified knowledge. However, when taken together they do not add up to a systematic representation of the phenomenon… they are “things to think about” but their relationship between them is loose or inexistent.

    It strikes me, as I write this, that this was the problem that I have always had with list-based books… as in “Seven Habits of Highly Effective People.” The list structure is a basic way to give coherence to things that may not even go together.

    However, Gawande’s book challenges such dismissive view. That’s why it’s so interesting.

    Now, on to the analysis: if lists can certainly do good things to you… can they be detrimental as well?

  4. Tom Hickey Says:

    “Now, here’s a question that I’ve been asking myself: can a list be thought of as model?”

    Yes, exactly. My friends list is a model, although not a mathematical one. It does contain a list of operating principles but it is also expressed graphically as a coherent system in which all the elements are related.

    The model is that of the dynamic of the market, and as such constitutes a “philosophy” of the market. I say “philosophy” since it is not an applied mathematical model, but a trader’s understanding of himself in relation to the market. So part of it is universal and part of it is personal. It is also a “philosophy” because it is a reflection on experience, the definition Socrates gave of philosophy.

    Otherwise, it is just a list, as you note, and there may not be an underlying rationale that corresponds to conditions. Or els, if it is just a trading algorithm, well, we see where that got us by ignoring fat tails.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: