Thinking in Bets: Making Smarter Decisions When You Don't Have All the Facts

Miles: The first of February, twenty fifteen. The Super Bowl, an average of a hundred and fourteen million people watching. Seattle is down four, on New England's one yard line. Second down, twenty six seconds, one timeout.

Miles: Everybody in the stadium is waiting for a handoff to the running back the whole league fears. Seattle throws instead. A cornerback named Malcolm Butler steps in front of the ball, and that's the game.

Miles: Before long the call has a title. The worst in Super Bowl history. Three years later a retired poker professional opens a book about deciding with that play, and her complaint is that almost nobody grading it was grading the call. They were grading the interception.

Miles: Today: Thinking in Bets, by Annie Duke. In two sentences: every decision is a wager on a future you can only partly see, so one bad ending says far less about a choice than it seems to. Judge the choice by what was knowable when it was made, which matters most to anybody whose results get read as a verdict on who they are.

Miles: Sixty Books. Presented by Stoar Consulting and Coaching.

Miles: Episode thirty eight, and you're listening to Sixty Books. I'm Miles. The arrangement holds: a single management book every time, sixty picked from a century of them, and the same pair of questions put to each. What does it offer people who lead, or report to, minds that run on a different plan. And what survives once AI is part of the ordinary working day. Nora brings the research, and today's author trained in it.

Miles: Nora, last time we left two books leaning on one assumption. Rumelt on strategy, Christensen on why strong companies stumble, and both taking it as given that a good call and a good outcome can be told apart.

Nora: And this book says that from the ending alone, mostly they can't. Christensen's publisher sums his firms up as doing everything right and still losing. On her terms that's no paradox at all, just a sound bet seen from the losing side.

Miles: So the ending is a poor witness.

Nora: A witness with one data point. And she'd know something about reading endings. She walked away from a doctorate in cognitive psychology a month before defending it, and finished it thirty two years later.

Miles: Thirty two years.

Nora: Yeah. Thirty two years. And by the logic of her own book, the fact that she finished tells you nothing about whether walking out was the right call at the time.

Miles: Give me the play the way she uses it.

Nora: Seattle trails twenty eight to twenty four, second and goal at the one. The call is a quick throw to a receiver cutting inside, and Butler reads it and takes the ball away.

Miles: And the verdict.

Nora: Fast and nearly unanimous. A senior writer at Sports Illustrated called it one of the worst calls in Super Bowl history, a Hall of Fame running back called it the worst in the history of the sport, and the league's own film unit later ranked it first among the worst plays ever.

Miles: So where does she come in?

Nora: With a number. She writes that over fifteen years of league data, a short pass from that spot was intercepted less than two percent of the time. And there's a clock reason: an incomplete pass stops the clock by itself, so Seattle could still hand off afterwards without burning its one timeout. Jim Harbaugh and Joe Namath were among those who defended it. Her point is that almost nobody was arguing about the call. An outcome she puts under two percent decided which story more than a hundred million viewers went home with.

Miles: And there's a name for that.

Nora: Borrowed from poker players. Resulting. Treating how a decision turned out as the measure of how good it was. The result arrives looking like a grade, and nobody checks whether the grade belongs to the choice or to the cards.

Miles: Now the author.

Nora: Annie Duke, born Anne Lederer in Concord, New Hampshire, in nineteen sixty five, to two parents who both played cards. Columbia, a double major in English and psychology. Then the University of Pennsylvania on a National Science Foundation fellowship, working on how young children learn language.

Miles: And then the month.

Nora: Nineteen ninety one. A month before her defense, she decides academic life isn't for her. Her brother is already a professional player, and the following year he sends her two thousand four hundred dollars and a stack of poker books.

Miles: How did that go?

Nora: Two decades among the world's top players, by her university's description. In two thousand four, a World Series of Poker bracelet, and that same year a tournament of champions that paid her two million dollars. In twenty ten she won a national heads up championship on network television, and she's still the only woman to have won it. She stepped back from professional play in twenty twelve.

Miles: And the books.

Nora: This one, February of twenty eighteen, from Portfolio, a business imprint at Penguin. Two hundred and eighty eight pages, a national bestseller on the publisher's account. Then How to Decide in twenty twenty and Quit in twenty twenty two. In twenty fourteen she co founded a nonprofit that brings decision education into middle and high schools. And the doctorate was completed five years after this book, so she wrote it trained in the field but without the letters. It leans on published research throughout, and it isn't itself research. That matters later.

Miles: The argument, in her order.

Nora: It starts in a taxi during the Second World War. John von Neumann, one of the mathematicians who built game theory, tells Jacob Bronowski that chess isn't a game. It's computation: every position has a right answer in principle. Real life, he says, is bluffing, and guessing what the other person thinks you'll do.

Miles: So life's the card table.

Nora: That's chapter one: life is poker, not chess. Two things decide how anything turns out, she says: the quality of your decisions and luck. In chess a loss means somebody made a mistake. In poker you can play a hand perfectly and lose, or badly and win. Working life sits on the poker side, and we keep grading it like chess.

Miles: And grading it like chess is resulting.

Nora: It is, and it has a partner: hindsight bias, the sense afterwards that the ending was obvious all along. She pictures the future as a tree with a lot of branches. Once one of them happens, we saw off the rest, and soon we can't remember they were ever there.

Miles: Chapter two.

Nora: Wanna Bet. Every belief is a bet, money or no money, and we place most of them badly because of how beliefs arrive. Here she draws on Daniel Gilbert: three experiments, published in nineteen ninety three. In two of them, people read about a criminal defendant, with some statements flagged as false. Under mental load, they believed the false statements anyway and let them sway their recommended prison terms.

Miles: Believe first, check later.

Nora: And if the checking gets interrupted, it never happens. Her prescription is cheap: put a number on your confidence. Not I'm sure, but I'm about seventy percent. And whenever you hear yourself certain, put the chapter title to yourself. Wanna bet?

Miles: Which must make for lively dinner parties.

Nora: She's clear it works as a question inside your own head, not a dare across the table. A bet makes you look at what you'd lose.

Miles: Chapter three.

Nora: Bet to Learn. Every outcome is feedback, but it has to be sorted first: my decision, or luck? She calls that fielding the outcome, and the sorting is rigged. We file our wins under skill and our losses under bad luck, and do the reverse for other people. That's self serving bias, and she spends a while on players who never improved because every loss went into the luck pile.

Miles: And the ones who did improve.

Nora: Did the opposite. Her example is the professional Phil Ivey, who wins a major tournament and spends the celebration dinner hunting through his own hands for the errors the win had covered up.

Miles: Chapter four.

Nora: The Buddy System. You can't sort alone, since the person grading your outcomes has the most riding on the answer. So she builds a small group with an explicit agreement: reward accuracy over agreement, hold each other to account, stay open to views that differ. The accountability piece comes from Jennifer Lerner and Philip Tetlock, a nineteen ninety nine review in Psychological Bulletin. It improves thinking only under narrow conditions: you learn before forming a view that you'll have to explain it, to somebody who cares about accuracy, knows the ground, and whose own opinion you don't know. Otherwise people mostly defend what they've said or drift toward what the listener wants. Keep the second name from that paper, by the way.

Miles: Chapter five.

Nora: Dissent to Win. She borrows four norms the sociologist Robert Merton described for science, spelled out as CUDOS. Share all the data, including what makes you look bad. Judge a claim on its evidence, not its source. Watch for conflicts of interest. And make doubt a habit of the whole group. As management, that's a team where I think you're wrong is the expected contribution.

Miles: And the last chapter.

Nora: Adventures in Mental Time Travel. The trouble she names is that the self deciding now isn't the self who lives with it. Then the tools. Ulysses contracts, after the sailor tied to the mast: commitments made calm that bind you when you're not. Ten ten ten, from the writer Suzy Welch: how will this feel in ten minutes, ten months, ten years. Tilt, the poker word for playing badly because the last hand stung. And backcasting and the premortem: imagine it succeeded, or failed, and work back to why.

Miles: We had the premortem with Kahneman, in episode twenty six, and he credited Gary Klein.

Nora: Same tool. What she adds is running it both ways and putting rough probabilities on the branches, so the plan carries its own odds.

Miles: Now the evidence. Start with the claim everything else sits on.

Nora: That people grade decisions by their outcomes, and it's the sturdiest thing in the book. Baron and Hershey, nineteen eighty eight, Journal of Personality and Social Psychology. Five studies. Undergraduates read about other people's decisions, medical choices and money gambles, with everything the decision maker knew laid out. When the outcome was good, they rated the thinking as better and the decision maker as more competent. Asked directly, they said outcomes shouldn't count. They counted them anyway.

Miles: Undergraduates in nineteen eighty eight. Still true?

Nora: It's been rerun. Aiyer, Feldman and colleagues, twenty twenty three, a preregistered replication of the first experiment, which had used twenty people. This time six hundred and ninety two adults online, each seeing one ending. The effect came back larger, from about three quarters of a standard deviation to a little over one, and held among people who'd said outcomes shouldn't matter.

Miles: And hindsight.

Nora: Fischhoff, nineteen seventy five. Readers got a short account of a real but obscure conflict, the British against the Gurkhas in eighteen fourteen. One group got no ending. Four groups were each told a different one. Whichever ending a group was told, it rated that ending as more likely than the others did, and the readers largely couldn't see they'd done it. A patient safety journal reprinted it in two thousand three as a classic.

Miles: And since then.

Nora: Guilbault and colleagues pooled ninety five studies in a two thousand four meta analysis. The average effect was about four tenths of a standard deviation: real and moderate. And smaller with real events or case histories than with trivia questions, which matters, because work is made of the first kind.

Miles: Does any of it show up outside a laboratory?

Nora: Three places. Nineteen ninety one, JAMA: a hundred and twelve practicing anesthesiologists rated the care in twenty one real cases that had gone badly. Each case was also rewritten with the opposite severity of harm and nothing else changed. Make the harm permanent instead of temporary, and ratings of appropriate care fell thirty one percentage points. Identical care. Then a twenty fifteen study of professional basketball: coaches change strategy more after a loss than a win, even a narrow loss that says almost nothing about the team. And the other football. Gauriot and Page took more than thirteen thousand shots that struck the post, about two thousand four hundred of which bounced in. Against near identical shots that bounced out, the scorers got higher match ratings from journalists and more playing time from their coaches in the next game.

Miles: Luck, paid out as skill.

Nora: About as close to a coin flip as the real world allows. So the first half of this book holds better than most of our shelf.

Miles: Then where does it slip?

Nora: In the poker. The deck is known, the odds can be worked out, and a serious player sees huge numbers of hands with feedback in minutes. Kahneman and Klein, American Psychologist, two thousand nine, set out when expert intuition can be trusted: a regular enough environment, and enough practice with clear feedback to learn it. Poker meets both. A reorganization, a hire or a product launch mostly meets neither.

Miles: So the two percent was a luxury.

Nora: Fifteen seasons of data on one situation. Most leaders never get a base rate for the decision in front of them, so telling luck from skill is far harder off the table than on it. And the two percent is her figure, from her account of league data. I couldn't check it independently, so hear it as hers.

Miles: And even on the table?

Nora: It takes volume. Steven Levitt and Thomas Miles studied the twenty ten World Series of Poker: seven hundred and twenty players flagged as highly skilled in advance, across fifty seven events, made back more than thirty percent on their entry fees on average, while everybody else lost about fifteen. Skill is real. But a German team put three hundred players through sixty hands each, experts and average players at the same tables, with the quality of each seat's cards fixed by design. The cards decided who left with money. Expertise showed mainly as losing less on bad hands.

Miles: So over a short run, luck takes it.

Nora: And one person's quarter is a short run. That cuts for the book's argument and against its favorite example at once: telling luck from skill takes repetitions that poker players get and most people deciding at work never do.

Miles: So, the tier statement for this one.

Nora: Thinking in Bets is practitioner literature, written for general readers by somebody trained in cognitive psychology, and it isn't peer reviewed research. The bias at its center is among the better replicated findings in that field, rerun as recently as twenty twenty three and found larger than first reported. The analogy it's built around travels less well than the cover suggests, because the card table hands you the odds and most of working life never does. Neither half of that is a remark about the author.

Miles: Our half.

Nora: It's in those field studies. A review that looks only at results is a draw: whoever's shot clipped the post and went in gets the better rating. Judging the process rather than the result is the difference between a fair review and a lottery. And I'll say plainly that what follows is my argument, built from that research, and not a finding about neurodivergent staff.

Miles: Why does it land harder on them?

Nora: Two steps, and only the second has evidence. First, when the result can't be trusted, reviewers fill the gap with impressions. Second, impressions are where some people lose before they've said a word. Sasson and colleagues, twenty seventeen, Scientific Reports, three studies. Observers watched brief clips of autistic and non autistic people in ordinary social situations. Within seconds their first impressions of the autistic participants were less favorable, and they were less willing to interact with them, for children and adults alike. Then the researchers gave observers only what was said, the words without the voice or the face, and the gap disappeared. The authors' reading: style, not substance, drove the impressions. Group findings, and not a description of anybody listening.

Miles: So a process review can go wrong as well.

Nora: Very easily, if process comes to mean how somebody came across in the meeting. It has to mean the reasoning: what was known, which options were on the table, what odds the person put on them, and why. On paper where possible, since on paper is where that gap closed.

Miles: And when do you collect the reasoning?

Nora: Before the result. That's Fischhoff turned into a practice. Once you know the ending you can't rebuild what you'd have thought without it, so the record has to exist first. Write the bet down when it's placed: the choice, the expected result, a rough percentage, and what would change your mind. Then the review compares the decision to the record, not the scoreboard.

Miles: And if nobody kept a record?

Nora: There's a repair with evidence behind it. Arkes and colleagues, nineteen eighty eight, Journal of Applied Psychology. A hundred and ninety four neuropsychologists answered a mailed case history, a man of fifty five with three possible diagnoses. Some were told which diagnosis was correct. Among those, fifty eight percent showed hindsight bias. Among those also asked to write down one piece of evidence for each of the three, forty one percent.

Miles: Not gone, then.

Nora: Yeah. Not gone, and clearly smaller, from a minute's work. So before anybody grades a bad outcome, they spend that minute arguing for the endings that didn't happen. And Lerner and Tetlock belong here: the person should know in advance that the review is of their reasoning, by someone who hasn't already decided.

Miles: Cautions.

Nora: Two. First, the one I owe every time: nobody has studied outcome bias in the reviews of neurodivergent employees. I've joined a literature on how evaluators treat luck to a literature on how observers treat style, and that join is mine, not the research's.

Miles: And the second.

Nora: Process review can fail the other way, and Duke names it herself: every loss filed under bad luck. A team that always concludes it made the right call and the dice were unkind has stopped learning. The record cuts both ways, which is what makes it fair.

Miles: Fair enough.

Miles: Sixty seconds, the way this show keeps them. Thinking in Bets says every decision is a bet on a future you can only partly see, that people grade those bets by how they landed, and that the grading is wrong often enough to cost you. For somebody leading a mind that runs on a different plan, or working inside one, this is Monday. Write the bet down before the result arrives: what you chose, what you expected, roughly how sure you were. Review people against that record and not against the scoreboard, because the scoreboard counts the shots that hit the post. Before reviewing a bad outcome, make the case for the endings that never came. Read reasoning on paper where you can, since that's where style stopped counting against substance. And hold good results to the same standard, so luck doesn't get promoted either.

Miles: The machines, then.

Nora: The standing caution on this whole class of tool runs after the close, in fixed wording, so I'll leave it there and stay with what Duke hands us.

Miles: Go on.

Nora: The bet record first. Everything in our half depends on writing the decision down before the result, and that's the step people skip because it's tedious. A tool can interview you for five minutes and turn the answers into a dated page: options, expectation, percentage, what would change your mind. The habit gets close to free.

Miles: And when the review comes round.

Nora: Give it the record without the outcome and ask it to grade the reasoning. Grade it yourself too, and only then open the result. Whatever you let the tool read, assume it uses. If the ending is in the file, it's in the grade.

Miles: And the buddy system?

Nora: That's where it needs handling. A study posted in twenty twenty three tested five widely used assistants across several writing tasks and found they tend to echo whatever view the user has stated, which the field calls sycophancy. That's the reverse of a truthseeking group.

Miles: So you don't ask it whether you're right.

Nora: You tell it you're seventy percent sure, and you ask for the strongest case for the other thirty. That's the Arkes repair, from something that never tires of being contrary. Same for the premortem: it can sketch the branches faster than a room can, and you supply the probabilities, because it has no base rate for your situation either.

Miles: Send us out.

Nora: With her two words. The tool will tell you yes all afternoon if you let it. The version worth having is the one you've told to look at your plan and ask you, wanna bet?

Miles: That's Thinking in Bets.

Miles: Next time: Superforecasting, by Philip Tetlock and Dan Gardner. If every decision is a bet, then some people ought to be measurably better at placing them, and that book goes looking for who they are.

Nora: The accountability researcher from chapter four, keeping score this time.

Miles: That one paid off, Nora. Thanks.

Nora: Always worth it.

Miles: Catch you next time.

Miles: Stoar Consulting and Coaching is where this show gets made. Logan Williamson is a Licensed Professional Counselor in Texas and Louisiana. What you heard was one book read with care, for leaders and for the people on their teams. None of it was coaching or counseling, and none of it was a verdict on anyone listening.

Nora: The sources for every claim are on the episode page, alongside the transcript. Another book lands in two days.

Miles: For the record, two points. These voices are synthetic. Miles and Nora aren't people, hold no license of any kind, and nothing in this episode was said about you. The scripts are drafted with AI assistance, then edited and approved by Logan Williamson, a Licensed Professional Counselor in Texas and Louisiana, at Stoar Consulting and Coaching.

Nora: And one note about the AI segment. The evidence that these tools help neurodivergent people at work is early. There are controlled trials now, but they measure things like conversation skill rather than how a job actually goes, and for work itself most of what exists is individual providers reporting success. So take it as promising and not proven. The tools also get rebuilt every few months, so whatever you set up today is built on something that moves.