What NHL General Managers and Owners Can (and Should) Learn from the Carter Racing Case Study
When the logic of 'win now' breaks down.
Quick note: I’m cribbing heavily from the ‘Learning to Drop Your Familiar Tools’ chapter in David Epstein’s ‘Range.’ Instead of reading me awkwardly summarize the case study, I recommend reading the book for yourself.
Harvard Business School has a fun case study called Carter Racing. In it, students are asked a question: should the Carter Racing team compete in the biggest race of the season which begins in one hour? The argument in favor of racing follows the same principles that NHL GM’s have to compete: sound business. For starters…
The team has placed top five in 12 of 24 races.
They’ve secured sponsorships due to their success.
They won won their last race.
A top five finish would earn the team $2M in extra sponsorship money (for analogy purposes this is an extremely useful number since teams make between $1.5 and $3 million per home game).
If they don’t race and withdraw, Carter Racing will lose some of their entry fee, and be in debt for $80,000.
On this day, the team will be on national TV.
There are also sound reasons not to race:
In seven of 24 races, the engine failed, damaging the car.
The team’s mechanic thinks the engine failure might be caused by temperature. When the turbocharger warms up on a cool day, it’s possible the engine can’t handle the drop in temperature.
The expected temperature for the day will be 40 degrees.
If there’s a correlation between engine failure and temperature, then the argument against racing seems like a no brainer too. Except there isn’t:
The mechanic offers a random sample of data. We see that there were three gasket breaks on cooler days and three on warmer days. Most students argue that it’s good data, but it’s not useful data.
In the end, students ultimately agree that Carter Racing is in the business of racing. Risks are involved no matter what, and this is a big opportunity on the highest stage - so get into the playoffs race! What’s interesting about the way this exercise plays out in the classroom is that students are given every opportunity to ask for more analytics information. Most don’t, trusting that they have all the info necessary, and in the absence of info, a good understanding of probability. Once the students make their decision, they’re given a piece of info they could have accessed if they had just asked the right questions. Here is each race plotted with the temperatures and corresponding engine failures. It looks like this:
The mechanic’s intuition was right: every single race below 65 degrees resulted in engine failure. After further analysis, students learn that here is a 99.4 percent probability that the engine will fail at 40 degrees. There’s a horrific twist to this exercise: the Carter Racing case study is based on NASA’s decision to launch the space shuttle Challenger, taken from the exchanges between the managers and engineers of NASA, and their rocket contractor, Morton Thiokol. The broken gaskets are stand-ins for the famous O-Rings, which were damaged by cooler temperatures. Just like the Carter Racing case study, NASA had the data to make the right choice…but nobody asked the right questions, trusting their familiar tools instead.
The analogy writes itself. Just as everyone agrees that the Carter Racing team should compete on the biggest stage, NHL teams need to make the playoffs. And the ‘engine failure’ can be a stand-in for a ton of things: players that could have brought back assets for a brighter future instead of being lost in the offseason for nothing (self-explanatory), the failure to develop NHL experience for prospects that have a future in the league over veterans who have no future with the team (Valeri Nichushkin in Dallas*), making costly moves to higher the ceiling on a team starting from a low floor (Chicago’s trade for Seth Jones), and the end result of how that affects draft position (think about the wins lost by drafting lower). Former Montreal coach Dominique Ducharme was recently interviewed by Marc Antoine Godin at The Athletic and boy what an interview…Ducharme silently maps out all the problems with Win Now, standing by all of his bad decisions — overplaying goaltender Jake Allen, giving more icetime at one point to a career AHLer over Cole Caufield, finding effective lines only to arbitrarily ditch them (“I was trying to squeeze as much as I could everywhere to try and get some results”) — because management didn’t tell him they were no longer in the business of Win Now. For whatever bizarre reason, Win Now meant sacrificing the development of young players.
I could write ten articles about Ducharme’s comments alone but let’s get back to the analogy. What makes the Carter Racing case study so relevant is that it’s a repudiation of every GM or Owner’s rational: which is that trying to Win Now is “good business.” It’s “good business” to win now, make the playoffs, sell more tickets, and be more relevant. But that’s not a philosophy. It’s not even a mission statement. It’s nothing more than the warning label on a baby stroller that says ‘remove child before folding.’ Of course it’s better to win than to lose, and it’s better make more money than less. But if it were strictly about money, then teams would do the hard math. But they don’t. Because business is not about the plus/minus of numbers. It’s about, as Daniel Kahneman reminds us, the psychological value of outcomes. It’s about broad vs narrow framing — what’s the cost of trying to win now (making a Cup run) versus the cost of trying to win later (i.e. rebuild) — and specifically, what is the cost of managing that dynamic?
But let’s use the language of money. Consider the current champs, the Colorado Avalanche. In the 2013-2014 season under Patrick Roy, COL made the playoffs after missing the previous three seasons. Were they suddenly good? No. Due to their mediocre shot quality, they were expected to score 145 goals at EV, which ranked 18th. Instead they scored 174, which ranked 4th. They were an awful possession team and Roy famously derided the analytics that said they were lucky rather than good. Sure enough, just like the three seasons before, COL would miss the playoffs in the three seasons after. But in that time, they acquired lots and lots of assets, getting a grab bag of players and $7M in cap relief for Ryan O’Reilly. Critical pieces of their blueline came from not trying to hold onto Matt Duchene. They could have held onto those players but instead they traded on-ice losses for off-ice wins, grabbing Bowen Byram, Mikko Rantanen, and Cale Makar in the draft (per Evolving-Hockey, they are collectively worth almost 10 extra wins in the standings). Since 2018 Colorado has 21 playoff wins at home, which is somewhere in the neighborhood of $42 million in house money (likely on the low end given how much prices go up during the playoffs). What do you think is the over/under on how much less they would have made if they had tried to keep one of O’Reilly or Duchene to salvage one of those three lost seasons?
Quick note: Few teams have done more to invest in analytics recently than Colorado, who brought in Dawson Sprigings — the expected goal expert all the way back in 2015 (!) — and Arik Parnass, who took a unique approach to analytics by focusing on special teams. It’s worth asking how much of their approach following Roy’s departure was driven by being a case study in pretending like the “analytics” weren’t signals of impending doom they should have heeded all along.
To be fair, there is more to being a GM or an owner than simply making hockey decisions. They aren’t just managing the cap, and their departments, but they’re also managing human relationships. Trading a player for a high draft pick might be obvious to most of us, but what about the player’s value in the locker room? What happens to a team’s chemistry when beloved players are sent packing just so ownership can lose now in order to win later? I imagine that’s demoralizing, especially for players who won’t be there once the rebuild is over. So no, I’m not pretending like this is all the simple matter of gaining assets when you can, however you can. There are aspects to these jobs I will never comprehend, and I don’t pretend it’s easy.
But don’t condescend to fans what “good business” means by telling us how important it is to win now. There’s no shame in losing. Every team except the Cup champs ends their season by losing. But there is shame in treating every race like you can’t lose. That’s how disasters, both big and small, happen.
*Two things can be true at once. Nichushkin needed a change of scenery. And Dallas didn’t have to give up on him. Recall that even before the Stars chose players like Cogliano and Comeau over him, Nuke was scratched for Travis Moen under Ruff.