Competing Against Luck: The Story of Innovation and Customer Choice

Clayton Christensen is probably the most name-checked author in business books. His theory of disruptive innovation has changed the way people think about and go about their business. It has become so embedded in business thought that you can't hardly get through a business conversation without the importance of "disruption" and "innovation" being raised. But it is important to get beyond the buzzwords, and dig into the books.

Well, Christensen and a cohort of coauthors now have a new theory of innovation—the "Jobs to be Done" theory—and a new book to explain it. What you'll learn in their new book, Competing Against Luck: The Story of Innovation and Customer Choice, is that customers are really interested in progress, not products. The following article, to explain this concept, is an adaptation from Chapter 2: “Progress, Not Products.”

◊◊◊◊◊

If you’re like many entrepreneurs and managers, the word “progress” might not spring to mind when you’re trying to innovate. Instead, you obsess about creating the perfect product with just the right combination of features and benefits to appeal to customers. Or you try to continually fine-tune your existing products so they’re more profitable or differentiated from those of your competitors. You think you know just what your customers would like, but in reality, it can feel pretty hit or miss. Place enough bets and—with a bit of luck—something will work out.

On paper, this makes no sense. Companies have never had more sophisticated tools and techniques at their disposal; there are more resources than ever deployed in reaching innovation goals. In 2015, according to an article in Strategy & Business, 1,000 publicly held companies spent $680 billion on research and development alone, a 5.1 percent increase over the previous year.

Innovation processes in many companies are structured and disciplined, and the talent applying them is highly skilled. There are careful stage-gates, rapid iterations, and checks and balances built into most organizations’ innovation processes. Risks are carefully calculated and mitigated. Principles like six-sigma have pervaded innovation process design so we now have precise measurements and strict requirements for new products to meet at each stage of their development. From the outside, it looks like companies have mastered an awfully precise, scientific process. But the results show that, for most companies, innovation is still hit or miss. They should be improving, but they are not. And worst of all, all this activity gives the illusion of progress, without actually causing it.

After decades of watching great companies fail over and over again, we’ve come to the conclusion that what has been missing in the discussions about how companies can create successful innovations is good theory. Companies are creating products and services without ever understanding the causal mechanism for innovation success. Good theory requires us to answer what causes what to happen. Innovation has, for too long, missed that critical question. But we believe the Theory of Jobs to be Done, at last, provides an answer.

Instead of knowing more and more about customers themselves, we need to know more and more about what “job” they are hiring products and services to do for them.

Let us explain: When we buy a product, we essentially “hire” something to get a job done. If it does the job well, when we are confronted with the same job, we hire that same product again. And if the product does a crummy job, we “fire” it and look around for something else we might hire to solve the problem.

Every day, stuff happens to us. Jobs arise in over lives that we need to get done. Some are little jobs, some are big ones. Some jobs surface unpredictably. Other times we know they’re coming. When we realize we have a job to do, we reach out and pull something into our lives to get the job done.

“Jobs to be Done” is not just a cute catch phrase; we mean something very specific with that term. First, jobs are about progress, not products. We define a “job” as the progress that a person is trying to make in a particular circumstance. This definition of a job is not simply a new way of categorizing customers or their problems—it’s key to understanding why they make the choices they make. The choice of the word “progress” is deliberate. It represents movement toward a goal or aspiration—a job is always a process to make progress.

Second, the idea of a “circumstance” is intrinsic to the definition of a job. A job can only be defined—and a solution created—relative to the specific context in which it arises. There are dozens of questions that could be important to answer in defining the circumstance of a job. “Where are you?” “When is it?” “Who are you with?” “While doing what?” “What were you doing half an hour ago? What will you be doing next?” “What social or cultural or political pressures exert influence?” And so on. Our notion of a circumstance can extend to other contextual factors, as well, such as life-stage, family, or financial status, to name a few. The circumstance is fundamental to defining the job because the nature of the progress desired will always be strongly influenced by the circumstance.

Finally, a job has an inherent complexity to it; jobs not only have functional dimensions, they have social and emotional dimensions. In so many innovations, the focus is entirely on the functional or practical need, but that never tells the full story. In reality, the social and emotional needs can far outweigh any functional desires. Think of how you would “hire” childcare. Yes, the functional dimensions of that job to be done are important: will the solution safely take care of your children in a location and manner that works well in your life? But the social and emotional probably weigh more heavily on your choice. “Will my babysitter report back to her mother that our house is messy?” or “Who will I trust with my children?”

These may seem like fine distinctions, but seeing innovation through the lens of Jobs to be Done, rather than features and benefits a product or service offers, changes everything.

Consider some of the most recent entrepreneurial success stories through this. Take Airbnb, for example. On the surface, Airbnb could be reduced to its function—providing a place to stay when travelling. On one level, it’s competing against hotels. And by traditional measures of quality in the hotel industry, Airbnb is a far inferior option. Who would pay to stay on an air mattress on the floor of a stranger’s apartment—or sleep in a stranger’s spare bed—rather than stay in the privacy of their own hotel room?

It turns out, lots of people. People weren’t hiring Airbnb because it’s a place to stay. They were hiring Airbnb because having a place to stay allows them to be someplace so they can participate in something in which they want to be part—and offer a more authentic local experience than a cookie-cutter, one-size-fits-the-world hotel chain.

Airbnb initially identified a job to be done in co-founder Brian Chesky’s own life. As a new college graduate in San Francisco, Chesky could barely afford his own rent let alone find the fees to attend a local design conference. When he realized that all the hotels in the area were sold out—and that there must be other aspiring designers with the same struggle he had—he came up with the idea to “rent” 3 air mattresses in his apartment to help fund his own conference attendance. He could imagine himself as one of those air mattress renters if he found himself in the same circumstances in another city. He might desperately want to participate in something, but he didn’t want to feel like a tourist or rack up credit card debt to make it happen.

Just because Airbnb didn’t stack up well compared to hotels or motels by traditional measures didn’t mean there wasn’t a very real struggle for progress for which Airbnb was a better option. The circumstances in which consumers would hire Airbnb are very different than those in which they’d hire a hotel. Airbnb isn’t just competing with hotels; it’s competing with staying with friends—or not making the trip at all.

In the world of innovation, many companies are stuck in a world of creating elaborate approximations, estimations, and extrapolations. Because we gather, fine-tune, and cross-reference all manner of data, it seems like we should be getting better and better at predicting success. But if we fail to understand why customers make the choices they make, we’re just getting better at a fundamentally flawed process. Without the right understanding of the causal mechanism behind why people make the choices they do, companies are forced to rely on an array of borrowed best practices, probabilistic tools, and tips and tricks that may have worked for other companies, but it can’t guarantee success. As you look at innovation through the lenses of the Jobs to be Done Theory, what you see is not the customer as critical to successful innovation, but the customer’s job to be done. It may seem like a small distinction, but it matters a great deal. In fact, it changes everything.

Adapted from Competing Against Luck: The Story of Innovation and Customer Choice by Clayton M Christensen, Taddy Hall, Karen Dillon, and David Duncan.
Copyright © by Clayton Christensen, Ridgway Harken Hall, Karen Dillion Perez, and David Duncan.
Published by HarperBusiness.
All rights reserved.

ABOUT THE AUTHORS

New York Times bestselling author Clayton Christensen is a Harvard Business School professor and world-renowned innovation strategy and growth expert. Thinkers50 named him 2013’s most influential living management thinker in the world.

Taddy Hall is a Principal at The Cambridge Group (part of The Nielsen Company) and Leader, The Nielsen Breakthrough Innovation Project. He has worked closely with some of the world’s leading companies on breakthrough innovation, including Anheuser Bush, PepsiCo, Kraft, Conagra, Big Heart Pet, and Nestlé.

Karen Dillon is the former editor of Harvard Business Review and Christensen’s co-author on New York Times bestseller How Will You Measure Your Life?

David Duncan is a senior partner at Innosight and a leading thinker and advisor on innovation strategy and growth. He works closely with top leaders at many of the world’s most iconic companies to help them navigate disruptive change, create sustainable growth, and build the organizations of the future.