What anomalies have taught us about disruption theory and theory-building itself.
Alan Hodgkin, a famous neurophysiologist responsible for describing how the voltage in neurons changes rapidly when they are stimulated (for which he won a Nobel Prize), would go around the laboratory each day visiting with each student or postdoctoral researcher working on one project or another. If you showed him data from yesterday’s experiments that were the expected result, he would nod approval and move on. The only way to get his attention was to have an anomalous result that stuck out. Then he would sit down, light his pipe, and go to work with you on what this could mean.
—Stuart Firestein, Ignorance: How It Drives Science
Once the “trajectories” insight helped me see the causal structure of disruption, I began to share what I had discovered in talks and in classes. At first, I set out to extend and validate the theory by showing that it held true in adjacent industries. But I found that successful application in this or that setting didn’t settle the question – there is always another industry, discipline, or country. Researchers and students and executives replied, in effect, “I think the theory of disruption is basically right. But what about this? Can the theory explain that?” This questioning has continued for the past 20 years. As a consequence, the theory of disruption has actually grown stronger; it has not been disproved or discredited. In this post, I’ll try to explain why this is so.
First let’s look at what it means to validate a theory. At one point, a friend of mine named Andy, whose training is in econometrics, asked me why I hadn’t tested the theory of disruption. In reply, I argued that there are two fundamentally different approaches to research in the social sciences. Andy’s approach—and that of many other academics—is to launch a search for the best way of explaining things, in order to get the best result. The search is often called a “performance hypothesis,” or, in the commonly used academic terminology, a “positive” approach. You might test the disruption hypothesis by collecting data about companies in industries that have been disrupted and in industries that had not been. This is essentially what I did in the early part of my data analysis, when I found that 35% of disruptive companies became successful as compared with only 6% of companies that followed a sustaining trajectory. Andy’s approach would then require him to collect data on how well the disruptors and disruptees performed in other industries, before and after the disruption occurred, and so on.
The other approach, I suggested, is akin to basic research. I had observed, documented, explained, and sought replication of a basic causal process. The logic behind this approach is this: if we understand causally how the world works, we have a higher probability of being successful in whatever we are trying to do. Thomas Kuhn (The Structure of Scientific Revolutions); Alfred Chandler (The Visible Hand); Joseph Bower (The Resource Allocation Process); Anita Elberse (Blockbusters); Jeffrey Pfeffer and Gerald R. Salancik (The External Control of Organizations) and James G. March (in a vast body of work) have been leaders in this approach. These thinkers do not have an agenda, a performance hypothesis that they are trying to prove or disprove. Instead they seek to understand causal mechanisms based on a small number of deep studies. As a statistician might say, they have a small N, not a big one. I am a disciple of this approach.
Because I took the second approach, I was confronted by anomalies to my theory—industries and situations where the theory didn’t seem to apply. Anomalies are valuable in theory building because, to paraphrase Thomas Kuhn, “the discovery of an anomaly is the enabling step toward less ambiguous description and measurement, and to identifying and improving the categorization scheme in a body of theory.” Researchers whose goal is to prove a theory’s validity often view discovery of an anomaly as failure; they therefore search for reasons to exclude outlying data points, hoping to get more significant measures of statistical fit. Researchers whose goal is to understand causality, by contrast, welcome anomalies, because there typically is more information in the outlying data points than in the ones that fit tidily into the model. Understanding these outliers or anomalies is generally the key to discovering problems in definition and measurement, and in formulating better categorization schemes. Researchers who seek to surface and resolve anomalies therefore tend to advance their fields more productively than those that seek to avoid them.
So the discovery of anomalies helped to sharpen my disruptive model. It opened up new ways of seeing what was happening in industries as diverse as steel and healthcare. It has also given me a lifetime full of interesting problems and new insights. What follows are a few examples, provided in intellectual (rather than chronological) sequence.
At about the same time, my friend and sometime coauthor Michael Raynor brought another anomaly to my attention, which we think originated with one of his students. “Hotels,” Michael posited, “don’t improve their performance in the same way disk drive makers do. They don’t go up-market. Look at the evidence. Holiday Inn started up at the inexpensive end of the market. But they haven’t added services that would drive their low-cost model up to higher price points. They can go up to another price point only if they emulate all of the amenities, the ballrooms, and the restaurants that the competitors at that price point are offering. This means that firms must try to succeed with a sustaining innovation strategy. They seem unable to disrupt the leaders above them.
“And the same thing goes for McDonald’s.,” he concluded. “They came into the low end of the market but they haven’t moved upmarket in a significant way.”
It took ten years for us to resolve this seeming contradiction, and here is how we now explain it. There are indeed some industries in which the two trajectories of improvement – supply and demand – are roughly the same. Disruption does not occur in those industries. This is why my friend could not prove that the trajectories were different: in some industries they are quite different, and in others they are essentially the same. The hotel, restaurant, branded food, and higher education industries historically have been among those in which the slopes of demand and supply are similar. Companies in industries like these have moved up-market to higher price tiers. But they can do so only by emulating competitors at those higher tiers, not by offering a disruptive technology or business model. McDonald’s was able to move up-market by investing in Chipotle, not by remodeling McDonald’s. Holiday Inn has moved up-market by establishing Crowne Plaza and then buying Embassy Suites Hotels – emulating, not disrupting, the Hilton, Sheraton, and Marriott chains.
Here’s another way of understanding the difference between disruptive and nondisruptive situations. Disruption occurs in markets where an innovator possesses something that we might call an “extensible core.” An extensible core is an element of a company’s processes or profit formula which at first can do only simple things, but which then can be improved to do ever-more sophisticated work without adding substantially to its costs. In the classic case of steel minimills, the minimills’ extensible core was their electric furnace. That allowed them to alloy the melt to produce consistent outputs regardless of the scrap inputs. The Intel microprocessor was the extensible core for computers. The business model itself was the extensible core for Target; by changing the merchandising mix on the store layout, the company could move up-market.
You can see the importance of an extensible core in higher education, another industry that has largely resisted the forces of disruption. Many entrants have started as state universities or community colleges, with charters to make higher education better in quality and lower in cost. At the outset, typically, they offered associate and bachelor degrees. Yet in most instances—and often within just a few years of their opening—these institutions start climbing up the academic ladder, offering masters and then doctoral degrees, providing higher quality education at higher cost. Universities apparently can climb the academic ladder only by behaving like the institutions above them, emulating their dormitories, their classroom buildings, their stadiums, their laboratories, and their faculty. It has been an expensive arms race, and the entrants have been unable able to disruptive the incumbents. With few exceptions, the top 20 universities of thirty years ago are still the top 20, and the second 50 are still in the second 50. So it remains, decade after decade.
There has been no disruptive effect in the higher education market because the trajectories of customers and innovators have not crossed. The only way a low-cost competitor could go up-market has been to take the incumbents head-on with a sustaining strategy. Most of the entrants would have failed by now but for the largesse of governments, which paid dearly to sustain all these competitors trying to climb the same academic ladder.
The question then was this: Is there a new extensible core – a technology or an innovation in business model – that could tilt the trajectory of higher education upwards? Could innovations in higher education intersect with a different customer demand trajectory, doing sufficiently sophisticated things without emulating the sophisticated costs of their entrenched competitors? The electric furnace played this role in steel, the microprocessor in the personal computer industry. In higher education, the extensible core for a disruptive entrant may turn out to be online learning, which is becoming broadly available. Real tuition at online colleges is falling. Accessibility and quality are improving. The steepness of the innovator’s trajectory is stunning, and disruption has begun in earnest. A similar change is occurring in the hotel industry, because of AirBnB and similar lodging-sharing enterprises. But it has not yet occurred in foods and restaurants.
The answer to our question as to whether disruption occurs everywhere is thus a resounding no. The diagram “Theory of Disruption 2.0” shows that the likelihood of disruption depends on two factors: the slopes of the two curves, and the existence of an extensible core in the sense defined earlier. If the trajectories of improvement and customer need remain close to parallel—that is, if they don’t cross—disruption is highly unlikely. But an intersection depends on the introduction of new technologies or new business models. When such an innovation appears, it sets the stage for an industry to metamorphose from one that has been immune into one that can be disrupted.
Does the vertical axis always measure performance?
Disruptive theory depended initially on a measure of performance—capacity in disk drives, output and mix of products in steel mills, and so on. But always using a performance metric on the vertical axis turns out to be misleading. Anomalies in airlines, restaurants, venture capital, and commercial lending have convinced us that the vertical axis in disruption should usually measure the profit formula. That is, it should measure the innovations that helped leading companies make more money in the way they are structured to make money, with the most-profitable products or services at the top and the least-profitable ones at the bottom.
We’ll summarize the cases in each of these industries.
Disruption never seemed to fit aviation, in part because it wasn’t clear what constituted “performance.” Deep-discount coach, full-fare coach, business class, first class—and then what? A few months after Seeing What’s Next was published, I received my Delta frequent flyer report. The report announced that in the previous quarter Delta had added eight new routes. They included, if I remember right, flights from Atlanta to Dubai, Atlanta to Johannesburg, and Dallas to Seoul—all long, lusciously profitable routes. On the next page the report listed an additional 31 new routes operated by regional airlines like Skywest. These were routes that went from Salt Lake City to Idaho Falls, from Atlanta to Birmingham, Alabama, and the like. Some of them were relatively long as well, such as Atlanta to Des Moines, Iowa.
All at once, everything seemed to fall into place. The vertical axis in aviation should measure the length of the route. Long routes are more profitable than short routes for major airlines. So the major airlines were resolutely going up-market, adding more long routes and outsourcing short routes to the regional airlines. And the regional airlines, which started on the shortest routes, were themselves resolutely going up-market, adding longer and longer routes. This suggested a fundamental improvement in the theory: the vertical axis needs to measure the profit formula of the leading companies in an industry. These companies will pursue whatever activity helps them make more money in the way they are structured to make money. If the innovation will cause the leading companies to make less money in the way they are structured to make money, they cannot reasonably pursue it. That opens the way for entrants.
Consulting firms. Here, too, insight stemmed from anomaly. A close friend from the Boston Consulting Group, where I worked for five years, visited me with a puzzle. When he and I joined the firm, nearly all of its projects were focused on strategy. Today, my friend said, less than 10% of the firm’s revenues come from strategy projects. Most of the revenue came from operational projects, especially post-merger integration. For example, a client was then paying BCG about $150 million for its guidance on integrating with another major company that the client had recently acquired.
“How can we get back into the strategy business?” my friend wondered.
“How much do you get for a strategy-related project for a client?” I asked.
“About $500,000,” he said.
It was like airlines. In consulting, the metric that I had mislabeled as quality or performance should have been the profit formula according to the established business model of the leading firms. BCG was going upmarket by offering services that earned the firm more money in the way it knew how to make money.
Venture capital and private equity. My friend Mitt Romney, later to become governor of Massachusetts and candidate for president, inadvertently taught me how to define the vertical metric in the venture capital industry. In 1984, I raised $2.5 million from four Boston-area venture capital funds to launch our high-end ceramics business. In the same year, Romney started a small fund called Bain Capital. The firm’s first investment was $4.5 million to start Staples.
In early 1985 Romney called me up, asking where we bought our stationery supplies. When I replied that we got them from an office supplies distributor in Lowell, Massachusetts, Mitt responded, “Could you please do me a favor? We have invested in a new type of discount stationery retailer, called Staples. Could you buy your supplies from Staples instead, just as a favor to me? Their store is on Soldiers Field Road, very close to the Harvard Business School campus.” I said that we would.
Think about this for a minute. Mitt Romney. Bain Capital. It was in Mitt’s interest to call a little, 30-employee startup and ask us to take our orders to Staples, just as a favor.
Not long thereafter, an HBS classmate called to say that he was trying to raise $3 million to $4 million to start a discount retailer firm. The venture was similar to Staples in its strategy, but it was in a different industry. He asked me to introduce him to Mitt Romney, and I agreed. When I described the idea to Mitt, he cut me off and announced, “We can’t waste our time on little deals like this any more. Our minimum deal size is $15 million.”
What had happened? Mitt and Bain Capital had made so much money so fast, that when they raised their second fund for 10 times the size of the first, they closed it within hours. They had so much more money per partner that they had to raise the minimum size of the deal. A firm that had been an early-stage investor had become a later-stage private equity investor, and it was then positioned to become a leveraged buyout shop like Kohlberg Kravis Roberts & Co. In terms of the disruptive model, the vertical axis was the size of the deal. Just as the minimills were resolutely focused on expanding into structural beams and rolled sheet steel, the partners of funds like Bain Capital were resolutely focused on ever-bigger deals.
After a good discussion, we realized that individual products take price or functionality positions in their markets, and they tend to stay. Because a company such as Coca-Cola doesn’t have a new extensible core, it must go up-market by adding new products at higher price points. As shown in figure TK, Toyota did not evolve its Corona-brand subcompact into a Lexus. Rather, it moved up-market as a corporation by layering the Corolla, Camry, Avalon, and then the Lexus on top of its initial product. The minimills moved up-market not by making rebar into structural steel. Rather, they added to the high end of their product line and pruned products from the low.
The market for drinks has created a huge premium market. For years, Coca Cola did not move upmarket, because as a general rule products do not go upmarket. Finally, Coca-Cola bought brands that other companies had developed and moved upmarket in that manner.
Bank lending. As a final illustration, consider the bank lending market. In 1970, at age 18, I applied for a credit card at a bank in Salt Lake City. A loan officer interviewed me for 30 minutes, asking questions about my life and my family that seemed irrelevant to my application. He must have judged me trustworthy enough: the bank issued a MasterCard with a maximum balance of $375.
At about the same time, a Minneapolis-based company called Fair-Isaac developed a formula (called a FICO score) by which a computer could assess the creditworthiness of borrowers. The first application of the Fair-Isaac formula was in Sears, Roebuck department stores. When its customers didn’t have enough cash to purchase what they needed, Sears would calculate their FICO scores. If they were deemed worthy, Sears then gave them its own credit card. Later, as banks gained more and more confidence in the FICO algorithm, they began issuing consumer loans via credit cards. They then applied this method to progressively bigger loans. Before long, most consumer lending was being done completely by computer, without the judgment of banks’ loan officers. Even car loans, mortgages, and (eventually) small-business loans were issued in this way. This, too, has been an important insight about the causal mechanism by which companies move upmarket.
In a Q&A session in a large hotel ballroom in Silicon Valley a few years ago, an engineer asked, “I can understand that the Intel microprocessor was a disruptive innovation relative to the wired logic circuit board made by minicomputer products such as Digital Equipment Corporation (DEC). But what drove the Intel’s processor cost down and its performance up were the machines made by companies like Applied Materials. It was their innovations, more than Intel’s, that made Moore’s Law happen. So was the equipment made by Applied Materials disruptive? Or sustaining?”
Another engineer in the same meeting asked, “In our fab we are trying to implement the Toyota Production System, with minimal work in process inventory. It is reducing costs, improving quality, and freeing up working capital. Is this a disruptive or sustaining innovation? Or is it both?”
These questions and others led me to realize that the theory of disruption had become too big. It was being used to explain too much. In response to these anomalies, we have been pruning the branches of disruption, trying to be clearer about what disruption is and is not. Disruption is not a theory of growth, nor is it a theory of efficiency; it is a theory of competitive response. Remember that disruption explains how competitors will respond to you – whether they will flee or fight. This always has been the theory’s central insight. We will have more to say on the boundaries and conditions for this insight in following entries.
Unfortunately, not every part of the original theory has revealed either a boundary or an important condition for application. One significant element of the theory of disruption, for example, is the idea of separation of business units. If the established leaders in an industry are being disrupted by an entrant, the established companies can respond successfully only if they set up completely independent business units under the corporate umbrella and give them an unbridled charter to kill the established units. If the corporation tries to address the disruption from within established business units, it will not succeed. From the time this piece of the theory was published, in 1998, we have not been able to find an anomaly to this assertion.
The fact that we have found no exceptions to the rule is not a triumph, however, because the 17 years without an anomaly have been a period with no improvement in the theory. So I and my students are truly searching for anomalies, where the established leader fought off the disruption without an independent business unit. It is as if we have put out a neon light announcing, “Anomalies Wanted.” If ever we find one, we might help managers create new businesses in cheaper and easier ways—thereby either bounding or improving our understanding of the conditions of the theory’s application. Incidentally, I cannot overstate the role that my students have played in the process of improving the theory of disruption. On the first day of every semester I describe how they need to prepare for class. I assign them the job of finding something that the theory of that day’s discussion cannot account for. Because of this “Anomalies Wanted” assignment in my class, my students have stepped up to the plate – and the theory has improved dramatically, as the examples summarized above show.
Reflections on anomalies
Despite having read Kuhn’s Structure of Scientific Revolutions and its wisdom about how science progresses—more than once—I didn’t feel the significance of anomalies in my bones at first. After I had articulated the theory, I sought to apply my theory in related areas. But what really supercharged the research was the search for answers to the anomalies people discovered. The act of going back into the dumpster of the phenomena to understand the origins of these anomalies drove improvement. In the Quality Movement, the articulation of a standard—and rigorous adherence to that standard—helps workers see anomalies, diagnose their origin, and improve all future work. The same thing happened to me. Once I had a causal theory, smart people asked really hard questions like the ones above. Those questions helped me understand the origin of the anomaly and its root cause. Put another way, the improvements to the theory of disruption didn’t come about in spite of anomalies but because of them.
All of those insights have been wonderful. But perhaps the biggest discovery of all was the “discovery” of anomaly itself. I’m now sharply attuned to anomalies because I know they provide the surest route to deepening our understanding of the world around us. I have even made a wooden sign that I display in my office that says “Anomalies Wanted.” It is a constant reminder that I should be on the prowl not for proof of my ideas, but for the things my ideas cannot explain. It’s as if you were playing baseball and the rules had changed, so that even striking out counted as a hit. Learning what the theory cannot explain strengthens that theory just the same as learning what it can explain. Without anomalies, I could never have seen the boundaries of the theory so clearly. Because I approached my thesis project in this manner, I unwittingly opened a lifelong research program that seems still to be gaining momentum.
 Of course, I couldn’t go out to create experimental replication, as can be done in the physical sciences. Rather, my replication started by testing my hypothesis on generations of disk drives. Since then, I have sought replication through application of the causal model to other phenomena. This post is principally about how repeated replication of this sort gave rise to anomalies that have clarified and delineated the causal process I first observed in disk drives.
 Later work by Michael Horn and myself in Disrupting Class resolved these early problems. Continuing work by Michael and his colleagues at the Christensen Institute has led to robust application and positive influence in educational policy and practice.