Dispatches from the Frontier

Breakthroughs and the Long Tail of Innovation

Lee Fleming, a researcher at Harvard Business School, has written an interesting article titled Breakthroughs and the "Long Tail" of Innovation in the Fall 2007 issue of MIT Sloan Management Review.  In it, Fleming analyzes the impact of the following variables (the "levers of invention") given his assumptions that "invention is essentially a process of recombinant search" and "Almost all inventions are useless; a few are of moderate value; and only a very, very few are breakthroughs."  (Click on image for larger view of Fleming's summary table.)

The Presence of Lone Inventors

My fieldwork consistently indicates that innovators working by themselves can be the source of more failures as well as more breakthroughs...[My analysis of a sample of all patented U.S. inventors since 1975 indicates] that lone inventors generate fewer novel combinations and that they combinations they create are less likely to be used, on average, by future inventors...But the future use of their work is also much more variable, such that they are more likely to be the source of a highly skewed outlier, thus bolstering the argument that they are indeed more likely to be the sources of radical innovations...

A less rigorous selection process for loners...means that they will have a greater variance of output [relative to collaborative inventors].  Because they are less constrained by convention and skeptical groupthink, lone inventors are more likely to invent (and not immediately dismiss) the radical breakthrough.  Thus, lone inventors are less creative on average and yet are also more likely to come up with a breakthrough.  In other words, lone inventors make fewer shots on average and tend to score both very low and extremely high.  They are on average both less successful—and more likely to be the source of breakthroughs.

This is consistent with our experience at EIP.  Over the last two years, we have screened over 2,000 inventions submitted by individual inventors and have made collaboration proposals on less than 1%.  On the other hand, using the Merwyn Concept Accelerator tool as our measuring stick, we have found that the best concepts submitted by individual inventors score at least a standard deviation higher than the average of concepts developed by collaborative, corporate teams.

Brokered Collaboration

Collaborations among inventors can be either brokered or cohesive.  Brokerage occurs when a single individual is the hub through which others interact.  In a cohesive collaboration, people develop separate and independent relationships with one another that do not include a central individual...

By brokering others, hub inventors gain first access to and control of information, enabling them to generate a greater number of new combinations.  Yet this same social networking structure also makes in inherently more difficult for others to understand a local inventor's idea in order to critique, transfer and evolve it.  Hence, brokers tend to generate more new combinations, but those innovations are less likely to be picked up and developed by others...

Neither brokered nor cohesive collaboration is inherently superior to the other; much depends on the organizational culture and the specific environment of the inventors.  For example, although brokerage is better for the generation of new combinations, cohesion confers strong marginal benefits in collaborations that lack trust or involve fresh information.

This rings true, as well.  A number of large companies that have embraced the idea of Open Innovation have designated certain people to broker connections among external and internal networks of inventors and experts.  Their organizational objective is to increase access to prospective breakthrough innovations.  However, in a number of circumstances, we've found that the person charged with being the "hub inventor" behaves in manner that preserves her proprietary control over the flow of information.  Because "lack of trust or...fresh information" is a common characteristic in this context, we've found that an effective implementation of Open Innovation requires the transformation of brokered collaborations into cohesive collaborations.

Diversity of Teams

Diversity helps generate more shots on goal although, on average, those shots are less successful.  But diversity also gives rise to new and unexplored combinations that increase the probability of a highly skewed breakthrough...

My own research studying more than 17,000 patents has found that the greater the divergence between collaborators' fields of expertise, the lower the overall quality of their output.  But multi-disciplinary collaboration increases the variance of the outcome, such that failures as well as breakthroughs are more likely.

This, too, makes sense.  Collaboration requires the cultivation of shared meaning.  People from diverse backgrounds view the world differently, use different language, and use the same language in different ways.  The development of effective boundary objects that effectively facilitate shared meaning is hard, subtle work.

Application of Science

My fieldwork, research and experience suggest that the scientific method and knowledge help provide a useful map of the technology landscape...

In effect, scientific methods make the process of invention less random, enabling inventors to make fewer but better shots on goal.

Or, as my partner Vandy Van Wagener would say, "Run the damned experiment."  The iterative resolution of uncertainty through prototyping, valid market research, and in-market tests are well-known techniques to the product development world, but surprisingly underutilized.  In previous posts, I've speculated as to why that might be.

On balance, I'm singing from Fleming's hymnal.  But, I would quibble with his use of the term "average score."  Although Fleming posits the existence of a "long tail" or power law distribution of market outcomes for inventions, he speaks of the "mean value" of outcomes.  As Art De Vany and others have observed, the mean of a power law distribution has little, if any, meaning.  Nevertheless, if one were to substitute "median" for "mean" in his article, I believe Fleming's analysis is apt.

One Entrepreneur's Take on VCs

ax - i - om n. A self-evident or universally recognized truth.

This morning, I came across a list of axioms regarding venture capitalists posted by an entrepreneur who calls himself "jetskier."  The following are my favorites.  Substitute "product capitalist" or "big company" for "VC" as you see fit.

• If you show a VC a potential $1B market, they won't believe you.  If you show a VC a potential $100M market, they will consider the market to be too small.
• Patents are important as a barrier to entry only if you don't have any.  Otherwise, they are considered unimportant.
• Investments outside the comfort zone will not be considered, even if potentially lucrative.  Investments inside the comfort zone will not be considered, as there is too much competitive pressure.
• Entrepreneurs are a dime-a-dozen; VCs are a rare breed.
• Any idea worthy of investment must be fatally flawed in some respect.
• No means "no," and yes means "maybe."

Critiques of the Red Queen Model

One of the immensely valuable aspects of a weblog is the opportunity it provides to test ideas.  On occasion, I've forwarded my version of a Red Queen model (7:13 video).  Polymath Benoit Mandelbrot might call it a "cartoon":

I use the term in the sense of the Renaissance fresco painters and tapestry designers: a preliminary sketch in which the artist tries out a few ideas, and which if successful becomes a pattern for the full ouvre to come.

The basic idea of my cartoon is pretty simple:  If the fundamental pace of an industry—its clockspeed—increases by, say, 10% per year, the compounded effect is a kind of innovation deficit.  Taken literally, one would have to "run" twice as fast today just to keep still.  I chose to define innovation as launching new products.  I've also inferred that the model supports a trend toward Open Innovation.  In this case, Open Innovation would be the leveraging of external capacity to launch new products.

Fortunately, the idea has prompted critical feedback.  For instance, John Hagel wrote:

One quibble I would make is with Dave's definition of innovation as "the adoption of products by customers".  This is a product-centric view of innovation and ignores the impact of process innovation (which also includes innovations in work practices).  In fact, process innovations are ultimately much more powerful in terms of generating business value because, if done right, they can generate a compounding effect of their own - they keep on giving, in contrast to most product innovations where the tyranny of product life cycles limits the potential value creation.  Rapid incremental process innovation combined with aggressive leveraging of third party resources may in fact hold the key to diminishing, if not overcoming, the Red Queen effect.

That's fair enough: innovation is more than launching new products.  On the other hand, I think that even my cartoon model accommodates John's point.  Companies generate revenues by selling products and services to users.  As the pace of change forces the "retirement" of products and enabling technologies, new products must be launched for the company to sustain itself and grow.  Although I didn't expand upon it in my cartoon, process innovations (as I understand the term) increase the capacity to launch new products by increasing capability.  (The way I use the terms is that capacity is the product of resources times capability, a kind of productivity rate.)  In other words, an increase in capability decreases the average cost to launch a successful new product.  So, absolutely, process innovations can increase capacity and, hence, mitigate the Red Queen effect.  Furthermore, because capability building is often subject to S-shaped learning curves, an investment in process innovations can yield attractive returns versus simply throwing more resources at the problem.

Kathleen Fasanella commented:

I found the Red Queen presentation problematic; the conclusions don't necessarily dictate the adoption of open innovation (read: confirmation bias).

Quite right.  Conceptually, there are considerable potential advantages of a closed (i.e., internally focused) model of innovation.  For instance, combining capacities within a single organization can make hand-offs between functions easier.  That's the essence of the classic transaction cost theory of the firm.  Nevertheless, I think John Hagel and his collaborator John Seely Brown are on to something:

Specialization requires connectivity and effective methods of coordination.  If enterprises cannot depend on other specialized entities to complement their own activities, they will avoid specialization themselves and suffer productivity penalties as a consequence...By connecting with other specialized institutions, we create an opportunity for leveraged capability building—getting better faster by working with others.

To the extent the Red Queen effect is real, it doesn't dictate the adoption of Open Innovation, I just think it's a good idea (but not the only good idea).

Although Richard Veryard's original critique of my Red Queen cartoon echoed John Hagel's, I find Richard's skepticism regarding the fundamental assumption of accelerating industry clockspeed the most intriguing:

I have always been wary of the common belief that technological change is accelerating.  I think this belief derives from a combination of proximity, selectivity and distorted perception.  I think we can sometimes be disproportionately impressed by the glamour of recent technology, and misled by the commercially-driven measures of intellectual property (such as volumes of patent activity and product releases).

I think there is evidence of accelerating clockspeed.  Nevertheless, Richard's skepticism raises a really interesting point: Is such acceleration constant?  In other words, might the evolution of innovation be punctuated?  As I understand it, Mandelbrot has hypothesized a kind of variable market pacing he's called "trading time" and "multifractal time".  He acknowledges that the concept "remains mostly speculation.  But it already permits some extraordinarily faithful reproductions of a financial market."

So, here's where I think this exploration is taking me:

• It's conceivable that different industries and markets feel alternatively fast and slow over time.
• An accelerating market fuels the need for innovation—of all kinds.
• An Open Innovation model offers the prospect of value in two guises.  First of all, it is a way to source compelling new products and services.  Secondly, collaboration among specialist organizations offers the potential of accelerated capability building.
• The degree of emphasis on Open Innovation models may wax and wane as a function of perceived changes in industry clockspeed.

Thus, the cartoon evolves.

Thanks for the feedback!

For more: The Only Sustainable Edge, The (Mis)Behavior of Markets, Open Innovation: Researching a New Paradigm