Research

Solving complex problems in medicine, engineering, and other technological domains often requires exploring multiple approaches, particularly when significant uncertainty exists about which one will lead to success. Conventional wisdom assumes that having many experimenters independently decide which approaches to pursue increases diversity and thus also the chances of finding a solution. However, if experimenters herd toward the most promising approach, this convergence may reduce diversity and thus the likelihood of solving the problem. In this paper, we develop a simple model to show that, holding the total number of experiments constant, markets dominated by a few large-scale experimenters, firms conducting multiple experiments, explore more diverse approaches than markets with many single-shot experimenters. Single-shot experimenters tend to converge on the most promising approach, while multi-experimenters are more likely to diversify to avoid the correlation inherent in pursuing multiple experiments within the same approach. We test our model's predictions using data from pharmaceutical R&D. Our analysis shows that increasing the average number of experiments per firm by one unit raises target diversity by over three standard deviations. In turn, a one-standard deviation increase from the mean in target diversity boosts the likelihood of at least one experiment reaching Phase 1 clinical trials by 25.9 percentage points. Our findings highlight the need for technology policies that optimize experiment allocation across firms to maximize approach diversity and market-level success.

Using detailed product- and invention-level data from the pharmaceutical industry, we demonstrate that firms with particularly high-selling blockbuster products concentrate their development efforts on new products that both target the same customer segments and are more likely to be technically similar to existing blockbuster products. This behavior, driven by an expectation of the stickiness of demand for existing product offerings, limits firms' incentives to invest in entirely new products targeting different customer segments. Our findings offer insights into how blockbuster products shape firms' customer segment and innovation choices, with implications for understanding the dynamics of technological change in R&D-intensive industries.