Publications

2020
Jin H. Paik, Steven Randazzo, and Jenny Hoffman. 6/2020. “AI in the Enterprise: How Do I Get Started?”.Abstract

While there are dispersed resources to learn more about artificial intelligence, there remains a need to cultivate a community of practitioners for cyclical exposure and knowledge sharing of best practices in the enterprise. That is why Laboratory for Innovation Science at Harvard launched the AI in the Enterprise series, which exposes managers and executives to interesting applications of AI and the decisions behind developing such tools. 

Moderated by HBS Professor and co-author of Competing in the Age of AI, Karim R. Lakhani, the most recent virtual session with over 240 attendees featured Rob May, General Partner at PJC, an early-stage venture capital firm, and founder of Inside AI, a premier source for information on AI, robotics and neurotechnology. Together, they discussed why we have seen a rise in interest in AI, what managers should consider when wading into the AI waters, and what steps they can take when it is time to do so. 

AI in Enterprise - How Do I Get Started (R May).pdf
Luke Boosey, Philip Brookins, and Dmitry Ryvkin. 5/13/2020. “Information Disclosure in Contests with Endogenous Entry: An Experiment.” Management Science. Publisher's VersionAbstract
We use a laboratory experiment to study the effects of disclosing the number of active participants in contests with endogenous entry. At the first stage, potential participants decide whether to enter competition, and at the second stage, entrants choose their investments. In a 2××2 design, we manipulate the size of the outside option, w, and whether the number of entrants is disclosed between the stages. Theory predicts more entry for lower w and the levels of entry and aggregate investment to be independent of disclosure in all cases. We find empirical entry frequencies decreasing with w. For aggregate investment, we find no effect of disclosure when w is low but a strong positive effect of disclosure when w is high. The difference is driven by substantial overinvestment in contests with a small, publicly known number of players contrasted by more restrained investment in contests in which the number of players is uncertain and may be small. The behavior under disclosure is explained by a combination of joy of winning and entry regret.
Timothy DeStefano, Richard Kneller, and Jonathan Timmis. 5/6/2020. “Cloud computing and firm growth.” VOX. Publisher's VersionAbstract
The last decade has seen a fundamental shift in the way firms access technology, from physical hardware towards cloud computing. This shift not only significantly reduces the cost of such technologies but also allows for the possibility of remote and simultaneous access. This column presents evidence on the impact of cloud adoption by firms using firm level data from the UK. There are marked differences in the effects on young and incumbent firms, where cloud adoption largely impacts the growth of young firms while it affects the geography of incumbent firms.
Roberto Verganti, Luca Vendraminelli, and Marco Iansiti. 3/19/2020. “Innovation and Design in the Age of Artificial Intelligence”. Publisher's VersionAbstract

At the heart of any innovation process lies a fundamental practice: the way people create ideas and solve problems. This “decision making” side of innovation is what scholars and practitioners refer to as “design”. Decisions in innovation processes have so far been taken by humans. What happens when they can be substituted by machines? Artificial Intelligence (AI) brings data and algorithms to the core of innovation processes. What are the implications of this diffusion of AI for our understanding of design and innovation? Is AI just another digital technology that, akin to many others, will not significantly question what we know about design? Or will it create transformations in design that current theoretical frameworks cannot capture?

This article proposes a framework for understanding design and innovation in the age of AI. We discuss the implications for design and innovation theory. Specifically, we observe that, as creative problem solving is significantly conducted by algorithms, human design increasingly becomes an activity of sense making, i.e. understanding which problems should or could be addressed. This shift in focus calls for new theories and brings design closer to leadership, which is, inherently, an activity of sense making.

Our insights are derived from and illustrated with two cases at the frontier of AI ‐‐ Netflix and AirBnB (complemented with analyses in Microsoft and Tesla) ‐‐, which point to two directions for the evolution of design and innovation in firms. First, AI enables an organization to overcome many past limitations of human‐intensive design processes, by improving the scalability of the process, broadening its scope across traditional boundaries, and enhancing its ability to learn and adapt on the fly. Second, and maybe more surprising, while removing these limitations, AI also appears to deeply enact several popular design principles. AI thus reinforces the principles of Design Thinking, namely: being people‐centered, abductive, and iterative. In fact, AI enables the creation of solutions that are more highly user‐centered than human‐based approaches (i.e., to an extreme level of granularity, designed for every single person); that are potentially more creative; and that are continuously updated through learning iterations across the entire product life cycle.

In sum, while AI does not undermine the basic principles of design, it profoundly changes the practice of design. Problem solving tasks, traditionally carried out by designers, are now automated into learning loops that operate without limitations of volume and speed. The algorithms embedded in these loops think in a radically different way than a designer who handles complex problems holistically with a systemic perspective. Algorithms instead handle complexity through very simple tasks, which are iterated continuously. This article discusses the implications of these insights for design and innovation management scholars and practitioners.

Marco Iansiti and Karim R. Lakhani. 3/3/2020. “From Disruption to Collision: The New Competitive Dynamics.” MIT Sloan Management Review.Abstract
In the age of AI, traditional businesses across the economy are being attacked by highly scalable data-driven companies whose operating models leverage network effects to deliver value.
Jin Paik, Martin Schöll, Rinat Sergeev, Steven Randazzo, and Karim R. Lakhani. 2/26/2020. “Innovation Contests for High-Tech Procurement.” Research-Technology Management, 63:2, 36-45. Publisher's VersionAbstract
Innovation managers rarely use crowdsourcing as an innovative instrument despite extensive academic and theoretical research. The lack of tools available to compare and measure crowdsourcing, specifically contests, against traditional methods of procuring goods and services is one barrier to adoption. Using ethnographic research to understand how managers solved their problems, we find that the crowdsourcing model produces higher costs in the framing phase but yields savings in the solving phase, whereas traditional procurement is downstream cost-intensive. Two case study examples with the National Aeronautics and Space Agency (NASA) and the United States Department of Energy demonstrate a potential total cost savings of 27 percent and 33 percent, respectively, using innovation contests. We provide a comprehensive evaluation framework for crowdsourcing contests developed from a high-tech industry perspective, which are applicable to other industries.
Christopher Stanton, Karim R. Lakhani, Jennifer L. Hoffman, Jin Hyun Paik, and Nina Cohodes. 1/13/2020. Freelancer, Ltd.. Harvard Business School Case. Harvard Business School.Abstract
Over the course of the 2010s, the rapid advancement of mobile technologies and the rise of online freelancing platforms seemed to portend a radical transformation of labor markets into on-demand, flexible talent pools. Even though several Fortune 500 companies-including Microsoft, Samsung, and General Electric-embraced digital labor solutions, enterprise adoption lagged far behind smaller businesses and startups. Despite the promising potential benefits, concerns persisted about navigating labor regulations, ensuring appropriate vetting, and guaranteeing the quality of work. Sarah Tang, the newly appointed Vice President of Enterprise at Freelancer, Ltd., took on the challenge of crafting the growth strategy, operations, and sales of Freelancer's services to Fortune 500 companies. What it would take to convince more enterprises of the potential of on-demand freelance labor that could help them hire skilled freelancers in volume or in multiple countries simultaneously? What did the future hold for open work practices between enterprises and digital labor markets?
Karim R. Lakhani, Hong Luo, and Laura Katsnelson. 1/2020. Market for Judgement: Creative Destruction Lab. Harvard Business School Case. Harvard Business School.
Andrea Blasco, Ted Natoli, Michael G. Endres, Rinat A. Sergeev, Steven Randazzo, Jin Paik, Max Macaluso, Rajiv Narayan, Karim R. Lakhani, and Aravind Subramaniam. 1/2020. “Improving Deconvolution Methods in Biology through Open Innovation Competitions: An Application to the Connectivity Map.” bioRxiv. Publisher's VersionAbstract
A recurring problem in biomedical research is how to isolate signals of distinct populations (cell types, tissues, and genes) from composite measures obtained by a single analyte or sensor. Existing computational deconvolution approaches work well in many specific settings, but they might be suboptimal in more general applications. Here, we describe new methods that were obtained via an open innovation competition. The goal of the competition was to characterize the expression of 1,000 genes from 500 composite measurements, which constitutes the approach of a new assay, called L1000, used to scale-up the Connectivity Map (CMap) — a catalog of millions of perturbational gene expression profiles. The competition used a novel dataset of 2,200 profiles and attracted 294 competitors from 20 countries. The top-nine performing methods ranged from machine learning approaches (Convolutional Neural Networks and Random Forests) to more traditional ones (Gaussian Mixtures and k-means). These solutions were faster and more accurate than the benchmark and likely have applications beyond gene expression.
2019
Peter Barrett and Karim R. Lakhani. 10/29/2019. Kymera Therapeutics: Building a Biotech Execution Plan. Harvard Business School Case. Harvard Business School. Publisher's Version
Feng Zhu, Sacha L. Schmidt, Karim R. Lakhani, and Sebastian Koppers. 10/24/2019. TSG Hoffenheim: Football in the Age of Analytics (B). Harvard Business School Case. Harvard Business School. Publisher's VersionAbstract
In 2015, Dietmar Hopp, owner of Germany's Bundesliga football team TSG Hoffenheim and co-founder of the global enterprise software company SAP, was considering how to ensure long-term sustainability and competitiveness for TSG Hoffenheim. While historically a small team from bottom rungs of the league, TSG Hoffenheim, with revenues of €60 million to €70 million, reached the top division of the Bundesliga in the 2008-2009 season thanks to a deliberate strategy focused on enhanced scouting, strong youth programs, and innovative technology and analytics that improved player development. In 2014 Hopp, who had personally invested €300 million in the club, built a "footbonaut," an automated training environment that collected data on players' skills and strengths. The tool, one of three in the world, helped scouts and coaches better assess and develop each player. Yet some managers felt the technology was a distraction, an investment too expensive for a team that was not yet cash-flow positive. The team finished the 2014-2015 season in eighth place, below the top division, and Hopp wondered whether the focus on technology and analytics was the right strategy to grow the club. He wondered if the "moneyball" approach-when a smaller team competed with wealthier teams by using statistical analysis to buy undervalued assets and sell overvalued assets-could work in football and if investments in technology could lead the team to financial independence.
Tarun Khanna, Karim Lakhani, Shubhangi Bhadada, Nabil Khan, Saba Dave, Rasim Alam, and Meena Hewett. 10/2019. “Crowdsourcing Memories: Mixed Methods Research by Cultural Insiders-Epistemological Outsiders.” Academy of Management Perspectives. Publisher's VersionAbstract
This paper examines the role that the two lead authors’ personal connections played in the research methodology and data collection for the Partition Stories Project - a mixed methods approach to revisiting the much-studied historical trauma of the Partition of British India in 1947. The Project collected survivors’ oral histories, a data type that is a mainstay of qualitative research, and subjected their narrative data to statistical analysis to detect aggregated trends. In this paper, the authors discuss the process of straddling the dichotomies of insider/outsider and qualitative/quantitative, address the “myth of informed objectivity”, and the need for hybrid research structures with the intent to innovate in humanities projects such as this. In presenting key learnings from the project, this paper highlights the tensions that the authors faced between positivist and interpretivist methods of inquiry, between “insider” and “outsider” categories of positionality, and in the quantification of qualitative oral history data. The paper concludes with an illustrative example from one of the lead authors’ past research experiences to suggest that the tensions of this project are general in occurrence and global in applicability, beyond the specifics of the Partition case study explored here.
John Winsor, Jin Paik, Michael Tushman, and Karim Lakhani. 10/2019. “Overcoming cultural resistance to open source innovation.” Strategy & Leadership, 47, 6, Pp. 28-33. Publisher's VersionAbstract

Purpose: This article offers insight on how to effectively help incumbent organizations prepare for global business shifts to open source and digital business models.

Design/methodology/approach: Discussion related to observation, experience and case studies related to incumbent organizations and their efforts to adopt open source models and business tools.

Findings: Companies that let their old culture reject the new risk becoming obsolete if doing so inhibits their rethinking of their future using powerful tools like crowdsourcing, blockchain, customer experience-based connections, integrating workflows with artificial intelligence (AI), automated technologies and digital business platforms. These new ways of working affect how and where work is done, access to information, an organization’s capacity for work and its efficiency. As important as technological proficiency is, managing the cultural shift required to embrace transformative industry architecture – the key to innovating new business models – may be the bigger challenge.

Research limitations/implications: Findings are based on original research and case studies. Insights are theoretically, based on additional study, interviews, and research, but need to be tested through additional case studies. Practical implications: The goal is to make the transition more productive and less traumatic for incumbent firms by providing a language and tested methods to help senior leaders use innovative technologies to build on their core even as they explore new business models.

Social implications: This article provides insights that will lead to more effective ideas for helping organizations adapt. Originality/value: This article is based on original research and case experience. That research and experience has then been analyzed and viewed through the lens of models that have been known to work. The result is original insights and findings that can be applied in new ways to further adoption within incumbent organizations.

10-1108_sl-08-2019-011441.pdf
Peter Barrett, Karim R. Lakhani, and Julia Kelley. 9/20/2019. Nimbus Therapeutics. Harvard Business School Case. Harvard Business School. Publisher's VersionAbstract
This case focuses on Nimbus Therapeutics, a biotechnology startup based in Cambridge, Massachusetts, as its leadership team tries to determine the company's long-term strategy. The startup's founders structured Nimbus as a limited liability company, which has given it more flexibility when it comes to funding and development partnerships. Does the operating structure still makes sense as Nimbus looks ahead to the future?
Karim R. Lakhani, Kerry Herman, and Julia Kelley. 9/16/2019. Obsidian: Product or Platform?. Harvard Business School Case. Harvard Business School. Publisher's Version
Yael Grushka-Cockayne and Karim R. Lakhani. 9/11/2019. 2U: Higher Education Rewired. Harvard Business School Case. Harvard Business School.Abstract
In its 2019 Partner Symposium, 2U, an online program management provider (OPM), showcased its new vision: "Career. Curriculum. Continuum. A construct for lifelong learning in the 21st century". 2U, founded in 2008, and went public in 2014, was looking to expand beyond their current degree offerings to include a wider range of programs, such as short courses, bootcamps, and professional certificates. Led by co-founder and CEO Chip Paucek, 2U believed that they were the strongest partner in the OPM market that could enable universities' digital transformation, allowing them to offer a variety of courses to a changing student profile. The universities, on the other hand, recognized that times were changing and that the appeal of a residential experience might be dwindling. Pressures of offering a more flexible learning format were mounting. Some schools were engaging in partnerships such as with 2U to get themselves online while others saw digital and online as the next evolution of instruction and that it was their responsibility to learn how to master it and own it. The case considers Paucek's challenge of leading a for-profit OPM. Was 2U growing in a way that risked alienating their most important stakeholders, the brand named universities themselves? Were the university leaders going to change their approach and start investing in the digital transformation themselves to avoid giving 2U a cut of their revenues?
Andrea Blasco, Michael G. Endres, Rinat A. Sergeev, Anup Jonchhe, Max Macaluso, Rajiv Narayan, Ted Natoli, Jin H. Paik, Bryan Briney, Chunlei Wu, Andrew I. Su, Aravind Subramanian, and Karim R. Lakhani. 9/2019. “Advancing Computational Biology and Bioinformatics Research Through Open Innovation Competitions.” PLOS One, 14, 9. Publisher's VersionAbstract
Open data science and algorithm development competitions over a unique avenue for rapid discovery of better computational strategies. We highlight three examples in computational biology and bioinformatics research where the use of competitions has yielded significant performance gains over established algorithms. These include algorithms for antibody clustering, imputing gene expression data, and querying the Connectivity Map (CMap). Performance gains are evaluated quantitatively using realistic, albeit sanitized, data sets. The solutions produced through these competitions are then examined with respect to their utility and the prospects for implementation in the field. We present the decision process and competition design considerations that lead to these successful outcomes as a model for researchers who want to use competitions and non-domain crowds as collaborators to further their research.
Peter Barrett, Karim R. Lakhani, Julia Kelley, and Kelly Herman. 8/28/2019. Synthetic Biology Investment Opportunity. Harvard Business School Case. Harvard Business School. Publisher's Version
Elizabeth E. Richard, Jeffrey R. Davis, Jin H. Paik, and Karim R. Lakhani. 4/25/2019. “Sustaining open innovation through a “Center of Excellence”.” Strategy & Leadership. Publisher's VersionAbstract

This paper presents NASA’s experience using a Center of Excellence (CoE) to scale and sustain an open innovation program as an effective problem-solving tool and includes strategic management recommendations for other organizations based on lessons learned.

This paper defines four phases of implementing an open innovation program: Learn, Pilot, Scale and Sustain. It provides guidance on the time required for each phase and recommendations for how to utilize a CoE to succeed. Recommendations are based upon the experience of NASA’s Human Health and Performance Directorate, and experience at the Laboratory for Innovation Science at Harvard running hundreds of challenges with research and development organizations.

Lessons learned include the importance of grounding innovation initiatives in the business strategy, assessing the portfolio of work to select problems most amenable to solving via crowdsourcing methodology, framing problems that external parties can solve, thinking strategically about early wins, selecting the right platforms, developing criteria for evaluation, and advancing a culture of innovation. Establishing a CoE provides an effective infrastructure to address both technical and cultural issues.

The NASA experience spanned more than seven years from initial learnings about open innovation concepts to the successful scaling and sustaining of an open innovation program; this paper provides recommendations on how to decrease this timeline to three years.

Raymond H. Mak, Michael G. Endres, Jin H. Paik, Rinat A. Sergeev, Hugo Aerts, Christopher L. Williams, Karim R. Lakhani, and Eva C. Guinan. 4/18/2019. “Use of Crowd Innovation to Develop an Artificial Intelligence–Based Solution for Radiation Therapy Targeting.” JAMA Oncology, 5, 5, Pp. 654-661. Publisher's VersionAbstract

Radiation therapy (RT) is a critical cancer treatment, but the existing radiation oncologist work force does not meet growing global demand. One key physician task in RT planning involves tumor segmentation for targeting, which requires substantial training and is subject to significant interobserver variation.

To determine whether crowd innovation could be used to rapidly produce artificial intelligence (AI) solutions that replicate the accuracy of an expert radiation oncologist in segmenting lung tumors for RT targeting.

We conducted a 10-week, prize-based, online, 3-phase challenge (prizes totaled $55 000). A well-curated data set, including computed tomographic (CT) scans and lung tumor segmentations generated by an expert for clinical care, was used for the contest (CT scans from 461 patients; median 157 images per scan; 77 942 images in total; 8144 images with tumor present). Contestants were provided a training set of 229 CT scans with accompanying expert contours to develop their algorithms and given feedback on their performance throughout the contest, including from the expert clinician.

Main Outcomes and Measures  The AI algorithms generated by contestants were automatically scored on an independent data set that was withheld from contestants, and performance ranked using quantitative metrics that evaluated overlap of each algorithm’s automated segmentations with the expert’s segmentations. Performance was further benchmarked against human expert interobserver and intraobserver variation.

A total of 564 contestants from 62 countries registered for this challenge, and 34 (6%) submitted algorithms. The automated segmentations produced by the top 5 AI algorithms, when combined using an ensemble model, had an accuracy (Dice coefficient = 0.79) that was within the benchmark of mean interobserver variation measured between 6 human experts. For phase 1, the top 7 algorithms had average custom segmentation scores (S scores) on the holdout data set ranging from 0.15 to 0.38, and suboptimal performance using relative measures of error. The average S scores for phase 2 increased to 0.53 to 0.57, with a similar improvement in other performance metrics. In phase 3, performance of the top algorithm increased by an additional 9%. Combining the top 5 algorithms from phase 2 and phase 3 using an ensemble model, yielded an additional 9% to 12% improvement in performance with a final S score reaching 0.68.

A combined crowd innovation and AI approach rapidly produced automated algorithms that replicated the skills of a highly trained physician for a critical task in radiation therapy. These AI algorithms could improve cancer care globally by transferring the skills of expert clinicians to under-resourced health care settings.

Pages