Incentives & Governance

Milena Tsvetkova, Sebastian Müller, Oana Vuculescu, Haylee Ham, and Rinat A Sergeev. 11/11/2022. “Relative Feedback Increases Disparities in Effort and Performance in Crowdsourcing Contests: Evidence from a Quasi-Experiment on Topcoder.” Proceedings of the ACM on Human-Computer Interaction, 6, CSW2, Pp. 1-27. Publisher's VersionAbstract
Rankings and leaderboards are often used in crowdsourcing contests and online communities to motivate individual contributions but feedback based on social comparison can also have negative effects. Here, we study the unequal effects of such feedback on individual effort and performance for individuals of different ability. We hypothesize that the effects of social comparison differ for top performers and bottom performers in a way that the inequality between the two increases. We use a quasi-experimental design to test our predictions with data from Topcoder, a large online crowdsourcing platform that publishes computer programming contests. We find that in contests where the submitted code is evaluated against others' submissions, rather than using an absolute scale, top performers increase their effort while bottom performers decrease it. As a result, relative scoring leads to better outcomes for those at the top but lower engagement for bottom performers. Our findings expose an important but overlooked drawback from using gamified competitions, rankings, and relative evaluations, with potential implications for crowdsourcing markets, online learning environments, online communities, and organizations in general.
Ademir Vrolijk and Zoe Szajnfarber. 12/20/2022. “The Opportunists in Innovation Contests Understanding Whom to Attract and How to Attract Them.” Research-Technology Management, 66, 1, Pp. 30-40. Publisher's VersionAbstract
Organizations increasingly turn to innovation contests for solutions to their complex problems. But these contests still face a fundamental inefficiency: they need to attract many participants to find the right solution, resulting in high costs and uncertainty. Studies have identified multiple dichotomies of successful and unsuccessful solver types, but these diverge. These studies also offer little guidance on how to attract successful solver types. We introduce the opportunist-transactor dichotomy, bridging whom to attract and how to attract them. Opportunists view the contest as a onramp to a new pursuit instead of a temporary undertaking. Characterizing solvers according to this new dichotomy was a better predictor of success than existing ones: in our context, most winners were opportunists. This type of solver was also reliably attracted by the seeker’s in-kind incentives, unlike those described by the other dichotomies. Our insights provide a deeper understanding of participants in complex contests and a concrete lever for influencing who shows up to solve.
Frank Nagle, James Dana, Jennifer Hoffman, Steven Randazzo, and Yanuo Zhou. 3/2/2022. Census II of Free and Open Source Software — Application Libraries. The Linux Foundation. Harvard Laboratory for Innovation Science (LISH) and Open Source Security Foundation (OpenSSF). Publisher's VersionAbstract

Free and Open Source Software (FOSS) has become a critical part of the modern economy. There are tens of millions of FOSS projects, many of which are built into software and products we use every day. However, it is difficult to fully understand the health, economic value, and security of FOSS because it is produced in a decentralized and distributed manner. This distributed development approach makes it unclear how much FOSS, and precisely what FOSS projects, are most widely used. This lack of understanding is a critical problem faced by those who want to help enhance the security of FOSS (e.g., companies, governments, individuals), yet do not know what projects to start with. This problem has garnered widespread attention with the Heartbleed and log4shell vulnerabilities that resulted in the susceptibility of hundreds of millions of devices to exploitation.

This report, Census II, is the second investigation into the widespread use of FOSS and aggregates data from over half a million observations of FOSS libraries used in production applications at thousands of companies, which aims to shed light on the most commonly used FOSS packages at the application library level. This effort builds on the Census I report that focused on the lower level critical operating system libraries and utilities, improving our understanding of the FOSS packages that software applications rely on. Such insights will help to identify critical FOSS packages to allow for resource prioritization to address security issues in this widely used software.

The Census II effort utilizes data from partner Software Composition Analysis (SCA) companies including Snyk, the Synopsys Cybersecurity Research Center (CyRC), and FOSSA, which partnered with Harvard to advance the state of open source research. Our goal is to not only identify the most widely used FOSS, but to also provide an example of how the distributed nature of FOSS requires a multi-party effort to fully understand the value and security of the FOSS ecosystem. Only through data-sharing, coordination, and investment will the value of this critical component of the digital economy be preserved for generations to come.

In addition to the detailed results on FOSS usage provided in the report, we identified five high-level findings: 1) the need for a standardized naming schema for software components, 2) the complexities associated with package versions, 3) much of the most widely used FOSS is developed by only a handful of contributors, 4) the increasing importance of individual developer account security, and 5) the persistence of legacy software in the open source space.

Philip Brookins, Dmitry Ryvkin, and Andrew Smyth. 3/8/2021. “Indefinitely repeated contests: An experimental study.” Experimental Economics . Publisher's VersionAbstract
We experimentally explore indefinitely repeated contests. Theory predicts more cooperation, in the form of lower expenditures, in indefinitely repeated contests with a longer expected time horizon. Our data support this prediction, although this result attenuates with contest experience. Theory also predicts more cooperation in indefinitely repeated contests compared to finitely repeated contests of the same expected length, and we find empirical support for this. Finally, theory predicts no difference in cooperation across indefinitely repeated winner-take-all and proportional-prize contests, yet we find evidence of less cooperation in the latter, though only in longer treatments with more contests played. Our paper extends the experimental literature on indefinitely repeated games to contests and, more generally, contributes to an infant empirical literature on behavior in indefinitely repeated games with “large” strategy spaces.
Philip Brookins and Paan Jindapon. 2/20/2021. “Risk preference heterogeneity in group contests.” Journal of Mathematical Economics. Publisher's VersionAbstract
We analyze the first model of a group contest with players that are heterogeneous in their risk preferences. In our model, individuals’ preferences are represented by a utility function exhibiting a generalized form of constant absolute risk aversion, allowing us to consider any combination of risk-averse, risk-neutral, and risk-loving players. We begin by proving equilibrium existence and uniqueness under both linear and convex investment costs. Then, we explore how the sorting of a compatible set of players by their risk attitudes into competing groups affects aggregate investment. With linear costs, a balanced sorting (i.e., minimizing the variance in risk attitudes across groups) always produces an aggregate investment level that is at least as high as an unbalanced sorting (i.e., maximizing the variance in risk attitudes across groups). Under convex costs, however, identifying which sorting is optimal is more nuanced and depends on preference and cost parameters.
Marco Iansiti and Karim R. Lakhani. 3/3/2020. “From Disruption to Collision: The New Competitive Dynamics.” MIT Sloan Management Review.Abstract
In the age of AI, traditional businesses across the economy are being attacked by highly scalable data-driven companies whose operating models leverage network effects to deliver value.
Roberto Verganti, Luca Vendraminelli, and Marco Iansiti. 3/19/2020. “Innovation and Design in the Age of Artificial Intelligence”. Publisher's VersionAbstract

At the heart of any innovation process lies a fundamental practice: the way people create ideas and solve problems. This “decision making” side of innovation is what scholars and practitioners refer to as “design”. Decisions in innovation processes have so far been taken by humans. What happens when they can be substituted by machines? Artificial Intelligence (AI) brings data and algorithms to the core of innovation processes. What are the implications of this diffusion of AI for our understanding of design and innovation? Is AI just another digital technology that, akin to many others, will not significantly question what we know about design? Or will it create transformations in design that current theoretical frameworks cannot capture?

This article proposes a framework for understanding design and innovation in the age of AI. We discuss the implications for design and innovation theory. Specifically, we observe that, as creative problem solving is significantly conducted by algorithms, human design increasingly becomes an activity of sense making, i.e. understanding which problems should or could be addressed. This shift in focus calls for new theories and brings design closer to leadership, which is, inherently, an activity of sense making.

Our insights are derived from and illustrated with two cases at the frontier of AI ‐‐ Netflix and AirBnB (complemented with analyses in Microsoft and Tesla) ‐‐, which point to two directions for the evolution of design and innovation in firms. First, AI enables an organization to overcome many past limitations of human‐intensive design processes, by improving the scalability of the process, broadening its scope across traditional boundaries, and enhancing its ability to learn and adapt on the fly. Second, and maybe more surprising, while removing these limitations, AI also appears to deeply enact several popular design principles. AI thus reinforces the principles of Design Thinking, namely: being people‐centered, abductive, and iterative. In fact, AI enables the creation of solutions that are more highly user‐centered than human‐based approaches (i.e., to an extreme level of granularity, designed for every single person); that are potentially more creative; and that are continuously updated through learning iterations across the entire product life cycle.

In sum, while AI does not undermine the basic principles of design, it profoundly changes the practice of design. Problem solving tasks, traditionally carried out by designers, are now automated into learning loops that operate without limitations of volume and speed. The algorithms embedded in these loops think in a radically different way than a designer who handles complex problems holistically with a systemic perspective. Algorithms instead handle complexity through very simple tasks, which are iterated continuously. This article discusses the implications of these insights for design and innovation management scholars and practitioners.

Best Management Practices

LISH is working to develop testable systems and methods to help open innovation (OI) practitioners explore techniques for best practices. To date, the lab has spent extensive time studying both contests and communities with profit companies, governments, academic research centers, and platforms. Research in these areas explore... Read more about Best Management Practices

Social Comparison and Contests

In this project, we study the effect of social comparison on inequality in contest environment. We hypothesize that the effects of social comparison on effort will differ for top-performers and bottom-performers in a way that inequality increases, where we consider inequality both in terms of the dispersion of outcomes... Read more about Social Comparison and Contests

Kevin J. Boudreau and Karim R. Lakhani. 2015. “'Open' Disclosure of Innovations, Incentives and Follow-on Reuse: Theory on Processes of Cumulative Innovation and a Field Experiment in Computational Biology.” Research Policy, 44, 1, Pp. 4-19. Publisher's VersionAbstract

Most of society's innovation systems – academic science, the patent system, open source, etc. – are “open” in the sense that they are designed to facilitate knowledge disclosure among innovators. An essential difference across innovation systems is whether disclosure is of intermediate progress and solutions or of completed innovations. We theorize and present experimental evidence linking intermediate versus final disclosure to an ‘incentives-versus-reuse’ tradeoff and to a transformation of the innovation search process. We find intermediate disclosure has the advantage of efficiently steering development towards improving existing solution approaches, but also has the effect of limiting experimentation and narrowing technological search. We discuss the comparative advantages of intermediate versus final disclosure policies in fostering innovation.

Karim R. Lakhani and Meredith L. Liu. 2012. Innovation at Charlotte-Mecklenburg Schools. Harvard Business School Case. Harvard Business School. Publisher's VersionAbstract

Following its 2011 win of the Broad Prize, the most prestigious award available for urban school districts, Charlotte-Mecklenburg Schools must hire a new superintendent. This case examines the context of a large urban public school district and how its Board of Education and superintendent were able to create an environment that successfully fostered innovation, using a variety of tools including policy, structure, tools, and culture. It explores the particular constraints and barriers of public education and how the district leadership navigated them. Covers issues such as the resistance to innovation in the public sector, the importance of leadership in building a culture of innovation, the use of autonomy and accountability to encourage individual creativity, the difficulty of managing multiple stakeholders, and the challenge of sustaining improvements over changes in leadership.

Pages