Introducing Continuous Experimentation in Large Software-Intensive Product and Service Organizations

Sezin Gizem Yaman, Myriam Munezero, Jürgen Münch, Fabian Fagerholm, Ossi Syd, Mika Aaltola, Christina Palmu, Tomi Männistö

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Software development in highly dynamic environments imposes high risks to development organizations. One such risk is that the developed software may be of only little or no value to customers, wasting the invested development efforts. Continuous experimentation, as an experiment-driven development approach, may reduce such development risks by iteratively testing product and service assumptions that are critical to the success of the software. Although several experiment-driven development approaches are available, there is little guidance available on how to introduce continuous experimentation into an organization. This article presents a multiple-case study that aims at better understanding the process of introducing continuous experimentation into an organization with an already established development process. The results from the study show that companies are open to adopting such an approach and learning throughout the introduction process. Several benefits were obtained, such as reduced development efforts, deeper customer insights, and better support for development decisions. Challenges included complex stakeholder structures, difficulties in defining success criteria, and building experimentation skills. Our findings indicate that organizational factors may limit the benefits of experimentation. Moreover, introducing continuous experimentation requires fundamental changes in how companies operate, and a systematic introduction process can increase the chances of a successful start.
Original languageEnglish
Article numberJSS10002
JournalThe Journal of Systems and Software
Volume133
Pages (from-to)195-211
ISSN0164-1212
DOIs
Publication statusPublished - 1 Nov 2017
MoE publication typeA1 Journal article-refereed

Bibliographical note

Corrigendum: DOI:10.1016/j.jss.2019.02.022

Fields of Science

  • 113 Computer and information sciences

Cite this