Building Blocks for Continuous Experimentation

Fabian Fagerholm, Hanna Mäenpää, Alejandro Sanchez Guinea, Jürgen Münch

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review


Abstract: Context: Development of software-intensive products and services increasingly occurs by continuously deploying product or service increments, such as new features and enhancements, to customers. Product and service developers need to continuously find out what customers want by direct customer feedback and observation of usage behaviour, rather than indirectly through up-front business analyses. Objective: This paper examines the preconditions for setting up an experimentation system for continuous customer experiments. It describes the building blocks required for such a system. Method: A model for continuous experimentation is analytically derived from prior work. The proposed model is validated against a case study examining a startup company. Results: Building blocks for a continuous experimentation system and infrastructure are presented. Conclusion: A suitable experimentation system requires at least the ability to release minimum viable products or features with suitable instrumentation, design and manage experiment plans, link experiment results with a product roadmap, and manage a flexible business strategy. The main challenges are proper and rapid design of experiments, advanced instrumentation of software to collect, analyse, and store relevant data, and the integration of experiment results in both the product development cycle and the software development process.
Original languageEnglish
Title of host publicationProceedings RCoSE 2014 Proceedings of the 1st International Workshop on Rapid Continuous Software Engineering
Number of pages9
Place of PublicationNew York
Publication date1 Jun 2014
ISBN (Print)978-1-4503-2856-2
Publication statusPublished - 1 Jun 2014
MoE publication typeA4 Article in conference proceedings
EventInternational Workshop on Rapid Continuous Software Engineering - Hyderabad, India
Duration: 31 May 20147 Jun 2014
Conference number: 1

Fields of Science

  • 113 Computer and information sciences

Cite this