Patterns of User Involvement in Experiment-Driven Software Development

Sezin Yaman, Fabian Fagerholm, Myriam Douce Munezero, Tomi Matti Männistö, Tommi Johannes Mikkonen

Research output: Contribution to journalArticleScientificpeer-review


Background: Experiments are often used as a means to continuously validate user needs and to aid in making software development decisions. Involving users in the development of software products benefits both the users and companies. How software companies efficiently involve users in both general development and in experiments remains unclear; however, it is especially important to determine the perceptions and attitudes held by practitioners in different roles in these companies.

Objective: We seek to: 1) explore how software companies involve users in software development and experimentation; 2) understand how developer, manager and UX designer roles perceive and involve users in experimentation; and 3) uncover systematic patterns in practitioners’ views on user involvement in experimentation. The study aims to reveal behaviors and perceptions that could support or undermine experiment-driven development, point out what skills could enhance experiment-driven development, and raise awareness of such issues for companies that wish to adopt experiment-driven development.

Methods: We conducted a survey within four Nordic software companies, inviting practitioners in three major roles: developers, managers, and UX designers. We asked the respondents to indicate how they involve users in their job function, as well as their perspectives regarding software experiments and ethics.

Results and Conclusion: We identified six patterns describing experimentation and user involvement. For instance, managers were associated with a cautious user notification policy, that is, to always let users know of an experiment they are subject to, and they also believe that users have to be convinced before taking part in experiments. We discovered that, due to lack of clear processes for involving users and the lack of a common understanding of ethics in experimentation, practitioners tend to rationalize their perceptions based on their own experiences. Our patterns were based on empirical evidence and they can be evaluated in different populations and contexts.
Original languageEnglish
Article number106244
JournalInformation and Software Technology
Number of pages19
Publication statusPublished - Apr 2020
MoE publication typeA1 Journal article-refereed

Fields of Science

  • 113 Computer and information sciences

Cite this