Initiating the Transition towards Continuous Experimentation : Empirical Studies with Software Development Teams and Practitioners

Software experiments are presently often used by big technology pioneers, such as Microsoft, Facebook and Google, in order to learn about their users and to guide their research and development activities. Continuous experimentation (CE) is reported to be an integral part of software development in these organisations, however, how they transitioned to the approach is not publicly shared. Therefore, there is a lack of guidance for other organisations that are willing to adopt CE. In the current competitive markets, investing time and money in a new approach might be risky for these organisations, especially if they do not know how to initiate this transition process. This dissertation focuses on how organisations can initiate the transition towards CE, i.e., an approach to enhance development decisions by running experiments in an iterative and sustainable fashion. The dissertation was designed to acquire descriptive and observational knowledge through empirical studies and was con- ducted in three main phases. First, we designed and ran multiple-case studies to investigate how CE can be introduced to existing software company development teams, who want to run their first systematic experiments. We extracted descriptive knowledge from the introduction process and composed lessons learned to act as guidelines. In the second phase, we conducted a survey study with practitioners from four Nordic software companies, in order to better understand their attitudes and perception towards experiment-driven development, user involvement and ethics. Examining the results at role-to-role levels gave us an understanding of commonalities and distinctions stemming from different job functions. Furthermore, we identified patterns from the data that describe what trends exist across the dataset with respect to experiment-driven software development. Finally, in the last phase of the study, we conducted a single-case study with a mobile gaming company to investigate how CE functions as an organisational mechanism throughout the development life-cycle. The findings show that transitioning towards CE is a learning process that can be facilitated well by guidance, utilising existing resources and starting with small experiments with potentially enormous impact. Furthermore, by investigating the point of view of practitioners, we observed that software experiments represent different concepts, for instance, A/B tests and user interviews. We also observed that the role of the practitioner has a big impact not only on how experiments are understood, but also how individuals perceive the ethics involved in the experiments. For example, while managers are more cautious about company-customer relationships, UX designers were found to allow exceptions to user notification during experiments. In addition, we discovered that companies might understand and adopt experiment-driven development differently, for in- stance, influenced by their business contexts. Lastly, by examining a company’s CE practices, we found that experiments can take different forms given the development stage, and the organisational mechanism can be established to fit both the needs of the business domain and organisational goals. One of the biggest challenges of adopting CE, inaccessible real users, can be overcome with alter- native methods, such as proxy users, especially early in the development, when experiments are important in determining product value. Highly competitive markets can put pressure on organisations to avoid risks and costs when adopting a new approach. In this dissertation, we learned that by and large, software organisations and development teams can initiate their transition towards CE in an efficient and economical way. Furthermore, we conclude that the transition is a learning process that improves with practice and has to adapt to the organisational goals and contexts. The influence of human factors, such as the finding that individual perception of experiments and ethics is correlated with job functions indicates that CE is a multi-disciplinary research field, where individuals should be studied as well as experimentation processes. Software engineering research needs further studies to validate the findings in different contexts.Suuret teknologiayritykset, kuten Microsoft, Facebook tai Google, tekevät jatkuvasti ohjelmistokokeiluja oppiakseen enemmän järjestelmiensä käyttäjistä ja ohjatakseen niiden perusteella yrityksen liiketoimintaa. Vuonna 2009 Google teki laajan kokeilun, jossa tutkittiin, mitkä käytetyistä 41 erilaisesta sinisen sävystä mainoslinkkien väreissä saivat käyttäjät useimmin seuraamaan heille esitettyjä linkkejä. Tutkimuksen seurauksena Google raportoi liikevaihtonsa kasvaneen liki 200 miljoonaa dollaria. Ohjelmistokokeilujen tiedetään olevan olennainen osa näiden yritysten ohjelmistokehitystyötä. Koska näissä kokeiluissa kerätään lisäksi paljon dataa sekä tuotteiden että palvelujen käyttäjistä, liittyy niihin monia huomioonotettavia asioita, kuten etiikkaan ja tietosuojaan liittyviä kysymyksiä. Esimerkiksi vuonna 2014 Facebook teki kokeilun, jossa oli mukana tuhansia käyttäjiä ja jossa testattiin sitä, kuinka muutokset käyttäjien uutisvirran sisällössä vaikuttavat heidän tunteisiinsa. Kokeilun tuloksena oli, että tällaisen muuntelun avulla voidaan tehdä käyttäjät joko iloisemmiksi tai surullisemmiksi. Facebookiin ja Cambridge Analyticaan liittyvä skandaali alkuvuonna 2018 koski yli 50 miljoonaa Facebookin käyttäjää ja heihin liittyvän datan käyttöä poliittisessa mainonnassa. Nämä edellä mainitut esimerkkitapaukset herättävät sekä vakavia eettisiä kysymyksiä että myös kysymyksiä liittyen käyttäjien yksityisyyteen: Kuinka tällaisia käyttäjäkokeiluja voidaan tehdä ilman käyttäjien nimenomaista suostumusta? Tutkimusta siitä, kuinka ohjelmistoyhtiöt päätyvät tekemään tällaisia kokeilututkimuksia tai millaisia eettisiä vaikutuksia tämänkaltaisilla ohjelmistokokeiluilla on, on hyvin vähän. Tämä väitöskirjatyö tarkastelee sitä, kuinka yrityksissä siirrytään käyttämään edellä kuvatun tyyppisiä ohjelmistokokeiluja tarkastelemalla sekä itse siirtymäprosessia että siihen liittyviä toiminnallisia tarpeita ja inhimillisiä tekijöitä, kuten eettisiä kysymyksiä. Väitöskirjatyössä tehdyt havainnot osoittavat, että siirtymä on oppimisprosessi, jota voidaan tehokkaasti ohjailla sopivalla ohjeistuksella, hyödyntämällä olemassa olevia resursseja sekä tekemällä ensin pienempiä kokeiluja, joilla voi olla mahdollisesti suuret vaikutukset, mutta joiden kustannukset ovat alhaisia. Edelleen ohjelmistoalan ammattilaisten käsityksistä tehty tutkimus osoittaa, että ammattilaisten roolilla on suuri vaikutus siihen, millaisina he kokevat tällaiset kokeilut ja kuinka he ymmärtävät niihin liittyvät eettiset ongelmat. Erityisesti yritysten työntekijät tarkastelevat kysymystä siitä, mitkä asiat ovat eettisesti hyväksyttäviä, nimenomaan omien työtehtäviensä näkökulmasta. Esimerkiksi johtajat ovat huolissaan yrityksen ja sen asiakkaiden välisistä suhteista ja he ovat sitä mieltä, että käyttäjille pitäisi aina tiedottaa etukäteen käynnissä olevista kokeiluista, kun taas käyttäjäkokemusta kehittävät suunnittelijat pitävät myös hyväksyttävänä sitä, että käyttäjiä informoidaan kokeiluista vasta jälkikäteen. Jos yrityksellä ei ole näihin edellä kuvattuihin kysymyksiin liittyen olemassa mitään selkeitä ja pysyviä sääntöjä, jää eettisten kysymysten arviointi yrityksen työntekijöiden oman tulkinnan varaan. Yritysten täytyykin työstää omat säännöt ja ohjeistukset datan keräämisestä, jotta voidaan varmistaa se, että ohjelmistoalan ammattilaisilla on käytössään selkeät eettiset ohjeistukset, jotka tukevat heidän työtään ja varmistavat sen, että toiminta on asiaan liittyvän lainsäädännön mukaista. Ohjelmistoalalla tarvitaankin lisää tutkimusta, jotta nyt tehdyn tutkimusten löydökset voidaan todentaa myös muissa ympäristöissä

Customer Involvement in Continuous Deployment: A Systematic Literature Review

Abstract. [Context and motivation] In order to build successful software products and services, customer involvement and an understanding of customers' requirements and behaviours during the development process are essential. [Question/Problem] Although continuous deployment is gaining attention in the software industry as an approach for continuously learning from customers, there is no common overview of the topic yet. [Principal ideas/results] To provide a common overview, we conduct a secondary study that explores the state of reported evidence on customer input during continuous deployment in software engineering, including the potential benefits, challenges, methods and tools of the field. [Contribution] We report on a systematic literature review covering 25 primary studies. Our analysis of these studies reveals that although customer involvement in continuous deployment is highly relevant in the software industry today, it has been relatively unexplored in academic research. The field is seen as beneficial, but there are a number of challenges related to it, such as misperceptions among customers. In addition to providing a comprehensive overview of the research field, we clarify the gaps in knowledge that need to be studied further

Multi-touch Tables in High School Science Education

Education is one important field where interactive tables can be used as supportive tools. While many studies introduce multi-touch displays for collaborative use, the requirements and benefits of using such a technology in science education are still unclear to teaching professionals. This thesis focuses on the role of multi-touch tables in high school science education. Three different aspects are examined: i) the aspects of multi-touch tables that are important in science education, ii) the facets of general research in interactive tables that also benefit research in multi-touch tables, and, iii) experimental multi-touch framework features that are useful in a high school science classroom. The study includes a comprehensive examination of background research in the field, followed by a case study that examines the perceptions regarding interactive tables among high school teachers and the development life-cycle of a multi-touch framework. The case study covers interviews and questionnaires with high school teachers. At the end of the thesis, the research questions are answered. Based on the study, collaborative group works, social interactions, inquiry-based learning and visual learning are the most important aspects of multi-touch tables in science education. Some facets of general research into interactive tabletops also relate to the study of multi-touch tables in education. The most important of these are space, interactivity, participation and orchestration. Experimental framework features, observation of laboratory experiments, visualization, and extendibility are useful for a high-school science classroom. To maximize the benefit of interactive tables, it is necessary to first investigate the purpose of the learning environment in order to understand the critical variables and constraints. Then, attentively constructing a learning environment's dynamics to support multi-touch tables can enhance high school science education

Strategic analysis of healthcare service quality using fuzzy AHP methodology

Managers in the service sector are under pressure to demonstrate that their services are customer-focused and that continuous performance improvement is being delivered. It is essential that customer expectations are properly understood and measured under the constraints that organizations must manage. The majority of the work to date has attempted to use the SERVQUAL (service quality) methodology in an effort to measure service quality. In this study, firstly the concept and factors of service quality are examined. Then a fuzzy AHP (analytic hierarchy process) is structured to evaluate the proposed service quality framework. A case study in healthcare sector in Turkey is presented to clarify the methodology. (C) 2011 Elsevier Ltd. All rights reserved

Design of a Flight Stabilizer for Fixed-Wing Aircrafts Using H-infinity Loop Shaping Method

9th International Conference on Electrical and Electronics Engineering (ELECO) (2015 : Bursa, Turkey)Autopilot systems for unmanned aerial vehicles (UAVs) and aircrafts provide flight missions without need of human input and make them more reliable and efficient. The first step of designing an autopilot is a stabilizer mode. Conventional autopilot systems have inner and outer loops. Stabilizer is the inner loop for an autopilot. In this paper designing an aircraft control system ensures good performance and robustness that allows control of roll, pitch and yaw angles will be declared. Aircraft dynamics are used to design the model of the control system in the MATLAB Simulink environment. Using loop shaping method to achieve stable and robust control system will be the strategy. Generated controllers to find the most effective one are embedded in the X-plane which is one of the realistic simulations

Introducing Continuous Experimentation in Large Software-Intensive Product and Service Organizations

Corrigendum: DOI:10.1016/j.jss.2019.02.022Software development in highly dynamic environments imposes high risks to development organizations. One such risk is that the developed software may be of only little or no value to customers, wasting the invested development efforts. Continuous experimentation, as an experiment-driven development approach, may reduce such development risks by iteratively testing product and service assumptions that are critical to the success of the software. Although several experiment-driven development approaches are available, there is little guidance available on how to introduce continuous experimentation into an organization. This article presents a multiple-case study that aims at better understanding the process of introducing continuous experimentation into an organization with an already established development process. The results from the study show that companies are open to adopting such an approach and learning throughout the introduction process. Several benefits were obtained, such as reduced development efforts, deeper customer insights, and better support for development decisions. Challenges included complex stakeholder structures, difficulties in defining success criteria, and building experimentation skills. Our findings indicate that organizational factors may limit the benefits of experimentation. Moreover, introducing continuous experimentation requires fundamental changes in how companies operate, and a systematic introduction process can increase the chances of a successful start.Peer reviewe

Screening for antitumor activity of various plant extracts on HeLa and C 4-I cell lines

Purpose: Cancer is a long process that leads the organism to death and is associated with the normal cells acquiring the ability to divide permanently. Nowadays, the use of natural products in cancer therapy has a great importance. In addition, working with plants that are endemic to Turkey and determining the biological activities of these plant extracts, is extremely important due to the potential for new drug development. There is no comparative study available in the literature on the antitumor effects of Colchicum sanguicolle, a new found species of the genus Colchicum in Turkey, Crateagus microphylla, of the genus Crateagus and Centaurea antiochia of the genus Centaurea. In this study, we tried to demonstrate the antitumor effect of these plant extracts on HeLa and C 4-1 cells

Biofilm Formation Research of Coagulase-Negative Staphylococci Isolates' Isolated from Blood and Hand Culture at Nanofilm Covered Micro Plaques by Plasma Polymerization Technique: An Experimental Model

Introduction: Coagulase-negative staphylococci (CNS) can protect themselves from the effects of antibiotics by producing biofilms through breeding on biomaterials, medical equipment and devices. It is possible to influence biofilm formation with the aid of various surface modifications. In our study, plasma polymerization method, which is a surface modification technique, was used. The plasma polymerization technique is an environmentally-friendly technique that allows you to modify the nanometer level only at the surface without affecting the stack using the fourth state of the material. The possibility to generate surfaces with different properties (hydrophilic, hydrophobic, biocompatible etc.) by the help of various monomers and gases has made this technique more popular. In this study, the effect of the microplate surfaces modified by three different monomers on the biofilm formation of CNS was investigated. Materials and Methods: A total of 60 isolated CNS isolates from blood and hand cultures were included into the study. As control strains, Staphylococcus epidermidis ATCC 35984, known to be biofilm positive, and S. epidermidis ATCC 12228 which do not form biofilm, were used. Slime formation was determined by the quantitative plaque assay method described by Christensen. In microplates, which were plain or modified by three different monomers, the biofilm formation behavior of all strains was investigated simultaneously and comparatively. Results: There was no difference in biofilm positivity between strains isolated from hand and blood. A total of 71.6% biofilm formation was observed on microplates, which were not coated with plasma technique, and on plasma-modified microplated surfaces, 80% (monomer: 3- mercaptopropionic acid), 65% (monomer: 2-hydroxyethyl methacylate) and 31.6% (monomer: ethylene glycol dimethacylate) biofilm formation was observed, respectively. It was found that ethylene glycol dimethacrylate in three monomers significantly inhibited biofilm formation when compared to other monomers. Conclusion: In recent years CNS, especially S. epidermidis has become the most frequently isolated bacteria in catheter infections and responsible for the 28% of nosocomial bacteremia. The widespread use of prosthetic and permanent devices has been shown as a reason for the increase in the frequency of this effect. In 90% of patients with S. epidermidis bacteremia, there is an intravascular catheter history. Biofilm is an extracellular structure containing water, proteins and carbohydrates and is responsible for the unwanted adhesion of microorganisms to host cells and artificial surfaces. The biofilm mechanism can be altered by the interaction between the material surface and the bacterial surface. In our study, in-vitro results were obtained showing the potential to reduce the risk of biofilm-associated infection by microorganism biofilm formation on modified surfaces with appropriate monomer selection

Continuous experimentation cookbook : an introduction to systematic experimentation for software-intensive businesses. - (DIMECC result publications ; 3) - (DIMECC publications series no. 15)

The business landscape is changing radically because of software. Companies in all industry sectors are continously finding new flexibilities in this programmable world. They are able to deliver new functionalities even after the product is already in the customer's hands. But success is far from guaranteed if they cannot validate their assumptions about what their customers actually need. A competitor with better knowledge of customer needs can disrupt the market in an instant. This book introduces continuous experimentation, an approach to continuously and systematically test assumptions about the company's product or service strategy and verify customers' needs through experiments. By observing how customers actually use the product or early versions of it, companies can make better development decisions and avoid potentially expensive and wasteful activities. The book explains the cycle of continuous experimentation, demonstrates its use through industry cases, provides advice on how to conduct experiments with recipes, tools, and models, and lists some common pitfalls to avoid. Use it to get started with continuous experimentation and make better product and service development decisions that are in-line with your customers' needs