The RIGHT Model for Continuous Experimentation

Context:Development of software-intensive products and services increasingly occurs by continuously deploying product or service increments, such as new features and enhancements, to customers. Product and service developers must continuously find out what customers want by direct customer feedback and usage behaviour observation. Objective: This paper examines the preconditions for setting up an experimentation system for continuous customer experiments. It describes the RIGHT Model for Continuous Experimentation (Rapid Iterative value creation Gained through High-frequency Testing), illustrating the building blocks required for such a system. Method: An initial model for continuous experimentation is analytically derived from prior work. The model is matched against empirical case study findings from two startup companies and further developed. Results: Building blocks for a continuous experimentation system and infrastructure are presented. Conclusions: A suitable experimentation system requires at least the ability to release minimum viable products or features with suitable instrumentation, design and manage experiment plans, link experiment results with a product roadmap, and manage a flexible business strategy. The main challenges are proper, rapid design of experiments, advanced instrumentation of software to collect, analyse, and store relevant data, and the integration of experiment results in both the product development cycle and the software development process.Peer reviewe

Breakdowns

We study a continuous-time game of strategic experimentation in which the players try to assess the failure rate of some new equipment or technology. Breakdowns occur at the jump times of a Poisson process whose unknown intensity is either high or low. In marked contrast to existing models, we find that the cooperative value function does not exhibit smooth pasting at the efficient cut-off belief. This finding extends to the boundaries between continuation and stopping regions in Markov perfect equilibria. We characterize the unique symmetric equilibrium, construct a class of asymmetric equilibria, and elucidate the impact of bad versus good Poisson news on equilibrium outcomes

Strategic Experimentation with Poisson Bandits

We study a game of strategic experimentation with two-armed bandits where the risky arm distributes lump-sum payoffs according to a Poisson process. Its intensity is either high or low, and unknown to the players. We consider Markov perfect equilibria with beliefs as the state variable. As the belief process is piecewise deterministic, payoff functions solve differential-difference equations. There is no equilibrium where all players use cut-off strategies, and all equilibria exhibit an `encouragement effect' relative to the single-agent optimum. We construct asymmetric equilibria in which players have symmetric continuation values at sufficiently optimistic beliefs yet take turns playing the risky arm before all experimentation stops. Owing to the encouragement effect, these equilibria Pareto dominate the unique symmetric one for sufficiently frequent turns. Rewarding the last experimenter with a higher continuation value increases the range of beliefs where players experiment, but may reduce average payoffs at more optimistic beliefs. Some equilibria exhibit an `anticipation effect': as beliefs become more pessimistic, the continuation value of a single experimenter increases over some range because a lower belief means a shorter wait until another player takes over

Strategic Experimentation with Exponential Bandits

This paper studies a game of strategic experimentation with two-armed bandits whose risky arm might yield a payoff only after some exponentially distributed random time. Because of free-riding, there is an inefficiently low level of experimentation in any equilibrium where the players use stationary Markovian strategies with posterior beliefs as the state variable. After characterizing the unique symmetric Markovian equilibrium of the game, which is in mixed strategies, we construct a variety of pure-strategy equilibria. There is no equilibrium where all players use simple cut-off strategies. Equilibria where players switch finitely often between the roles of experimenter and free-rider all lead to the same pattern of information acquisition; the efficiency of these equilibria depends on the way players share the burden of experimentation among them. In equilibria where players switch roles infinitely often, they can acquire an approximately efficient amount of information, but the rate at which it is acquired still remains inefficient; moreover, the expected payoff of an experimenter exhibits the novel feature that it rises as players become more pessimistic. Finally, over the range of beliefs where players use both arms a positive fraction of the time, the symmetric equilibrium is dominated by any asymmetric one in terms of aggregate payoffs

Negatively Correlated Bandits

We analyze a two-player game of strategic experimentation with two-armed bandits. Each player has to decide in continuous time whether to use a safe arm with a known payoff or a risky arm whose likelihood of delivering payoffs is initially unknown. The quality of the risky arms is perfectly negatively correlated between players. In marked contrast to the case where both risky arms are of the same type, we find that learning will be complete in any Markov perfect equilibrium if the stakes exceed a certain threshold, and that all equilibria are in cutoff strategies. For low stakes, the equilibrium is unique, symmetric, and coincides with the planner's solution. For high stakes, the equilibrium is unique, symmetric, and tantamount to myopic behavior. For intermediate stakes, there is a continuum of equilibria

Strategic Learning in Teams

This paper analyzes a two-player game of strategic experimentation with three-armed exponential bandits in continuous time. Players face replica bandits, with one arm that is safe in that it generates a known payoff, whereas the likelihood of the risky arms’ yielding a positive payoff is initially unknown. It is common knowledge that the types of the two risky arms are perfectly negatively correlated. I show that the efficient policy is incentive-compatible if, and only if, the stakes are high enough. Moreover, learning will be complete in any Markov perfect equilibrium with continuous value functions if, and only if, the stakes exceed a certain threshold

Experimentation in Two-Sided Markets

We study optimal experimentation by a monopolistic platform in a two-sided market framework. The platform provider faces uncertainty about the strength of the externality each side is exerting on the other. It maximizes the expected present value of its profit stream in a continuous-time infinite-horizon framework by setting participation fees or quantities on both sides. We show that a price-setting platform provider sets a fee lower than the myopically optimal level on at least one side of the market, and on both sides if the two externalities are of approximately equal strenght. If the externality that one side exerts is sufficiently weaker than the externality it experiences, the optimal fee on this side exceeds the myopically optimal level. We obtain analogous results for expected prives when the platform provider chooses quantities. While the optimal policy does not admin closed-form representations in general, we identify special cases in which the undiscounted limit of the model can be solved in closed form

Considerations about Continuous Experimentation for Resource-Constrained Platforms in Self-Driving Vehicles

Autonomous vehicles are slowly becoming reality thanks to the efforts of many academic and industrial organizations. Due to the complexity of the software powering these systems and the dynamicity of the development processes, an architectural solution capable of supporting long-term evolution and maintenance is required. Continuous Experimentation (CE) is an already increasingly adopted practice in software-intensive web-based software systems to steadily improve them over time. CE allows organizations to steer the development efforts by basing decisions on data collected about the system in its field of application. Despite the advantages of Continuous Experimentation, this practice is only rarely adopted in cyber-physical systems and in the automotive domain. Reasons for this include the strict safety constraints and the computational capabilities needed from the target systems. In this work, a concept for using Continuous Experimentation for resource-constrained platforms like a self-driving vehicle is outlined.Comment: Copyright 2017 Springer. Paper submitted and accepted at the 11th European Conference on Software Architecture. 8 pages, 1 figure. Published in Lecture Notes in Computer Science vol 10475 (Springer), https://link.springer.com/chapter/10.1007/978-3-319-65831-5_

Modeling of Complex Parts for Industrial WaterJet Cleaning

Industrial high-pressure waterjet cleaning is common to many industries. The modeling in this paper functions inside a collaborative robotic framework for high mix, low volume processes where human robot collaboration is beneficial. Automation of pressure washing is desirable for economic and ergonomic reasons. An automated cleaning system needs path simulation and analysis to give the operator insight into the predicted cleaning performance of the system. In this paper, ablation, the removal of a substrate coating by waterjet, is modeled for robotic cleaning operations. The model is designed to work with complex parts often found in spray cleaning operations, namely parts containing hidden portions, holes, or concavities. Experimentation is used to validate and calibrate the ablation model to yield accurate evaluations for how well every feature of a part is cleaned based on the cumulative effect of water affecting the part surface. The ablation model will provide the foundation for optimizing process parameters for robotic waterjet cleaning