When can the two-armed bandit algorithm be trusted?

We investigate the asymptotic behavior of one version of the so-called two-armed bandit algorithm. It is an example of stochastic approximation procedure whose associated ODE has both a repulsive and an attractive equilibrium, at which the procedure is noiseless. We show that if the gain parameter is constant or goes to 0 not too fast, the algorithm does fall in the noiseless repulsive equilibrium with positive probability, whereas it always converges to its natural attractive target when the gain parameter goes to zero at some appropriate rates depending on the parameters of the model. We also elucidate the behavior of the constant step algorithm when the step goes to 0. Finally, we highlight the connection between the algorithm and the Polya urn. An application to asset allocation is briefly described

How fast is the bandit?

International audienceIn this paper we investigate the rate of convergence of the so-called two-armed bandit algorithm in a financial context of asset allocation. The behaviour of the algorithm turns out to be highly non-standard: no CLT whatever the time scale, possible existence of two rate regimes

Learning to deal with COTS (commercial off the shelf)

With the advent of model based development technologies, dependence of COTS in software development has increased considerably. Use of COTS is considered economical and practical when it comes to integration of various software components. However COTS are trapped with some pitfalls. COTS provided are not usually accompanied by models or extensive specifications. This approach makes usage & integration of COTS components with in house developed software components a very challenging task. Conformance of the implementation with the specification forms the basis for our approach. In this thesis, we analyze an approach where the model is extracted from the COTS software that greatly aids in integration.;We developed a system that extracts the state machine model from the COTS software using Dana Angluin\u27s L* Algorithm. We also developed a hierarchical approach of viewing the state machine model by static analysis of assembly code

Regret bounds for Narendra-Shapiro bandit algorithms

Narendra-Shapiro (NS) algorithms are bandit-type algorithms that have been introduced in the sixties (with a view to applications in Psychology or learning automata), whose convergence has been intensively studied in the stochastic algorithm literature. In this paper, we adress the following question: are the Narendra-Shapiro (NS) bandit algorithms competitive from a \textit{regret} point of view? In our main result, we show that some competitive bounds can be obtained for such algorithms in their penalized version (introduced in \cite{Lamberton_Pages}). More precisely, up to an over-penalization modification, the pseudo-regret $\bar{R}_n$ related to the penalized two-armed bandit algorithm is uniformly bounded by $C \sqrt{n}$ (where $C$ is made explicit in the paper). \noindent We also generalize existing convergence and rates of convergence results to the multi-armed case of the over-penalized bandit algorithm, including the convergence toward the invariant measure of a Piecewise Deterministic Markov Process (PDMP) after a suitable renormalization. Finally, ergodic properties of this PDMP are given in the multi-armed case

Memory requirements for the detection of impostor nodes in wireless sensor networks

This paper shows how it is at least in principle possible to detect impostor nodes in wireless sensor networks with a quite simplistic detection algorithm by purely statistical means and merely from external observation without any knowledge of the impostor’s internal composition. This method, however, requires considerable volumes of internal memory for any WSN node on which such detection algorithms are supposed to be implemented.http://www.easychair.org/publications/EPiC/Computingam2023Computer Scienc

Information Processing, Computation and Cognition

Computation and information processing are among the most fundamental notions in cognitive science. They are also among the most imprecisely discussed. Many cognitive scientists take it for granted that cognition involves computation, information processing, or both – although others disagree vehemently. Yet different cognitive scientists use ‘computation’ and ‘information processing’ to mean different things, sometimes without realizing that they do. In addition, computation and information processing are surrounded by several myths; first and foremost, that they are the same thing. In this paper, we address this unsatisfactory state of affairs by presenting a general and theory-neutral account of computation and information processing. We also apply our framework by analyzing the relations between computation and information processing on one hand and classicism and connectionism/computational neuroscience on the other. We defend the relevance to cognitive science of both computation, at least in a generic sense, and information processing, in three important senses of the term. Our account advances several foundational debates in cognitive science by untangling some of their conceptual knots in a theory-neutral way. By leveling the playing field, we pave the way for the future resolution of the debates’ empirical aspects

Synthesis of Event-Based Controllers for Software Engineering

Behavioural modelling has been widely used to aid in the design of concurrent systems. Behaviour models have shown to be useful to uncover design errors in early stages of the development process. However, building correct behaviour models is costly and requires significant experience. Controller synthesis offers a way to build models that are correct by construction. Existing software engineering techniques for synthesising controllers have various limitations. Such limitations can be seen as restrictions in the expressiveness of the controller goals and environment model, or in the relation between the controllable and monitored actions. The main aim of this thesis is the development of novel techniques overcoming known limitations of previous approaches and methodological guidelines for synthesising useful controllers. This thesis establishes the framework for controller synthesis techniques that support event-based models, expressive goal specifications, distinguish controllable from monitored actions and guarantee achievement of the desired goals. Together with these techniques, methodological guidelines are proposed to help in building more accurate descriptions of the environment and more effective controllers. In addition, this thesis presents a tool that implements the proposed techniques. Evaluation of the techniques has been conducted using the tool to model known case studies from the literature, showing that by allowing more expressive controller goals and environment models, and explicitly distinguishing controllable and monitored actions such case studies can be more accurately modelled and solutions guaranteeing satisfaction of the goals can be achieved