Thompson Sampling: An Asymptotically Optimal Finite Time Analysis

The question of the optimality of Thompson Sampling for solving the stochastic multi-armed bandit problem had been open since 1933. In this paper we answer it positively for the case of Bernoulli rewards by providing the first finite-time analysis that matches the asymptotic rate given in the Lai and Robbins lower bound for the cumulative regret. The proof is accompanied by a numerical comparison with other optimal policies, experiments that have been lacking in the literature until now for the Bernoulli case.Comment: 15 pages, 2 figures, submitted to ALT (Algorithmic Learning Theory

Solution of Dual Fuzzy Equations Using a New Iterative Method

In this paper, a new hybrid scheme based on learning algorithm of fuzzy neural network (FNN) is offered in order to extract the approximate solution of fully fuzzy dual polynomials (FFDPs). Our FNN in this paper is a five-layer feed-back FNN with the identity activation function. The input-output relation of each unit is defined by the extension principle of Zadeh. The output from this neural network, which is also a fuzzy number, is numerically compared with the target output. The comparison of the feed-back FNN method with the feed-forward FNN method shows that the less error is observed in the feed-back FNN method. An example based on applications are given to illustrate the concepts, which are discussed in this paper

A Stochastic Search on the Line-Based Solution to Discretized Estimation

Recently, Oommen and Rueda [11] presented a strategy by which the parameters of a binomial/multinomial distribution can be estimated when the underlying distribution is nonstationary. The method has been referred to as the Stochastic Learning Weak Estimator (SLWE), and is based on the principles of continuous stochastic Learning Automata (LA). In this paper, we consider a new family of stochastic discretized weak estimators pertinent to tracking time-varying binomial distributions. As opposed to the SLWE, our proposed estimator is discretized , i.e., the estimate can assume only a finite number of values. It is well known in the field of LA that discretized schemes achieve faster convergence speed than their corresponding continuous counterparts. By virtue of discretization, our estimator realizes extremely fast adjustments of the running estimates by jumps, and it is thus able to robustly, and very quickly, track changes in the parameters of the distribution after a switch has occurred in the environment. The design principle of our strategy is based on a solution, pioneered by Oommen [7], for the Stochastic Search on the Line (SSL) problem. The SSL solution proposed in [7], assumes the existence of an Oracle which informs the LA whether to go “right” or “left”. In our application domain, in order to achieve efficient estimation, we have to first infer (or rather simulate ) such an Oracle. In order to overcome this difficulty, we rather intelligently construct an “Artificial Oracle” that suggests whether we are to increase the current estimate or to decrease it. The paper briefly reports conclusive experimental results that demonstrate the ability of the proposed estimator to cope with non-stationary environments with a high adaptation rate, and with an accuracy that depends on its resolution. The results which we present are, to the best of our knowledge, the first reported results that resolve the problem of discretized weak estimation using a SSL-based solution

Learning automaton based on-line discovery and tracking of spatio-temporal event patterns

Discovering and tracking of spatio-temporal patterns in noisy sequences of events is a difficult task that has become increasingly pertinent due to recent advances in ubiquitous computing, such as community-based social networking applications. The core activities for applications of this class include the sharing and notification of events, and the importance and usefulness of these functionalites increases as event-sharing expands into larger areas of one's life. Ironical

Scalable Independent Multi-level Distribution in Multimedia Content Analysis

Due to the limited processing resources available on a typical host, monolithic multimedia content analysis applications are often restricted to simple content analysis tasks, covering a small number of media streams. This limitation on processing resources can often be reduced by parallelizing and distributing an application, utilizing the processing resources on several hosts. However, multimedia content analysis applications consist of multiple logical levels, such as streaming, filtering, feature extraction, and classification. This complexity makes parallelization and distribution a difficult task, as each logical level may require special purpose techniques. In this paper we propose a component-based framework where each logical level can be parallelized and distributed independently

Effects of medetomidine, a novel antifouling agent, on the burrowing bivalve Abra nitida (Muller)

The effects of medetomidine, a novel antifouling candidate, on the burrowing bivalve Abra nitida were studied. The burrowing behaviour, sediment reworking activity and faeces production were assessed after 24 h exposure of A. nitida to sublethal concentrations of medetomidine. Medetomidine caused a significant decrease in the burrowing response and in the sediment reworking activity. The median effective concentrations (EC50) were 430 nM (86 mu g/l) and 4.4 nM (0.9 mu g/l), respectively. No effects on the faeces production were detected. Although significant effects of medetomidine on A. nitida were registered, a reversibility of the effects was observed when 24 hexposed animals were incubated in clean seawater and sediment for 24 h. Considerations relating to the future commercialisation of medetomidine for antifouling purposes are discussed