Towards Minimax Online Learning with Unknown Time Horizon

We consider online learning when the time horizon is unknown. We apply a minimax analysis, beginning with the fixed horizon case, and then moving on to two unknown-horizon settings, one that assumes the horizon is chosen randomly according to some known distribution, and the other which allows the adversary full control over the horizon. For the random horizon setting with restricted losses, we derive a fully optimal minimax algorithm. And for the adversarial horizon setting, we prove a nontrivial lower bound which shows that the adversary obtains strictly more power than when the horizon is fixed and known. Based on the minimax solution of the random horizon setting, we then propose a new adaptive algorithm which "pretends" that the horizon is drawn from a distribution from a special family, but no matter how the actual horizon is chosen, the worst-case regret is of the optimal rate. Furthermore, our algorithm can be combined and applied in many ways, for instance, to online convex optimization, follow the perturbed leader, exponential weights algorithm and first order bounds. Experiments show that our algorithm outperforms many other existing algorithms in an online linear optimization setting

Reliability of Dynamic Load Scheduling with Solar Forecast Scenarios

This paper presents and evaluates the performance of an optimal scheduling algorithm that selects the on/off combinations and timing of a finite set of dynamic electric loads on the basis of short term predictions of the power delivery from a photovoltaic source. In the algorithm for optimal scheduling, each load is modeled with a dynamic power profile that may be different for on and off switching. Optimal scheduling is achieved by the evaluation of a user-specified criterion function with possible power constraints. The scheduling algorithm exploits the use of a moving finite time horizon and the resulting finite number of scheduling combinations to achieve real-time computation of the optimal timing and switching of loads. The moving time horizon in the proposed optimal scheduling algorithm provides an opportunity to use short term (time moving) predictions of solar power based on advection of clouds detected in sky images. Advection, persistence, and perfect forecast scenarios are used as input to the load scheduling algorithm to elucidate the effect of forecast errors on mis-scheduling. The advection forecast creates less events where the load demand is greater than the available solar energy, as compared to persistence. Increasing the decision horizon leads to increasing error and decreased efficiency of the system, measured as the amount of power consumed by the aggregate loads normalized by total solar power. For a standalone system with a real forecast, energy reserves are necessary to provide the excess energy required by mis-scheduled loads. A method for battery sizing is proposed for future work.Comment: 6 pager, 4 figures, Syscon 201

Collision-aware Task Assignment for Multi-Robot Systems

We propose a novel formulation of the collision-aware task assignment (CATA) problem and a decentralized auction-based algorithm to solve the problem with optimality bound. Using a collision cone, we predict potential collisions and introduce a binary decision variable into the local reward function for task bidding. We further improve CATA by implementing a receding collision horizon to address the stopping robot scenario, i.e. when robots are confined to their task location and become static obstacles to other moving robots. The auction-based algorithm encourages the robots to bid for tasks with collision mitigation considerations. We validate the improved task assignment solution with both simulation and experimental results, which show significant reduction of overlapping paths as well as deadlocks

Efficient Computer Network Anomaly Detection by Changepoint Detection Methods

We consider the problem of efficient on-line anomaly detection in computer network traffic. The problem is approached statistically, as that of sequential (quickest) changepoint detection. A multi-cyclic setting of quickest change detection is a natural fit for this problem. We propose a novel score-based multi-cyclic detection algorithm. The algorithm is based on the so-called Shiryaev-Roberts procedure. This procedure is as easy to employ in practice and as computationally inexpensive as the popular Cumulative Sum chart and the Exponentially Weighted Moving Average scheme. The likelihood ratio based Shiryaev-Roberts procedure has appealing optimality properties, particularly it is exactly optimal in a multi-cyclic setting geared to detect a change occurring at a far time horizon. It is therefore expected that an intrusion detection algorithm based on the Shiryaev-Roberts procedure will perform better than other detection schemes. This is confirmed experimentally for real traces. We also discuss the possibility of complementing our anomaly detection algorithm with a spectral-signature intrusion detection system with false alarm filtering and true attack confirmation capability, so as to obtain a synergistic system.Comment: 7 pages, 6 figures, to appear in "IEEE Journal of Selected Topics in Signal Processing

A parametric LQ approach to multiobjective control system design

The synthesis of a constant parameter output feedback control law of constrained structure is set in a multiple objective linear quadratic regulator (MOLQR) framework. The use of intuitive objective functions such as model-following ability and closed-loop trajectory sensitivity, allow multiple objective decision making techniques, such as the surrogate worth tradeoff method, to be applied. For the continuous-time deterministic problem with an infinite time horizon, dynamic compensators as well as static output feedback controllers can be synthesized using a descent Anderson-Moore algorithm modified to impose linear equality constraints on the feedback gains by moving in feasible directions. Results of three different examples are presented, including a unique reformulation of the sensitivity reduction problem

Rotting bandits are not harder than stochastic ones

In stochastic multi-armed bandits, the reward distribution of each arm is assumed to be stationary. This assumption is often violated in practice (e.g., in recommendation systems), where the reward of an arm may change whenever is selected, i.e., rested bandit setting. In this paper, we consider the non-parametric rotting bandit setting, where rewards can only decrease. We introduce the filtering on expanding window average (FEWA) algorithm that constructs moving averages of increasing windows to identify arms that are more likely to return high rewards when pulled once more. We prove that for an unknown horizon $T$, and without any knowledge on the decreasing behavior of the $K$ arms, FEWA achieves problem-dependent regret bound of $\widetilde{\mathcal{O}}(\log{(KT)}),$ and a problem-independent one of $\widetilde{\mathcal{O}}(\sqrt{KT})$. Our result substantially improves over the algorithm of Levine et al. (2017), which suffers regret $\widetilde{\mathcal{O}}(K^{1/3}T^{2/3})$. FEWA also matches known bounds for the stochastic bandit setting, thus showing that the rotting bandits are not harder. Finally, we report simulations confirming the theoretical improvements of FEWA