Maximizing the Probability of Arriving on Time: A Practical Q-Learning Method

The stochastic shortest path problem is of crucial importance for the development of sustainable transportation systems. Existing methods based on the probability tail model seek for the path that maximizes the probability of arriving at the destination before a deadline. However, they suffer from low accuracy and/or high computational cost. We design a novel Q-learning method where the converged Q-values have the practical meaning as the actual probabilities of arriving on time so as to improve accuracy. By further adopting dynamic neural networks to learn the value function, our method can scale well to large road networks with arbitrary deadlines. Experimental results on real road networks demonstrate the significant advantages of our method over other counterparts

Information Rates of ASK-Based Molecular Communication in Fluid Media

This paper studies the capacity of molecular communications in fluid media, where the information is encoded in the number of transmitted molecules in a time-slot (amplitude shift keying). The propagation of molecules is governed by random Brownian motion and the communication is in general subject to inter-symbol interference (ISI). We first consider the case where ISI is negligible and analyze the capacity and the capacity per unit cost of the resulting discrete memoryless molecular channel and the effect of possible practical constraints, such as limitations on peak and/or average number of transmitted molecules per transmission. In the case with a constrained peak molecular emission, we show that as the time-slot duration increases, the input distribution achieving the capacity per channel use transitions from binary inputs to a discrete uniform distribution. In this paper, we also analyze the impact of ISI. Crucially, we account for the correlation that ISI induces between channel output symbols. We derive an upper bound and two lower bounds on the capacity in this setting. Using the input distribution obtained by an extended Blahut-Arimoto algorithm, we maximize the lower bounds. Our results show that, over a wide range of parameter values, the bounds are close.Comment: 31 pages, 8 figures, Accepted for publication on IEEE Transactions on Molecular, Biological, and Multi-Scale Communication

Adaptive and Resilient Revenue Maximizing Dynamic Resource Allocation and Pricing for Cloud-Enabled IoT Systems

Cloud computing is becoming an essential component of modern computer and communication systems. The available resources at the cloud such as computing nodes, storage, databases, etc. are often packaged in the form of virtual machines (VMs) to be used by remotely located client applications for computational tasks. However, the cloud has a limited number of VMs available, which have to be efficiently utilized to generate higher productivity and subsequently generate maximum revenue. Client applications generate requests with computational tasks at random times with random complexity to be processed by the cloud. The cloud service provider (CSP) has to decide whether to allocate a VM to a task at hand or to wait for a higher complexity task in the future. We propose a threshold-based mechanism to optimally decide the allocation and pricing of VMs to sequentially arriving requests in order to maximize the revenue of the CSP over a finite time horizon. Moreover, we develop an adaptive and resilient framework based that can counter the effect of realtime changes in the number of available VMs at the cloud server, the frequency and nature of arriving tasks on the revenue of the CSP.Comment: American Control Conference (ACC 2018

The Submodular Secretary Problem Goes Linear

During the last decade, the matroid secretary problem (MSP) became one of the most prominent classes of online selection problems. Partially linked to its numerous applications in mechanism design, substantial interest arose also in the study of nonlinear versions of MSP, with a focus on the submodular matroid secretary problem (SMSP). So far, O(1)-competitive algorithms have been obtained for SMSP over some basic matroid classes. This created some hope that, analogously to the matroid secretary conjecture, one may even obtain O(1)-competitive algorithms for SMSP over any matroid. However, up to now, most questions related to SMSP remained open, including whether SMSP may be substantially more difficult than MSP; and more generally, to what extend MSP and SMSP are related. Our goal is to address these points by presenting general black-box reductions from SMSP to MSP. In particular, we show that any O(1)-competitive algorithm for MSP, even restricted to a particular matroid class, can be transformed in a black-box way to an O(1)-competitive algorithm for SMSP over the same matroid class. This implies that the matroid secretary conjecture is equivalent to the same conjecture for SMSP. Hence, in this sense SMSP is not harder than MSP. Also, to find O(1)-competitive algorithms for SMSP over a particular matroid class, it suffices to consider MSP over the same matroid class. Using our reductions we obtain many first and improved O(1)-competitive algorithms for SMSP over various matroid classes by leveraging known algorithms for MSP. Moreover, our reductions imply an O(loglog(rank))-competitive algorithm for SMSP, thus, matching the currently best asymptotic algorithm for MSP, and substantially improving on the previously best O(log(rank))-competitive algorithm for SMSP