Markov Chain Monitoring

In networking applications, one often wishes to obtain estimates about the number of objects at different parts of the network (e.g., the number of cars at an intersection of a road network or the number of packets expected to reach a node in a computer network) by monitoring the traffic in a small number of network nodes or edges. We formalize this task by defining the 'Markov Chain Monitoring' problem. Given an initial distribution of items over the nodes of a Markov chain, we wish to estimate the distribution of items at subsequent times. We do this by asking a limited number of queries that retrieve, for example, how many items transitioned to a specific node or over a specific edge at a particular time. We consider different types of queries, each defining a different variant of the Markov chain monitoring. For each variant, we design efficient algorithms for choosing the queries that make our estimates as accurate as possible. In our experiments with synthetic and real datasets we demonstrate the efficiency and the efficacy of our algorithms in a variety of settings.Comment: 13 pages, 10 figures, 1 tabl

Fleet management strategies for urban Mobility-on-Demand systems

In recent years, the paradigm of personal urban mobility has radically evolved as an increasing number of Mobility-on-Demand (MoD) systems continue to revolutionize urban transportation. Hailed as the future of sustainable transportation, with significant implications on urban planning, these systems typically utilize a fleet of shared vehicles such as bikes, electric scooters, cars, etc., and provide a centralized matching platform to deliver point-to-point mobility to passengers. In this dissertation, we study MoD systems along three operational directions – (1) modeling: developing analytical models that capture the rich stochasticity of passenger demand and its impact on the fleet distribution, (2) economics: devising strategies to maximize revenue, and (3) control: developing coordination mechanisms aimed at optimizing platform throughput. First, we focus on the metropolitan bike-sharing systems where platforms typically do not have access to real-time location data to ascertain the exact spatial distribution of their fleet. We formulate the problem of accurately predicting the fleet distribution as a Markov Chain monitoring problem on a graph representation of a city. Specifically, each monitor provides information on the exact number of bikes transitioning to a specific node or traversing a specific edge at a particular time. Under budget constraints on the number of such monitors, we design efficient algorithms to determine appropriate monitoring operations and demonstrate their efficacy over synthetic and real datasets. Second, we focus on the revenue maximization strategies for individual strategic driving partners on ride-hailing platforms. Under the key assumption that large-scale platform dynamics are agnostic to the actions of an individual strategic driver, we propose a series of dynamic programming-based algorithms to devise contingency plans that maximize the expected earnings of a driver. Using robust optimization techniques, we rigorously reason about and analyze the sensitivity of such strategies to perturbations in passenger demand distributions. Finally, we address the problem of large-scale fleet management. Recent approaches for the fleet management problem have leveraged model-free deep reinforcement learning (RL) based algorithms to tackle complex decision-making problems. However, such methods suffer from a lack of explainability and often fail to generalize well. We consider an explicit need-based coordination mechanism to propose a non-deep RL-based algorithm that augments tabular Q-learning with a combinatorial optimization problem. Empirically, a case study on the New York City taxi demand enables a rigorous assessment of the value, robustness, and generalizability of the proposed approaches

Exploring the Landscape of Ubiquitous In-home Health Monitoring: A Comprehensive Survey

Ubiquitous in-home health monitoring systems have become popular in recent years due to the rise of digital health technologies and the growing demand for remote health monitoring. These systems enable individuals to increase their independence by allowing them to monitor their health from the home and by allowing more control over their well-being. In this study, we perform a comprehensive survey on this topic by reviewing a large number of literature in the area. We investigate these systems from various aspects, namely sensing technologies, communication technologies, intelligent and computing systems, and application areas. Specifically, we provide an overview of in-home health monitoring systems and identify their main components. We then present each component and discuss its role within in-home health monitoring systems. In addition, we provide an overview of the practical use of ubiquitous technologies in the home for health monitoring. Finally, we identify the main challenges and limitations based on the existing literature and provide eight recommendations for potential future research directions toward the development of in-home health monitoring systems. We conclude that despite extensive research on various components needed for the development of effective in-home health monitoring systems, the development of effective in-home health monitoring systems still requires further investigation.Comment: 35 pages, 5 figure