Analysis of buffer allocations in time-dependent and stochastic flow lines

This thesis reviews and classifies the literature on the Buffer Allocation Problem under steady-state conditions and on performance evaluation approaches for queueing systems with time-dependent parameters. Subsequently, new performance evaluation approaches are developed. Finally, a local search algorithm for the derivation of time-dependent buffer allocations is proposed. The algorithm is based on numerically observed monotonicity properties of the system performance in the time-dependent buffer allocations. Numerical examples illustrate that time-dependent buffer allocations represent an adequate way of minimizing the average WIP in the flow line while achieving a desired service level

Structured inference and sequential decision-making with Gaussian processes

Sequential decision-making is a central ability of intelligent agents interacting with an environment, including humans, animals, and animats. When those agents operate in complex systems, they need to be endowed with automatic decision-making frameworks quantifying the system uncertainty and the utility of different actions while allowing them to sequentially update their beliefs about the environment. When agents also aim at manipulating a system, they need to understand the data-generating mechanism. This requires accounting for causality which allows evaluating counterfactual scenarios while increasing interpretability and generalizability of an algorithm. Sequential causal decision making algorithms require an accurate surrogate model for the causal system and an acquisition function that based on its properties allows selecting actions. In this thesis, I tackle both components through the Bayesian framework which enables probabilistic reasoning while handling uncertainty in a principled manner. I consider Gaussian process (gp) models for both inference and causal decision-making as they provide a flexible framework capable of capturing a variety of data distributions. I first focus on developing scalable gp models incorporating structure in the likelihood and accounting for complex dependencies in the posteriors. These are indeed crucial properties of surrogate models used within decision-making algorithms. Particularly, I investigate models for point data as many realworld problems involve events and they present significant computational and methodological challenges. I then study how such models can incorporate causal structure and can be used to select actions based on cause-effect relationships. I focus on multi-task gp models, Bayesian Optimization, and Active Learning and show how they can be generalized to capture causality

Slowness learning for curiosity-driven agents

In the absence of external guidance, how can a robot learn to map the many raw pixels of high-dimensional visual inputs to useful action sequences? I study methods that achieve this by making robots self-motivated (curious) to continually build compact representations of sensory inputs that encode different aspects of the changing environment. Previous curiosity-based agents acquired skills by associating intrinsic rewards with world model improvements, and used reinforcement learning (RL) to learn how to get these intrinsic rewards. But unlike in previous implementations, I consider streams of high-dimensional visual inputs, where the world model is a set of compact low-dimensional representations of the high-dimensional inputs. To learn these representations, I use the slowness learning principle, which states that the underlying causes of the changing sensory inputs vary on a much slower time scale than the observed sensory inputs. The representations learned through the slowness learning principle are called slow features (SFs). Slow features have been shown to be useful for RL, since they capture the underlying transition process by extracting spatio-temporal regularities in the raw sensory inputs. However, existing techniques that learn slow features are not readily applicable to curiosity-driven online learning agents, as they estimate computationally expensive covariance matrices from the data via batch processing. The first contribution called the incremental SFA (IncSFA), is a low-complexity, online algorithm that extracts slow features without storing any input data or estimating costly covariance matrices, thereby making it suitable to be used for several online learning applications. However, IncSFA gradually forgets previously learned representations whenever the statistics of the input change. In open-ended online learning, it becomes essential to store learned representations to avoid re- learning previously learned inputs. The second contribution is an online active modular IncSFA algorithm called the curiosity-driven modular incremental slow feature analysis (Curious Dr. MISFA). Curious Dr. MISFA addresses the forgetting problem faced by IncSFA and learns expert slow feature abstractions in order from least to most costly, with theoretical guarantees. The third contribution uses the Curious Dr. MISFA algorithm in a continual curiosity-driven skill acquisition framework that enables robots to acquire, store, and re-use both abstractions and skills in an online and continual manner. I provide (a) a formal analysis of the working of the proposed algorithms; (b) compare them to the existing methods; and (c) use the iCub humanoid robot to demonstrate their application in real-world environments. These contributions together demonstrate that the online implementations of slowness learning make it suitable for an open-ended curiosity-driven RL agent to acquire a repertoire of skills that map the many raw pixels of high-dimensional images to multiple sets of action sequences

LIPIcs, Volume 261, ICALP 2023, Complete Volume

LIPIcs, Volume 261, ICALP 2023, Complete Volum