Computing stationary probability distributions and large deviation rates for constrained random walks. The undecidability results

Our model is a constrained homogeneous random walk in a nonnegative orthant Z_+^d. The convergence to stationarity for such a random walk can often be checked by constructing a Lyapunov function. The same Lyapunov function can also be used for computing approximately the stationary distribution of this random walk, using methods developed by Meyn and Tweedie. In this paper we show that, for this type of random walks, computing the stationary probability exactly is an undecidable problem: no algorithm can exist to achieve this task. We then prove that computing large deviation rates for this model is also an undecidable problem. We extend these results to a certain type of queueing systems. The implication of these results is that no useful formulas for computing stationary probabilities and large deviations rates can exist in these systems

Multi-Policy Decision Making for Reliable Navigation in Dynamic Uncertain Environments

Navigating everyday social environments, in the presence of pedestrians and other dynamic obstacles remains one of the key challenges preventing mobile robots from leaving carefully designed spaces and entering our daily lives. The complex and tightly-coupled interactions between these agents make the environment dynamic and unpredictable, posing a formidable problem for robot motion planning. Trajectory planning methods, supported by models of typical human behavior and personal space, often produce reasonable behavior. However, they do not account for the future closed-loop interactions of other agents with the trajectory being constructed. As a consequence, the trajectories are unable to anticipate cooperative interactions (such as a human yielding), or adverse interactions (such as the robot blocking the way). Ideally, the robot must account for coupled agent-agent interactions while reasoning about possible future outcomes, and then take actions to advance towards its navigational goal without inconveniencing nearby pedestrians. Multi-Policy Decision Making (MPDM) is a novel framework for autonomous navigation in dynamic, uncertain environments where the robot's trajectory is not explicitly planned, but instead, the robot dynamically switches between a set of candidate closed-loop policies, allowing it to adapt to different situations encountered in such environments. The candidate policies are evaluated based on short-term (five-second) forward simulations of samples drawn from the estimated distribution of the agents' current states. These forward simulations and thereby the cost function, capture agent-agent interactions as well as agent-robot interactions which depend on the ego-policy being evaluated. In this thesis, we propose MPDM as a new method for navigation amongst pedestrians by dynamically switching from amongst a library of closed-loop policies. Due to real-time constraints, the robot's emergent behavior is directly affected by the quality of policy evaluation. Approximating how good a policy is based on only a few forward roll-outs is difficult, especially with the large space of possible pedestrian configurations and the sensitivity of the forward simulation to the sampled configurations. Traditional methods based on Monte-Carlo sampling often missed likely, high-cost outcomes, resulting in an over-optimistic evaluation of a policy and unreliable emergent behavior. By re-formulating policy evaluation as an optimization problem and enabling the quick discovery of potentially dangerous outcomes, we make MPDM more reliable and risk-aware. Even with the increased reliability, a major limitation is that MPDM requires the system designer to provide a set of carefully hand-crafted policies as it can evaluate only a few policies reliably in real-time. We radically enhance the expressivity of MPDM by allowing policies to have continuous-valued parameters, while simultaneously satisfying real-time constraints by quickly discovering promising policy parameters through a novel iterative gradient-based algorithm. Overall, we reformulate the traditional motion planning problem and paint it in a very different light --- as a bilevel optimization problem where the robot repeatedly discovers likely high-cost outcomes and adapts its policy parameters avoid these outcomes. We demonstrate significant performance benefits through extensive experiments in simulation as well as on a physical robot platform operating in a semi-crowded environment.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/150017/1/dhanvinm_1.pd

An Improved Algorithm for Sequential Information-Gathering Decisions in Design under Uncertainty

In engineering decision making, particularly in design, engineers must make decisions under varying levels of uncertainty. While not always the case, oftentimes one of the options available to an engineer is the ability to gather information that will reduce the uncertainty. With the reduced uncertainty, the engineer then returns to the same decision with more information. This sequential information-gathering decision problem is difficult to analyze and solve because the engineer must predict the value of gathering information in order to determine if the value outweighs the cost of the resources expended to gather the information. In practice, heuristics, intuition, and deadlines are often used to decide whether or not to gather information. A more complete and formal approach for quantifying the value of gathering information would benefit engineers in design decision making. Recent work proposed that a Partially Observable Markov Decision Process (POMDP) is an appropriate formalism for modeling sequential information-gathering decisions. A POMDP appears capable of capturing the salient features of such decisions. However, existing POMDP solution algorithms scale poorly with problem size. This thesis introduces an improved algorithm for solving POMDPs that takes advantage of certain characteristics inherent to information-gathering decision problems. The new algorithm is orders of magnitude faster and also is capable of handling specific problem parameters that existing methods cannot. The improvement is shown with a detailed case study, where the case study also performs a comparison of using the POMDP formalism for solving information-gathering decision problems to widely known approximate methods, such as Expected Value of Information methods. The study demonstrates that the use of the POMDP formalism, along with the improved algorithm, provides a valuable method for solving certain information-gathering decision problems

Regularity Properties and Pathologies of Position-Space Renormalization-Group Transformations

We reconsider the conceptual foundations of the renormalization-group (RG) formalism, and prove some rigorous theorems on the regularity properties and possible pathologies of the RG map. Regarding regularity, we show that the RG map, defined on a suitable space of interactions (= formal Hamiltonians), is always single-valued and Lipschitz continuous on its domain of definition. This rules out a recently proposed scenario for the RG description of first-order phase transitions. On the pathological side, we make rigorous some arguments of Griffiths, Pearce and Israel, and prove in several cases that the renormalized measure is not a Gibbs measure for any reasonable interaction. This means that the RG map is ill-defined, and that the conventional RG description of first-order phase transitions is not universally valid. For decimation or Kadanoff transformations applied to the Ising model in dimension $d \ge 3$, these pathologies occur in a full neighborhood $\{ \beta > \beta_0 ,\, |h| < \epsilon(\beta) \}$ of the low-temperature part of the first-order phase-transition surface. For block-averaging transformations applied to the Ising model in dimension $d \ge 2$, the pathologies occur at low temperatures for arbitrary magnetic-field strength. Pathologies may also occur in the critical region for Ising models in dimension $d \ge 4$. We discuss in detail the distinction between Gibbsian and non-Gibbsian measures, and give a rather complete catalogue of the known examples. Finally, we discuss the heuristic and numerical evidence on RG pathologies in the light of our rigorous theorems.Comment: 273 pages including 14 figures, Postscript, See also ftp.scri.fsu.edu:hep-lat/papers/9210/9210032.ps.

Information processing in biology

To survive, organisms must respond appropriately to a variety of challenges posed by a dynamic and uncertain environment. The mechanisms underlying such responses can in general be framed as input-output devices which map environment states (inputs) to associated responses (output. In this light, it is appealing to attempt to model these systems using information theory, a well developed mathematical framework to describe input-output systems. Under the information theoretical perspective, an organism’s behavior is fully characterized by the repertoire of its outputs under different environmental conditions. Due to natural selection, it is reasonable to assume this input-output mapping has been fine tuned in such a way as to maximize the organism’s fitness. If that is the case, it should be possible to abstract away the mechanistic implementation details and obtain the general principles that lead to fitness under a certain environment. These can then be used inferentially to both generate hypotheses about the underlying implementation as well as predict novel responses under external perturbations. In this work I use information theory to address the question of how biological systems generate complex outputs using relatively simple mechanisms in a robust manner. In particular, I will examine how communication and distributed processing can lead to emergent phenomena which allow collective systems to respond in a much richer way than a single organism could

Analysis of Embedded Controllers Subject to Computational Overruns

Microcontrollers have become an integral part of modern everyday embedded systems, such as smart bikes, cars, and drones. Typically, microcontrollers operate under real-time constraints, which require the timely execution of programs on the resource-constrained hardware. As embedded systems are becoming increasingly more complex, microcontrollers run the risk of violating their timing constraints, i.e., overrunning the program deadlines. Breaking these constraints can cause severe damage to both the embedded system and the humans interacting with the device. Therefore, it is crucial to analyse embedded systems properly to ensure that they do not pose any significant danger if the microcontroller overruns a few deadlines.However, there are very few tools available for assessing the safety and performance of embedded control systems when considering the implementation of the microcontroller. This thesis aims to fill this gap in the literature by presenting five papers on the analysis of embedded controllers subject to computational overruns. Details about the real-time operating system's implementation are included into the analysis, such as what happens to the controller's internal state representation when the timing constraints are violated. The contribution includes theoretical and computational tools for analysing the embedded system's stability, performance, and real-time properties.The embedded controller is analysed under three different types of timing violations: blackout events (when no control computation is completed during long periods), weakly-hard constraints (when the number of deadline overruns is constrained over a window), and stochastic overruns (when violations of timing constraints are governed by a probabilistic process). These scenarios are combined with different implementation policies to reduce the gap between the analysis and its practical applicability. The analyses are further validated with a comprehensive experimental campaign performed on both a set of physical processes and multiple simulations.In conclusion, the findings of this thesis reveal that the effect deadline overruns have on the embedded system heavily depends the implementation details and the system's dynamics. Additionally, the stability analysis of embedded controllers subject to deadline overruns is typically conservative, implying that additional insights can be gained by also analysing the system's performance

An Improved Algorithm for Sequential Information-Gathering Decisions in Design under Uncertainty

In engineering decision making, particularly in design, engineers must make decisions under varying levels of uncertainty. While not always the case, oftentimes one of the options available to an engineer is the ability to gather information that will reduce the uncertainty. With the reduced uncertainty, the engineer then returns to the same decision with more information. This sequential information-gathering decision problem is difficult to analyze and solve because the engineer must predict the value of gathering information in order to determine if the value outweighs the cost of the resources expended to gather the information. In practice, heuristics, intuition, and deadlines are often used to decide whether or not to gather information. A more complete and formal approach for quantifying the value of gathering information would benefit engineers in design decision making. Recent work proposed that a Partially Observable Markov Decision Process (POMDP) is an appropriate formalism for modeling sequential information-gathering decisions. A POMDP appears capable of capturing the salient features of such decisions. However, existing POMDP solution algorithms scale poorly with problem size. This thesis introduces an improved algorithm for solving POMDPs that takes advantage of certain characteristics inherent to information-gathering decision problems. The new algorithm is orders of magnitude faster and also is capable of handling specific problem parameters that existing methods cannot. The improvement is shown with a detailed case study, where the case study also performs a comparison of using the POMDP formalism for solving information-gathering decision problems to widely known approximate methods, such as Expected Value of Information methods. The study demonstrates that the use of the POMDP formalism, along with the improved algorithm, provides a valuable method for solving certain information-gathering decision problems

Information processing in biology

To survive, organisms must respond appropriately to a variety of challenges posed by a dynamic and uncertain environment. The mechanisms underlying such responses can in general be framed as input-output devices which map environment states (inputs) to associated responses (output. In this light, it is appealing to attempt to model these systems using information theory, a well developed mathematical framework to describe input-output systems. Under the information theoretical perspective, an organism’s behavior is fully characterized by the repertoire of its outputs under different environmental conditions. Due to natural selection, it is reasonable to assume this input-output mapping has been fine tuned in such a way as to maximize the organism’s fitness. If that is the case, it should be possible to abstract away the mechanistic implementation details and obtain the general principles that lead to fitness under a certain environment. These can then be used inferentially to both generate hypotheses about the underlying implementation as well as predict novel responses under external perturbations. In this work I use information theory to address the question of how biological systems generate complex outputs using relatively simple mechanisms in a robust manner. In particular, I will examine how communication and distributed processing can lead to emergent phenomena which allow collective systems to respond in a much richer way than a single organism could

Automatic Algorithm Selection for Complex Simulation Problems

To select the most suitable simulation algorithm for a given task is often difficult. This is due to intricate interactions between model features, implementation details, and runtime environment, which may strongly affect the overall performance. The thesis consists of three parts. The first part surveys existing approaches to solve the algorithm selection problem and discusses techniques to analyze simulation algorithm performance.The second part introduces a software framework for automatic simulation algorithm selection, which is evaluated in the third part.Die Auswahl des passendsten Simulationsalgorithmus für eine bestimmte Aufgabe ist oftmals schwierig. Dies liegt an der komplexen Interaktion zwischen Modelleigenschaften, Implementierungsdetails und Laufzeitumgebung. Die Arbeit ist in drei Teile gegliedert. Der erste Teil befasst sich eingehend mit Vorarbeiten zur automatischen Algorithmenauswahl, sowie mit der Leistungsanalyse von Simulationsalgorithmen. Der zweite Teil der Arbeit stellt ein Rahmenwerk zur automatischen Auswahl von Simulationsalgorithmen vor, welches dann im dritten Teil evaluiert wird