Bayesian Nonparametric Methods for Partially-Observable Reinforcement Learning

Making intelligent decisions from incomplete information is critical in many applications: for example, robots must choose actions based on imperfect sensors, and speech-based interfaces must infer a user’s needs from noisy microphone inputs. What makes these tasks hard is that often we do not have a natural representation with which to model the domain and use for choosing actions; we must learn about the domain’s properties while simultaneously performing the task. Learning a representation also involves trade-offs between modeling the data that we have seen previously and being able to make predictions about new data. This article explores learning representations of stochastic systems using Bayesian nonparametric statistics. Bayesian nonparametric methods allow the sophistication of a representation to scale gracefully with the complexity in the data. Our main contribution is a careful empirical evaluation of how representations learned using Bayesian nonparametric methods compare to other standard learning approaches, especially in support of planning and control. We show that the Bayesian aspects of the methods result in achieving state-of-the-art performance in decision making with relatively few samples, while the nonparametric aspects often result in fewer computations. These results hold across a variety of different techniques for choosing actions given a representation

Linear Parametric Model Checking of Timed Automata

We present an extension of the model checker Uppaal capable of synthesizing linear parameter constraints for the correctness ofparametric timed automata. The symbolic representation of the (parametric) state-space is shown to be correct. A second contribution of thispaper is the identification of a subclass of parametric timed automata(L/U automata), for which the emptiness problem is decidable, contraryto the full class where it is know to be undecidable. Also we present anumber of lemmas enabling the verification effort to be reduced for L/Uautomata in some cases. We illustrate our approach by deriving linearparameter constraints for a number of well-known case studies from theliterature (exhibiting a flaw in a published paper)

Applications of complex adaptive systems approaches to coastal systems

This thesis investigatesth e application of complex adaptives ystemsa pproaches (e. g. Artificial Neural Networks and Evolutionary Computation) to the study of coastal hydrodynamica nd morphodynamicb ehaviour.T raditionally, nearshorem orphologicalc oastal systems tudiesh ave developeda n understandingo f thosep hysicalp rocesseso ccurringo n both short temporal, and small spatial scales with a large degree of success. The associated approachesa nd conceptsu sedt o study the coastals ystema t theses calesh ave Primarily been linear in nature.H owever,w hent hesea pproachetso studyingt he coastals ystema re extendedto investigating larger temporal and spatial scales,w hich are commensuratew ith the aims of coastal managementr, esults have had less success.T he lack of successi n developing an understandingo f large scalec oastalb ehaviouri s to a large extent attributablet o the complex behavioura ssociatedw ith the coastals ystem.I bis complexity arises as a result of both the stochastic and chaotic nature of the coastal system. This allows small scale system understandingto be acquiredb ut preventst he Largers caleb ehaviourt o be predictede ffectively. This thesis presentsf our hydro-morphodynamicc ase studies to demonstratet he utility of complex adaptives ystema pproachesfo r studying coastals ystems.T he first two demonstrate the application of Artificial Neural Networks, whilst the latter two illustrate the application of EvolutionaryC omputation.C aseS tudy #I considerst he natureo f the discrepancyb etweent he observedl ocation of wave breakingp atternso ver submergeds andbarsa nd the actual sandbar locations.A rtificial Neural Networks were able to quantitativelyc orrectt he observedlo cations to produce reliable estimates of the actual sand bar locations. Case Study #2 considers the developmenot f an approachf or the discriminationo f shorelinel ocation in video imagesf or the productiono f intertidal mapso f the nearshorer egion. In this caset he systemm odelledb y the Artificial Neural Network is the nature of the discrimination model carried out by the eye in delineating a shoreline feature between regions of sand and water. The Artificial Neural Network approachw as shownt o robustly recognisea rangeo f shorelinef eaturesa t a variety of beaches and hydrodynamic settings. Case Study #3 was the only purely hydrodynamic study consideredin the thesis.I t investigatedth e use of Evolutionary Computationt o provide means of developing a parametric description of directional wave spectra in both reflective and nonreflective conditions. It is shown to provide a unifying approach which produces results which surpassedth ose achievedb y traditional analysisa pproachese vent hough this may not strictly have been considered as a fidly complex system. Case Study #4 is the most ambitious applicationa nd addressetsh e needf or data reductiona s a precursorw hen trying to study large scalem orphodynamicd ata sets.I t utilises EvolutionaryC omputationa pproachesto extractt he significant morphodynamic variability evidenced in both directly and remotely sampled nearshorem orphologiesS. ignificantd atar eductioni s achievedw hilst reWning up to 90% of the original variability in the data sets. These case studies clearly demonstrate the ability of complex adaptive systems to be successfidly applied to coastal system studies. This success has been shown to equal and sometimess urpasst he results that may be obtained by traditional approachesT. he strong performance of Complex Adaptive System approaches is closely linked to the level of complexity or non-linearity of the system being studied. Based on a qualitative evaluation, Evolutionary Computation was shown to demonstrate an advantage over Artificial Neural Networks in terms of the level of new insights which may be obtained. However, utility also needs to consider general ease of applicability and ease of implementation of the study approach.I n this sense,A rtificial Neural Networks demonstratem ore utility for the study of coastals ystems.T he qualitative assessmenatp proachu sedt o evaluatet he cases tudiesi n this thesis, may be used as a guide for choosingt he appropriatenesso f either Artificial Neural Networks or Evolutionary Computation for future coastal system studies

Distance Oracles for Time-Dependent Networks

We present the first approximate distance oracle for sparse directed networks with time-dependent arc-travel-times determined by continuous, piecewise linear, positive functions possessing the FIFO property. Our approach precomputes $(1+\epsilon)-$approximate distance summaries from selected landmark vertices to all other vertices in the network. Our oracle uses subquadratic space and time preprocessing, and provides two sublinear-time query algorithms that deliver constant and $(1+\sigma)-$approximate shortest-travel-times, respectively, for arbitrary origin-destination pairs in the network, for any constant $\sigma > \epsilon$. Our oracle is based only on the sparsity of the network, along with two quite natural assumptions about travel-time functions which allow the smooth transition towards asymmetric and time-dependent distance metrics.Comment: A preliminary version appeared as Technical Report ECOMPASS-TR-025 of EU funded research project eCOMPASS (http://www.ecompass-project.eu/). An extended abstract also appeared in the 41st International Colloquium on Automata, Languages, and Programming (ICALP 2014, track-A

Formal Methods for Autonomous Systems

Formal methods refer to rigorous, mathematical approaches to system development and have played a key role in establishing the correctness of safety-critical systems. The main building blocks of formal methods are models and specifications, which are analogous to behaviors and requirements in system design and give us the means to verify and synthesize system behaviors with formal guarantees. This monograph provides a survey of the current state of the art on applications of formal methods in the autonomous systems domain. We consider correct-by-construction synthesis under various formulations, including closed systems, reactive, and probabilistic settings. Beyond synthesizing systems in known environments, we address the concept of uncertainty and bound the behavior of systems that employ learning using formal methods. Further, we examine the synthesis of systems with monitoring, a mitigation technique for ensuring that once a system deviates from expected behavior, it knows a way of returning to normalcy. We also show how to overcome some limitations of formal methods themselves with learning. We conclude with future directions for formal methods in reinforcement learning, uncertainty, privacy, explainability of formal methods, and regulation and certification

Automatic Gridding for DNA Microarray Image Using Image Projection Profile

DNA microarray is powerful tool and widely used in many areas. DNA microarray is produced from control and test tissue sample cDNAs, which are labeled with two different fluorescent dyes. After hybridization using a laser scanner, microarray images are obtained. Image analysis play an important role in extracting fluorescence intensity from microarray image. First step in microarray image analysis is addressing, that is finding areas in the image on which contain one spot using gird lines. This step can be done by either manually or automatically. In this paper we propose an efficient and simple automatic gridding for microarray image analysis using image projection profile, base on fact that microarray image has local minimum and maximum intensity at background and foreground areas respectively. Grid lines are obtained by finding local minimum of vertical and horizontal projection profile. This algorithm has been implemented in MATLAB and tested with several microarray image