Verification and control of partially observable probabilistic systems

We present automated techniques for the verification and control of partially observable, probabilistic systems for both discrete and dense models of time. For the discrete-time case, we formally model these systems using partially observable Markov decision processes; for dense time, we propose an extension of probabilistic timed automata in which local states are partially visible to an observer or controller. We give probabilistic temporal logics that can express a range of quantitative properties of these models, relating to the probability of an event’s occurrence or the expected value of a reward measure. We then propose techniques to either verify that such a property holds or synthesise a controller for the model which makes it true. Our approach is based on a grid-based abstraction of the uncountable belief space induced by partial observability and, for dense-time models, an integer discretisation of real-time behaviour. The former is necessarily approximate since the underlying problem is undecidable, however we show how both lower and upper bounds on numerical results can be generated. We illustrate the effectiveness of the approach by implementing it in the PRISM model checker and applying it to several case studies from the domains of task and network scheduling, computer security and planning

Certified Reinforcement Learning with Logic Guidance

This paper proposes the first model-free Reinforcement Learning (RL) framework to synthesise policies for unknown, and continuous-state Markov Decision Processes (MDPs), such that a given linear temporal property is satisfied. We convert the given property into a Limit Deterministic Buchi Automaton (LDBA), namely a finite-state machine expressing the property. Exploiting the structure of the LDBA, we shape a synchronous reward function on-the-fly, so that an RL algorithm can synthesise a policy resulting in traces that probabilistically satisfy the linear temporal property. This probability (certificate) is also calculated in parallel with policy learning when the state space of the MDP is finite: as such, the RL algorithm produces a policy that is certified with respect to the property. Under the assumption of finite state space, theoretical guarantees are provided on the convergence of the RL algorithm to an optimal policy, maximising the above probability. We also show that our method produces ''best available'' control policies when the logical property cannot be satisfied. In the general case of a continuous state space, we propose a neural network architecture for RL and we empirically show that the algorithm finds satisfying policies, if there exist such policies. The performance of the proposed framework is evaluated via a set of numerical examples and benchmarks, where we observe an improvement of one order of magnitude in the number of iterations required for the policy synthesis, compared to existing approaches whenever available.Comment: This article draws from arXiv:1801.08099, arXiv:1809.0782

An integration framework for managing rich organisational process knowledge

The problem we have addressed in this dissertation is that of designing a pragmatic framework for integrating the synthesis and management of organisational process knowledge which is based on domain-independent AI planning and plan representations. Our solution has focused on a set of framework components which provide methods, tools and representations to accomplish this task.In the framework we address a lifecycle of this knowledge which begins with a methodological approach to acquiring information about the process domain. We show that this initial domain specification can be translated into a common constraint-based model of activity (based on the work of Tate, 1996c and 1996d) which can then be operationalised for use in an AI planner. This model of activity is ontologically underpinned and may be expressed with a flexible and extensible language based on a sorted first-order logic. The model combines perspectives covering both the space of behaviour as well as the space of decisions. Synthesised or modified processes/plans can be translated to and from the common representation in order to support knowledge sharing, visualisation and mixed-initiative interaction.This work united past and present Edinburgh research on planning and infused it with perspectives from design rationale, requirements engineering, and process knowledge sharing. The implementation has been applied to a portfolio of scenarios which include process examples from business, manufacturing, construction and military operations. An archive of this work is available at: http://www.aiai.ed.ac.uk/~oplan/cpf

Altered visual perception near the hands: a critical review of attentional and neurophysiological models

Visual perception changes as a function of hand proximity. While various theoretical accounts have been offered for this alteration (attentional prioritisation, bimodal cell involvement, detailed evaluation, and magnocellular neuron input enhancement), the current literature lacks consensus on these mechanisms. The purpose of this review, therefore, is to critically review the existing body of literature in light of these distinct theoretical accounts. We find that a growing number of results support the magnocellular (M-cell) enhancement account, and are difficult to reconcile with general attention-based explanations. Despite this key theoretical development in the field, there has been some ambiguity with interpretations offered in recent papers, for example, equating the existing attentional and M-cell based explanations, when in fact they make contrasting predictions. We therefore highlight the differential predictions arising from the distinct theoretical accounts. Importantly, however, we also offer novel perspectives that synthesises the role of attention and neurophysiological mechanisms in understanding altered visual perception near the hands. We envisage that this theoretical development will ensure that the field can progress from documenting behavioural differences, to a consensus on the underlying visual and neurophysiological mechanisms.This research was supported by an Australian Research Council (ARC) Discovery Early Career Research Award (DE140101734) awarded to S.C.G., ARC Discovery Project (DP110104553) awarded to M.E, a Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery Grants awarded S.F. and J.P

Towards a Theory of Constrained Relativism: Comparing and Combining the Work of Pierre Bourdieu, Mary Douglas and Michael Thompson, and Alan Fiske

In this article, I seek to compare Pierre Bourdieu\'s theory of practice, the cultural theory developed by Mary Douglas and Michael Thompson, and the relational models theory pioneered by Alan Fiske, and attempt to sketch how these theories could possibly be combined. I argue that the three theories are among the most interesting conceptual enterprises in the social sciences of the last few decades, as they all represent –quite similar– syntheses of long-standing social-science dualisms, such as objectivism vs. subjectivism, social structure vs. free will, functionalism vs. social conflict, etc. Besides these commonalities, I spell out the relative strengths and weaknesses of each of these approaches. This allows me to conclude by considering whether, and how, it might be possible to synthesise these syntheses by picking the most interesting features of the three theories, and avoiding their less appealing ones.[No keywords]

Causal Chain Analysis in Systematic Reviews of International Development Interventions

Understanding the extent to which an intervention ‘works’ can provide compelling evidence to decision-makers, although without an accompanying explanation of how an intervention works, this evidence can be difficult to apply in other settings, ultimately impeding its usefulness in making judicious and evidence-informed decisions. In this paper, we describe causal chain analysis as involving the development of a logic model, which outlines graphically a hypothesis of how an intervention leads to a change in an outcome. This logic model is then used to anchor subsequent decisions in the systematic review process, including decisions on synthesis. In this paper, we outline the steps taken in building a logic model, which usually consists of a series of boxes depicting intervention components and processes, outputs, and outcomes with arrows depicting connecting relationships. The nature of these connecting relationships and their basis in causality are considered, through a focus on complex causal relationships and the way in which contextual factors about the intervention setting or population may moderate these. We also explore the way in which specific combinations of intervention components may lead to successful interventions. Evidence synthesis techniques are discussed in the context of causal chain analysis, and their usefulness in exploring different parts of the causal chain or different types of relationship. The approaches outlined in this paper aim to assist systematic reviewers in producing findings that are useful to decision-makers and practitioners, and in turn, help to confirm existing theories or develop entirely new ways of understanding how interventions effect change

Hardward and algorithm architectures for real-time additive synthesis

Additive synthesis is a fundamental computer music synthesis paradigm tracing its origins to the work of Fourier and Helmholtz. Rudimentary implementation linearly combines harmonic sinusoids (or partials) to generate tones whose perceived timbral characteristics are a strong function of the partial amplitude spectrum. Having evolved over time, additive synthesis describes a collection of algorithms each characterised by the time-varying linear combination of basis components to generate temporal evolution of timbre. Basis components include exactly harmonic partials, inharmonic partials with time-varying frequency or non-sinusoidal waveforms each with distinct spectral characteristics. Additive synthesis of polyphonic musical instrument tones requires a large number of independently controlled partials incurring a large computational overhead whose investigation and reduction is a key motivator for this work. The thesis begins with a review of prevalent synthesis techniques setting additive synthesis in context and introducing the spectrum modelling paradigm which provides baseline spectral data to the additive synthesis process obtained from the analysis of natural sounds. We proceed to investigate recursive and phase accumulating digital sinusoidal oscillator algorithms, defining specific metrics to quantify relative performance. The concepts of phase accumulation, table lookup phase-amplitude mapping and interpolated fractional addressing are introduced and developed and shown to underpin an additive synthesis subclass - wavetable lookup synthesis (WLS). WLS performance is simulated against specific metrics and parameter conditions peculiar to computer music requirements. We conclude by presenting processing architectures which accelerate computational throughput of specific WLS operations and the sinusoidal additive synthesis model. In particular, we introduce and investigate the concept of phase domain processing and present several “pipeline friendly” arithmetic architectures using this technique which implement the additive synthesis of sinusoidal partials

Measurements, Models, Systems and Design

531 s.