Rewriting and narrowing for constructor systems with call-time choice semantics

Non-confluent and non-terminating constructor-based term rewrite systems are useful for the purpose of specification and programming. In particular, existing functional logic languages use such kind of rewrite systems to define possibly non-strict non-deterministic functions. The semantics adopted for non-determinism is call-time choice, whose combination with non-strictness is a non trivial issue, addressed years ago from a semantic point of view with the Constructor-based Rewriting Logic (CRWL), a well-known semantic framework commonly accepted as suitable semantic basis of modern functional logic languages. A drawback of CRWL is that it does not come with a proper notion of one-step reduction, which would be very useful to understand and reason about how computations proceed. In this paper we develop thoroughly the theory for the first order version of letrewriting, a simple reduction notion close to that of classical term rewriting, but extended with a let-binding construction to adequately express the combination of call-time choice with non-strict semantics. Let-rewriting can be seen as a particular textual presentation of term graph rewriting. We investigate the properties of let-rewriting, most remarkably their equivalence with respect to a conservative extension of the CRWL-semantics coping with let-bindings, and we show by some case studies that having two interchangeable formal views (reduction/semantics) of the same language is a powerful reasoning tool. After that, we provide a notion of let-narrowing which is adequate for call-time choice as proved by soundness and completeness results of let-narrowing with respect to letre writing. Moreover, we relate those let-rewriting and let-narrowing relations (and hence CRWL) with ordinary term rewriting and narrowing, providing in particular soundness and completeness of let-rewriting with respect to term rewriting for a class of programs which are deterministic in a semantic sense

GemPy 1.0 : open-source stochastic geological modeling and inversion

Peer reviewedPublisher PD

Partial Identification in Nonseparable Binary Response Models with Endogenous Regressors

This paper considers (partial) identification of a variety of counterfactual parameters in binary response models with possibly endogenous regressors. Our framework allows for nonseparable index functions with multi-dimensional latent variables, and does not require parametric distributional assumptions. We leverage results on hyperplane arrangements and cell enumeration from the literature on computational geometry in order to provide a tractable means of computing the identified set. We demonstrate how various functional form, independence, and monotonicity assumptions can be imposed as constraints in our optimization procedure to tighten the identified set. Finally, we apply our method to study the effects of health insurance on the decision to seek medical treatment

Deep Generative Models for Natural Language

Generative models aim to simulate the process by which a set of data is generated. They are intuitive, interpretable, and naturally suited to learning from unlabelled data. This is particularly appealing in natural language processing, where labels are often costly to obtain and can require significant manual input from trained annotators. However, traditional generative modelling approaches can often be inflexible due to the need to maintain tractable maximum likelihood training. On the other hand, deep learning methods are powerful, flexible, and have achieved significant success on a wide variety of natural language processing tasks. In recent years, algorithms have been developed for training generative models that incorporate neural networks to parametrise their conditional distributions. These approaches aim to take advantage of the intuitiveness and interpretability of generative models as well as the power and flexibility of deep learning. In this work, we investigate how to leverage such algorithms in order to develop deep generative models for natural language. Firstly, we present an attention-based latent variable model, trained using unlabelled data, for learning representations of sentences. Experiments such as missing word imputation and sentence similarity matching suggest that the representations are able to learn semantic information about the sentences. We then present an RNN-based latent variable model for per- forming machine translation. Trained using semi-supervised learning, our approach achieves strong results even with very limited labelled data. Finally, we present a locally-contextual conditional random field for performing sequence labelling tasks. Our method consistently outperforms the linear chain conditional random field and achieves state of the art performance on two out of the four tasks evaluated

Incorporating prior knowledge into deep neural networks without handcrafted features

Deep learning (DL) is currently the largest area of research within artificial intelligence (AI). This success can largely be attributed to the data-driven nature of the DL algorithms themselves: unlike previous approaches in AI which required handcrafting and significant human intervention, DL models can be implemented and trained with little to no human involvement. The lack of handcrafting, however, can be a two-edged sword. DL algorithms are notorious for producing uninterpretable features, generalising badly to new tasks and relying on extraordinarily large datasets for training. In this thesis, on the assumption that these shortcomings are symptoms of the under-constrained training setup of deep networks, we address the question of how to incorporate knowledge into DL algorithms without reverting to complete handcrafting in order to train more data efficient algorithms. % In this thesis we consider different alternatives to this problem. We start by motivating this line of work with an example of a DL architecture which, inspired by symbolic AI, aims at extracting symbols from a simple environment and using those for quickly learning downstream tasks. Our proof-of-concept model shows that it is possible to address some of the data efficiency issues as well as obtaining more interpretable representations by reasoning at this higher level of abstraction. Our second approach for data-efficiency is based on pre-training: the idea is to pre-train some parts of the DL network on a different, but related, task to first learn useful feature extractors. For our experiments we pre-train the encoder of a reinforcement learning agent on a 3D scene prediction task and then use the features produced by the encoder to train a simulated robot arm on a reaching task. Crucially, unlike previous approaches that could only learn from fixed view-points, we are able to train an agent using observations captured from randomly changing positions around the robot arm, without having to train a separate policy for each observation position. Lastly, we focus on how to build in prior knowledge through the choice of dataset. To this end, instead of training DL models on a single dataset, we train them on a distribution over datasets that captures the space of tasks we are interested in. This training regime produces models that can flexibly adapt to any dataset within the distribution at test time. Crucially they only need a small number of observations in order to adapt their predictions, thus addressing the data-efficiency challenge at test time. We call this family of meta-learning models for few-shot prediction Neural Processes (NPs). In addition to successfully learning how to carry out few-shot regression and classification, NPs produce uncertainty estimates and can generate coherent samples at arbitrary resolutions.Open Acces

Una implementación en Haskell de un lenguaje funcional no determinista

En este trabajo se propone e implementa un lenguaje funcional que incorpora características habituales de este paradigma, como es el orden superior, ajuste de patrones o evaluación perezosa. Añade características no tan habituales como es el indeterminismo, expresado mediante funciones no deterministas, es decir, funciones que para unos argumentos dados pueden devolver más de un resultado. Este tipo de funciones existen en los llamados lenguajes lógico funcionales, que adoptan para el indeterminismo una semántica de compartición (call-time choice), en contraste con la semántica de no compartición (run-time choice) más típica de otros formalismos, como son los sistemas de reescritura. Lo específico del lenguaje nuevo que se va a proponer e implementar, Sharade, es que combinará ambos tipos de semánticas mediante primitivas que existirán en el propio lenguaje. La implementación del lenguaje está realizada íntegramente en Haskell, en contraste con las implementaciones clásicas de los lenguajes lógico funcionales, habitualmente basadas en Prolog. Haskell es un lenguaje funcional puro con un sistema de tipos muy fuerte, siendo el lenguaje de referencia en el ámbito de la programación funcional con evaluación perezosa. Aparte de fases auxiliares como son el análisis sintáctico y la inferencia de tipos, lo esencial de la implementación consiste en un proceso de traducción de programas fuente Sharade en programas objeto Haskell, aprovechando así muchas de las características de este último como lenguaje funcional. Los programas objeto hacen uso intensivo de programación monádica, a través en particular de una biblioteca Haskell para la programación con indeterminismo con call-time choice y evaluación perezosa, biblioteca ya existente pero que ha debido ser adaptada por nosotros para actualizarla a las nuevas versiones de Haskell

Algorithmic Regulation using AI and Blockchain Technology

This thesis investigates the application of AI and blockchain technology to the domain of Algorithmic Regulation. Algorithmic Regulation refers to the use of intelligent systems for the enabling and enforcement of regulation (often referred to as RegTech in financial services). The research work focuses on three problems: a) Machine interpretability of regulation; b) Regulatory reporting of data; and c) Federated analytics with data compliance. Uniquely, this research was designed, implemented, tested and deployed in collaboration with the Financial Conduct Authority (FCA), Santander, RegulAItion and part funded by the InnovateUK RegNet project. I am a co-founder of RegulAItion. / Using AI to Automate the Regulatory Handbook: In this investigation we propose the use of reasoning systems for encoding financial regulation as machine readable and executable rules. We argue that our rules-based “white-box” approach is needed, as opposed to a “black-box” machine learning approach, as regulators need explainability and outline the theoretical foundation needed to encode regulation from the FCA Handbook into machine readable semantics. We then present the design and implementation of a production-grade regulatory reasoning system built on top of the Java Expert System Shell (JESS) and use it to encode a subset of regulation (consumer credit regulation) from the FCA Handbook. We then perform an empirical evaluation, with the regulator, of the system based on its performance and accuracy in handling 600 “real- world” queries and compare it with its human equivalent. The findings suggest that the proposed approach of using reasoning systems not only provides quicker responses, but also more accurate results to answers from queries that are explainable. / SmartReg: Using Blockchain for Regulatory Reporting: In this investigation we explore the use of distributed ledgers for real-time reporting of data for compliance between firms and regulators. Regulators and firms recognise the growing burden and complexity of regulatory reporting resulting from the lack of data standardisation, increasing complexity of regulation and the lack of machine executable rules. The investigation presents a) the design and implementation of a permissioned Quorum-Ethereum based regulatory reporting network that makes use of an off-chain reporting service to execute machine readable rules on banks’ data through smart contracts b) a means for cross border regulators to share reporting data with each other that can be used to given them a true global view of systemic risk c) a means to carry out regulatory reporting using a novel pull-based approach where the regulator is able to directly “pull” relevant data out of the banks’ environments in an ad-hoc basis- enabling regulators to become more active when addressing risk. We validate the approach and implementation of our system through a pilot use case with a bank and regulator. The outputs of this investigation have informed the Digital Regulatory Reporting initiative- an FCA and UK Government led project to improve regulatory reporting in the financial services. / RegNet: Using Federated Learning and Blockchain for Privacy Preserving Data Access In this investigation we explore the use of Federated Machine Learning and Trusted data access for analytics. With the development of stricter Data Regulation (e.g. GDPR) it is increasingly difficult to share data for collective analytics in a compliant manner. We argue that for data compliance, data does not need to be shared but rather, trusted data access is needed. The investigation presents a) the design and implementation of RegNet- an infrastructure for trusted data access in a secure and privacy preserving manner for a singular algorithmic purpose, where the algorithms (such as Federated Learning) are orchestrated to run within the infrastructure of data owners b) A taxonomy for Federated Learning c) The tokenization and orchestration of Federated Learning through smart contracts for auditable governance. We validate our approach and the infrastructure (RegNet) through a real world use case, involving a number of banks, that makes use of Federated Learning with Epsilon-Differential Privacy for improving the performance of an Anti-Money-Laundering classification model