The Phase Diagram of 1-in-3 Satisfiability Problem

We study the typical case properties of the 1-in-3 satisfiability problem, the boolean satisfaction problem where a clause is satisfied by exactly one literal, in an enlarged random ensemble parametrized by average connectivity and probability of negation of a variable in a clause. Random 1-in-3 Satisfiability and Exact 3-Cover are special cases of this ensemble. We interpolate between these cases from a region where satisfiability can be typically decided for all connectivities in polynomial time to a region where deciding satisfiability is hard, in some interval of connectivities. We derive several rigorous results in the first region, and develop the one-step--replica-symmetry-breaking cavity analysis in the second one. We discuss the prediction for the transition between the almost surely satisfiable and the almost surely unsatisfiable phase, and other structural properties of the phase diagram, in light of cavity method results.Comment: 30 pages, 12 figure

Explainable Deep Learning

Il grande successo che il Deep Learning ha ottenuto in ambiti strategici per la nostra società quali l'industria, la difesa, la medicina etc., ha portanto sempre più realtà a investire ed esplorare l'utilizzo di questa tecnologia. Ormai si possono trovare algoritmi di Machine Learning e Deep Learning quasi in ogni ambito della nostra vita. Dai telefoni, agli elettrodomestici intelligenti fino ai veicoli che guidiamo. Quindi si può dire che questa tecnologia pervarsiva è ormai a contatto con le nostre vite e quindi dobbiamo confrontarci con essa. Da questo nasce l’eXplainable Artificial Intelligence o XAI, uno degli ambiti di ricerca che vanno per la maggiore al giorno d'oggi in ambito di Deep Learning e di Intelligenza Artificiale. Il concetto alla base di questo filone di ricerca è quello di rendere e/o progettare i nuovi algoritmi di Deep Learning in modo che siano affidabili, interpretabili e comprensibili all'uomo. Questa necessità è dovuta proprio al fatto che le reti neurali, modello matematico che sta alla base del Deep Learning, agiscono come una scatola nera, rendendo incomprensibile all'uomo il ragionamento interno che compiono per giungere ad una decisione. Dato che stiamo delegando a questi modelli matematici decisioni sempre più importanti, integrandole nei processi più delicati della nostra società quali, ad esempio, la diagnosi medica, la guida autonoma o i processi di legge, è molto importante riuscire a comprendere le motivazioni che portano questi modelli a produrre determinati risultati. Il lavoro presentato in questa tesi consiste proprio nello studio e nella sperimentazione di algoritmi di Deep Learning integrati con tecniche di Intelligenza Artificiale simbolica. Questa integrazione ha un duplice scopo: rendere i modelli più potenti, consentendogli di compiere ragionamenti o vincolandone il comportamento in situazioni complesse, e renderli interpretabili. La tesi affronta due macro argomenti: le spiegazioni ottenute grazie all'integrazione neuro-simbolica e lo sfruttamento delle spiegazione per rendere gli algoritmi di Deep Learning più capaci o intelligenti. Il primo macro argomento si concentra maggiormente sui lavori svolti nello sperimentare l'integrazione di algoritmi simbolici con le reti neurali. Un approccio è stato quelli di creare un sistema per guidare gli addestramenti delle reti stesse in modo da trovare la migliore combinazione di iper-parametri per automatizzare la progettazione stessa di queste reti. Questo è fatto tramite l'integrazione di reti neurali con la Programmazione Logica Probabilistica (PLP) che consente di sfruttare delle regole probabilistiche indotte dal comportamento delle reti durante la fase di addestramento o ereditate dall'esperienza maturata dagli esperti del settore. Queste regole si innescano allo scatenarsi di un problema che il sistema rileva durate l'addestramento della rete. Questo ci consente di ottenere una spiegazione di cosa è stato fatto per migliorare l'addestramento una volta identificato un determinato problema. Un secondo approccio è stato quello di far cooperare sistemi logico-probabilistici con reti neurali per la diagnosi medica da fonti di dati eterogenee. La seconda tematica affrontata in questa tesi tratta lo sfruttamento delle spiegazioni che possiamo ottenere dalle rete neurali. In particolare, queste spiegazioni sono usate per creare moduli di attenzione che aiutano a vincolare o a guidare le reti neurali portandone ad avere prestazioni migliorate. Tutti i lavori sviluppati durante il dottorato e descritti in questa tesi hanno portato alle pubblicazioni elencate nel Capitolo 14.2.The great success that Machine and Deep Learning has achieved in areas that are strategic for our society such as industry, defence, medicine, etc., has led more and more realities to invest and explore the use of this technology. Machine Learning and Deep Learning algorithms and learned models can now be found in almost every area of our lives. From phones to smart home appliances, to the cars we drive. So it can be said that this pervasive technology is now in touch with our lives, and therefore we have to deal with it. This is why eXplainable Artificial Intelligence or XAI was born, one of the research trends that are currently in vogue in the field of Deep Learning and Artificial Intelligence. The idea behind this line of research is to make and/or design the new Deep Learning algorithms so that they are interpretable and comprehensible to humans. This necessity is due precisely to the fact that neural networks, the mathematical model underlying Deep Learning, act like a black box, making the internal reasoning they carry out to reach a decision incomprehensible and untrustable to humans. As we are delegating more and more important decisions to these mathematical models, it is very important to be able to understand the motivations that lead these models to make certain decisions. This is because we have integrated them into the most delicate processes of our society, such as medical diagnosis, autonomous driving or legal processes. The work presented in this thesis consists in studying and testing Deep Learning algorithms integrated with symbolic Artificial Intelligence techniques. This integration has a twofold purpose: to make the models more powerful, enabling them to carry out reasoning or constraining their behaviour in complex situations, and to make them interpretable. The thesis focuses on two macro topics: the explanations obtained through neuro-symbolic integration and the exploitation of explanations to make the Deep Learning algorithms more capable or intelligent. The neuro-symbolic integration was addressed twice, by experimenting with the integration of symbolic algorithms with neural networks. A first approach was to create a system to guide the training of the networks themselves in order to find the best combination of hyper-parameters to automate the design of these networks. This is done by integrating neural networks with Probabilistic Logic Programming (PLP). This integration makes it possible to exploit probabilistic rules tuned by the behaviour of the networks during the training phase or inherited from the experience of experts in the field. These rules are triggered when a problem occurs during network training. This generates an explanation of what was done to improve the training once a particular issue was identified. A second approach was to make probabilistic logic systems cooperate with neural networks for medical diagnosis on heterogeneous data sources. The second topic addressed in this thesis concerns the exploitation of explanations. In particular, the explanations one can obtain from neural networks are used in order to create attention modules that help in constraining and improving the performance of neural networks. All works developed during the PhD and described in this thesis have led to the publications listed in Chapter 14.2

Towards hybrid primary intersubjectivity: a neural robotics library for human science

Human-robot interaction is becoming an interesting area of research in cognitive science, notably, for the study of social cognition. Interaction theorists consider primary intersubjectivity a non-mentalist, pre-theoretical, non-conceptual sort of processes that ground a certain level of communication and understanding, and provide support to higher-level cognitive skills. We argue this sort of low level cognitive interaction, where control is shared in dyadic encounters, is susceptible of study with neural robots. Hence, in this work we pursue three main objectives. Firstly, from the concept of active inference we study primary intersubjectivity as a second person perspective experience characterized by predictive engagement, where perception, cognition, and action are accounted for an hermeneutic circle in dyadic interaction. Secondly, we propose an open-source methodology named \textit{neural robotics library} (NRL) for experimental human-robot interaction, and a demonstration program for interacting in real-time with a virtual Cartesian robot (VCBot). Lastly, through a study case, we discuss some ways human-robot (hybrid) intersubjectivity can contribute to human science research, such as to the fields of developmental psychology, educational technology, and cognitive rehabilitation

Benefit transfers of cultural heritage values - how far can we go?

Assessing the economic values attached to alternative land uses, when cultural heritage goods are at stake, makes the valuation process more articulated. Economic elicitation of cultural heritage values is quite a recent practice. Not many case studies have applied non-market valuation techniques, such as contingent valuation methods or travel cost methods, to derive monetary estimates of cultural goods attribute and even fewer applications have been policy oriented. Being a relatively recent research field, the first applications have mainly dealt with the challenges faced by the valuation techniques and the validity and reliability of results. These studies, particularly contingent valuation ones, have very high implementation costs. Hence, to obtain primary estimates of cultural values, agencies need to spend a great deal of money and time. Since these resources are scarce, there is an impinging need to consider the possibility of transferring benefit estimates from a specific “study site” for which data has been collected, to a “policy site” for which there is little or no information. An important question often addressed in the literature is what we can learn from individual case studies for a next case study. How general are the results of case study research? Can we transfer findings from a set of rather similar case studies to a new case study? This question is known as the benefit transfer (or value transfer) issue and seeks to investigate under which (general and specific) conditions common findings from various case studies are more or less valid for a new given case at a distinct site. Knowledge acquisition in the social sciences, and hence also in economics, is usually based on a reductionist approach, which eliminates many person-specific, object-specific or site-specific characteristics of a phenomenon, but the major advantage is that it allows for generalization through a common standardized approach that is applicable to a larger population. This methodology lies also at the heart of meta-analysis, which seeks to synthesize research findings from different case studies (van den Bergh et al. 1997, van den Bergh and Button 1997, 1999). Through the use of common relevant descriptors (behavioural, methodological, contextual) it is possible to draw inferences from a large sample of cases. For value transfer (also commonly named ‘benefit transfer’) the possibility of using meta-analysis is of major importance (Bal and Nijkamp 1998a). The basic idea of value transfer is that knowledge accumulated over time may be subjected to a transfer to a new, similar type of study. For the use of knowledge on a new similar study, it would be ideal if almost identical site characteristics could be transferred without any manipulation and if, at the same time, typical site-unique characteristics could be taken into account: that is, if it were possible to adapt derived variables for these site-unique characteristics.Value transfer studies in cultural heritage economics are rather rare, and the idea itself is quite controversial. In this paper we offer a concise – and certainly not exhausting – review of some recent value transfer studies in this area, with a particular view to spatial variability and transferability. We discuss limits and potentialities of benefit transfer approach for cultural values, aiming to raise debate on the topic. We acknowledge the local nature of cultural values and the strict relationship with the population to which the specific heritage belongs, but we focus on the more universally shared values that are embedded in cultural heritage and on possible ways of expressing them in terms of priorities and clusters. More research is needed in this direction before dismissing the possibility to apply benefit transfer in the case of cultural values estimates.

A corpus-based comparative pragmatic analysis of Irish English and Canadian English

This PhD thesis is a comparative study of the spoken grammar of Irish and Canadian Englishes within the framework of Variational Pragmatics at the formal level, used to study the pragmatic variation (the intra-varietal differences) in terms of forms and pragmatic functions. It is a study of spoken grammar as a whole (in a comparative and representative way between and across two varieties of English). Corpus linguistics is used as a methodological tool in order to conduct this research, exploring the nature of spoken grammar usage in both varieties comparatively in relation to their pragmatic functions and forms. The study illustrates an iterative approach in which top-down and bottom-up processes are used to establish pragmatic markers and their pragmatic functions in spoken grammar in the two varieties. Top-down analysis employs a framework for spoken grammar based on existing literature while the bottom-up process is based on micro-analysis of the data. The corpora used in the study are the spoken components of two International Corpus of English (ICE) corpora, namely ICE-Ireland and ICE-Canada comprising 600,000 words each (approximately). Methodologically, this study is not purely corpus-based nor corpus-driven but employs both methods. This iterative approach aligns with the notions of corpus-based versus corpus-driven linguistics and perspectives. Corpus tools are used to generate wordlists of the top 100 most frequent word and cluster lists. These are then analysed through qualitative analysis in order to identify whether or not they are a part of the spoken grammar. This process results in a candidate list that can then be functionally categorised and compared across varieties in terms of forms and functions. Specifically, the study offers insights on pragmatic markers: discourse markers, response tokens, questions, hedges and stance markers in Irish and Canadian English. The results offer a baseline description of the commonalities and differences in terms of spoken grammar and pragmatics across the two varieties of English which may have application to the study of other varieties of English. Also, the prominent forms of spoken grammar across these two varieties can be further explored from a macro-social perspective (e.g. age, gender, or social class) and a micro-social perspective (e.g. social distance or social dominance) and how these interplay with pragmatic choices.N

Biased landscapes for random Constraint Satisfaction Problems

The typical complexity of Constraint Satisfaction Problems (CSPs) can be investigated by means of random ensembles of instances. The latter exhibit many threshold phenomena besides their satisfiability phase transition, in particular a clustering or dynamic phase transition (related to the tree reconstruction problem) at which their typical solutions shatter into disconnected components. In this paper we study the evolution of this phenomenon under a bias that breaks the uniformity among solutions of one CSP instance, concentrating on the bicoloring of k-uniform random hypergraphs. We show that for small k the clustering transition can be delayed in this way to higher density of constraints, and that this strategy has a positive impact on the performances of Simulated Annealing algorithms. We characterize the modest gain that can be expected in the large k limit from the simple implementation of the biasing idea studied here. This paper contains also a contribution of a more methodological nature, made of a review and extension of the methods to determine numerically the discontinuous dynamic transition threshold.Comment: 32 pages, 16 figure

Natural Language Syntax Complies with the Free-Energy Principle

Natural language syntax yields an unbounded array of hierarchically structured expressions. We claim that these are used in the service of active inference in accord with the free-energy principle (FEP). While conceptual advances alongside modelling and simulation work have attempted to connect speech segmentation and linguistic communication with the FEP, we extend this program to the underlying computations responsible for generating syntactic objects. We argue that recently proposed principles of economy in language design - such as "minimal search" criteria from theoretical syntax - adhere to the FEP. This affords a greater degree of explanatory power to the FEP - with respect to higher language functions - and offers linguistics a grounding in first principles with respect to computability. We show how both tree-geometric depth and a Kolmogorov complexity estimate (recruiting a Lempel-Ziv compression algorithm) can be used to accurately predict legal operations on syntactic workspaces, directly in line with formulations of variational free energy minimization. This is used to motivate a general principle of language design that we term Turing-Chomsky Compression (TCC). We use TCC to align concerns of linguists with the normative account of self-organization furnished by the FEP, by marshalling evidence from theoretical linguistics and psycholinguistics to ground core principles of efficient syntactic computation within active inference