From Maxout to Channel-Out: Encoding Information on Sparse Pathways

Motivated by an important insight from neural science, we propose a new framework for understanding the success of the recently proposed "maxout" networks. The framework is based on encoding information on sparse pathways and recognizing the correct pathway at inference time. Elaborating further on this insight, we propose a novel deep network architecture, called "channel-out" network, which takes a much better advantage of sparse pathway encoding. In channel-out networks, pathways are not only formed a posteriori, but they are also actively selected according to the inference outputs from the lower layers. From a mathematical perspective, channel-out networks can represent a wider class of piece-wise continuous functions, thereby endowing the network with more expressive power than that of maxout networks. We test our channel-out networks on several well-known image classification benchmarks, setting new state-of-the-art performance on CIFAR-100 and STL-10, which represent some of the "harder" image classification benchmarks.Comment: 10 pages including the appendix, 9 figure

Access to recorded interviews: A research agenda

Recorded interviews form a rich basis for scholarly inquiry. Examples include oral histories, community memory projects, and interviews conducted for broadcast media. Emerging technologies offer the potential to radically transform the way in which recorded interviews are made accessible, but this vision will demand substantial investments from a broad range of research communities. This article reviews the present state of practice for making recorded interviews available and the state-of-the-art for key component technologies. A large number of important research issues are identified, and from that set of issues, a coherent research agenda is proposed

PATTERN FORMATION AND PHASE TRANSITION OF CONNECTIVITY IN TWO DIMENSIONS

This dissertation is devoted to the study and analysis of different types of emergent behavior in physical systems. Emergence is a phenomenon that has fascinated researchers from various fields of science and engineering. From the emergence of global pandemics to the formation of reaction-diffusion patterns, the main feature that connects all these diverse systems is the appearance of a complex global structure as a result of collective interactions of simple underlying components. This dissertation will focus on two types of emergence in physical systems: emergence of long-range connectivity in networks and emergence and analysis of complex patterns. The most prominent theory which deals with the emergence of long-range connectivity is the percolation theory. This dissertation employs many concepts from the percolation theory to study connectivity transitions in various systems. Ordinary percolation theory is founded upon two main assumptions, namely locality and independence of the underlying components. In Chapters 2 and 3, we relax these assumptions in different manners and show that relaxing these assumptions leads to irregular behaviors such as appearance of different universality classes and, in some instances, violation of universality. Chapter 2 deals with relaxing the assumption of locality of interactions. In this Chapter, we define a hierarchy of various measures of robust connectivity. We study the phase transition of these robustness metrics as a function of site/bond occupation/removal probability on the square lattice. Furthermore, we perform extensive numerical analysis and extract these robustness metrics\u27 critical thresholds and critical behaviors. We show that some of these robustness metrics do not fall under the regular percolation universality class. The extensive numerical results in this work can serve as a foundation for any researcher who aims to design/study various degrees of connectivity in networks. In Chapter 3, we study the non-equilibrium phase transition of long-range connectivity in a multi-particle interacting system on the square lattice. The interactions between different particles translate to relaxing the assumption of independence in the percolation theory. Using extensive numerical simulations, we show that the phase transition observed in this system violates the regular concept of universality. However, it conforms well with the concept of weak-universality recently introduced in the literature. We observe that by varying inter-particle interaction strength in our model, one can control the critical behavior of this phase transition. These observations could be pivotal in studying phase transitions and universality classes. Chapter 4 focuses on the analysis of reaction-diffusion patterns. We utilize a multitude of machine learning algorithms to analyze reaction-diffusion patterns. In particular, we address two main problems using these techniques, namely, pattern regression and pattern classification. Given an observed instance of a pattern with a known generative function, in the pattern regression task, we aim to predict the specific set of reaction-diffusion parameters (i.e. diffusion constant) which can reproduce the observed pattern. We employ supervised learning techniques to successfully solve this problem and show the performance of our model in some real-world instances. We also address the task of pattern classification. In this task, we are interested in grouping different instances of similar patterns together. This task is usually performed visually by the researcher studying certain natural phenomena. However, this method is tedious and can be inconsistent among different researchers. We utilize supervised and unsupervised machine learning algorithms to classify patterns of the Gray-Scott model. We show that our methods show outstanding performance both in supervised and unsupervised settings. The methods introduced in this Chapter could bridge the gaps between researchers studying patterns in different fields of science and engineering

Formally analysing the concepts of domestic violence.

The types of police inquiries performed these days are incredibly diverse. Often data processing architectures are not suited to cope with this diversity since most of the case data is still stored as unstructured text. In this paper Formal Concept Analysis (FCA) is showcased for its exploratory data analysis capabilities in discovering domestic violence intelligence from a dataset of unstructured police reports filed with the regional police Amsterdam-Amstelland in the Netherlands. From this data analysis it is shown that FCA can be a powerful instrument to operationally improve policing practice. For one, it is shown that the definition of domestic violence employed by the police is not always as clear as it should be, making it hard to use it effectively for classification purposes. In addition, this paper presents newly discovered knowledge for automatically classifying certain cases as either domestic or non-domestic violence is. Moreover, it provides practical advice for detecting incorrect classifications performed by police officers. A final aspect to be discussed is the problems encountered because of the sometimes unstructured way of working of police officers. The added value of this paper resides in both using FCA for exploratory data analysis, as well as with the application of FCA for the detection of domestic violence.Formal concept analysis (FCA); Domestic violence; Knowledge discovery in databases; Text mining; Exploratory data analysis; Knowledge enrichment; Concept discovery;

LearnFCA: A Fuzzy FCA and Probability Based Approach for Learning and Classification

Formal concept analysis(FCA) is a mathematical theory based on lattice and order theory used for data analysis and knowledge representation. Over the past several years, many of its extensions have been proposed and applied in several domains including data mining, machine learning, knowledge management, semantic web, software development, chemistry ,biology, medicine, data analytics, biology and ontology engineering. This thesis reviews the state-of-the-art of theory of Formal Concept Analysis(FCA) and its various extensions that have been developed and well-studied in the past several years. We discuss their historical roots, reproduce the original definitions and derivations with illustrative examples. Further, we provide a literature review of it’s applications and various approaches adopted by researchers in the areas of dataanalysis, knowledge management with emphasis to data-learning and classification problems. We propose LearnFCA, a novel approach based on FuzzyFCA and probability theory for learning and classification problems. LearnFCA uses an enhanced version of FuzzyLattice which has been developed to store class labels and probability vectors and has the capability to be used for classifying instances with encoded and unlabelled features. We evaluate LearnFCA on encodings from three datasets - mnist, omniglot and cancer images with interesting results and varying degrees of success. Adviser: Dr Jitender Deogu

LEARNFCA: A FUZZY FCA AND PROBABILITY BASED APPROACH FOR LEARNING AND CLASSIFICATION

Formal concept analysis(FCA) is a mathematical theory based on lattice and order theory used for data analysis and knowledge representation. Over the past several years, many of its extensions have been proposed and applied in several domains including data mining, machine learning, knowledge management, semantic web, software development, chemistry ,biology, medicine, data analytics, biology and ontology engineering. This thesis reviews the state-of-the-art of theory of Formal Concept Analysis(FCA) and its various extensions that have been developed and well-studied in the past several years. We discuss their historical roots, reproduce the original definitions and derivations with illustrative examples. Further, we provide a literature review of it’s applications and various approaches adopted by researchers in the areas of dataanalysis, knowledge management with emphasis to data-learning and classification problems. We propose LearnFCA, a novel approach based on FuzzyFCA and probability theory for learning and classification problems. LearnFCA uses an enhanced version of FuzzyLattice which has been developed to store class labels and probability vectors and has the capability to be used for classifying instances with encoded and unlabelled features. We evaluate LearnFCA on encodings from three datasets - mnist, omniglot and cancer images with interesting results and varying degrees of success. Adviser: Jitender Deogu

Exploiting transitivity in probabilistic models for ontology learning

Nel natural language processing (NLP) catturare il significato delle parole è una delle sfide a cui i ricercatori sono largamente interessati. Le reti semantiche di parole o concetti, che strutturano in modo formale la conoscenza, sono largamente utilizzate in molte applicazioni. Per essere effettivamente utilizzate, in particolare nei metodi automatici di apprendimento, queste reti semantiche devono essere di grandi dimensioni o almeno strutturare conoscenza di domini molto specifici. Il nostro principale obiettivo è contribuire alla ricerca di metodi di apprendimento di reti semantiche concentrandosi in differenti aspetti. Proponiamo un nuovo modello probabilistico per creare o estendere reti semantiche che prende contemporaneamente in considerazine sia le evidenze estratte nel corpus sia la struttura della rete semantiche considerata nel training. In particolare il nostro modello durante l'apprendimento sfrutta le proprietà strutturali, come la transitività, delle relazioni che legano i nodi della nostra rete. La formulazione della probabilità che una data relazione tra due istanze appartiene alla rete semantica dipenderà da due probabilità: la probabilità diretta stimata delle evidenze del corpus e la probabilità indotta che deriva delle proprietà strutturali della relazione presa in considerazione. Il modello che proponiano introduce alcune innovazioni nella stima di queste probabilità. Proponiamo anche un modello che può essere usato per apprendere conoscenza in differenti domini di interesse senza un grande effort aggiuntivo per l'adattamento. In particolare, nell'approccio che proponiamo, si apprende un modello da un dominio generico e poi si sfrutta tale modello per estrarre nuova conoscenza in un dominio specifico. Infine proponiamo Semantic Turkey Ontology Learner (ST-OL): un sistema di apprendimento di ontologie incrementale. Mediante ontology editor, ST-OL fornisce un efficiente modo di interagire con l'utente finale e inserire le decisioni di tale utente nel loop dell'apprendimento. Inoltre il modello probabilistico integrato in ST-OL permette di sfruttare la transitività delle relazioni per indurre migliori modelli di estrazione. Mediante degli esperimenti dimostriamo che tutti i modelli che proponiamo danno un reale contributo ai differenti task che consideriamo migliorando le prestazioni.Capturing word meaning is one of the challenges of natural language processing (NLP). Formal models of meaning such as semantic networks of words or concepts are knowledge repositories used in a variety of applications. To be effectively used, these networks have to be large or, at least, adapted to specific domains. Our main goal is to contribute practically to the research on semantic networks learning models by covering different aspects of the task. We propose a novel probabilistic model for learning semantic networks that expands existing semantic networks taking into accounts both corpus-extracted evidences and the structure of the generated semantic networks. The model exploits structural properties of target relations such as transitivity during learning. The probability for a given relation instance to belong to the semantic networks of words depends both on its direct probability and on the induced probability derived from the structural properties of the target relation. Our model presents some innovations in estimating these probabilities. We also propose a model that can be used in different specific knowledge domains with a small effort for its adaptation. In this approach a model is learned from a generic domain that can be exploited to extract new informations in a specific domain. Finally, we propose an incremental ontology learning system: Semantic Turkey Ontology Learner (ST-OL). ST-OL addresses two principal issues. The first issue is an efficient way to interact with final users and, then, to put the final users decisions in the learning loop. We obtain this positive interaction using an ontology editor. The second issue is a probabilistic learning semantic networks of words model that exploits transitive relations for inducing better extraction models. ST-OL provides a graphical user interface and a human- computer interaction workflow supporting the incremental leaning loop of our learning semantic networks of words