Hybrid image representation methods for automatic image annotation: a survey

In most automatic image annotation systems, images are represented with low level features using either global methods or local methods. In global methods, the entire image is used as a unit. Local methods divide images into blocks where fixed-size sub-image blocks are adopted as sub-units; or into regions by using segmented regions as sub-units in images. In contrast to typical automatic image annotation methods that use either global or local features exclusively, several recent methods have considered incorporating the two kinds of information, and believe that the combination of the two levels of features is beneficial in annotating images. In this paper, we provide a survey on automatic image annotation techniques according to one aspect: feature extraction, and, in order to complement existing surveys in literature, we focus on the emerging image annotation methods: hybrid methods that combine both global and local features for image representation

A Simple Post-Processing Technique for Improving Readability Assessment of Texts using Word Mover's Distance

Assessing the proper difficulty levels of reading materials or texts in general is the first step towards effective comprehension and learning. In this study, we improve the conventional methodology of automatic readability assessment by incorporating the Word Mover's Distance (WMD) of ranked texts as an additional post-processing technique to further ground the difficulty level given by a model. Results of our experiments on three multilingual datasets in Filipino, German, and English show that the post-processing technique outperforms previous vanilla and ranking-based models using SVM

Book Reviews and the Consolidation of Genre

Some literary scholars have claimed that predictive models can measure the strength of the boundaries that separate different cultural categories—genres, for instance, or market segments. But interpreting textual models as evidence about the strength of a cultural distinction has seemed a questionable move to many readers. We use book reviews to test this inference. Are the similarities between fictional texts purely verbal phenomena, or do they reflect social categories that are also legible (although expressed differently) in readers' responses to those texts? We find that the subject and genre categories most strongly marked in fictional texts are also the categories most strongly marked in reviews of fiction. The correlation is strong; r > .8

Post-training discriminative pruning for RBMs

One of the major challenges in the area of artificial neural networks is the identification of a suitable architecture for a specific problem. Choosing an unsuitable topology can exponentially increase the training cost, and even hinder network convergence. On the other hand, recent research indicates that larger or deeper nets can map the problem features into a more appropriate space, and thereby improve the classification process, thus leading to an apparent dichotomy. In this regard, it is interesting to inquire whether independent measures, such as mutual information, could provide a clue to finding the most discriminative neurons in a network. In the present work we explore this question in the context of Restricted Boltzmann Machines, by employing different measures to realize post-training pruning. The neurons which are determined by each measure to be the most discriminative, are combined and a classifier is applied to the ensuing network to determine its usefulness. We find that two measures in particular seem to be good indicators of the most discriminative neurons, producing savings of generally more than 50% of the neurons, while maintaining an acceptable error rate. Further, it is borne out that starting with a larger network architecture and then pruning is more advantageous than using a smaller network to begin with. Finally, a quantitative index is introduced which can provide information on choosing a suitable pruned network.Fil: Sánchez Gutiérrez, Máximo. Universidad Autónoma Metropolitana; MéxicoFil: Albornoz, Enrique Marcelo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional. Universidad Nacional del Litoral. Facultad de Ingeniería y Ciencias Hídricas. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional; ArgentinaFil: Rufiner, Hugo Leonardo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional. Universidad Nacional del Litoral. Facultad de Ingeniería y Ciencias Hídricas. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional; Argentina. Universidad Nacional de Entre Ríos; ArgentinaFil: Close, John Goddard. Universidad Autónoma Metropolitana; Méxic

Hate Me Not: Detecting Hate Inducing Memes in Code Switched Languages

The rise in the number of social media users has led to an increase in the hateful content posted online. In countries like India, where multiple languages are spoken, these abhorrent posts are from an unusual blend of code-switched languages. This hate speech is depicted with the help of images to form “Memes which create a long-lasting impact on the human mind. In this paper, we take up the task of hate and offense detection from multimodal data, i.e. images (Memes) that contain text in code-switched languages. We firstly present a novel triply annotated Indian political Memes (IPM) dataset, which comprises memes from various Indian political events that have taken place post-independence and are classified into three distinct categories. We also propose a binary-channelled CNN cum LSTM based model to process the images using the CNN model and text using the LSTM model to get state-of-the-art results for this task

IDENTIFIKASI KATA KUNCI PADA KONTEN PUBLIKASI JURNAL ILMIAH UNTUK STUDI KASUS PENCARIAN PUBLIKASI ONLINE ITS (POMITS)

Publikasi Online ITS (POMITS) adalah jurnal yang diperuntukkan sebagai jurnal publikasi bagi mahasiswa program sarjana ITS. Artikel yang terbit di dalamnya sudah cukup banyak dan seringkali diperlukan sebagai bahan referensi untuk penelitian mahasiswa lainnya. Proses pencarian yang ada saat ini masih berdasarkan judul, abstrak, nama penulis, dan kata kunci. Data-data tersebut masih dimasukkan secara manual oleh penulis. Proses ini memungkinkan adanya pemilihan kata kunci yang kurang sesuai. Sehingga diperlukan suatu upaya agar pemilihan kata kunci tersebut bisa lebih tepat dan merepresentasikan artikel tersebut.Tujuan dari penelitian ini adalah melakukan identifikasi kata kunci dalam artikel secara otomatis. Kata kunci tersebut dibedakan menjadi perangkat lunak yang digunakan, metode, dan kata kunci lain yang representatif. Dengan adanya identifikasi ini, pencarian artikel dapat mengembalikan hasil pencarian yang lebih tepat. Masalah ini dapat diatasi dengan menggunakan Named Entity Recognition (NER). Namun, model NER bahasa Indonesia yang dimiliki SpaCy masih belum tersedia, maka diperlukan pembangunan model NER tersebut.Dalam penelitian ini, identifikasi setiap anotasi kata kunci pada konten POMITS menjadi metadata dilakukan dengan mendeteksi named entity berupa perangkat lunak, metode, dan kata kunci representatif menggunakan model NER. Hasil anotasi NER disimpan dalam bentuk pasangan triplets pada triple store Apache Jena Fuseki. Selanjutnya, triple store tersebut dapat digunakan untuk menjawab pencarian tentang perangkat lunak, metode, dan kata kunci. Berdasarkan hasil pengujian, sistem berhasil mendeteksi entitas NER serta menyimpan anotasi dalam bentuk pasangan triplets pada Apache Jena Fuseki. Identifikasi kata kunci menghasilkan rata-rata nilai presisi 84,76% dan recall 63.59%

Learning and inference with Wasserstein metrics

Thesis: Ph. D., Massachusetts Institute of Technology, Department of Brain and Cognitive Sciences, 2018.Cataloged from PDF version of thesis.Includes bibliographical references (pages 131-143).This thesis develops new approaches for three problems in machine learning, using tools from the study of optimal transport (or Wasserstein) distances between probability distributions. Optimal transport distances capture an intuitive notion of similarity between distributions, by incorporating the underlying geometry of the domain of the distributions. Despite their intuitive appeal, optimal transport distances are often difficult to apply in practice, as computing them requires solving a costly optimization problem. In each setting studied here, we describe a numerical method that overcomes this computational bottleneck and enables scaling to real data. In the first part, we consider the problem of multi-output learning in the presence of a metric on the output domain. We develop a loss function that measures the Wasserstein distance between the prediction and ground truth, and describe an efficient learning algorithm based on entropic regularization of the optimal transport problem. We additionally propose a novel extension of the Wasserstein distance from probability measures to unnormalized measures, which is applicable in settings where the ground truth is not naturally expressed as a probability distribution. We show statistical learning bounds for both the Wasserstein loss and its unnormalized counterpart. The Wasserstein loss can encourage smoothness of the predictions with respect to a chosen metric on the output space. We demonstrate this property on a real-data image tagging problem, outperforming a baseline that doesn't use the metric. In the second part, we consider the probabilistic inference problem for diffusion processes. Such processes model a variety of stochastic phenomena and appear often in continuous-time state space models. Exact inference for diffusion processes is generally intractable. In this work, we describe a novel approximate inference method, which is based on a characterization of the diffusion as following a gradient flow in a space of probability densities endowed with a Wasserstein metric. Existing methods for computing this Wasserstein gradient flow rely on discretizing the underlying domain of the diffusion, prohibiting their application to problems in more than several dimensions. In the current work, we propose a novel algorithm for computing a Wasserstein gradient flow that operates directly in a space of continuous functions, free of any underlying mesh. We apply our approximate gradient flow to the problem of filtering a diffusion, showing superior performance where standard filters struggle. Finally, we study the ecological inference problem, which is that of reasoning from aggregate measurements of a population to inferences about the individual behaviors of its members. This problem arises often when dealing with data from economics and political sciences, such as when attempting to infer the demographic breakdown of votes for each political party, given only the aggregate demographic and vote counts separately. Ecological inference is generally ill-posed, and requires prior information to distinguish a unique solution. We propose a novel, general framework for ecological inference that allows for a variety of priors and enables efficient computation of the most probable solution. Unlike previous methods, which rely on Monte Carlo estimates of the posterior, our inference procedure uses an efficient fixed point iteration that is linearly convergent. Given suitable prior information, our method can achieve more accurate inferences than existing methods. We additionally explore a sampling algorithm for estimating credible regions.by Charles Frogner.Ph. D