Deep Neural Networks for Visual Reasoning, Program Induction, and Text-to-Image Synthesis.

Deep neural networks excel at pattern recognition, especially in the setting of large scale supervised learning. A combination of better hardware, more data, and algorithmic improvements have yielded breakthroughs in image classification, speech recognition and other perception problems. The research frontier has shifted towards the weak side of neural networks: reasoning, planning, and (like all machine learning algorithms) creativity. How can we advance along this frontier using the same generic techniques so effective in pattern recognition; i.e. gradient descent with backpropagation? In this thesis I develop neural architectures with new capabilities in visual reasoning, program induction and text-to-image synthesis. I propose two models that disentangle the latent visual factors of variation that give rise to images, and enable analogical reasoning in the latent space. I show how to augment a recurrent network with a memory of programs that enables the learning of compositional structure for more data-efficient and generalizable program induction. Finally, I develop a generative neural network that translates descriptions of birds, flowers and other categories into compelling natural images.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/135763/1/reedscot_1.pd

Gender bias in machine learning for sentiment analysis

This is an accepted manuscript of an article published by Emerald Publishing Limited in Online Information Review on 01/01/2018, available online: https://doi.org/10.1108/OIR-05-2017-0153 The accepted version of the publication may differ from the final published version.Purpose: This paper investigates whether machine learning induces gender biases in the sense of results that are more accurate for male authors than for female authors. It also investigates whether training separate male and female variants could improve the accuracy of machine learning for sentiment analysis. Design/methodology/approach: This article uses ratings-balanced sets of reviews of restaurants and hotels (3 sets) to train algorithms with and without gender selection. Findings: Accuracy is higher on female-authored reviews than on male-authored reviews for all data sets, so applications of sentiment analysis using mixed gender datasets will over represent the opinions of women. Training on same gender data improves performance less than having additional data from both genders. Practical implications: End users of sentiment analysis should be aware that its small gender biases can affect the conclusions drawn from it and apply correction factors when necessary. Users of systems that incorporate sentiment analysis should be aware that performance will vary by author gender. Developers do not need to create gender-specific algorithms unless they have more training data than their system can cope with. Originality/value: This is the first demonstration of gender bias in machine learning sentiment analysis

Crowdsourcing a Word-Emotion Association Lexicon

Even though considerable attention has been given to the polarity of words (positive and negative) and the creation of large polarity lexicons, research in emotion analysis has had to rely on limited and small emotion lexicons. In this paper we show how the combined strength and wisdom of the crowds can be used to generate a large, high-quality, word-emotion and word-polarity association lexicon quickly and inexpensively. We enumerate the challenges in emotion annotation in a crowdsourcing scenario and propose solutions to address them. Most notably, in addition to questions about emotions associated with terms, we show how the inclusion of a word choice question can discourage malicious data entry, help identify instances where the annotator may not be familiar with the target term (allowing us to reject such annotations), and help obtain annotations at sense level (rather than at word level). We conducted experiments on how to formulate the emotion-annotation questions, and show that asking if a term is associated with an emotion leads to markedly higher inter-annotator agreement than that obtained by asking if a term evokes an emotion

Sentiment Analysis: An Overview from Linguistics

Sentiment analysis is a growing field at the intersection of linguistics and computer science, which attempts to automatically determine the sentiment, or positive/negative opinion, contained in text. Sentiment can be characterized as positive or negative evaluation expressed through language. Common applications of sentiment analysis include the automatic determination of whether a review posted online (of a movie, a book, or a consumer product) is positive or negative towards the item being reviewed. Sentiment analysis is now a common tool in the repertoire of social media analysis carried out by companies, marketers and political analysts. Research on sentiment analysis extracts information from positive and negative words in text, from the context of those words, and the linguistic structure of the text. This brief survey examines in particular the contributions that linguistic knowledge can make to the problem of automatically determining sentiment

Meaning and individual minds : the case of if

PhD ThesisTraditionally (e.g. Sperber & Wilson 1995, Levinson 2000, Jackendoff 2002, Chomsky 2005a), linguistic expressions have meaning in virtue of having linguistic semantic properties. It is often claimed that linguistic semantics is functionally distinct from but related to the semantics of thought. In particular, linguistic semantics is assumed to be deterministically (necessarily and always) decoded in utterance interpretation and fed, as a basic premise, to pragmatic processing. Linguistic semantics is supposed to aid (i.e. constrain) utterance interpretation insofar as it is at least ‘widely’ shared among speech community members (Carston 2002). However, it has been suggested that linguistic semantics is problematic (e.g. Burton-Roberts 2005, Gibbs 2002, Recanati 2005). This thesis argues that the notion of linguistic semantics, as well as the process of deterministic decoding of such content, is implausible and explores the consequences of this claim for a theory of meaning and utterance interpretation. In the first part, I raise questions about the nature of semantics (externalism or internalism) as well as its structure (atomism, molecularism or holism). In line with the Representational Hypothesis (e.g. Burton-Roberts 2012), I maintain that thought is the only locus of semantics and that meaning is not a property of linguistic expressions, but a cognitive relation between an uttered word and semantics (of thought). I argue that whereas semantic content is holistic, meaning (in the sense of Burton-Roberts) is locally – i.e. contextually – constrained to a degree which, all things being equal, allows for successful communication. I argue that utterance interpretation is a wholly pragmatic inferential process, immediately constrained by a personal (i.e. holistic) inference about the communicative intention of a particular speaker in a particular conversational context. I claim that such a process of utterance interpretation can be implemented in terms of Hintzman’s (1986) multiple-trace theory of memory. In the second part, I illustrate my argument by an analysis of the relation between the word if and Material Implication (MI). I show that the claim (e.g. Grice 1989, Noh 2000) that if semantically encodes MI cannot be maintained. I argue that the application of MI has to be pragmatically determined and, therefore, when MI applies, it does so at the level of (holistic) thought – not at the (anyway problematic) linguistic semantic level. I explain the interpretation of conditionals in terms of Horton & Gerrig’s (2005) extension of a multiple-trace theory of memory into the study of common ground. I also discuss the implications of a wholly pragmatic account of utterance interpretation for the distinction between explicit and implicit communication.PhD bursary I received from the School of English Literature, Language and Linguistics and for conference grants offered by the School and by the Centre for Research in Linguistics and Language Sciences

The Multimodal Sentiment Analysis in Car Reviews (MuSe-CaR) Dataset: Collection, Insights and Improvements

Truly real-life data presents a strong, but exciting challenge for sentiment and emotion research. The high variety of possible `in-the-wild' properties makes large datasets such as these indispensable with respect to building robust machine learning models. A sufficient quantity of data covering a deep variety in the challenges of each modality to force the exploratory analysis of the interplay of all modalities has not yet been made available in this context. In this contribution, we present MuSe-CaR, a first of its kind multimodal dataset. The data is publicly available as it recently served as the testing bed for the 1st Multimodal Sentiment Analysis Challenge, and focused on the tasks of emotion, emotion-target engagement, and trustworthiness recognition by means of comprehensively integrating the audio-visual and language modalities. Furthermore, we give a thorough overview of the dataset in terms of collection and annotation, including annotation tiers not used in this year's MuSe 2020. In addition, for one of the sub-challenges - predicting the level of trustworthiness - no participant outperformed the baseline model, and so we propose a simple, but highly efficient Multi-Head-Attention network that exceeds using multimodal fusion the baseline by around 0.2 CCC (almost 50 % improvement).Comment: accepted versio

Mental content : consequences of the embodied mind paradigm

The central difference between objectivist cognitivist semantics and embodied cognition consists in the fact that the latter is, in contrast to the former, mindful of binding meaning to context-sensitive mental systems. According to Lakoff/Johnson's experientialism, conceptual structures arise from preconceptual kinesthetic image-schematic and basic-level structures. Gallese and Lakoff introduced the notion of exploiting sensorimotor structures for higherlevel cognition. Three different types of X-schemas realise three types of environmentally embedded simulation: Areas that control movements in peri-personal space; canonical neurons of the ventral premotor cortex that fire when a graspable object is represented; the firing of mirror neurons while perceiving certain movements of conspecifics. ..