Combination of Domain Knowledge and Deep Learning for Sentiment Analysis of Short and Informal Messages on Social Media

Sentiment analysis has been emerging recently as one of the major natural language processing (NLP) tasks in many applications. Especially, as social media channels (e.g. social networks or forums) have become significant sources for brands to observe user opinions about their products, this task is thus increasingly crucial. However, when applied with real data obtained from social media, we notice that there is a high volume of short and informal messages posted by users on those channels. This kind of data makes the existing works suffer from many difficulties to handle, especially ones using deep learning approaches. In this paper, we propose an approach to handle this problem. This work is extended from our previous work, in which we proposed to combine the typical deep learning technique of Convolutional Neural Networks with domain knowledge. The combination is used for acquiring additional training data augmentation and a more reasonable loss function. In this work, we further improve our architecture by various substantial enhancements, including negation-based data augmentation, transfer learning for word embeddings, the combination of word-level embeddings and character-level embeddings, and using multitask learning technique for attaching domain knowledge rules in the learning process. Those enhancements, specifically aiming to handle short and informal messages, help us to enjoy significant improvement in performance once experimenting on real datasets.Comment: A Preprint of an article accepted for publication by Inderscience in IJCVR on September 201

Stability of singular jump-linear systems with a large state space : a two-time-scale approach

This paper considers singular systems that involve both continuous dynamics and discrete events with the coefficients being modulated by a continuous-time Markov chain. The underlying systems have two distinct characteristics. First, the systems are singular, that is, characterized by a singular coefficient matrix. Second, the Markov chain of the modulating force has a large state space. We focus on stability of such hybrid singular systems. To carry out the analysis, we use a two-time-scale formulation, which is based on the rationale that, in a large-scale system, not all components or subsystems change at the same speed. To highlight the different rates of variation, we introduce a small parameter ε>0. Under suitable conditions, the system has a limit. We then use a perturbed Lyapunov function argument to show that if the limit system is stable then so is the original system in a suitable sense for ε small enough. This result presents a perspective on reduction of complexity from a stability point of view

Differences in the cellular response to acute spinal cord injury between developing and mature rats highlights the potential significance of the inflmmatory response

© 2017 Sutherland, Mathews, Mao, Nguyen and Gorrie. There exists a trend for a better functional recovery from spinal cord injury (SCI) in younger patients compared to adults, which is also reported for animal studies; however, the reasons for this are yet to be elucidated. The post injury tissue microenvironment is a complex milieu of cells and signals that interact on multiple levels. Inflammation has been shown to play a significant role in this post injury microenvironment. Endogenous neural progenitor cells (NPC), in the ependymal layer of the central canal, have also been shown to respond and migrate to the lesion site. This study used a mild contusion injury model to compare adult (9 week), juvenile (5 week) and infant (P7) Sprague-Dawley rats at 24 h, 1, 2, and 6 weeks post-injury (n = 108). The innate cells of the inflammatory response were examined using counts of ED1/IBA1 labeled cells. This found a decreased inflammatory response in the infants, compared to the adult and juvenile animals, demonstrated by a decreased neutrophil infiltration and macrophage and microglial activation at all 4 time points. Two other prominent cellular contributors to the post-injury microenvironment, the reactive astrocytes, which eventually form the glial scar, and the NPC were quantitated using GFAP and Nestin immunohistochemistry. After SCI in all 3 ages there was an obvious increase in Nestin staining in the ependymal layer, with long basal processes extending into the parenchyma. This was consistent between age groups early post injury then deviated at 2 weeks. The GFAP results also showed stark differences between the mature and infant animals. These results point to significant differences in the inflammatory response between infants and adults that may contribute to the better recovery indicated by other researchers, as well as differences in the overall injury progression and cellular responses. This may have important consequences if we are able to mirror and manipulate this response in patients of all ages; however much greater exploration in this area is required

HOMOE: A Memory-Based and Composition-Aware Framework for Zero-Shot Learning with Hopfield Network and Soft Mixture of Experts

Compositional Zero-Shot Learning (CZSL) has emerged as an essential paradigm in machine learning, aiming to overcome the constraints of traditional zero-shot learning by incorporating compositional thinking into its methodology. Conventional zero-shot learning has difficulty managing unfamiliar combinations of seen and unseen classes because it depends on pre-defined class embeddings. In contrast, Compositional Zero-Shot Learning uses the inherent hierarchies and structural connections among classes, creating new class representations by combining attributes, components, or other semantic elements. In our paper, we propose a novel framework that for the first time combines the Modern Hopfield Network with a Mixture of Experts (HOMOE) to classify the compositions of previously unseen objects. Specifically, the Modern Hopfield Network creates a memory that stores label prototypes and identifies relevant labels for a given input image. Following this, the Mixture of Expert models integrates the image with the fitting prototype to produce the final composition classification. Our approach achieves SOTA performance on several benchmarks, including MIT-States and UT-Zappos. We also examine how each component contributes to improved generalization