Convex Cauchy Schwarz Independent Component Analysis for Blind Source Separation

We present a new high performance Convex Cauchy Schwarz Divergence (CCS DIV) measure for Independent Component Analysis (ICA) and Blind Source Separation (BSS). The CCS DIV measure is developed by integrating convex functions into the Cauchy Schwarz inequality. By including a convexity quality parameter, the measure has a broad control range of its convexity curvature. With this measure, a new CCS ICA algorithm is structured and a non parametric form is developed incorporating the Parzen window based distribution. Furthermore, pairwise iterative schemes are employed to tackle the high dimensional problem in BSS. We present two schemes of pairwise non parametric ICA algorithms, one is based on gradient decent and the second on the Jacobi Iterative method. Several case study scenarios are carried out on noise free and noisy mixtures of speech and music signals. Finally, the superiority of the proposed CCS ICA algorithm is demonstrated in metric comparison performance with FastICA, RobustICA, convex ICA (C ICA), and other leading existing algorithms.Comment: 13 page

Mining Data with Feature Interactions

abstract: Models using feature interactions have been applied successfully in many areas such as biomedical analysis, recommender systems. The popularity of using feature interactions mainly lies in (1) they are able to capture the nonlinearity of the data compared with linear effects and (2) they enjoy great interpretability. In this thesis, I propose a series of formulations using feature interactions for real world problems and develop efficient algorithms for solving them. Specifically, I first propose to directly solve the non-convex formulation of the weak hierarchical Lasso which imposes weak hierarchy on individual features and interactions but can only be approximately solved by a convex relaxation in existing studies. I further propose to use the non-convex weak hierarchical Lasso formulation for hypothesis testing on the interaction features with hierarchical assumptions. Secondly, I propose a type of bi-linear models that take advantage of interactions of features for drug discovery problems where specific drug-drug pairs or drug-disease pairs are of interest. These models are learned by maximizing the number of positive data pairs that rank above the average score of unlabeled data pairs. Then I generalize the method to the case of using the top-ranked unlabeled data pairs for representative construction and derive an efficient algorithm for the extended formulation. Last but not least, motivated by a special form of bi-linear models, I propose a framework that enables simultaneously subgrouping data points and building specific models on the subgroups for learning on massive and heterogeneous datasets. Experiments on synthetic and real datasets are conducted to demonstrate the effectiveness or efficiency of the proposed methods.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Modeling Text with Graph Convolutional Network for Cross-Modal Information Retrieval

Cross-modal information retrieval aims to find heterogeneous data of various modalities from a given query of one modality. The main challenge is to map different modalities into a common semantic space, in which distance between concepts in different modalities can be well modeled. For cross-modal information retrieval between images and texts, existing work mostly uses off-the-shelf Convolutional Neural Network (CNN) for image feature extraction. For texts, word-level features such as bag-of-words or word2vec are employed to build deep learning models to represent texts. Besides word-level semantics, the semantic relations between words are also informative but less explored. In this paper, we model texts by graphs using similarity measure based on word2vec. A dual-path neural network model is proposed for couple feature learning in cross-modal information retrieval. One path utilizes Graph Convolutional Network (GCN) for text modeling based on graph representations. The other path uses a neural network with layers of nonlinearities for image modeling based on off-the-shelf features. The model is trained by a pairwise similarity loss function to maximize the similarity of relevant text-image pairs and minimize the similarity of irrelevant pairs. Experimental results show that the proposed model outperforms the state-of-the-art methods significantly, with 17% improvement on accuracy for the best case.Comment: 7 pages, 11 figure

Muscle synergy analysis of lower-limb movements

Dissertação de mestrado integrado em Biomedical Engineering (área de especialização em Medical Electronics)Neurological disorders and trauma often lead to impaired lower-limb motor coordination. Understanding how muscles combine to produce movement can directly benefit assistive solutions to those afflicted with these impairments. A theory in neuromusculoskeletal research, known as muscle synergies, has shown promising results in applications for this field. This hypothesis postulates that the Central Nervous System controls motor tasks through the time-variant combinations of modules (or synergies), each representing the co-activation of a group of muscles. There is, however, no unifying, evidence-based framework to ascertain muscle synergies, as synergy extraction methods vary greatly in the literature. Publications also focus on gait analysis, leaving a knowledge gap when concerning motor tasks important to daily life such as sitting and standing. The purpose of this dissertation is the development of a robust, evidence-based, task-generic synergy extraction framework unifying the divergent methodologies of this field of study, and to use this framework to study healthy muscle synergies on several activities of daily living: walking, sit-to-stand, stand-to-sit and knee flexion and extension. This was achieved by designing and implementing a cross-validated Non-Negative Matrix Factorization process and applying it to muscle electrical activity data. A preliminary study was undertaken to tune this configuration regarding cross-validating proportions, data structuring prior to factorization and evaluating criteria quantifying accuracy in modularity findings. Muscle synergies results were then investigated for different performing speeds to determine if their structure differed, and for consistency across subjects, to ascertain if a common set of muscle synergies underlay control on all subjects equally. Results revealed that the implemented framework was consistent in its ability to capture modularity (p < 0:05). The movements’ synergies also did not differ across the studied range of speeds (except one module in Knee Flexion) (p < 0:05). Additionally, a common set of muscle synergies was present across several subjects (p < 0:05), but shared commonality across every participant was only observed for the walking trials, for which much larger amounts of data were collected. Overall, the established framework is versatile and applicable for different lower-limb movements; muscle synergies findings for the examined movements may also be used as control references in assistive devices.As perturbações e traumas neurológicos afetam frequentemente a coordenação motora dos membros inferiores. Uma teoria recente em investigação neuromusculo-esquelética, denominada de sinergias musculares, tem demonstrado resultados promissores em soluções de assistência à população afetada por estes distúrbios. Esta teoria propõe que o Sistema Nervoso Central controla as tarefas motoras através de combinações variantes no tempo de módulos (ou sinergias), sendo que cada um representa a co-ativação de um grupo de músculos. No entanto, não existe nenhum processo uniformizante, empiricamente justificado para determinar sinergias musculares, porque os métodos de extração de sinergias variam muito na literatura. Para além disso, as publicações normalmente focam-se em análise da marcha, deixando uma lacuna de conhecimento em tarefas motoras do dia-a-dia, tais como sentar e levantar. O objetivo desta dissertação é o desenvolvimento de um processo robusto, genérico e empiricamente justificado de extração de sinergias em várias tarefas motoras, unindo as metodologias divergentes neste campo de estudo, e subsequentemente utilizar este processo para estudar sinergias musculares de sujeitos saudáveis em várias atividades do dia-a-dia: marcha, erguer-se de pé partir de uma posição sentada, sentar-se a partir de uma posição de pé e extensão e flexão do joelho. Isto foi alcançado através da implementação de um processo de cross-validated Non-Negative Matrix Factorization e subsequente aplicação em dados de atividade elétrica muscular. Um estudo preliminar foi realizado para configurar este processo relativamente às proporções de cross-validation, estruturação de dados antes da fatorização e seleção de critério que quantifique o sucesso da representação modular dos dados. Os resultados da extração de sinergias de diferentes velocidades de execução foram depois examinados no sentido de descobrir se este fator influenciava a estrutura dos módulos motores, assim como se semelhanças entre as sinergias de diferentes sujeitos apontavam para um conjunto comum de sinergias musculares subjacente ao controlo do movimento. Os resultados revelaram que o processo implementado foi consistente na sua capacidade de capturar a modularidade nos dados recolhidos (p < 0:05). As sinergias de todos os movimentos também não diferiram para toda a gama de velocidades estudada (exceto um módulo na flexão do joelho) (p < 0:05). Por fim, um conjunto comum de sinergias musculares esteve presente em vários sujeitos (p < 0:05), mas só esteve presente em todos os sujeitos de igual forma para a marcha, para a qual a quantidade de dados recolhida foi muito maior. Globalmente, o processo implementado é versátil e aplicável a diferentes movimentos dos membros inferiores; os resultados das sinergias musculares para os movimentos examinados podem também ser utilizado como referências de controlo para dispositivos de assistência

Real Time Changes in Monetary Policy

This paper investigates potential changes in monetary policy over the last decades using a nonparametric vector autoregression model. In the proposed model, the conditional mean and variance are time-dependent and estimated using a nonparametric local linear method, which allows for different forms of nonlinearity, conditional heteroskedasticity, and non-normality. Our results suggest that there have been gradual and abrupt changes in the variances of shocks, in the monetary transmission mechanism, and in the Fed’s reaction function. The response of output was strongest during Volcker’s disinflationary period and has since been slowly decreasing over time. There have been some abrupt changes in the response of inflation, especially in the early 1980s, but we can not conclude that it is weaker now than in previous periods. Finally, we find significant evidence that policy was passive during some parts of Burn’s period, and active during Volcker’s disinflationary period and Greenspan’s period. However, we find that the uncovered behavior of the parameters is more complex than general conclusions suggest, since they display considerable nonlinearities over time. A particular appeal of the recursive estimation of the proposed VAR-ARCH is the detection of discrete local deviations as well as more gradual ones, without smoothing the timing or magnitude of the changes.Monetary Policy, Taylor Rule, Local Estimation, Nonlinearity, Nonparametric, Monetary Policy; Taylor Rule; Local Estimation; Nonlinearity; Nonparametric; Structural Vector Autoregression; Autoregressive Conditional Heteroskedasticity;

Network Enhancement: a general method to denoise weighted biological networks

Networks are ubiquitous in biology where they encode connectivity patterns at all scales of organization, from molecular to the biome. However, biological networks are noisy due to the limitations of measurement technology and inherent natural variation, which can hamper discovery of network patterns and dynamics. We propose Network Enhancement (NE), a method for improving the signal-to-noise ratio of undirected, weighted networks. NE uses a doubly stochastic matrix operator that induces sparsity and provides a closed-form solution that increases spectral eigengap of the input network. As a result, NE removes weak edges, enhances real connections, and leads to better downstream performance. Experiments show that NE improves gene function prediction by denoising tissue-specific interaction networks, alleviates interpretation of noisy Hi-C contact maps from the human genome, and boosts fine-grained identification accuracy of species. Our results indicate that NE is widely applicable for denoising biological networks

A Comprehensive Survey on Cross-modal Retrieval

In recent years, cross-modal retrieval has drawn much attention due to the rapid growth of multimodal data. It takes one type of data as the query to retrieve relevant data of another type. For example, a user can use a text to retrieve relevant pictures or videos. Since the query and its retrieved results can be of different modalities, how to measure the content similarity between different modalities of data remains a challenge. Various methods have been proposed to deal with such a problem. In this paper, we first review a number of representative methods for cross-modal retrieval and classify them into two main groups: 1) real-valued representation learning, and 2) binary representation learning. Real-valued representation learning methods aim to learn real-valued common representations for different modalities of data. To speed up the cross-modal retrieval, a number of binary representation learning methods are proposed to map different modalities of data into a common Hamming space. Then, we introduce several multimodal datasets in the community, and show the experimental results on two commonly used multimodal datasets. The comparison reveals the characteristic of different kinds of cross-modal retrieval methods, which is expected to benefit both practical applications and future research. Finally, we discuss open problems and future research directions.Comment: 20 pages, 11 figures, 9 table

Semantic Concept Co-Occurrence Patterns for Image Annotation and Retrieval.

Describing visual image contents by semantic concepts is an effective and straightforward way to facilitate various high level applications. Inferring semantic concepts from low-level pictorial feature analysis is challenging due to the semantic gap problem, while manually labeling concepts is unwise because of a large number of images in both online and offline collections. In this paper, we present a novel approach to automatically generate intermediate image descriptors by exploiting concept co-occurrence patterns in the pre-labeled training set that renders it possible to depict complex scene images semantically. Our work is motivated by the fact that multiple concepts that frequently co-occur across images form patterns which could provide contextual cues for individual concept inference. We discover the co-occurrence patterns as hierarchical communities by graph modularity maximization in a network with nodes and edges representing concepts and co-occurrence relationships separately. A random walk process working on the inferred concept probabilities with the discovered co-occurrence patterns is applied to acquire the refined concept signature representation. Through experiments in automatic image annotation and semantic image retrieval on several challenging datasets, we demonstrate the effectiveness of the proposed concept co-occurrence patterns as well as the concept signature representation in comparison with state-of-the-art approaches

Recent Advance in Content-based Image Retrieval: A Literature Survey

The explosive increase and ubiquitous accessibility of visual data on the Web have led to the prosperity of research activity in image search or retrieval. With the ignorance of visual content as a ranking clue, methods with text search techniques for visual retrieval may suffer inconsistency between the text words and visual content. Content-based image retrieval (CBIR), which makes use of the representation of visual content to identify relevant images, has attracted sustained attention in recent two decades. Such a problem is challenging due to the intention gap and the semantic gap problems. Numerous techniques have been developed for content-based image retrieval in the last decade. The purpose of this paper is to categorize and evaluate those algorithms proposed during the period of 2003 to 2016. We conclude with several promising directions for future research.Comment: 22 page

Similarity of Semantic Relations

There are at least two kinds of similarity. Relational similarity is correspondence between relations, in contrast with attributional similarity, which is correspondence between attributes. When two words have a high degree of attributional similarity, we call them synonyms. When two pairs of words have a high degree of relational similarity, we say that their relations are analogous. For example, the word pair mason:stone is analogous to the pair carpenter:wood. This paper introduces Latent Relational Analysis (LRA), a method for measuring relational similarity. LRA has potential applications in many areas, including information extraction, word sense disambiguation, and information retrieval. Recently the Vector Space Model (VSM) of information retrieval has been adapted to measuring relational similarity, achieving a score of 47% on a collection of 374 college-level multiple-choice word analogy questions. In the VSM approach, the relation between a pair of words is characterized by a vector of frequencies of predefined patterns in a large corpus. LRA extends the VSM approach in three ways: (1) the patterns are derived automatically from the corpus, (2) the Singular Value Decomposition (SVD) is used to smooth the frequency data, and (3) automatically generated synonyms are used to explore variations of the word pairs. LRA achieves 56% on the 374 analogy questions, statistically equivalent to the average human score of 57%. On the related problem of classifying semantic relations, LRA achieves similar gains over the VSM