35,444 research outputs found
Design of Kernels in Convolutional Neural Networks for Image Classification
Despite the effectiveness of Convolutional Neural Networks (CNNs) for image
classification, our understanding of the relationship between shape of
convolution kernels and learned representations is limited. In this work, we
explore and employ the relationship between shape of kernels which define
Receptive Fields (RFs) in CNNs for learning of feature representations and
image classification. For this purpose, we first propose a feature
visualization method for visualization of pixel-wise classification score maps
of learned features. Motivated by our experimental results, and observations
reported in the literature for modeling of visual systems, we propose a novel
design of shape of kernels for learning of representations in CNNs. In the
experimental results, we achieved a state-of-the-art classification performance
compared to a base CNN model [28] by reducing the number of parameters and
computational time of the model using the ILSVRC-2012 dataset [24]. The
proposed models also outperform the state-of-the-art models employed on the
CIFAR-10/100 datasets [12] for image classification. Additionally, we analyzed
the robustness of the proposed method to occlusion for classification of
partially occluded images compared with the state-of-the-art methods. Our
results indicate the effectiveness of the proposed approach. The code is
available in github.com/minogame/caffe-qhconv
Graph Embedding with Rich Information through Heterogeneous Network
Graph embedding has attracted increasing attention due to its critical
application in social network analysis. Most existing algorithms for graph
embedding only rely on the typology information and fail to use the copious
information in nodes as well as edges. As a result, their performance for many
tasks may not be satisfactory. In this paper, we proposed a novel and general
framework of representation learning for graph with rich text information
through constructing a bipartite heterogeneous network. Specially, we designed
a biased random walk to explore the constructed heterogeneous network with the
notion of flexible neighborhood. The efficacy of our method is demonstrated by
extensive comparison experiments with several baselines on various datasets. It
improves the Micro-F1 and Macro-F1 of node classification by 10% and 7% on Cora
dataset.Comment: 9 pages, 7 figures, 4 table
Exploring Shared Structures and Hierarchies for Multiple NLP Tasks
Designing shared neural architecture plays an important role in multi-task
learning. The challenge is that finding an optimal sharing scheme heavily
relies on the expert knowledge and is not scalable to a large number of diverse
tasks. Inspired by the promising work of neural architecture search (NAS), we
apply reinforcement learning to automatically find possible shared architecture
for multi-task learning. Specifically, we use a controller to select from a set
of shareable modules and assemble a task-specific architecture, and repeat the
same procedure for other tasks. The controller is trained with reinforcement
learning to maximize the expected accuracies for all tasks. We conduct
extensive experiments on two types of tasks, text classification and sequence
labeling, which demonstrate the benefits of our approach.Comment:
Visual and Textual Sentiment Analysis Using Deep Fusion Convolutional Neural Networks
Sentiment analysis is attracting more and more attentions and has become a
very hot research topic due to its potential applications in personalized
recommendation, opinion mining, etc. Most of the existing methods are based on
either textual or visual data and can not achieve satisfactory results, as it
is very hard to extract sufficient information from only one single modality
data. Inspired by the observation that there exists strong semantic correlation
between visual and textual data in social medias, we propose an end-to-end deep
fusion convolutional neural network to jointly learn textual and visual
sentiment representations from training examples. The two modality information
are fused together in a pooling layer and fed into fully-connected layers to
predict the sentiment polarity. We evaluate the proposed approach on two widely
used data sets. Results show that our method achieves promising result compared
with the state-of-the-art methods which clearly demonstrate its competency.Comment: Accepted as oral presentation by ICIP201
NeuNetS: An Automated Synthesis Engine for Neural Network Design
Application of neural networks to a vast variety of practical applications is
transforming the way AI is applied in practice. Pre-trained neural network
models available through APIs or capability to custom train pre-built neural
network architectures with customer data has made the consumption of AI by
developers much simpler and resulted in broad adoption of these complex AI
models. While prebuilt network models exist for certain scenarios, to try and
meet the constraints that are unique to each application, AI teams need to
think about developing custom neural network architectures that can meet the
tradeoff between accuracy and memory footprint to achieve the tight constraints
of their unique use-cases. However, only a small proportion of data science
teams have the skills and experience needed to create a neural network from
scratch, and the demand far exceeds the supply. In this paper, we present
NeuNetS : An automated Neural Network Synthesis engine for custom neural
network design that is available as part of IBM's AI OpenScale's product.
NeuNetS is available for both Text and Image domains and can build neural
networks for specific tasks in a fraction of the time it takes today with human
effort, and with accuracy similar to that of human-designed AI models.Comment: 14 pages, 12 figures. arXiv admin note: text overlap with
arXiv:1806.0025
A short review on Applications of Deep learning for Cyber security
Deep learning is an advanced model of traditional machine learning. This has
the capability to extract optimal feature representation from raw input
samples. This has been applied towards various use cases in cyber security such
as intrusion detection, malware classification, android malware detection, spam
and phishing detection and binary analysis. This paper outlines the survey of
all the works related to deep learning based solutions for various cyber
security use cases. Keywords: Deep learning, intrusion detection, malware
detection, Android malware detection, spam & phishing detection, traffic
analysis, binary analysis.Comment: 15 page
Data Curation with Deep Learning [Vision]
Data curation - the process of discovering, integrating, and cleaning data -
is one of the oldest, hardest, yet inevitable data management problems. Despite
decades of efforts from both researchers and practitioners, it is still one of
the most time consuming and least enjoyable work of data scientists. In most
organizations, data curation plays an important role so as to fully unlock the
value of big data. Unfortunately, the current solutions are not keeping up with
the ever-changing data ecosystem, because they often require substantially high
human cost. Meanwhile, deep learning is making strides in achieving remarkable
successes in multiple areas, such as image recognition, natural language
processing, and speech recognition. In this vision paper, we explore how some
of the fundamental innovations in deep learning could be leveraged to improve
existing data curation solutions and to help build new ones. In particular, we
provide a thorough overview of the current deep learning landscape, and
identify interesting research opportunities and dispel common myths. We hope
that the synthesis of these important domains will unleash a series of research
activities that will lead to significantly improved solutions for many data
curation tasks
BENCHIP: Benchmarking Intelligence Processors
The increasing attention on deep learning has tremendously spurred the design
of intelligence processing hardware. The variety of emerging intelligence
processors requires standard benchmarks for fair comparison and system
optimization (in both software and hardware). However, existing benchmarks are
unsuitable for benchmarking intelligence processors due to their non-diversity
and nonrepresentativeness. Also, the lack of a standard benchmarking
methodology further exacerbates this problem. In this paper, we propose
BENCHIP, a benchmark suite and benchmarking methodology for intelligence
processors. The benchmark suite in BENCHIP consists of two sets of benchmarks:
microbenchmarks and macrobenchmarks. The microbenchmarks consist of
single-layer networks. They are mainly designed for bottleneck analysis and
system optimization. The macrobenchmarks contain state-of-the-art industrial
networks, so as to offer a realistic comparison of different platforms. We also
propose a standard benchmarking methodology built upon an industrial software
stack and evaluation metrics that comprehensively reflect the various
characteristics of the evaluated intelligence processors. BENCHIP is utilized
for evaluating various hardware platforms, including CPUs, GPUs, and
accelerators. BENCHIP will be open-sourced soon.Comment: 37pages, 14 figure
Automatic Question-Answering Using A Deep Similarity Neural Network
Automatic question-answering is a classical problem in natural language
processing, which aims at designing systems that can automatically answer a
question, in the same way as human does. In this work, we propose a deep
learning based model for automatic question-answering. First the questions and
answers are embedded using neural probabilistic modeling. Then a deep
similarity neural network is trained to find the similarity score of a pair of
answer and question. Then for each question, the best answer is found as the
one with the highest similarity score. We first train this model on a
large-scale public question-answering database, and then fine-tune it to
transfer to the customer-care chat data. We have also tested our framework on a
public question-answering database and achieved very good performance
Large Scale Font Independent Urdu Text Recognition System
OCR algorithms have received a significant improvement in performance
recently, mainly due to the increase in the capabilities of artificial
intelligence algorithms. However, this advancement is not evenly distributed
over all languages. Urdu is among the languages which did not receive much
attention, especially in the font independent perspective. There exists no
automated system that can reliably recognize printed Urdu text in images and
videos across different fonts. To help bridge this gap, we have developed
Qaida, a large scale data set with 256 fonts, and a complete Urdu lexicon. We
have also developed a Convolutional Neural Network (CNN) based classification
model which can recognize Urdu ligatures with 84.2% accuracy. Moreover, we
demonstrate that our recognition network can not only recognize the text in the
fonts it is trained on but can also reliably recognize text in unseen (new)
fonts. To this end, this paper makes following contributions: (i) we introduce
a large scale, multiple fonts based data set for printed Urdu text
recognition;(ii) we have designed, trained and evaluated a CNN based model for
Urdu text recognition; (iii) we experiment with incremental learning methods to
produce state-of-the-art results for Urdu text recognition. All the experiment
choices were thoroughly validated via detailed empirical analysis. We believe
that this study can serve as the basis for further improvement in the
performance of font independent Urdu OCR systems
- …