144 research outputs found
Deep Expander Networks: Efficient Deep Networks from Graph Theory
Efficient CNN designs like ResNets and DenseNet were proposed to improve
accuracy vs efficiency trade-offs. They essentially increased the connectivity,
allowing efficient information flow across layers. Inspired by these
techniques, we propose to model connections between filters of a CNN using
graphs which are simultaneously sparse and well connected. Sparsity results in
efficiency while well connectedness can preserve the expressive power of the
CNNs. We use a well-studied class of graphs from theoretical computer science
that satisfies these properties known as Expander graphs. Expander graphs are
used to model connections between filters in CNNs to design networks called
X-Nets. We present two guarantees on the connectivity of X-Nets: Each node
influences every node in a layer in logarithmic steps, and the number of paths
between two sets of nodes is proportional to the product of their sizes. We
also propose efficient training and inference algorithms, making it possible to
train deeper and wider X-Nets effectively.
Expander based models give a 4% improvement in accuracy on MobileNet over
grouped convolutions, a popular technique, which has the same sparsity but
worse connectivity. X-Nets give better performance trade-offs than the original
ResNet and DenseNet-BC architectures. We achieve model sizes comparable to
state-of-the-art pruning techniques using our simple architecture design,
without any pruning. We hope that this work motivates other approaches to
utilize results from graph theory to develop efficient network architectures.Comment: ECCV'1
Scalable Full Flow with Learned Binary Descriptors
We propose a method for large displacement optical flow in which local
matching costs are learned by a convolutional neural network (CNN) and a
smoothness prior is imposed by a conditional random field (CRF). We tackle the
computation- and memory-intensive operations on the 4D cost volume by a
min-projection which reduces memory complexity from quadratic to linear and
binary descriptors for efficient matching. This enables evaluation of the cost
on the fly and allows to perform learning and CRF inference on high resolution
images without ever storing the 4D cost volume. To address the problem of
learning binary descriptors we propose a new hybrid learning scheme. In
contrast to current state of the art approaches for learning binary CNNs we can
compute the exact non-zero gradient within our model. We compare several
methods for training binary descriptors and show results on public available
benchmarks.Comment: GCPR 201
Identification of Nursing Management Planning Standards in Iran
BACKGROUND: In recent century, planning is one of the most important care skills for health service development.
Nurses should be ready scientifically for managing situations in order to develop and perform appropriate models for
patient care. It is necessary for manager to know about the process and standards of planning and how to apply them in
real conditions. With regard to importance of health care planning and lack of nursing management planning accreditation
in Iran.
METHODS: This triangulating research was carried out between 2004 and 2006. Fifty professional nursing managers in
different level of Iran medical universities and central hospitals, having experience in nursing management at least for
five years, and an MS or BS degree in nursing management, were included in a study through purposed sampling. At
first, a pilot study with an open questionnaire was conducted in Isfahan and Shiraz and then the study went on in 3
phases including searching for international standards in method by 15 professional nursing manager after consensus on
70% and sending the final was used for data analysis. At last standards for nursing management planning were obtained
in Iran.
RESULTS: 48 standards of nursing management planning were obtained. The findings showed that most of the standards
were accepted (90%) and there were not any standards with agreement lower than 70%. These standards, accompanied
with the standards of other nursing units, can be used for quality improvement. The researcher suggests the ministry of
health to use these findings and other related ones to accredit the nursing system and to identify its deficiencies
Detection of denial of service attacks against domain name system using neural networks
In this paper we introduce an intrusion detection system for Denial of Service (DoS) attacks against Domain Name System (DNS). Our system architecture consists of two most important parts: a statistical preprocessor and a neural network classifier. The preprocessor extracts required statistical features in a shorttime frame from traffic received by the target name server. We compared three different neural networks for detecting and classifying different types of DoS attacks. The proposed system is evaluated in a simulated network and showed that the best performed neural network is a feed-forward backpropagation with an accuracy of 99%
End-to-End Localization and Ranking for Relative Attributes
We propose an end-to-end deep convolutional network to simultaneously
localize and rank relative visual attributes, given only weakly-supervised
pairwise image comparisons. Unlike previous methods, our network jointly learns
the attribute's features, localization, and ranker. The localization module of
our network discovers the most informative image region for the attribute,
which is then used by the ranking module to learn a ranking model of the
attribute. Our end-to-end framework also significantly speeds up processing and
is much faster than previous methods. We show state-of-the-art ranking results
on various relative attribute datasets, and our qualitative localization
results clearly demonstrate our network's ability to learn meaningful image
patches.Comment: Appears in European Conference on Computer Vision (ECCV), 201
Foothill: A Quasiconvex Regularization for Edge Computing of Deep Neural Networks
Deep neural networks (DNNs) have demonstrated success for many supervised
learning tasks, ranging from voice recognition, object detection, to image
classification. However, their increasing complexity might yield poor
generalization error that make them hard to be deployed on edge devices.
Quantization is an effective approach to compress DNNs in order to meet these
constraints. Using a quasiconvex base function in order to construct a binary
quantizer helps training binary neural networks (BNNs) and adding noise to the
input data or using a concrete regularization function helps to improve
generalization error. Here we introduce foothill function, an infinitely
differentiable quasiconvex function. This regularizer is flexible enough to
deform towards and penalties. Foothill can be used as a binary
quantizer, as a regularizer, or as a loss. In particular, we show this
regularizer reduces the accuracy gap between BNNs and their full-precision
counterpart for image classification on ImageNet.Comment: Accepted in 16th International Conference of Image Analysis and
Recognition (ICIAR 2019
BATS: Binary ArchitecTure Search
This paper proposes Binary ArchitecTure Search (BATS), a framework that
drastically reduces the accuracy gap between binary neural networks and their
real-valued counterparts by means of Neural Architecture Search (NAS). We show
that directly applying NAS to the binary domain provides very poor results. To
alleviate this, we describe, to our knowledge, for the first time, the 3 key
ingredients for successfully applying NAS to the binary domain. Specifically,
we (1) introduce and design a novel binary-oriented search space, (2) propose a
new mechanism for controlling and stabilising the resulting searched
topologies, (3) propose and validate a series of new search strategies for
binary networks that lead to faster convergence and lower search times.
Experimental results demonstrate the effectiveness of the proposed approach and
the necessity of searching in the binary space directly. Moreover, (4) we set a
new state-of-the-art for binary neural networks on CIFAR10, CIFAR100 and
ImageNet datasets. Code will be made available
https://github.com/1adrianb/binary-nasComment: accepted to ECCV 202
3DQ: Compact Quantized Neural Networks for Volumetric Whole Brain Segmentation
Model architectures have been dramatically increasing in size, improving
performance at the cost of resource requirements. In this paper we propose 3DQ,
a ternary quantization method, applied for the first time to 3D Fully
Convolutional Neural Networks (F-CNNs), enabling 16x model compression while
maintaining performance on par with full precision models. We extensively
evaluate 3DQ on two datasets for the challenging task of whole brain
segmentation. Additionally, we showcase our method's ability to generalize on
two common 3D architectures, namely 3D U-Net and V-Net. Outperforming a variety
of baselines, the proposed method is capable of compressing large 3D models to
a few MBytes, alleviating the storage needs in space critical applications.Comment: Accepted to MICCAI 201
(Dynamic) Prompting might be all you need to repair Compressed LLMs
Large language models (LLMs), while transformative for NLP, come with
significant computational demands, underlining the need for efficient,
training-free compression. Notably, despite the marked improvement in
training-free compression for the largest of LLMs, our tests using LLaMA-7B and
OPT-6.7b highlight a significant performance drop in several realistic
downstream tasks. Investigation into the trade-off between resource-intensive
post-compression re-training highlights the prospect of prompt-driven recovery
as a lightweight adaption tool. However, existing studies, confined mainly to
perplexity evaluations and simple tasks, fail to offer unequivocal confidence
in the scalability and generalizability of prompting. We tackle this
uncertainty in two key ways. First, we uncover the vulnerability of naive
prompts in LLM compression as an over-reliance on a singular prompt per input.
In response, we propose inference-time dynamic prompting (IDP), a mechanism
that autonomously chooses from a set of curated prompts based on the context of
each individual input. Second, we delve into a scientific understanding of why
"prompting might be all you need post-LLM compression." Our findings suggest
that compression does not irretrievably erase LLM model knowledge but displace
it, necessitating a new inference path. IDP effectively redirects this path,
enabling the model to tap into its inherent yet displaced knowledge and thereby
recover performance. Empirical tests affirm the value of IDP, demonstrating an
average performance improvement of 1.24% across nine varied tasks spanning
multiple knowledge domains
Organic and Conventional Vegetable Production in Oklahoma
This study compares he profitability and risk related to conventional and organic vegetable production systems A linear programming model was used to find the optimal mix of vegetables in both production systems. And a target MOTAD (minimization of total absolute deviation) model was used to perform risk analysis in both organic and conventional production systemsCrop Production/Industries, Research Methods/ Statistical Methods,
- …