158,419 research outputs found
Learning Accurate Performance Predictors for Ultrafast Automated Model Compression
In this paper, we propose an ultrafast automated model compression framework
called SeerNet for flexible network deployment. Conventional
non-differen-tiable methods discretely search the desirable compression policy
based on the accuracy from exhaustively trained lightweight models, and
existing differentiable methods optimize an extremely large supernet to obtain
the required compressed model for deployment. They both cause heavy
computational cost due to the complex compression policy search and evaluation
process. On the contrary, we obtain the optimal efficient networks by directly
optimizing the compression policy with an accurate performance predictor, where
the ultrafast automated model compression for various computational cost
constraint is achieved without complex compression policy search and
evaluation. Specifically, we first train the performance predictor based on the
accuracy from uncertain compression policies actively selected by efficient
evolutionary search, so that informative supervision is provided to learn the
accurate performance predictor with acceptable cost. Then we leverage the
gradient that maximizes the predicted performance under the barrier complexity
constraint for ultrafast acquisition of the desirable compression policy, where
adaptive update stepsizes with momentum are employed to enhance optimality of
the acquired pruning and quantization strategy. Compared with the
state-of-the-art automated model compression methods, experimental results on
image classification and object detection show that our method achieves
competitive accuracy-complexity trade-offs with significant reduction of the
search cost.Comment: Accepted to IJC
Bayesian Optimization with Unknown Constraints
Recent work on Bayesian optimization has shown its effectiveness in global
optimization of difficult black-box objective functions. Many real-world
optimization problems of interest also have constraints which are unknown a
priori. In this paper, we study Bayesian optimization for constrained problems
in the general case that noise may be present in the constraint functions, and
the objective and constraints may be evaluated independently. We provide
motivating practical examples, and present a general framework to solve such
problems. We demonstrate the effectiveness of our approach on optimizing the
performance of online latent Dirichlet allocation subject to topic sparsity
constraints, tuning a neural network given test-time memory constraints, and
optimizing Hamiltonian Monte Carlo to achieve maximal effectiveness in a fixed
time, subject to passing standard convergence diagnostics.Comment: 14 pages, 3 figure
(k,q)-Compressed Sensing for dMRI with Joint Spatial-Angular Sparsity Prior
Advanced diffusion magnetic resonance imaging (dMRI) techniques, like
diffusion spectrum imaging (DSI) and high angular resolution diffusion imaging
(HARDI), remain underutilized compared to diffusion tensor imaging because the
scan times needed to produce accurate estimations of fiber orientation are
significantly longer. To accelerate DSI and HARDI, recent methods from
compressed sensing (CS) exploit a sparse underlying representation of the data
in the spatial and angular domains to undersample in the respective k- and
q-spaces. State-of-the-art frameworks, however, impose sparsity in the spatial
and angular domains separately and involve the sum of the corresponding sparse
regularizers. In contrast, we propose a unified (k,q)-CS formulation which
imposes sparsity jointly in the spatial-angular domain to further increase
sparsity of dMRI signals and reduce the required subsampling rate. To
efficiently solve this large-scale global reconstruction problem, we introduce
a novel adaptation of the FISTA algorithm that exploits dictionary
separability. We show on phantom and real HARDI data that our approach achieves
significantly more accurate signal reconstructions than the state of the art
while sampling only 2-4% of the (k,q)-space, allowing for the potential of new
levels of dMRI acceleration.Comment: To be published in the 2017 Computational Diffusion MRI Workshop of
MICCA
- …