Finding the SWEET Spot: Analysis and Improvement of Adaptive Inference in Low Resource Settings

Adaptive inference is a simple method for reducing inference costs. The method works by maintaining multiple classifiers of different capacities, and allocating resources to each test instance according to its difficulty. In this work, we compare the two main approaches for adaptive inference, Early-Exit and Multi-Model, when training data is limited. First, we observe that for models with the same architecture and size, individual Multi-Model classifiers outperform their Early-Exit counterparts by an average of 2.3%. We show that this gap is caused by Early-Exit classifiers sharing model parameters during training, resulting in conflicting gradient updates of model weights. We find that despite this gap, Early-Exit still provides a better speed-accuracy trade-off due to the overhead of the Multi-Model approach. To address these issues, we propose SWEET (Separating Weights in Early Exit Transformers), an Early-Exit fine-tuning method that assigns each classifier its own set of unique model weights, not updated by other classifiers. We compare SWEET's speed-accuracy curve to standard Early-Exit and Multi-Model baselines and find that it outperforms both methods at fast speeds while maintaining comparable scores to Early-Exit at slow speeds. Moreover, SWEET individual classifiers outperform Early-Exit ones by 1.1% on average. SWEET enjoys the benefits of both methods, paving the way for further reduction of inference costs in NLP.Comment: Proceedings of ACL 202

Reconstruction of Quantitative Acoustic Microscopy Images from RF Signals Sampled at Innovation Rate

The principle of quantitative acoustic microscopy (QAM) is to form two-dimensional acoustic parameter maps from a collection of radiofrequency (RF) signals acquired by raster scanning a biological sample. Despite their relatively simple structure, i.e. two main reflections, QAM RF signals are currently sampled at very high frequencies, e.g., at 2.5 GHz for QAM system employing a single-element transducer with a center frequency of 250-MHz. The use of such high sampling frequencies is challenging because of the potentially large amount of acquired data and the cost of the necessary analog to digital converters. In this work, we propose a sampling scheme based on the finite rate of innovation theory that exploits the limited numbers of degrees of freedom of QAM RF signals and allows the reconstruction of accurate acoustic maps from a very limited number of samples

Approximate message passing reconstruction of quantitative acoustic microscopy images.

A novel framework for compressive sensing (CS) data acquisition and reconstruction in quantitative acoustic microscopy (QAM) is presented. Three different CS patterns, adapted to the specifics of QAM systems, were investigated as an alternative to the current raster-scanning approach. They consist of diagonal sampling, a row random, and a spiral scanning pattern and can all significantly reduce both the acquisition time and the amount of sampled data. For subsequent image reconstruction, we design and implement an innovative technique, whereby a recently proposed approximate message passing method is adapted to account for the underlying data statistics. A Cauchy maximum a posteriori image denoising algorithm is thus employed to account for the non-Gaussianity of QAM wavelet coefficients. The proposed methods were tested retrospectively on experimental data acquired with a 250- or 500-MHz QAM system. The experimental data were obtained from a human lymph node sample (250 MHz) and human cornea (500 MHz). Reconstruction results showed that the best performance is obtained using a spiral sensing pattern combined with the Cauchy denoiser in the wavelet domain. The spiral sensing matrix reduced the number of spatial samples by a factor of 2 and led to an excellent peak signal-to-noise ratio of 43.21 dB when reconstructing QAM speed-of-sound images of a human lymph node. These results demonstrate that the CS approach could significantly improve scanning time, while reducing costs of future QAM systems