USM-Lite: Quantization and Sparsity Aware Fine-tuning for Speech Recognition with Universal Speech Models

End-to-end automatic speech recognition (ASR) models have seen revolutionary quality gains with the recent development of large-scale universal speech models (USM). However, deploying these massive USMs is extremely expensive due to the enormous memory usage and computational cost. Therefore, model compression is an important research topic to fit USM-based ASR under budget in real-world scenarios. In this study, we propose a USM fine-tuning approach for ASR, with a low-bit quantization and N:M structured sparsity aware paradigm on the model weights, reducing the model complexity from parameter precision and matrix topology perspectives. We conducted extensive experiments with a 2-billion parameter USM on a large-scale voice search dataset to evaluate our proposed method. A series of ablation studies validate the effectiveness of up to int4 quantization and 2:4 sparsity. However, a single compression technique fails to recover the performance well under extreme setups including int2 quantization and 1:4 sparsity. By contrast, our proposed method can compress the model to have 9.4% of the size, at the cost of only 7.3% relative word error rate (WER) regressions. We also provided in-depth analyses on the results and discussions on the limitations and potential solutions, which would be valuable for future studies.Comment: Accepted by ICASSP 2024. Preprin

Lu-177-PSMA dosimetry for kidneys and tumors based on SPECT images at two imaging time points

BackgroundPersonalized dosimetry for Lu-177-PSMA treatment requires multiple-time-point SPECT/CT scans to calculate time-integrated activity (TIA). This study evaluates two-time-point (TTP) methods for TIA calculation for kidneys and tumors.MethodsA total of 18 patients treated with 3.7-7.4 GBq Lu-177 PSMA-617 were analyzed retrospectively, including 18 sets of left and right kidneys, as well as 45 tumors. Four quantitative SPECT/CT (4TP) were acquired at 2 h, 20 h, 40 h, 60 h (n = 11), or 200 h (n = 7) after treatment, and they were fit bi-exponentially as reference. The TTP method was fitted by a mono-exponential washout function using two selected imaging time points for kidneys. For tumors, one uptake and one washout phase were modeled, assuming linear (type I) and same (type II) uptake phase between 0 h to the first time point and mono-exponential washout thereafter. Two single-time-point (STP) methods were also implemented for comparison. TIA calculated by TTP and STP methods were compared with reference to the 4TP TIA.ResultsFor the kidneys, the TTP methods using 20 h-60 h and 40 h-200 h had smaller mean absolute errors of 8.05 ± 6.05% and 4.95 ± 3.98%, respectively, as compared to other combinations of time points and STP methods. For tumors, the type I and type II TTP methods using 20h−60 h and 40–200 h had smaller mean absolute errors of 6.14 ± 5.19% and 12.22 ± 4.44%, and 8.31 ± 7.16% and 4.48 ± 7.10%, respectively, as compared to other TTP and STP methods.ConclusionThe TTP methods based on later imaging time demonstrated fewer errors than the STP methods in kidney and tumor TIA. Imaging at 20 h−60 h and 40 h−200 h could simplify the dosimetry procedures with fewer TIA estimation errors

JaxPruner: A concise library for sparsity research

This paper introduces JaxPruner, an open-source JAX-based pruning and sparse training library for machine learning research. JaxPruner aims to accelerate research on sparse neural networks by providing concise implementations of popular pruning and sparse training algorithms with minimal memory and latency overhead. Algorithms implemented in JaxPruner use a common API and work seamlessly with the popular optimization library Optax, which, in turn, enables easy integration with existing JAX based libraries. We demonstrate this ease of integration by providing examples in four different codebases: Scenic, t5x, Dopamine and FedJAX and provide baseline experiments on popular benchmarks.Comment: Jaxpruner is hosted at http://github.com/google-research/jaxprune

Monte-Carlo Simulation for H(γ, pn)n Experiment with High Intensity Gamma-ray Source (HIγS)

<p>Experiment to measure the differential cross-section for photodisintegration of triton is being developed at Triangle Universities Nuclear Laboratory (TUNL). The goal of this experiment is to provide data for assessing the theoretical treatment of meson-exchange currents in photodisintegration of nuclei and for investigating long-range features of three-nucleon interactions. Measurements will be performed using a linearly polarized gamma-ray beam at the High Intensity Gamma-ray Source (HIγS) at TUNL. This thesis describes the Monte-Carlo simulation developed as a tool for aiding in the optimization of the experimental design and the data analysis software. The Monte-Carlo simulation is important for guiding the data acquisition strategy and interpretation of data. The simulation is based on the GEANT4 toolkit and theoretical cross-section predictions for current experimental setup.</p>Thesi

Nanocomposites of cobalt sulfide embedded carbon nanotubes with enhanced supercapacitor performance

CoS is one of the ideal electrode materials for supercapacitor, but its long-term stability and electrochemical performance needed to be improved before its successful application. Uniformly embedding carbon nanotubes (CNTs) inside the CoS matrix can provide numerous and effective diffusion paths of electrons and electrolyte ions, which can reduce the charge-transfer resistance and effectively improve the electrochemical performance of CoS. In this work, nanocomposites of Co2(CO3)(OH)2 and CNTs were prepared using a facile hydrothermal method, and then were transformed into CoS1.29@CNTs nanocomposites via an ion-exchange process. The carbon nanotubes were uniformly embedded inside the CoS1.29 matrix. When the amount of CNTs was 6.1 wt%, the CoS1.29@CNTs electrode exhibited a higher specific capacitance (99.7 mAh g-1) than that of CoS1.29 electrode (84.1 mAh g-1) at a current density of 1 A g-1 measured in 2 M KOH electrolyte. The asymmetric supercapacitor assembled with the [email protected]% electrode and an activated carbon (AC) electrode exhibited an energy density of 39.1 Wh kg-1 at a power density of 399.9 W kg-1. Moreover, the specific capacitance of the [email protected]%//AC device maintained 91.3 % of its original value after 2000 cycles at a current density of 3 A g-1

Ultrafast Response/Recovery and High Selectivity of H2S Gas Sensor Based on α-Fe2O3 Nano-Ellipsoids from One-Step Hydrothermal Synthesis

Ultrafast response/recovery and high selectivity of gas sensors are critical for real-time and online monitoring of hazardous gases. In this work, α-Fe2O3 nano-ellipsoids were synthesized using a facile one-step hydrothermal method and investigated as highly sensitive H2S sensing materials. The nano-ellipsoids have an average long axis diameter of 275 nm and an average short axis diameter of 125 nm. H2S gas sensors fabricated using the α-Fe2O3 nano-ellipsoids showed excellent H2S sensing performance at an optimum working temperature of 260 ℃. The response and recovery times were 0.8 s/2.2 s for H2S gas with a concentration of 50 ppm, which are much faster than those of H2S gas sensors reported in literature. The α-Fe2O3 nano-ellipsoid based sensors also showed a high selectivity to H2S compared to other commonly investigated gases including NH3, CO, NO2, H2, CH2Cl2 and ethanol. In addition, the sensors exhibited high response values to different concentrations of H2S with a detection limit as low as 100 ppb, as well as excellent repeatability and long-term stability