Efficient model compression with Random Operation Access Specific Tile (ROAST) hashing

Advancements in deep learning are often associated with increasing model sizes. The model size dramatically affects the deployment cost and latency of deep models. For instance, models like BERT cannot be deployed on edge devices and mobiles due to their sheer size. As a result, most advances in Deep Learning are yet to reach the edge. Model compression has sought much-deserved attention in literature across natural language processing, vision, and recommendation domains. This paper proposes a model-agnostic, cache-friendly model compression approach: Random Operation Access Specific Tile (ROAST) hashing. ROAST collapses the parameters by clubbing them through a lightweight mapping. Notably, while clubbing these parameters, ROAST utilizes cache hierarchies by aligning the memory access pattern with the parameter access pattern. ROAST is up to $\sim 25 \times$ faster to train and $\sim 50 \times$ faster to infer than the popular parameter sharing method HashedNet. Additionally, ROAST introduces global weight sharing, which is empirically and theoretically superior to local weight sharing in HashedNet, and can be of independent interest in itself. With ROAST, we present the first compressed BERT, which is $100\times - 1000\times$ smaller but does not result in quality degradation. These compression levels on universal architecture like transformers are promising for the future of SOTA model deployment on resource-constrained devices like mobile and edge device

Token-Weighted RNN-T for Learning from Flawed Data

ASR models are commonly trained with the cross-entropy criterion to increase the probability of a target token sequence. While optimizing the probability of all tokens in the target sequence is sensible, one may want to de-emphasize tokens that reflect transcription errors. In this work, we propose a novel token-weighted RNN-T criterion that augments the RNN-T objective with token-specific weights. The new objective is used for mitigating accuracy loss from transcriptions errors in the training data, which naturally appear in two settings: pseudo-labeling and human annotation errors. Experiments results show that using our method for semi-supervised learning with pseudo-labels leads to a consistent accuracy improvement, up to 38% relative. We also analyze the accuracy degradation resulting from different levels of WER in the reference transcription, and show that token-weighted RNN-T is suitable for overcoming this degradation, recovering 64%-99% of the accuracy loss

Mechanistic Insight into Intestinal α-Synuclein Aggregation in Parkinsons Disease Using a Laser-Printed Electrochemical Sensor.

Aggregated deposits of the protein α-synuclein and depleting levels of dopamine in the brain correlate with Parkinsons disease development. Treatments often focus on replenishing dopamine in the brain; however, the brain might not be the only site requiring attention. Aggregates of α-synuclein appear to accumulate in the gut years prior to the onset of any motor symptoms. Enteroendocrine cells (specialized gut epithelial cells) may be the source of intestinal α-synuclein, as they natively express this protein. Enteroendocrine cells are constantly exposed to gut bacteria and their metabolites because they border the gut lumen. These cells also express the dopamine metabolic pathway and form synapses with vagal neurons, which innervate the gut and brain. Through this connection, Parkinsons disease pathology may originate in the gut and spread to the brain over time. Effective therapeutics to prevent this disease progression are lacking due to a limited understanding of the mechanisms by which α-synuclein aggregation occurs in the gut. We previously proposed a gut bacterial metabolic pathway responsible for the initiation of α-synuclein aggregation that is dependent on the oxidation of dopamine. Here, we develop a new tool, a laser-induced graphene-based electrochemical sensor chip, to track α-synuclein aggregation and dopamine level over time. Using these sensor chips, we evaluated diet-derived catechols dihydrocaffeic acid and caffeic acid as potential inhibitors of α-synuclein aggregation. Our results suggest that these molecules inhibit dopamine oxidation. We also found that these dietary catechols inhibit α-synuclein aggregation in STC-1 enteroendocrine cells. These findings are critical next steps to reveal new avenues for targeted therapeutics to treat Parkinsons disease, specifically in the context of functional foods that may be used to reshape the gut environment

Light Field Salient Object Detection: A Review and Benchmark

Salient object detection (SOD) is a long-standing research topic in computer vision and has drawn an increasing amount of research interest in the past decade. This paper provides the first comprehensive review and benchmark for light field SOD, which has long been lacking in the saliency community. Firstly, we introduce preliminary knowledge on light fields, including theory and data forms, and then review existing studies on light field SOD, covering ten traditional models, seven deep learning-based models, one comparative study, and one brief review. Existing datasets for light field SOD are also summarized with detailed information and statistical analyses. Secondly, we benchmark nine representative light field SOD models together with several cutting-edge RGB-D SOD models on four widely used light field datasets, from which insightful discussions and analyses, including a comparison between light field SOD and RGB-D SOD models, are achieved. Besides, due to the inconsistency of datasets in their current forms, we further generate complete data and supplement focal stacks, depth maps and multi-view images for the inconsistent datasets, making them consistent and unified. Our supplemental data makes a universal benchmark possible. Lastly, because light field SOD is quite a special problem attributed to its diverse data representations and high dependency on acquisition hardware, making it differ greatly from other saliency detection tasks, we provide nine hints into the challenges and future directions, and outline several open issues. We hope our review and benchmarking could help advance research in this field. All the materials including collected models, datasets, benchmarking results, and supplemented light field datasets will be publicly available on our project site https://github.com/kerenfu/LFSOD-Survey