Minimal random code learning: Getting bits back from compressed model parameters

While deep neural networks are a highly successful model class, their large memory footprint puts considerable strain on energy consumption, communication bandwidth, and storage requirements. Consequently, model size reduction has become an utmost goal in deep learning. A typical approach is to train a set of deterministic weights, while applying certain techniques such as pruning and quantization, in order that the empirical weight distribution becomes amenable to Shannon-style coding schemes. However, as shown in this paper, relaxing weight determinism and using a full variational distribution over weights allows for more efficient coding schemes and consequently higher compression rates. In particular, following the classical bits-back argument, we encode the network weights using a random sample, requiring only a number of bits corresponding to the Kullback-Leibler divergence between the sampled variational distribution and the encoding distribution. By imposing a constraint on the Kullback-Leibler divergence, we are able to explicitly control the compression rate, while optimizing the expected loss on the training set. The employed encoding scheme can be shown to be close to the optimal information-theoretical lower bound, with respect to the employed variational family. Our method sets new state-of-the-art in neural network compression, as it strictly dominates previous approaches in a Pareto sense: On the benchmarks LeNet-5/MNIST and VGG-16/CIFAR-10, our approach yields the best test performance for a fixed memory budget, and vice versa, it achieves the highest compression rates for a fixed test performance

Compressing images by encoding their latent representations with relative entropy coding

Variational Autoencoders (VAEs) have seen widespread use in learned image compression. They are used to learn expressive latent representations on which downstream compression methods can operate with high efficiency. Recently proposed 'bits-back' methods can indirectly encode the latent representation of images with codelength close to the relative entropy between the latent posterior and the prior. However, due to the underlying algorithm, these methods can only be used for lossless compression, and they only achieve their nominal efficiency when compressing multiple images simultaneously; they are inefficient for compressing single images. As an alternative, we propose a novel method, Relative Entropy Coding (REC), that can directly encode the latent representation with codelength close to the relative entropy for single images, supported by our empirical results obtained on the Cifar10, ImageNet32 and Kodak datasets. Moreover, unlike previous bits-back methods, REC is immediately applicable to lossy compression, where it is competitive with the state-of-the-art on the Kodak dataset

Seasonal change of thyroid histomorphological structure and hormone production in yellowfin seabream (Acanthopagrus latus) in the Persian Gulf

Seasonal changes of the thyroid gland structure and hormones secretion was examined in yellowfin seabream (Acanthopagrus latus) in the northwest of Persian Gulf (Musa creek). Thyroid gland composed of follicles scattered around the ventral aorta, near the gills. Follicular cells varied according to secretion of the gland during warm and cold seasons. Thyroid hormones (Triidothyronine [T3] and Thyroxine [T4]) were detected in the fish serum in levels ranged from 4.09-1.30 ng/mL for T3 and from 1.10-0.21 ng/mL for (T4) in the warm and cold seasons, respectively. The results showed that the height of thyroid epithelium and plasma concentration of thyroid hormones (thyroid activity) in A. latus increased significantly during spring and summer. The peak of these factors occurred in midsummer (August). Then, the thyroid activity decreased significantly during autumn and early winter from October to December according to decrease of temperature. T3 and T4 increased significantly from January to April

Quality of Life in Hungarian Parents of Autistic Individuals

Purpose: Parents of autistic individuals have been known to have a lower overall quality of life (QQL) than those of typically developing children. We present the first Hungarian large-sample study whose objective was to explore the differences in QOL between parents of autistic individuals (AS) and those of neurotypical (NT) persons. Methods: Based on the ABCX model we developed a questionnaire comprising standardized scales to characterize the life of parents involved. Our data came from parents of 842 individuals (ASD = 521, NT = 321) between 0 and 49 years. Battery deployed standardized instruments to examine quality of life (WHO-QQL BREF and Quality of Life in Autism questionnaire, QOLA). We assessed the families’ socio-economic/demographic characteristics, parents’ psychological well-being, the autistic/neurotypical individuals’ characteristics, and the interventions. Results: Our data showed significantly lower QOL in parents of autistic individuals in all domains of questionnaires. We analyzed 20 relevant factors to uncover the predictors of parental QOL. We confirmed the existence of most but not all predictors present in earlier literature and identified intervention-related predictors. Conclusion: Our study confirms the importance of supporting parents in their role, and of providing health and social supports that focus on quality of life, in addition to child care

Learning Colour Representations of Search Queries

Image search engines rely on appropriately designed ranking features that capture various aspects of the content semantics as well as the historic popularity. In this work, we consider the role of colour in this relevance matching process. Our work is motivated by the observation that a significant fraction of user queries have an inherent colour associated with them. While some queries contain explicit colour mentions (such as 'black car' and 'yellow daisies'), other queries have implicit notions of colour (such as 'sky' and 'grass'). Furthermore, grounding queries in colour is not a mapping to a single colour, but a distribution in colour space. For instance, a search for 'trees' tends to have a bimodal distribution around the colours green and brown. We leverage historical clickthrough data to produce a colour representation for search queries and propose a recurrent neural network architecture to encode unseen queries into colour space. We also show how this embedding can be learnt alongside a cross-modal relevance ranker from impression logs where a subset of the result images were clicked. We demonstrate that the use of a query-image colour distance feature leads to an improvement in the ranker performance as measured by users' preferences of clicked versus skipped images.Comment: Accepted as a full paper at SIGIR 202

Risk factors for venous thromboembolism in immunoglobulin light chain amyloidosis

Patients with immunoglobulin light chain amyloidosis are at risk for both thrombotic and bleeding complications. While the hemostatic defects have been extensively studied, less is known about thrombotic complications in this disease. This retrospective study examined the frequency of venous thromboembolism in 929 patients with immunoglobulin light chain amyloidosis presenting to a single referral center, correlated risk of venous thromboembolism with clinical and laboratory factors, and examined complications of anticoagulation in this population. Sixty-five patients (7%) were documented as having at least one venous thromboembolic event. Eighty percent of these patients had events within one year prior to or following diagnosis. Lower serum albumin was associated with increased risk of VTE, with a hazard ratio of 4.30 (CI 1.60–11.55; P=0.0038) for serum albumin less than 3 g/dL compared to serum albumin greater than 4 g/dL. Severe bleeding complications were observed in 5 out of 57 patients with venous thromboembolism undergoing treatment with anticoagulation. Prospective investigation should be undertaken to better risk stratify these patients and to determine the optimal strategies for prophylaxis against and management of venous thromboembolism