Less is More -- Towards parsimonious multi-task models using structured sparsity

Model sparsification in deep learning promotes simpler, more interpretable models with fewer parameters. This not only reduces the model's memory footprint and computational needs but also shortens inference time. This work focuses on creating sparse models optimized for multiple tasks with fewer parameters. These parsimonious models also possess the potential to match or outperform dense models in terms of performance. In this work, we introduce channel-wise l1/l2 group sparsity in the shared convolutional layers parameters (or weights) of the multi-task learning model. This approach facilitates the removal of extraneous groups i.e., channels (due to l1 regularization) and also imposes a penalty on the weights, further enhancing the learning efficiency for all tasks (due to l2 regularization). We analyzed the results of group sparsity in both single-task and multi-task settings on two widely-used Multi-Task Learning (MTL) datasets: NYU-v2 and CelebAMask-HQ. On both datasets, which consist of three different computer vision tasks each, multi-task models with approximately 70% sparsity outperform their dense equivalents. We also investigate how changing the degree of sparsification influences the model's performance, the overall sparsity percentage, the patterns of sparsity, and the inference time.Comment: Under revie

Multi-Task Meta Learning: learn how to adapt to unseen tasks

This work proposes Multi-task Meta Learning (MTML), integrating two learning paradigms Multi-Task Learning (MTL) and meta learning, to bring together the best of both worlds. In particular, it focuses simultaneous learning of multiple tasks, an element of MTL and promptly adapting to new tasks, a quality of meta learning. It is important to highlight that we focus on heterogeneous tasks, which are of distinct kind, in contrast to typically considered homogeneous tasks (e.g., if all tasks are classification or if all tasks are regression tasks). The fundamental idea is to train a multi-task model, such that when an unseen task is introduced, it can learn in fewer steps whilst offering a performance at least as good as conventional single task learning on the new task or inclusion within the MTL. By conducting various experiments, we demonstrate this paradigm on two datasets and four tasks: NYU-v2 and the taskonomy dataset for which we perform semantic segmentation, depth estimation, surface normal estimation, and edge detection. MTML achieves state-of-the-art results for three out of four tasks for the NYU-v2 dataset and two out of four for the taskonomy dataset. In the taskonomy dataset, it was discovered that many pseudo-labeled segmentation masks lacked classes that were expected to be present in the ground truth; however, our MTML approach was found to be effective in detecting these missing classes, delivering good qualitative results. While, quantitatively its performance was affected due to the presence of incorrect ground truth labels. The the source code for reproducibility can be found at https://github.com/ricupa/MTML-learn-how-to-adapt-to-unseen-tasks

Evaluation of Cardioprotective activity of Asparagus racemosus against Doxorubicin induced cardiotoxicity in albino rats: an experimental study

Background: Cardiotoxicity is one of the most feared side effects of anticancer agents like Doxorubicin. Asparagus racemosus is a widely used medicinal plant in Indian system of medicine known for its antioxidant activity. In certain studies ethanol extract of Asparagus racemosus has shown to possess cardioprotective activity in experimental animals while in some other studies cardioprotective potential of Asparagus racemosus has not been demonstrated. Therefore, due to the controversial action, the present study was designed to explore the cardioprotective effect of aqueous effect of Asparagus racemosus against doxorubicin induced cardiotoxity.Methods: Following approval from Institutional Animal Ethics Committee of L.L.R.M Medical College registered under CPCSEA, India, this study was conducted in which 30 rats were randomized into five groups of six rats each. Group I received 2 ml/kg b.w. normal saline p.o for 21 days, group II apart from receiving pellet diet and normal saline for 21 days were treated with Doxorubicin in a single dose of 20 mg/kg intraperitoneally on the 21st day, group III and group IV received aqueous extract of Asparagus racemosus in doses of 250 mg/kg/day and 500 mg/kg/day respectively p.o. for 21 days followed by administration of Doxorubicin (20 mg/kg i.p.) on the 21st day, Group V received Carvedilol in doses of  30 mg/kg/day p.o. for 21 days followed by administration of Doxorubicin (20mg/kg i.p) on the 21st day. Then they were anaesthetized and blood sample was collected from abdominal aorta for performing blood test i.e. Creatinine kinase MB fraction (CK-MB), Lactate dehydrogenase (LDH), Serum glutamate oxaloacetate transaminase (SGOT), Serum glutamate pyruvate transaminase (SGPT). After blood collection the animals were sacrificed and heart was dissected out for histopathological study. The data obtained was organized and analysed by suitable statistical methods i.e. ANOVA followed by Post Hoc test.Results: CK-MB, LDH, SGOT and SGPT levels were found to be significantly raised (p<0.001) in Doxorubicin treated group. Asparagus racemosus pretreated groups exhibited significant limitation (p<0.001) in rise in levels of CK-MB,LDH,SGOT and SGPT levels in a dose dependent manner following Doxorubicin administration which were comparable to the group treated with the standard cardioprotective drug Carvedilol. Histopathological changes further corroborated cardioprotective potential of Asparagus racemosus.Conclusions: The present study concluded that aqueous extract of Asparagus racemosus possess cardioprotective potential against Doxorubicin induced cardiotoxicity

TheNorth @ HaSpeeDe 2: BERT-based Language Model Fine-tuning for Italian Hate Speech Detection

This report was written to describe the systems that were submitted by the team “TheNorth” for the HaSpeeDe 2 shared task organised within EVALITA 2020. To address the main task which is hate speech detection, we fine-tuned BERT-based models. We evaluated both multilingual and Italian language models trained with the data provided and additional data. We also studied the contributions of multitask learning considering both hate speech detection and stereotype detection tasks

A Systematic Performance Analysis of Deep Perceptual Loss Networks: Breaking Transfer Learning Conventions

Deep perceptual loss is a type of loss function in computer vision that aims to mimic human perception by using the deep features extracted from neural networks. In recent years, the method has been applied to great effect on a host of interesting computer vision tasks, especially for tasks with image or image-like outputs, such as image synthesis, segmentation, depth prediction, and more. Many applications of the method use pretrained networks, often convolutional networks, for loss calculation. Despite the increased interest and broader use, more effort is needed toward exploring which networks to use for calculating deep perceptual loss and from which layers to extract the features. This work aims to rectify this by systematically evaluating a host of commonly used and readily available, pretrained networks for a number of different feature extraction points on four existing use cases of deep perceptual loss. The use cases of perceptual similarity, super-resolution, image segmentation, and dimensionality reduction, are evaluated through benchmarks. The benchmarks are implementations of previous works where the selected networks and extraction points are evaluated. The performance on the benchmarks, and attributes of the networks and extraction points are then used as a basis for an in-depth analysis. This analysis uncovers insight regarding which architectures provide superior performance for deep perceptual loss and how to choose an appropriate extraction point for a particular task and dataset. Furthermore, the work discusses the implications of the results for deep perceptual loss and the broader field of transfer learning. The results show that deep perceptual loss deviates from two commonly held conventions in transfer learning, which suggests that those conventions are in need of deeper analysis

Functional Knowledge Transfer with Self-supervised Representation Learning

This work investigates the unexplored usability of self-supervised representation learning in the direction of functional knowledge transfer. In this work, functional knowledge transfer is achieved by joint optimization of self-supervised learning pseudo task and supervised learning task, improving supervised learning task performance. Recent progress in self-supervised learning uses a large volume of data, which becomes a constraint for its applications on small-scale datasets. This work shares a simple yet effective joint training framework that reinforces human-supervised task learning by learning self-supervised representations just-in-time and vice versa. Experiments on three public datasets from different visual domains, Intel Image, CIFAR, and APTOS, reveal a consistent track of performance improvements on classification tasks during joint optimization. Qualitative analysis also supports the robustness of learnt representations. Source code and trained models are available on GitHub.Comment: Accepted at IEEE International Conference on Image Processing (ICIP 2023

Anomaly Detection in Natural Scene Images Based on Enhanced Fine-Grained Saliency and Fuzzy Logic

This paper proposes a simple yet effective method for anomaly detection in natural scene images improving natural scene text detection and recognition. In the last decade, there has been significant progress towards text detection and recognition in natural scene images. However, in cases where there are logos, company symbols, or other decorative elements for text, existing methods do not perform well. This work considers such misclassified components, which are part of the text as anomalies, and presents a new idea for detecting such anomalies in the text for improving text detection and recognition in natural scene images. The proposed method considers the result of the existing text detection method as input for segmenting characters or components based on saliency map and rough set theory. For each segmented component, the proposed method extracts feature from the saliency map based on density, pixel distribution, and phase congruency to classify text and non-text components by exploring a fuzzy-based classifier. To verify the effectiveness of the method, we have performed experiments on several benchmark datasets of natural scene text detection, namely, MSRATD-500 and SVT. Experimental results show the efficacy of the proposed method over the existing ones for text detection and recognition in these datasets

Strong impact of TGF-&#946;1 gene polymorphisms on breast cancer risk in Indian women: a case-control and population-based study

Introduction: TGF-&#946;1 is a multi-functional cytokine that plays an important role in breast carcinogenesis. Critical role of TGF-&#946;1 signaling in breast cancer progression is well documented. Some TGF-&#946;1 polymorphisms influence its expression; however, their impact on breast cancer risk is not clear. Methods: We analyzed 1222 samples in a candidate gene-based genetic association study on two distantly located and ethnically divergent case-control groups of Indian women, followed by a population-based genetic epidemiology study analyzing these polymorphisms in other Indian populations. The c.29C&#62;T (Pro10Leu, rs1982073 or rs1800470) and c.74G&#62;C (Arg25Pro, rs1800471) polymorphisms in the TGF-&#946;1 gene were analyzed using direct DNA sequencing, and peripheral level of TGF-&#946;1 were measured by ELISA. Results: c.29C&#62;T substitution increased breast cancer risk, irrespective of ethnicity and menopausal status. On the other hand, c.74G&#62;C substitution reduced breast cancer risk significantly in the north Indian group (p  =  0.0005) and only in the pre-menopausal women. The protective effect of c.74G&#62;C polymorphism may be ethnicity-specific, as no association was seen in south Indian group. The polymorphic status of c.29C&#62;T was comparable among Indo-Europeans, Dravidians and Tibeto-Burmans. Interestingly, we found that Tibeto-Burmans lack polymorphism at c.74G&#62;C locus as true for the Chinese populations. However, the Brahmins of Nepal (Indo-Europeans) showed polymorphism in 2.08% of alleles. Mean TGF-&#946;1 was significantly elevated in patients in comparison to controls (p&#60;0.001). Conclusion: c.29C&#62;T and c.74G&#62;C polymorphisms in the TGF-&#946;1 gene significantly affect breast cancer risk, which correlates with elevated TGF-&#946;1 level in the patients. The c.29C&#62;T locus is polymorphic across ethnically different populations, but c.74G&#62;C locus is monomorphic in Tibeto-Burmans and polymorphic in other Indian populations