Cost-effective On-device Continual Learning over Memory Hierarchy with Miro

Continual learning (CL) trains NN models incrementally from a continuous stream of tasks. To remember previously learned knowledge, prior studies store old samples over a memory hierarchy and replay them when new tasks arrive. Edge devices that adopt CL to preserve data privacy are typically energy-sensitive and thus require high model accuracy while not compromising energy efficiency, i.e., cost-effectiveness. Our work is the first to explore the design space of hierarchical memory replay-based CL to gain insights into achieving cost-effectiveness on edge devices. We present Miro, a novel system runtime that carefully integrates our insights into the CL framework by enabling it to dynamically configure the CL system based on resource states for the best cost-effectiveness. To reach this goal, Miro also performs online profiling on parameters with clear accuracy-energy trade-offs and adapts to optimal values with low overhead. Extensive evaluations show that Miro significantly outperforms baseline systems we build for comparison, consistently achieving higher cost-effectiveness.Comment: This paper is to be published in the 29th Annual International Conference on Mobile Computing and Networking (ACM MobiCom 23

Efficient Multi-Scale Stereo-Matching Network Using Adaptive Cost Volume Filtering

While recent deep learning-based stereo-matching networks have shown outstanding advances, there are still some unsolved challenges. First, most state-of-the-art stereo models employ 3D convolutions for 4D cost volume aggregation, which limit the deployment of networks for resource-limited mobile environments owing to heavy consumption of computation and memory. Although there are some efficient networks, most of them still require a heavy computational cost to incorporate them to mobile computing devices in real-time. Second, most stereo networks indirectly supervise cost volumes through disparity regression loss by using the softargmax function. This causes problems in ambiguous regions, such as the boundaries of objects, because there are many possibilities for unreasonable cost distributions which result in overfitting problem. A few works deal with this problem by generating artificial cost distribution using only the ground truth disparity value that is insufficient to fully regularize the cost volume. To address these problems, we first propose an efficient multi-scale sequential feature fusion network (MSFFNet). Specifically, we connect multi-scale SFF modules in parallel with a cross-scale fusion function to generate a set of cost volumes with different scales. These cost volumes are then effectively combined using the proposed interlaced concatenation method. Second, we propose an adaptive cost-volume-filtering (ACVF) loss function that directly supervises our estimated cost volume. The proposed ACVF loss directly adds constraints to the cost volume using the probability distribution generated from the ground truth disparity map and that estimated from the teacher network which achieves higher accuracy. Results of several experiments using representative datasets for stereo matching show that our proposed method is more efficient than previous methods. Our network architecture consumes fewer parameters and generates reasonable disparity maps with faster speed compared with the existing state-of-the art stereo models. Concretely, our network achieves 1.01 EPE with runtime of 42 ms, 2.92 M parameters, and 97.96 G FLOPs on the Scene Flow test set. Compared with PSMNet, our method is 89% faster and 7% more accurate with 45% fewer parameters

Sibylla: To Retry or Not To Retry on Deep Learning Job Failure

GPUs are highly contended resources in shared clusters for deep learning (DL) training. However, our analysis with a real-world trace reveals that a non-negligible number of jobs running on the cluster undergo failures and are blindly retried by the job scheduler. Unfortunately, these job failures often repeat and waste GPU resources, limiting effective GPU utilization across the cluster. In this paper, we introduce Sibylla which informs whether an observed failure of DL training will repeat or not upon retry on the failure. Sibylla employs a machine learning model based on RNNs that trains on stdout and stderr logs of failed jobs and can continuously update the model on new log messages without hand-constructing labels for the new training samples. With Sibylla, the job scheduler is learning-enhanced, performing a retry for a failed job only when it is highly likely to succeed with the retry. We evaluate the effectiveness of Sibylla under a variety of scenarios using trace-driven simulations. Sibylla improves cluster utilization and reduces job completion time (JCT) by up to 15%

Expression of uncharacterized male germ cell-specific genes and discovery of novel sperm-tail proteins in mice.

The identification and characterization of germ cell-specific genes are essential if we hope to comprehensively understand the mechanisms of spermatogenesis and fertilization. Here, we searched the mouse UniGene databases and identified 13 novel genes as being putatively testis-specific or -predominant. Our in silico and in vitro analyses revealed that the expressions of these genes are testis- and germ cell-specific, and that they are regulated in a stage-specific manner during spermatogenesis. We generated antibodies against the proteins encoded by seven of the genes to facilitate their characterization in male germ cells. Immunoblotting and immunofluorescence analyses revealed that one of these proteins was expressed only in testicular germ cells, three were expressed in both testicular germ cells and testicular sperm, and the remaining three were expressed in sperm of the testicular stages and in mature sperm from the epididymis. Further analysis of the latter three proteins showed that they were all associated with cytoskeletal structures in the sperm flagellum. Among them, MORN5, which is predicted to contain three MORN motifs, is conserved between mouse and human sperm. In conclusion, we herein identify 13 authentic genes with male germ cell-specific expression, and provide comprehensive information about these genes and their encoded products. Our finding will facilitate future investigations into the functional roles of these novel genes in spermatogenesis and sperm functions

Factors affecting the changes in antihypertensive medications in patients with hypertension

As frequent changes in anti-hypertensive (HTN) medications may reduce adherence to the treatments, identifying modifiable factors leading to changes in anti-HTN medications can help clinicians optimize treatment strategies for individual patients. We performed this study to explore the pattern of anti-HTN medications and to identify factors that are associated with the changes in anti-HTN medications. To this end, we used a clinical database of Seoul National University Hospital, extracted, transformed, and loaded by the observational medical outcomes partnership common data model. Demographic and all recorded clinical diagnoses, medications, and procedures data of eligible subjects were collected. Of 636 subjects who were eligible for this study, 297 subjects with a record of >= 1 anti-HTN medication changes and other 297 subjects without a record of medication change were selected for the study population. High diastolic blood pressure (adjusted odds ratio [OR]: 1.02, 95% confidence interval [CI]: 1.001-1.040, p = 0.040), arrhythmia (adjusted OR: 10.01, 95% CI: 1.86-185.57, p = 0.030), and angina pectoris with antianginal agents (adjusted OR: 4.85, CI: 1.05-23.89, p = 0.046) were associated with the changes in anti-HTN medications, indicating that any patients with these covariates require additional attention to reduce the likelihood of changing anti-HTN medications.Y

Impaired fasting glucose and development of chronic kidney disease in non-diabetic population: a Mendelian randomization study

Introduction Diabetes mellitus is a risk factor of chronic kidney disease (CKD); however, the relationship between fasting glucose and CKD remains controversial in non-diabetic population. This study aimed to assess causal relationship between genetically predicted fasting glucose and incident CKD.Research design and methods This study included 5909 participants without diabetes and CKD from the Korean Genome Epidemiology Study. The genetic risk score (GRS9) was calculated using nine genetic variants associated with fasting glucose in previous genome-wide association studies. Incident CKD was defined as estimated glomerular filtration rate (eGFR) <60 mL/min/1.73 m2 and/or proteinuria (≥1+). The causal relationship between fasting glucose and CKD was evaluated using the Mendelian randomization (MR) approach.Results The GRS9 was strongly associated with fasting glucose (β, 1.01; p<0.001). During a median follow-up of 11.6 years, 490 (8.3%) CKD events occurred. However, GRS9 was not significantly different between participants with CKD events and those without. After adjusting for confounding factors, fasting glucose was not associated with incident CKD (OR 0.990; 95% CI 0.977 to 1.002; p=0.098). In the MR analysis, GRS9 was not associated with CKD development (OR per 1 SD increase, 1.179; 95% CI 0.819 to 1.696; p=0.376). Further evaluation using various other MR methods and strict CKD criteria (decrease in the eGFR of ≥30% to a value of <60 mL/min/1.73 m2) found no significant relationship between GRS9 and incident CKD.Conclusions Fasting glucose was not causally associated with CKD development in non-diabetic population

Expression of uncharacterized male germ cell-specific genes and discovery of novel sperm-tail proteins in mice

<div><p>The identification and characterization of germ cell-specific genes are essential if we hope to comprehensively understand the mechanisms of spermatogenesis and fertilization. Here, we searched the mouse UniGene databases and identified 13 novel genes as being putatively testis-specific or -predominant. Our <i>in silico</i> and <i>in vitro</i> analyses revealed that the expressions of these genes are testis- and germ cell-specific, and that they are regulated in a stage-specific manner during spermatogenesis. We generated antibodies against the proteins encoded by seven of the genes to facilitate their characterization in male germ cells. Immunoblotting and immunofluorescence analyses revealed that one of these proteins was expressed only in testicular germ cells, three were expressed in both testicular germ cells and testicular sperm, and the remaining three were expressed in sperm of the testicular stages and in mature sperm from the epididymis. Further analysis of the latter three proteins showed that they were all associated with cytoskeletal structures in the sperm flagellum. Among them, MORN5, which is predicted to contain three MORN motifs, is conserved between mouse and human sperm. In conclusion, we herein identify 13 authentic genes with male germ cell-specific expression, and provide comprehensive information about these genes and their encoded products. Our finding will facilitate future investigations into the functional roles of these novel genes in spermatogenesis and sperm functions.</p></div

Image_2_Factors affecting the changes in antihypertensive medications in patients with hypertension.PDF

As frequent changes in anti-hypertensive (HTN) medications may reduce adherence to the treatments, identifying modifiable factors leading to changes in anti-HTN medications can help clinicians optimize treatment strategies for individual patients. We performed this study to explore the pattern of anti-HTN medications and to identify factors that are associated with the changes in anti-HTN medications. To this end, we used a clinical database of Seoul National University Hospital, extracted, transformed, and loaded by the observational medical outcomes partnership common data model. Demographic and all recorded clinical diagnoses, medications, and procedures data of eligible subjects were collected. Of 636 subjects who were eligible for this study, 297 subjects with a record of ≥1 anti-HTN medication changes and other 297 subjects without a record of medication change were selected for the study population. High diastolic blood pressure (adjusted odds ratio [OR]: 1.02, 95% confidence interval [CI]: 1.001–1.040, p = 0.040), arrhythmia (adjusted OR: 10.01, 95% CI: 1.86–185.57, p = 0.030), and angina pectoris with antianginal agents (adjusted OR: 4.85, CI: 1.05–23.89, p = 0.046) were associated with the changes in anti-HTN medications, indicating that any patients with these covariates require additional attention to reduce the likelihood of changing anti-HTN medications.</p

Characterization of three sperm tail proteins.

<p>A. Sperm from the epididymis and vas deferens were treated with 1% NP-40 or 1% Triton X-100 and then centrifuged. Soluble and insoluble fractions were subjected to immunoblot analysis. ADAM2 and α-tubulin were used to verify the soluble and insoluble fractions, respectively. The tested proteins failed to solubilize with these detergents. S, supernatant after centrifugation; P, pellet after centrifugation. B. Sperm were treated with 2, 3, 4, or 6 M urea and then centrifuged. Soluble and insoluble fractions were subjected to Western blot analysis, with α-tubulin detected as a loading control. MORN5 and Mm.271255 were found to be insoluble in urea.</p