Are All Losses Created Equal: A Neural Collapse Perspective

While cross entropy (CE) is the most commonly used loss to train deep neural networks for classification tasks, many alternative losses have been developed to obtain better empirical performance. Among them, which one is the best to use is still a mystery, because there seem to be multiple factors affecting the answer, such as properties of the dataset, the choice of network architecture, and so on. This paper studies the choice of loss function by examining the last-layer features of deep networks, drawing inspiration from a recent line work showing that the global optimal solution of CE and mean-square-error (MSE) losses exhibits a Neural Collapse phenomenon. That is, for sufficiently large networks trained until convergence, (i) all features of the same class collapse to the corresponding class mean and (ii) the means associated with different classes are in a configuration where their pairwise distances are all equal and maximized. We extend such results and show through global solution and landscape analyses that a broad family of loss functions including commonly used label smoothing (LS) and focal loss (FL) exhibits Neural Collapse. Hence, all relevant losses(i.e., CE, LS, FL, MSE) produce equivalent features on training data. Based on the unconstrained feature model assumption, we provide either the global landscape analysis for LS loss or the local landscape analysis for FL loss and show that the (only!) global minimizers are neural collapse solutions, while all other critical points are strict saddles whose Hessian exhibit negative curvature directions either in the global scope for LS loss or in the local scope for FL loss near the optimal solution. The experiments further show that Neural Collapse features obtained from all relevant losses lead to largely identical performance on test data as well, provided that the network is sufficiently large and trained until convergence.Comment: 32 page, 10 figures, NeurIPS 202

Identification and target prediction of miRNAs specifically expressed in rat neural tissue

<p>Abstract</p> <p>Background</p> <p>MicroRNAs (miRNAs) are a large group of RNAs that play important roles in regulating gene expression and protein translation. Several studies have indicated that some miRNAs are specifically expressed in human, mouse and zebrafish tissues. For example, miR-1 and miR-133 are specifically expressed in muscles. Tissue-specific miRNAs may have particular functions. Although previous studies have reported the presence of human, mouse and zebrafish tissue-specific miRNAs, there have been no detailed reports of rat tissue-specific miRNAs. In this study, Home-made rat miRNA microarrays which established in our previous study were used to investigate rat neural tissue-specific miRNAs, and mapped their target genes in rat tissues. This study will provide information for the functional analysis of these miRNAs.</p> <p>Results</p> <p>In order to obtain as complete a picture of specific miRNA expression in rat neural tissues as possible, customized miRNA microarrays with 152 selected miRNAs from miRBase were used to detect miRNA expression in 14 rat tissues. After a general clustering analysis, 14 rat tissues could be clearly classified into neural and non-neural tissues based on the obtained expression profiles with p values < 0.05. The results indicated that the miRNA profiles were different in neural and non-neural tissues. In total, we found 30 miRNAs that were specifically expressed in neural tissues. For example, miR-199a was specifically expressed in neural tissues. Of these, the expression patterns of four miRNAs were comparable with those of Landgraf et al., Bak et al., and Kapsimani et al. Thirty neural tissue-specific miRNAs were chosen to predict target genes. A total of 1,475 target mRNA were predicted based on the intersection of three public databases, and target mRNA's pathway, function, and regulatory network analysis were performed. We focused on target enrichments of the dorsal root ganglion (DRG) and olfactory bulb. There were four Gene Ontology (GO) functions and five KEGG pathways significantly enriched in DRG. Only one GO function was significantly enriched in the olfactory bulb. These targets are all predictions and have not been experimentally validated.</p> <p>Conclusion</p> <p>Our work provides a global view of rat neural tissue-specific miRNA profiles and a target map of miRNAs, which is expected to contribute to future investigations of miRNA regulatory mechanisms in neural systems.</p

Holographic Storage of Biphoton Entanglement

Coherent and reversible storage of multi-photon entanglement with a multimode quantum memory is essential for scalable all-optical quantum information processing. Although single photon has been successfully stored in different quantum systems, storage of multi-photon entanglement remains challenging because of the critical requirement for coherent control of photonic entanglement source, multimode quantum memory, and quantum interface between them. Here we demonstrate a coherent and reversible storage of biphoton Bell-type entanglement with a holographic multimode atomic-ensemble-based quantum memory. The retrieved biphoton entanglement violates Bell's inequality for 1 microsecond storage time and a memory-process fidelity of 98% is demonstrated by quantum state tomography.Comment: 5 pages, 4 figures, accepted by Phys. Rev. Let

荆州区农村人口初发糖尿病胰岛功能的现状跟踪调查*

Objective: To study the change of islet function in patients with incipient diabetic characteristics through incipient diabetic tracking observation of the islet function in patients of Jingzhou area. Methods: Selection of 1220 cases of patients with diabetes mellitus in Jingzhou area as research object at the beginning, 12 months follow-up, the clinic after 3 months, 6 months and 12 months, all patients to detect blood sugar change, c-peptide release quantity, calculate insulin secretion index (HOMA -β) and insulin resistance index (HOMA IR), summarizes the characteristic of islet function in patients with changes. Results: ① The patients restored to basic standard blood sugar in 3 months by drug treatment, and the patient's blood glucose levels not seen obvious fluctuation after 6 months and 12 months; ② During follow-up, patients with diabetes sustained c-peptide release quantity reduction, and in three months after treatment, c-peptide release decreased obviously, and see a doctor at 6 months and 12 months after the comparison, the difference was statistically significant (P &lt; 0.05). ③ During follow-up, insulin capacity was decreasing among patients with diabetes, within three months after the doctor had the greatest reduction, the difference was statistically significant (P &lt; 0.05); ④ During follow-up, island hormone decreasing index, insulin resistance index continued to rise among patients with diabetes, and 6 months and 12 months, the most significant variations in 3 months (P &lt; 0.05). Conclusion: With the extension of course, the pancreatic islet function in patients with early onset diabetes decreased gradually. It could be proved that there is a significant correlation between the two and especially seen in obvious function decline of pancreatic islets among the patients within 3 months.目的  通过对荆州区初发糖尿病患者的胰岛功能进行跟踪观察，探讨发现初发糖尿病患者胰岛功能的变化特点。方法  选取荆州区1220例初发糖尿病患者作为观察对象，跟踪随访12个月，在就诊后的3个月、6个月及12个月时，全部患者检测血糖变化、Ｃ-肽释放量，计算胰岛素分泌指数（HOMA-β）及胰岛素抵抗指数（HOMA-IR），观察总结患者的胰岛功能变化特点。结果  （1）通过药物治疗，患者血糖在3个月时基本达标，6个月及12个月时，患者的血糖水平未见明显波动；（2）随访期间，糖尿病患者Ｃ-肽释放量持续降低，且在就诊后3个月内，Ｃ-肽释放量下降明显，与就诊后6个月时及12个月时比较，差异有统计学意义（P＜0.05）。（3）随访期间，糖尿病患者胰岛素放量持续降低，就诊后3个月内下降最明显，差异有统计学意义（P＜0.05）；（4）随访期间，糖尿病患者胰岛素分泌指数持续降低，胰岛素抵抗指数持续升高，且与6个月时和12个月时比较，3个月时变化幅度最为显著（P＜0.05）。结论  随着病程的延长，初发糖尿病患者胰岛功能逐渐降低，二者具有显著相关性，且3个月内患者的胰岛功能下降最为显著