Implicit Kernel Attention

\textit{Attention} computes the dependency between representations, and it encourages the model to focus on the important selective features. Attention-based models, such as Transformers and graph attention networks (GAT) are widely utilized for sequential data and graph-structured data. This paper suggests a new interpretation and generalized structure of the attention in Transformer and GAT. For the attention in Transformer and GAT, we derive that the attention is a product of two parts: 1) the RBF kernel to measure the similarity of two instances and 2) the exponential of $L^{2}$ norm to compute the importance of individual instances. From this decomposition, we generalize the attention in three ways. First, we propose implicit kernel attention with an implicit kernel function, instead of manual kernel selection. Second, we generalize $L^{2}$ norm as the $L^{p}$ norm. Third, we extend our attention to structured multi-head attention. Our generalized attention shows better performance on classification, translation, and regression tasks

Modelling and Simulation of a River-Crossing Operation via Discrete Event Simulation with Engineering Details

From a military standpoint, a river is an area that should be avoided in a potential engagement because of lack of cover and the necessity of dividing the unit while crossing. Thus, a key point of a river-crossing operation is speed. Many efforts have been made to enable faster river crossing by improvement of tactics, techniques, and procedures (TTP). However, improvements in TTP are evaluated by modelling and simulation much less frequently than are the toe-to-toe engagements between two opposing forces, and to our knowledge, this is the first simulation model of brigade-level river crossing with engineering details. This study presents a simulation model of the river-crossing operation, applies real world parameters, and evaluates which tactics are preferable in a particular operational environments. This analysis has led to new operational methods of river crossing that have been suggested by experienced subject-matter experts. For instance, the current Republic of Korea Army Field Manual dictates to rotate river-crossing rafts in all situations, but our experiment suggests that no rotation is preferable when the width of river is less than 400 m based on the statistical analyses, which includes the regression-based meta-modelling and the ANOVA, of our simulation model that embodies the engineering details of river-crossing equipment.Defence Science Journal, Vol. 65, No. 2, March 2015, pp.135-143, DOI:http://dx.doi.org/10.14429/dsj.65.814

Refining Generative Process with Discriminator Guidance in Score-based Diffusion Models

The proposed method, Discriminator Guidance, aims to improve sample generation of pre-trained diffusion models. The approach introduces a discriminator that gives explicit supervision to a denoising sample path whether it is realistic or not. Unlike GANs, our approach does not require joint training of score and discriminator networks. Instead, we train the discriminator after score training, making discriminator training stable and fast to converge. In sample generation, we add an auxiliary term to the pre-trained score to deceive the discriminator. This term corrects the model score to the data score at the optimal discriminator, which implies that the discriminator helps better score estimation in a complementary way. Using our algorithm, we achive state-of-the-art results on ImageNet 256x256 with FID 1.83 and recall 0.64, similar to the validation data's FID (1.68) and recall (0.66). We release the code at https://github.com/alsdudrla10/DG.Comment: International Conference on Machine Learning (ICML23

Label-Noise Robust Diffusion Models

Conditional diffusion models have shown remarkable performance in various generative tasks, but training them requires large-scale datasets that often contain noise in conditional inputs, a.k.a. noisy labels. This noise leads to condition mismatch and quality degradation of generated data. This paper proposes Transition-aware weighted Denoising Score Matching (TDSM) for training conditional diffusion models with noisy labels, which is the first study in the line of diffusion models. The TDSM objective contains a weighted sum of score networks, incorporating instance-wise and time-dependent label transition probabilities. We introduce a transition-aware weight estimator, which leverages a time-dependent noisy-label classifier distinctively customized to the diffusion process. Through experiments across various datasets and noisy label settings, TDSM improves the quality of generated samples aligned with given conditions. Furthermore, our method improves generation performance even on prevalent benchmark datasets, which implies the potential noisy labels and their risk of generative model learning. Finally, we show the improved performance of TDSM on top of conventional noisy label corrections, which empirically proving its contribution as a part of label-noise robust generative models. Our code is available at: https://github.com/byeonghu-na/tdsm.Comment: Accepted at ICLR 202

Autonomous control of terminal erythropoiesis via physical interactions among erythroid cells

AbstractIn vitro erythropoiesis has been studied extensively for its application in the manufacture of transfusable erythrocytes. Unfortunately, culture conditions have not been as effective as in vivo growth conditions, where bone marrow macrophages are known to be a key regulator of erythropoiesis. This study focused on the fact that some erythroblasts are detached from macrophages and only contact other erythroblasts. We hypothesized that additional factors regulate erythroblasts, likely through either physical contact or secreted factors. To further elucidate these critical factors, human erythroblasts derived from cord blood were cultured at high density to mimic marrow conditions. This growth condition resulted in a significantly increased erythroid enucleation rate and viability. We found several novel contact-related signals in erythroblasts: intercellular adhesion molecule-4 (ICAM-4) and deleted in liver cancer-1 (DLC-1). DLC-1, a Rho-GTPase-activating protein, has not previously been reported in erythroid cells, but its interaction with ICAM-4 was demonstrated here. We further confirmed the presence of a secreted form of human ICAM-4 for the first time. When soluble ICAM-4 was added to media, cell viability and enucleation increased with decreased nuclear dysplasia, suggesting that ICAM-4 is a key factor in contact between cells. These results highlight potential new mechanisms for autonomous control of erythropoiesis. The application of these procedures to erythrocyte manufacturing could enhance in vitro erythrocyte production for clinical use

Renal transplantation in a patient with Bartter syndrome and glomerulosclerosis

Bartter syndrome (BS) is a clinically and genetically heterogeneous inherited renal tube disorder characterized by renal salt wasting, hypokalemic metabolic alkalosis and normotensive hyperreninemic hyperaldosteronism. There have been several case reports of BS complicated by focal segmental glomerulosclerosis (FSGS). Here, we have reported the case of a BS patient who developed FSGS and subsequent end-stage renal disease (ESRD) and provided a brief literature review. The patient presented with classic BS at 3 months of age and developed proteinuria at 7 years. Renal biopsy performed at 11 years of age revealed a FSGS perihilar variant. Hemodialysis was initiated at 11 years of age, and kidney transplantation was performed at 16 years of age. The post-transplantation course has been uneventful for more than 3 years with complete disappearance of BS without the recurrence of FSGS. Genetic study revealed a homozygous p.Trp(TGG)610Stop(TGA) mutation in the CLCNKB gene. In summary, BS may be complicated by secondary FSGS due to the adaptive response to chronic salt-losing nephropathy, and FSGS may progress to ESRD in some patients. Renal transplantation in patients with BS and ESRD results in complete remission of BS

Antitumorigenic effect of atmospheric-pressure dielectric barrier discharge on human colorectal cancer cells via regulation of Sp1 transcription factor

Human colorectal cancer cell lines (HT29 and HCT116) were exposed to dielectric barrier discharge (DBD) plasma at atmospheric pressure to investigate the anticancer capacity of the plasma. The dose- and time-dependent effects of DBDP on cell viability, regulation of transcription factor Sp1, cell-cycle analysis, and colony formation were investigated by means of MTS assay, DAPI staining, propidium iodide staining, annexin V-FITC staining, Western blot analysis, RT-PCR analysis, fluorescence microscopy, and anchorage-independent cell transformation assay. By increasing the duration of plasma dose times, significant reductions in the levels of both Sp1 protein and Sp1 mRNA were observed in both cell lines. Also, expression of negative regulators related to the cell cycle (such as p53, p21, and p27) was increased and of the positive regulator cyclin D1 was decreased, indicating that the plasma treatment led to apoptosis and cell-cycle arrest. In addition, the sizes and quantities of colony formation were significantly suppressed even though two cancer promoters, such as TPA and epidermal growth factor, accompanied the plasma treatment. Thus, plasma treatment inhibited cell viability and colony formation by suppressing Sp1, which induced apoptosis and cell-cycle arrest in these two human colorectal cancer cell lines.1