69 research outputs found
Neural Diffeomorphic Non-uniform B-spline Flows
Normalizing flows have been successfully modeling a complex probability
distribution as an invertible transformation of a simple base distribution.
However, there are often applications that require more than invertibility. For
instance, the computation of energies and forces in physics requires the second
derivatives of the transformation to be well-defined and continuous. Smooth
normalizing flows employ infinitely differentiable transformation, but with the
price of slow non-analytic inverse transforms. In this work, we propose
diffeomorphic non-uniform B-spline flows that are at least twice continuously
differentiable while bi-Lipschitz continuous, enabling efficient
parametrization while retaining analytic inverse transforms based on a
sufficient condition for diffeomorphism. Firstly, we investigate the sufficient
condition for Ck-2-diffeomorphic non-uniform kth-order B-spline
transformations. Then, we derive an analytic inverse transformation of the
non-uniform cubic B-spline transformation for neural diffeomorphic non-uniform
B-spline flows. Lastly, we performed experiments on solving the force matching
problem in Boltzmann generators, demonstrating that our C2-diffeomorphic
non-uniform B-spline flows yielded solutions better than previous spline flows
and faster than smooth normalizing flows. Our source code is publicly available
at https://github.com/smhongok/Non-uniform-B-spline-Flow.Comment: Accepted to AAAI 202
The Pandemic, Ecological Justice, and Zhu Xi’s Philosophy
COVID-19 has brought many changes to society and encouraged mankind to reflect on its civilization. The pandemic has revealed that our health care systems and community solidarity are far more fragile than we believed. It made us rethink the solidarity of human civilization and community, and more fundamentally, reconsider the global ecosystem beyond human society. This paper claims that COVID-19 was an inevitable result of the anthropocentric perspective, and argues that it is necessary to change the perception to an ecological worldview and practice ecological justice in order to solve this situation. First, it analyses the ecological reasons for the regular outbreak of zoonotic diseases, including COVID-19, and then it examines Naess’s deep ecology with regard to a fundamental change of perception, but also finds several weaknesses in this. Third, this paper focuses on Zhu Xi’s philosophy in order to compensate for the weaknesses of deep ecology. It argues for the importance of human roles and obligations in relation to the safety and health of the environment based on his philosophy, and explains ecological justice by applying his social equality theory to ecology. Finally, it sheds new light on Zhu Xi’s theory of investigation of things (gewu æ ¼ç‰©) as a practical way of implementing ecological justice
On Exact Inversion of DPM-Solvers
Diffusion probabilistic models (DPMs) are a key component in modern
generative models. DPM-solvers have achieved reduced latency and enhanced
quality significantly, but have posed challenges to find the exact inverse
(i.e., finding the initial noise from the given image). Here we investigate the
exact inversions for DPM-solvers and propose algorithms to perform them when
samples are generated by the first-order as well as higher-order DPM-solvers.
For each explicit denoising step in DPM-solvers, we formulated the inversions
using implicit methods such as gradient descent or forward step method to
ensure the robustness to large classifier-free guidance unlike the prior
approach using fixed-point iteration. Experimental results demonstrated that
our proposed exact inversion methods significantly reduced the error of both
image and noise reconstructions, greatly enhanced the ability to distinguish
invisible watermarks and well prevented unintended background changes
consistently during image editing. Project page:
\url{https://smhongok.github.io/inv-dpm.html}.Comment: 16 page
DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation
Transformer is a deep learning language model widely used for natural
language processing (NLP) services in datacenters. Among transformer models,
Generative Pre-trained Transformer (GPT) has achieved remarkable performance in
text generation, or natural language generation (NLG), which needs the
processing of a large input context in the summarization stage, followed by the
generation stage that produces a single word at a time. The conventional
platforms such as GPU are specialized for the parallel processing of large
inputs in the summarization stage, but their performance significantly degrades
in the generation stage due to its sequential characteristic. Therefore, an
efficient hardware platform is required to address the high latency caused by
the sequential characteristic of text generation.
In this paper, we present DFX, a multi-FPGA acceleration appliance that
executes GPT-2 model inference end-to-end with low latency and high throughput
in both summarization and generation stages. DFX uses model parallelism and
optimized dataflow that is model-and-hardware-aware for fast simultaneous
workload execution among devices. Its compute cores operate on custom
instructions and provide GPT-2 operations end-to-end. We implement the proposed
hardware architecture on four Xilinx Alveo U280 FPGAs and utilize all of the
channels of the high bandwidth memory (HBM) and the maximum number of compute
resources for high hardware efficiency. DFX achieves 5.58x speedup and 3.99x
energy efficiency over four NVIDIA V100 GPUs on the modern GPT-2 model. DFX is
also 8.21x more cost-effective than the GPU appliance, suggesting that it is a
promising solution for text generation workloads in cloud datacenters.Comment: Extension of HOTCHIPS 2022 and accepted in MICRO 202
Echocardiographic View Classification with Integrated Out-of-Distribution Detection for Enhanced Automatic Echocardiographic Analysis
In the rapidly evolving field of automatic echocardiographic analysis and
interpretation, automatic view classification is a critical yet challenging
task, owing to the inherent complexity and variability of echocardiographic
data. This study presents ECHOcardiography VIew Classification with
Out-of-Distribution dEtection (ECHO-VICODE), a novel deep learning-based
framework that effectively addresses this challenge by training to classify 31
classes, surpassing previous studies and demonstrating its capacity to handle a
wide range of echocardiographic views. Furthermore, ECHO-VICODE incorporates an
integrated out-of-distribution (OOD) detection function, leveraging the
relative Mahalanobis distance to effectively identify 'near-OOD' instances
commonly encountered in echocardiographic data. Through extensive
experimentation, we demonstrated the outstanding performance of ECHO-VICODE in
terms of view classification and OOD detection, significantly reducing the
potential for errors in echocardiographic analyses. This pioneering study
significantly advances the domain of automated echocardiography analysis and
exhibits promising prospects for substantial applications in extensive clinical
research and practice
Optimization, Modification and Applications of Gold Nanoparticles as the Substrates of Surface Enhanced Raman Spectroscopy
Surface enhanced Raman spectroscopy (SERS) is one of the techniques that overcomes the poor intensity of Raman scattering by utilizing the metal surface to enhance the Raman scattering. So far, silver (Ag), gold (Au) and copper (Cu) have been demonstrated to provide good enhancement for Raman signals. Many studies have proved that SERS is a powerful technique. However, the origin of the enhancement still needs clarification. More importantly, how to further improve the SERS enhancement through optimization of the SERS substrates and technique is a long and enduring challenge. Chapter 3 is dedicated to find out the optimal size of AuNPs. Au NPs with different sizes were synthesized from 17 nm to 80 nm. The SERS activities of AuNPs were tested using target molecules of 4-aminothiophenol and 4-nitrothiophenol. The experiments were performed under three different conditions: same number of AuNPs, same surface area of AuNPs, and same concentration of AuNPs. For the same number of AuNPs, it showed linear relationship between the enhancement factor and sizes of AuNPs. However, in case of same surface area and concentration, the maximum enhancement was achieved around 50 nm AuNPs. These results were identical for both molecules, which indicate that the conclusions might be also applicable to other analytes. More importantly, the highest SERS enhancement can be achieved with AuNPs of 50 nm while they introduce minimum toxicity to the biological samples. Once the optimal size of AuNPs was found, the work in chapter 4 was dedicated to find the way to improve the SERS enhancement of molecules that do not have strong affinity toward the surface of AuNPs. The affinity was improved by using 2-mercaptoethanol as a linking molecule (2MELM) since the thiol group of 2ME can be strongly adsorbed on the surface of AuNPs, and the other end of 2ME has a hydroxyl group that can induce intermolecular forces toward the molecules of interest. Three molecules were chosen as the target molecule (TM): benzoic acid, cyclohexanol and 1,3-cyclohexanediol. When the results were compared between the substrates with LM and without LM, the spectra of benzoic acid did not display any difference. However, the spectra of the other two TMs show higher enhancement with presence of LM than that without. The results from this simple modification method shine light on the possible characterization or detection of molecules that have low affinity toward the metal surface using SERS. SERS was also applied to study the intermolecular interaction in the field of Material Science. For the first part of the chapter 5, the interaction between vitamin B12 and metal organic frame (MOF) was studied. The results confirmed the encapsulation and also, strong interactions between the two components were observed from the shift of vibrational modes of both VB12 and Tb-MOF upon encapsulation. Second part of the chapter 5 was dedicated to study the functionalization of single wall carbon nanotubes in polymer network. The results confirm the successfully functionalized carbon nanotube. SERS was used to detect different biomarkers through collaborations and the results are shown in Chapter 6. The first part was neurotransmitters, dopamin, melatonin and serotonin. All of them showed low μM range as their detection limit. Other two parts are the priliminary results from studying caffeine and nicotine. The results were promsing exhibiting detection limits in low nM to sub nM range. None of these results were performed with optimized condition, therefore, when the conditions are optimized, it is very probable to have even lower detection limits
Optimal Size of Gold Nanoparticles for Surface-Enhanced Raman Spectroscopy under Different Conditions
Gold nanoparticles have been used as effective surface-enhanced Raman spectroscopy (SERS) substrates for decades. However, the origin of the enhancement and the effect of the size of nanoparticles still need clarification. Here, gold nanoparticles with different sizes from 17 to 80 nm were synthesized and characterized, and their SERS enhancement toward both 4-aminothiophenol and 4-nitrothiophenol was examined. For the same number of nanoparticles, the enhancement factor generated from the gold nanoparticles increases as the size of nanoparticles increases. Interestingly, when the concentration of gold or the total surface area of gold nanoparticles was kept the same, the optimal size of gold nanoparticles was found out to be around 50 nm when the enhancement factor reached a maximum. The same size effect was observed for both 4-aminothiophenol and 4-nitrothiophenol, which suggests that the conclusions drawn in this study might also be applicable to other adsorbates during SERS measurements
Optimal Size of Gold Nanoparticles for Surface-Enhanced Raman Spectroscopy under Different Conditions
Gold nanoparticles have been used as effective surface-enhanced Raman spectroscopy (SERS) substrates for decades. However, the origin of the enhancement and the effect of the size of nanoparticles still need clarification. Here, gold nanoparticles with different sizes from 17 to 80 nm were synthesized and characterized, and their SERS enhancement toward both 4-aminothiophenol and 4-nitrothiophenol was examined. For the same number of nanoparticles, the enhancement factor generated from the gold nanoparticles increases as the size of nanoparticles increases. Interestingly, when the concentration of gold or the total surface area of gold nanoparticles was kept the same, the optimal size of gold nanoparticles was found out to be around 50 nm when the enhancement factor reached a maximum. The same size effect was observed for both 4-aminothiophenol and 4-nitrothiophenol, which suggests that the conclusions drawn in this study might also be applicable to other adsorbates during SERS measurements
- …