Improving Sample Quality of Diffusion Models Using Self-Attention Guidance

Following generative adversarial networks (GANs), a de facto standard model for image generation, denoising diffusion models (DDMs) have been actively researched and attracted strong attention due to their capability to generate images with high quality and diversity. However, the way the internal self-attention mechanism works inside the UNet of DDMs is under-explored. To unveil them, in this paper, we first investigate the self-attention operations within the black-boxed diffusion models and build hypotheses. Next, we verify the hypotheses about the self-attention map by conducting frequency analysis and testing the relationships with the generated objects. In consequence, we find out that the attention map is closely related to the quality of generated images. On the other hand, diffusion guidance methods based on additional information such as labels are proposed to improve the quality of generated images. Inspired by these methods, we present label-free guidance based on the intermediate self-attention map that can guide existing pretrained diffusion models to generate images with higher fidelity. In addition to the enhanced sample quality when used alone, we show that the results are further improved by combining our method with classifier guidance on ImageNet 128x128.Comment: Project Page: https://ku-cvlab.github.io/Self-Attention-Guidanc

Improving ionic conductivity of von-Alpen-type NASICON ceramic electrolytes via magnesium doping

NASICON (sodium (Na) superionic conductor) compounds have attracted considerable attention as promising solid electrolyte materials for advanced Na-based batteries. In this study, we investigated the improvement in ionic conductivities of von-Alpen-type NASICON (vA-NASICON) ceramic electrolytes by introducing a magnesium ion (Mg2+) as a heterogeneous element. The optimal Mg-doped vA-NASICON exhibited a high ionic conductivity of 3.64×10−3 S·cm−1, which was almost 80% higher than that of un-doped vA-NASICON. The changes in physicochemical properties of the vA-NASICONs through the Mg introduction were systematically analyzed, and their effects on the ionic conductivities of the vA-NASICON were studied in detail. When the optimal ratio of Mg2+ was used in a synthetic process, the relative density (96.6%) and grain boundary ionic conductivity (σgb) were maximized, which improved the total ionic conductivity (σt) of the vA-NASICON. However, when Mg2+ was introduced in excess, the ionic conductivity decreased because of the formation of an undesired sodium magnesium phosphate (NaxMgyPO4) secondary phase. The results of this study are expected to be effectively applied in the development of advanced sodium-based solid electrolytes with high ionic conductivities

Currency crises and the evolution of foreign exchange market: Evidence from minimum spanning tree

We examined the time series properties of the foreign exchange market for 1990-2008 in relation to the history of the currency crises using the minimum spanning tree (MST) approach and made several meaningful observations about the MST of currencies. First, around currency crises, the mean correlation coefficient between currencies decreased whereas the normalized tree length increased. The mean correlation coefficient dropped dramatically passing through the Asian crisis and remained at the lowered level after that. Second, the Euro and the US dollar showed a strong negative correlation after 1997, implying that the prices of the two currencies moved in opposite directions. Third, we observed that Asian countries and Latin American countries moved away from the cluster center (USA) passing through the Asian crisis and Argentine crisis, respectively. (C) 2010 Elsevier B.V. All rights reserved.ONNELA JP, 2003, PHYS REV E, V68MANTEGNA RN, 2000, INRO ECONOPHYSICS COMantegna RN, 1999, EUR PHYS J B, V11, P193Bonanno G, 2004, EUR PHYS J B, V38, P363, DOI 10.1140/epjb/e2004-00129-6McDonald M, 2005, PHYS REV E, V72, DOI 10.1103/PhysRevE.72.046106Mizuno T, 2006, PHYSICA A, V364, P336, DOI 10.1016/j.physa.2005.08.079Coelho R, 2007, PHYSICA A, V376, P455, DOI 10.1016/j.physa.2006.10.045Eom C, 2007, PHYSICA A, V383, P139, DOI 10.1016/j.physa.2007.04.102Tola V, 2008, J ECON DYN CONTROL, V32, P235, DOI 10.1016/j.jedc.2007.01.034Didier T, 2008, J POLICY MODEL, V30, P775, DOI 10.1016/j.jpolmod.2007.09.008Kaminsky GL, 1999, AM ECON REV, V89, P473WEST DB, 1996, INTRO GRAPH THEORYEICHENGREEN B, 1995, ECON POLICY, V21, P249PAPADIMITRIOU CH, 1982, COMBINATORIAL OPTIMIFREEMAN LC, 1977, SOCIOMETRY, V40, P35

Comparing national innovation system among the USA, Japan, and Finland to improve Korean deliberation organization for national science and technology policy

As the Korean government expands the budget for national research and development, the need for an institute that deliberates, coordinates, and operates research development and its budget has increased. In response to these demands, the National Science and Technology Council (NSTC) was recently established. However, to achieve a creative economy, which is the economic system where value is based on novel imaginative qualities rather than the traditional resources of land, labor, and capital, more efficiently, fundamental research regarding the current state of the Korean national administration system of science and technology in Korea is required. Accordingly, this study first analyzes the function and organizational structure of the NSTC in Korea. Second, it investigates the current state of the NSTC in other countries. Finally, the study derives several implications for improving NSTC operation based on the benchmarking study and suggests an operational improvement plan for NSTC with respect to enhancement of function, operation of organization, human resource management, and improvement of the relationships between other departments. The study contributes to analyze the current state of the NSTC in Korea and science and technology (S&T) Councils in other major countries, systematically and in detail. In addition, based on benchmarking study, this study derived operational improvement of NSTC in Korea with four perspectives, including enhancement of function, operation of organization, human resource management, and improvement of the relationships between other departments

Identifying the Policy Direction of National R&D Programs Based on Data Envelopment Analysis and Diversity Index Approach

The Korean government has been continuously conducting diverse national R&D programs to discover new growth engines. The Republic of Korea is one of the countries with the largest investment in national R&D, but its efficiency was relatively low. In response, this study established a framework to identify the characteristics and direction of outstanding R&D programs. In this study, the performance of the R&D programs was identified in the sub-program unit. The efficiency of the national R&D program was analyzed using the data envelopment analysis model through the outputs of the national R&D programs such as papers and patents. However, patent and paper output would take time to be realized. Therefore, this study also calculated the diversity index of R&D programs to identify their potential expected performance. This study applied the suggested framework in the electric vehicle fields, which is one of the core growth engines of South Korea. A list of outstanding programs was identified from the National Institute of Science and Technology Information (NTIS) data. Additionally, this study also discovered the main technology areas and their current issues of outstanding and brand-new R&D programs. These results could contribute to suggesting the policy direction to conduct high-performance national R&D programs

Introduction of Infection Prevention Tracheal Intubation Protocol during the COVID-19 Pandemic Is Not Associated with First-Pass Success Rates of Endotracheal Intubation in the Emergency Department: A Before-and-After Comparative Study

Aerosols and droplets have put healthcare workers performing airway management at high risk of contracting coronavirus disease 2019 (COVID-19). Experts have developed endotracheal intubation (ETI) guidelines and protocols to protect intubators from infection. We aimed to determine whether changes in the emergency department (ED) intubation protocol to prevent COVID-19 infection were associated with first-pass success (FPS) rates in ETI. We used data from the airway management registries in two academic EDs. The study was divided into pre-pandemic (January 2018 to January 2020) and pandemic (February 2020 to February 2022) periods. We selected 2476 intubation cases, including 1151 and 1325 cases recorded before and during the pandemic, respectively. During the pandemic, the FPS rate was 92.2%, which did not change significantly, and major complications increased slightly but not significantly compared with the pre-pandemic period. The OR for the FPS of applying infection prevention intubation protocols was 0.72 (p = 0.069) in a subgroup analysis, junior emergency physicians (PGY1 residents) had an FPS of less than 80% regardless of pandemic protocol implementation. The FPS rate of senior emergency physicians in physiologically difficult airways decreased significantly during the pandemic (98.0% to 88.5%). In conclusion, the FPS rate and complications for adult ETI performed by emergency physicians using COVID-19 infection prevention intubation protocols were similar to pre-pandemic conditions

LSim: Fine-Grained Simulation Framework for Large-Scale Performance Evaluation

As large-scale workloads with massive parallelism emerge, the demand for large-scale systems such as datacenters and supercomputers is rising sharply. To accurately design a large-scale system, architects heavily rely on performance modeling at design phases. However, modeling a large-scale workload without a large-scale system is a challenging problem. This paper presents LSim, a framework for large-scale performance evaluation. Based on the captured behavior within small-scale workload traces, LSim extrapolates the behavior of the workload on a large-scale system. To do so, we propose two techniques: (1) representative trace model and (2) function latency model to synthesize a trace and to predict the latency of functions in the synthesized trace, respectively.N

Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidance

Recent studies have demonstrated that diffusion models are capable of generating high-quality samples, but their quality heavily depends on sampling guidance techniques, such as classifier guidance (CG) and classifier-free guidance (CFG). These techniques are often not applicable in unconditional generation or in various downstream tasks such as image restoration. In this paper, we propose a novel sampling guidance, called Perturbed-Attention Guidance (PAG), which improves diffusion sample quality across both unconditional and conditional settings, achieving this without requiring additional training or the integration of external modules. PAG is designed to progressively enhance the structure of samples throughout the denoising process. It involves generating intermediate samples with degraded structure by substituting selected self-attention maps in diffusion U-Net with an identity matrix, by considering the self-attention mechanisms' ability to capture structural information, and guiding the denoising process away from these degraded samples. In both ADM and Stable Diffusion, PAG surprisingly improves sample quality in conditional and even unconditional scenarios. Moreover, PAG significantly improves the baseline performance in various downstream tasks where existing guidances such as CG or CFG cannot be fully utilized, including ControlNet with empty prompts and image restoration such as inpainting and deblurring.Comment: Project page is available at https://ku-cvlab.github.io/Perturbed-Attention-Guidanc

Wafer‐Scale Synthesis of Mixed‐Dimensional Heterostructures via Manipulating Platinization Conditions

Abstract 2D van der Waals (vdW) hetero integration, which features exotic interplanar interactions derived from mixed‐dimensional heterostructures, is an emergent platform for implementing high‐performance electronics and broadband/wavelength‐tunable photodetectors. However, the production of large‐area 2D spatially homogeneous transition‐metal dichalcogenides (TMDs) and elucidation of the electrostatic dynamics governing the interlayer interactions are two paramount prerequisites for realizing practical 2D‐TMD‐heterostructure‐based photodetectors. Here, a wafer‐scale synthesis of mixed‐dimensional Pt–MoS2‐based vdW heterostructures is unprecedentedly demonstrated by manipulating the platinization conditions. The rationally designed platinization yields dimensionality‐tailored Pt, including Pt nanofilm, Pt nanoparticles, and Pt atoms, with MoS2 as host platform. From density functional theory calculations, this study insights that Mo vacancy sites on the MoS2 surface are thermo‐dynamically favorable sites for Pt with an adsorption energy of −2.25 eV, then Pt clusters are sequentially formed neighboring the specific Pt‐substituted position with a formation energy of 1.30 eV. Intensive microscopic and spectroscopic analyses reveal the structural, chemical, and electrical features, validating the proposed dynamics‐related mechanism. The dimensionality‐tailored vdW heterostructures exhibit outstanding optoelectrical properties with excellent photoresponsivity (2.04 mA W−1) and highly sensitive detectivity (9.82 × 106 cm Hz1/2 W−1)