Towards Flexible Inductive Bias via Progressive Reparameterization Scheduling

There are two de facto standard architectures in recent computer vision: Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). Strong inductive biases of convolutions help the model learn sample effectively, but such strong biases also limit the upper bound of CNNs when sufficient data are available. On the contrary, ViT is inferior to CNNs for small data but superior for sufficient data. Recent approaches attempt to combine the strengths of these two architectures. However, we show these approaches overlook that the optimal inductive bias also changes according to the target data scale changes by comparing various models' accuracy on subsets of sampled ImageNet at different ratios. In addition, through Fourier analysis of feature maps, the model's response patterns according to signal frequency changes, we observe which inductive bias is advantageous for each data scale. The more convolution-like inductive bias is included in the model, the smaller the data scale is required where the ViT-like model outperforms the ResNet performance. To obtain a model with flexible inductive bias on the data scale, we show reparameterization can interpolate inductive bias between convolution and self-attention. By adjusting the number of epochs the model stays in the convolution, we show that reparameterization from convolution to self-attention interpolates the Fourier analysis pattern between CNNs and ViTs. Adapting these findings, we propose Progressive Reparameterization Scheduling (PRS), in which reparameterization adjusts the required amount of convolution-like or self-attention-like inductive bias per layer. For small-scale datasets, our PRS performs reparameterization from convolution to self-attention linearly faster at the late stage layer. PRS outperformed previous studies on the small-scale dataset, e.g., CIFAR-100.Comment: Accepted at VIPriors ECCVW 2022, camera-ready versio

Addressing Negative Transfer in Diffusion Models

Diffusion-based generative models have achieved remarkable success in various domains. It trains a model on denoising tasks that encompass different noise levels simultaneously, representing a form of multi-task learning (MTL). However, analyzing and improving diffusion models from an MTL perspective remains under-explored. In particular, MTL can sometimes lead to the well-known phenomenon of $\textit{negative transfer}$, which results in the performance degradation of certain tasks due to conflicts between tasks. In this paper, we aim to analyze diffusion training from an MTL standpoint, presenting two key observations: $\textbf{(O1)}$ the task affinity between denoising tasks diminishes as the gap between noise levels widens, and $\textbf{(O2)}$ negative transfer can arise even in the context of diffusion training. Building upon these observations, our objective is to enhance diffusion training by mitigating negative transfer. To achieve this, we propose leveraging existing MTL methods, but the presence of a huge number of denoising tasks makes this computationally expensive to calculate the necessary per-task loss or gradient. To address this challenge, we propose clustering the denoising tasks into small task clusters and applying MTL methods to them. Specifically, based on $\textbf{(O2)}$, we employ interval clustering to enforce temporal proximity among denoising tasks within clusters. We show that interval clustering can be solved with dynamic programming and utilize signal-to-noise ratio, timestep, and task affinity for clustering objectives. Through this, our approach addresses the issue of negative transfer in diffusion models by allowing for efficient computation of MTL methods. We validate the proposed clustering and its integration with MTL methods through various experiments, demonstrating improved sample quality of diffusion models.Comment: 22 pages, 12 figures, under revie

Towards Practical Plug-and-Play Diffusion Models

Diffusion-based generative models have achieved remarkable success in image generation. Their guidance formulation allows an external model to plug-and-play control the generation process for various tasks without fine-tuning the diffusion model. However, the direct use of publicly available off-the-shelf models for guidance fails due to their poor performance on noisy inputs. For that, the existing practice is to fine-tune the guidance models with labeled data corrupted with noises. In this paper, we argue that this practice has limitations in two aspects: (1) performing on inputs with extremely various noises is too hard for a single model; (2) collecting labeled datasets hinders scaling up for various tasks. To tackle the limitations, we propose a novel strategy that leverages multiple experts where each expert is specialized in a particular noise range and guides the reverse process at its corresponding timesteps. However, as it is infeasible to manage multiple networks and utilize labeled data, we present a practical guidance framework termed Practical Plug-And-Play (PPAP), which leverages parameter-efficient fine-tuning and data-free knowledge transfer. We exhaustively conduct ImageNet class conditional generation experiments to show that our method can successfully guide diffusion with small trainable parameters and no labeled data. Finally, we show that image classifiers, depth estimators, and semantic segmentation models can guide publicly available GLIDE through our framework in a plug-and-play manner

A Robot Operating System Framework for Secure UAV Communications

To perform advanced operations with unmanned aerial vehicles (UAVs), it is crucial that components other than the existing ones such as flight controller, network devices, and ground control station (GCS) are also used. The inevitable addition of hardware and software to accomplish UAV operations may lead to security vulnerabilities through various vectors. Hence, we propose a security framework in this study to improve the security of an unmanned aerial system (UAS). The proposed framework operates in the robot operating system (ROS) and is designed to focus on sev-eral perspectives, such as overhead arising from additional security elements and security issues essential for flight missions. The UAS is operated in a nonnative and native ROS environment. The performance of the proposed framework in both environments is verified through experiments. © 2021 by the authors. Licensee MDPI, Basel, Switzerland.1

KIOM-79, an Inhibitor of AGEs–Protein Cross-linking, Prevents Progression of Nephropathy in Zucker Diabetic Fatty Rats

Advanced glycation end products (AGEs) have been implicated in the development of diabetic complications, including diabetic nephropathy. KIOM-79, an 80% ethanolic extract obtained from parched Puerariae Radix, gingered Magnolia Cortex, Glycyrrhiza Radix and Euphorbia Radix, was investigated for its effects on the development of renal disease in Zucker diabetic fatty rats, an animal model of type 2 diabetes. In vitro inhibitory effect of KIOM-79 on AGEs cross-linking was examined by enzyme-linked immunosorbent assay (ELISA). KIOM-79 (50 mg/kg/day) was given to Zucker diabetic fatty rats for 13 weeks. Body and kidney weight, blood glucose, glycated hemoglobin, urinary albumin and creatinine excretions were monitored. Kidney histopathology, collagen accumulation, fibrinogen and transforming growth factor-beta 1 (TGF-β1) expression were also examined. KIOM-79 reduced blood glucose, kidney weight, histologic renal damage and albuminuria in Zucker diabetic fatty rats. KIOM-79 prevented glomerulosclerosis, tubular degeneration, collagen deposition and podocyte apoptosis. In the renal cortex, TGF-β1, fibronectin mRNA and protein were significantly reduced by KIOM-79 treatment. KIOM-79 reduces AGEs accumulation in vivo, AGE–protein cross-linking and protein oxidation. KIOM-79 could be beneficial in preventing the progression of diabetic glomerularsclerosis in type 2 diabetic rats by attenuating AGEs deposition in the glomeruli

Posterior Epidural Migration of a Lumbar Intervertebral Disc Fragment Resembling a Spinal Tumor: A Case Report

Posterior epidural migration of a lumbar intervertebral disc fragment (PEMLIF) is uncommon because of anatomical barriers. It is difficult to diagnose PEMLIF definitively because of its relatively rare incidence and the ambiguity of radiological findings resembling spinal tumors. This case report describes a 76-year-old man with sudden-onset weakness and pain in both legs. Electromyography revealed bilateral lumbosacral polyradiculopathy with a mass-like lesion in L2-3 dorsal epidural space on lumbosacral magnetic resonance imaging (MRI). The lesion showed peripheral rim enhancement on T1-weighted MRI with gadolinium administration. The patient underwent decompressive L2-3 central laminectomy, to remove the mass-like lesion. The excised lesion was confirmed as an intervertebral disc. The possibility of PEMLIF should be considered when rim enhancement is observed in the epidural space on MRI scans and electrodiagnostic features of polyradiculopathy with sudden symptoms of cauda equina syndrome

Association of pathway mutation with survival after recurrence in colorectal cancer patients treated with adjuvant fluoropyrimidine and oxaliplatin chemotherapy

Background Although the prognostic biomarkers associated with colorectal cancer (CRC) survival are well known, there are limited data on the association between the molecular characteristics and survival after recurrence (SAR). The purpose of this study was to assess the association between pathway mutations and SAR. Methods Of the 516 patients with stage III or high risk stage II CRC patients treated with surgery and adjuvant chemotherapy, 87 who had distant recurrence were included in the present study. We analyzed the association between SAR and mutations of 40 genes included in the five critical pathways of CRC (WNT, P53, RTK-RAS, TGF-β, and PI3K). Results Mutation of genes within the WNT, P53, RTK-RAS, TGF-β, and PI3K pathways were shown in 69(79.3%), 60(69.0%), 57(65.5%), 21(24.1%), and 19(21.8%) patients, respectively. Patients with TGF-β pathway mutation were younger and had higher incidence of mucinous adenocarcinoma (MAC) histology and microsatellite instability-high. TGF-β pathway mutation (median SAR of 21.6 vs. 44.4 months, p = 0.021) and MAC (20.0 vs. 44.4 months, p = 0.003) were associated with poor SAR, and receiving curative resection after recurrence was associated with favorable SAR (Not reached vs. 23.6 months, p <  0.001). Mutations in genes within other critical pathways were not associated with SAR. When MAC was excluded as a covariate, multivariate analysis revealed TGF-β pathway mutation and curative resection after distant recurrence as an independent prognostic factor for SAR. The impact of TGF-β pathway mutations were predicted using the PROVEAN, SIFT, and PolyPhen-2. Among 25 mutations, 23(92.0%)-24(96.0%) mutations were predicted to be damaging mutation. Conclusions Mutation in genes within TGF-β pathway may have negative prognostic role for SAR in CRC. Other pathway mutations were not associated with SAR.This research was supported by the Seoul National University Hospital (SNUH) Research Fund (03–2014-0440) and a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HI14C1277 and HI13C2163). The funding bodies had no influence on the design of the study and collection, analysis, and interpretation of data and in writing the manuscript

보안된 UAV 통신을 위한 로봇 운영 체제 프레임워크

To perform advanced operations with unmanned aerial vehicles (UAVs), it is crucial that components other than the existing ones such as flight controller, network devices, and GCS are used. The feature of obstacle avoidance is added to the pre-existing simple waypoint missions to ensure the commercialization of UAVs. However, this feature requires additional hardware and software to recognize obstacles based on radar or lidar. The inevitable addition of components to accomplish this functionality may lead to security vulnerabilities through various vectors. Hence, we propose a security framework in this study to improve the security of UAS. The proposed framework operates in the ROS (robot operating system) and is designed to focus on several perspectives such as overhead arising from additional security elements and security issues essential for flight missions. The UAS is operated in a non-native and native ROS environment. The performance of the proposed framework in both environments is verified through experiments.YⅠ. INTRODUCTION 1 Ⅱ. BACKGROUND 3 2.1 Unmanned Aerial System (UAS) 3 2.2 Robot Operating System (ROS) 5 2.3 Rosbridge 6 2.4 Safety tool of ROS 6 Ⅲ. VULNEARABILITY DEFINITION OF ROBOT OPERATING SYSTEM 9 3.1 Vulnerability in CPS 9 3.2 UAS data transmission layer model in CPS perspective 10 3.3 Model of ROS-based UAS 11 3.4 Vulnerability of ROS-based UAS 13 Ⅳ. RELATED WORK 15 Ⅴ. PROPOSED METHOD 17 5.1 Registration of a new node 18 5.2 Signature with HMAC 19 5.3 Performance and Conceptual Comparison 21 Ⅵ. TEST 23 6.1 Experiment environment of UAS 23 6.2 Experiment on native ROS attack 25 6.3 Experiment on non-native ROS attack 28 Ⅶ. CONCLUSION 32 REFERENCES 33 SUMMARY (Korean) 35본 논문은 ROS가 적용된 UAS에서의 취약점을 분석하고, 이를 방어할 수 있는 보안 프레임 워크를 제시한다. 최근 여러 상업매체 또는 군에서 동작하는 UAV 는 개별적으로 동작하는 전통적인 임베디드와 달리 컴퓨팅 시스템과 물리시스템이 밀접한 상호작용을 하며 동작한다. 이러한 특성을 갖는 UAV를 운용하는 시스템인 UAS 는 CPS 의 한 범주에 속한다. CPS 는 계산, 네트워킹, 물리적 프로세스가 하나의 피드백 루프로 통합된 시스템을 의미한다. 이 중 네트워킹을 담당하는 데이터 전송 계층은 장소, 상황, 하드웨어, 소프트웨어 등 다양한 통신 네트워크의 접근성을 갖는 CPS 의 특성으로 인해 취약한 계층이다. 이러한 사실은 CPS 의 일부인 UAS 도 같은 취약점을 갖고 있음을 의미한다. 최근 UAV 의 사용은 단순한 비행 미션 수행뿐만 아니라 군집비행, 자율비행 등의 특수한 미션을 수행한다. 해당 미션을 위해서는 컴퓨팅 능력이 있는 하드웨어 및 소프트웨어가 UAS 에 추가 되어야한다. 그 소프트웨어 중 하나로 ROS가 사용되며, 이는 보안적 요소의 부재로 공격자가 로봇 시스템을 망가뜨리는 것을 허용한다. 우리는 ROS 의 보안을 위해 기존에 연구된 내용을 조사하였으며, 해당 방법이 UAS 에 적용 가능한지 여부를 분석하였다. 우리는 시간에 민감한 UAS 에 적용할 수 있는 가벼운 오버헤드를 갖는 보안 프레임워크를 제안한다. 또한 ROS 를 활용할 수 있는 두 가지 접근법인 native ROS 와 non-native ROS 환경에서 실제 실험을 통해 보안 프레임워크의 성능을 검증한다MasterdCollectio