SDDNet: Style-guided Dual-layer Disentanglement Network for Shadow Detection

Despite significant progress in shadow detection, current methods still struggle with the adverse impact of background color, which may lead to errors when shadows are present on complex backgrounds. Drawing inspiration from the human visual system, we treat the input shadow image as a composition of a background layer and a shadow layer, and design a Style-guided Dual-layer Disentanglement Network (SDDNet) to model these layers independently. To achieve this, we devise a Feature Separation and Recombination (FSR) module that decomposes multi-level features into shadow-related and background-related components by offering specialized supervision for each component, while preserving information integrity and avoiding redundancy through the reconstruction constraint. Moreover, we propose a Shadow Style Filter (SSF) module to guide the feature disentanglement by focusing on style differentiation and uniformization. With these two modules and our overall pipeline, our model effectively minimizes the detrimental effects of background color, yielding superior performance on three public datasets with a real-time inference speed of 32 FPS.Comment: Accepted by ACM MM 202

Generative adversarial network for low‐light image enhancement

Low-light image enhancement is rapidly gaining research attention due to the increasing demands of extreme visual tasks in various applications. Although numerous methods exist to enhance image qualities in low light, it is still undetermined how to trade-off between the human observation and computer vision processing. In this work, an effective generative adversarial network structure is proposed comprising both the densely residual block (DRB) and the enhancing block (EB) for low-light image enhancement. Specifically, the proposed end-to-end image enhancement method, consisting of a generator and a discriminator, is trained using the hyper loss function. The DRB adopts the residual and dense skip connections to connect and enhance the features extracted from different depths in the network while the EB receives unique multi-scale features to ensure feature diversity. Additionally, increasing the feature sizes allows the discriminator to further distinguish between fake and real images from the patch levels. The merits of the loss function are also studied to recover both contextual and local details. Extensive experimental results show that our method is capable of dealing with extremely low-light scenes and the realistic feature generator outperforms several state-of-the-art methods in a number of qualitative and quantitative evaluation tests

단백질 복합체 구조예측과 설계를 위한 방법 개발

학위논문(박사) -- 서울대학교대학원 : 자연과학대학 화학부, 2023. 8. 석차옥.단백질 사이의 상호작용은 다양한 생체 내에서 다양한 대사과정과 신호전달 과정에서 중요한 역할을 한다. 이런 역할 때문에 단백질간 상호작용은 질병의 발병 과정에 관련되어 있기 때문에 중요한 치료 표적으로 지목된다. 이러한 상호작용을 원자 수준 구조로 이해하는 것은 단백질의 기능과 특성에 대해 깊은 이해가 가능하게 하고 이를 토대로 분자 약물 또는 단백질 약물을 개발과 개량에 결정적 도움을 줄 수 있다. 이런 맥락에서 컴퓨터를 활용한 단백질 복합체 구조예측 및 상호작용 연구는 주목받아왔다. 최근에 Alphafold2와 RoseTTAFold와 같은 딥러닝 기반의 구조 예측 프로그램의 등장하며 단백질의 구조예측에 대한 성능은 상당히 많이 향상되었다. 그러나 여전히 많은 발전이 필요한 영역들이 남아있으며 특히, 유의미한 다중 단백질서열 정렬(multiple sequence alignment)나 유의미한 정보를 담고 있는 서열 임베딩이 없으면 구조예측 성능이 많이 떨어진다. 그리고 단백질 약물 개발도 단백질 서열공간과 구조 공간을 동시에 예측하고 최적화해야 하는 문제로 상당히 복잡하다. 이 논문에서는 뛰어난 단백질 복합체 구조예측 소프트웨어인 GALAXY에 대해 포괄적으로 다루어 보고, 단백질 구조 예측 및 단백질 복합체 설계 분야의 문제점들을 해결할 수 있는 새로운 두 가지 방법론을 소개한다. 첫째로, AlphaFold2와 구조 공간 담금질(CSA)에 영감을 받아 상보성 결정 영역(CDR) H3 고리를 예측하는 새로운 딥러닝 모델을 소개한다. 이 모델은 구조 예측을 위한 새로운 개념의 모델 구조를 도입하며, 단백질-단백질 결합구조 예측과 일반적인 단백질 구조 예측으로 그 응용 분야를 확장할 가능성을 보여준다.. 둘째로, 'H-map'이라는 새로운 프로그램을 . 이는 표적 단백질의 국소 표면과 강한 상호작용을 해서 결합할 수 있는 아미노산의 종류를 알려주는 H-map 이라는 새로운 프로그램을 소개한다.Protein-protein interactions play a vital role in numerous biological processes and often serve as therapeutic targets due to their involvement in disease pathogenesis. Comprehending the atomistic intricacies of these interactions can lead to the discovery of regulatory molecules for disease-related biological processes and the rational design of proteins for therapeutic applications. The emergence of deep learning-based techniques, such as Alphafold2, RoseTTAFold, and RFdiffusion, has substantially advanced our capabilities in protein structure prediction and design. However, several challenges persist in these domains. Deep learning tools, while transformative, still exhibit limitations, particularly in the absence of strong guiding information for overall conformations, such as those contained in multiple sequence alignment or sequence embedding. Moreover, the protein design problem is quite complex in nature because it requires concurrent optimization in the sequence space and the conformation space. This thesis first provides a comprehensive review of the GALAXY protein modeling package, a highly effective software for protein oligomer structure prediction, and further illuminates the path towards novel breakthroughs in the field of protein structure prediction and protein binder design. Two new methods are then proposed to address the persistent challenges in these areas. First, a novel deep learning model, inspired by the AlphaFold2 structure module and conformational space annealing (CSA) global optimization, is introduced as a technique for predicting the structures of antibody complementarity determining region (CDR) H3 loops. This deep neural network model introduces a novel framework for structure prediction, implying the potential applicability to other prediction domains involving great molecular complexity such as protein-protein docking and ab initio protein structure prediction. Second, we present a new deep neural network amino acid generator called 'H-map' on the surface of the target protein considering the local environment of the target protein only, unlike other methods that require backbone structures of a potential binder.TABLE OF CONTENTS ABSTRACT i TABLE OF CONTENTS iii LIST OF FIGURES viii LIST OF TABLES x 1. Introduction 1 2. Protein Oligomer Structure Prediction with GALAXY Software 4 2.1. Introduction 4 2.2. Brief Introduction of Galaxy Software for Predicting Protein-Protein Complex Structure 6 2.2.1. Overall pipeline for predict protein-protein complex structure with GALAXY Package 6 2.2.2. Protein monomer structure modeling 8 2.2.3. Protein-protein complex structure modeling 9 2.2.4. Protein structure refinement 14 2.3. Applications I: SARS-CoV2-Spike protein structure prediction 16 2.3.1. Introduction 16 2.3.2. Full-length SARS-CoV-2 S protein model building 21 2.3.3. Predicting characteristic stalk movement of the S protein consists of two highly flexible linkers 23 2.4. Applications II: participating in CASP and CAPRI blind prediction experiments 27 2.5. Applications III: prediction of GPCR-peptide complexes 30 2.6. Conclustion 34 3. Deep-Learning based Antibody H3 Loop Structure Predicition Inspired by Alphafold2 and Genetic Algorithm 35 3.1. Introduction 35 3.2. Methods 37 3.2.1. Brief introduction of the overall method 37 3.2.2. Dataset preparation for method training and testing 39 3.2.2.1 Preparation of antibody structure set 39 3.2.2.2. Preparation of general dimer loop set 39 3.2.3. Benchmark set and training set 40 3.2.3.1. IgFold benchmark set 40 3.2.3.2. In-house test set 41 3.2.3.3. Training set and validation set 41 3.2.4. Loop structure prediction neural network architecture 41 3.2.4.1. PerturbInitialStructure : initial loop structure generation moduler for further evolution 42 3.2.4.2. SingleFeatureEmbedder: feature embedding module 44 3.2.4.3. RecycleSingleFeature module 45 3.2.4.4. PairwiseFeatureEmbedder module 45 3.2.4.5. IPAEncoder module 47 3.2.4.6. Cross-over module 48 3.2.4.7. TriangularPairwiseFeature module 51 3.2.4.8. IPAModule 52 3.2.4.9. TorsionAnglePredictior module 53 3.2.4.10. LDDTPredictior module 53 3.2.5. Loss function 54 3.2.6. Training procedure 55 3.2.6.1. Preparation of input 55 3.2.6.2. Data augmentation 56 3.2.6.3. Fine-tuning 57 3.3. Results and discussion 57 3.3.1. Results of CDR H3 loop structure prediction on the benchmark set 57 3.3.2. Evaluate the effect of multi-seed strategy 60 3.3.3. Evolving predicted structures through iterative optimization 60 3.4. Conclusion 69 4. H-map: Amino Acid Generator for Designing and Scoring Protein Binders without Backbone Structure Information 70 4.1. Introduction 70 4.2. Method 74 4.2.1. Overall workflow of the Hmap method 74 4.2.2. Dataset preparation for training and testing 75 4.2.2.1. Amino acid type reconstruction 75 4.2.2.2. Protein-protein docking reranking set 76 4.2.2.3. Mutation effect prediction set: SKEMPI2 76 4.2.3. Algorithm architecture of Hmap 77 4.2.3.1. Input preparation 77 4.2.3.2. SE(3)-Transformer 82 4.2.3.3. Final node-embedding processing 84 4.2.3.4. Loss function 84 4.2.4. Training procedure 89 4.2.5. Performance comparison with our methods 91 4.3. Results and Discussion 91 4.3.1. Performance of amino acid type reconstruction 91 4.3.2. Functional group center position prediction task 95 4.3.3. Protein-protein docking decoy reranking task 95 4.3.4. Mutation effect prediction task 96 4.4. Conclusion 100 5. Conclusion 101 BIBLIOGRAPHY 103 국문초록 111박

Automated liver tissues delineation based on machine learning techniques: A survey, current trends and future orientations

There is no denying how machine learning and computer vision have grown in the recent years. Their highest advantages lie within their automation, suitability, and ability to generate astounding results in a matter of seconds in a reproducible manner. This is aided by the ubiquitous advancements reached in the computing capabilities of current graphical processing units and the highly efficient implementation of such techniques. Hence, in this paper, we survey the key studies that are published between 2014 and 2020, showcasing the different machine learning algorithms researchers have used to segment the liver, hepatic-tumors, and hepatic-vasculature structures. We divide the surveyed studies based on the tissue of interest (hepatic-parenchyma, hepatic-tumors, or hepatic-vessels), highlighting the studies that tackle more than one task simultaneously. Additionally, the machine learning algorithms are classified as either supervised or unsupervised, and further partitioned if the amount of works that fall under a certain scheme is significant. Moreover, different datasets and challenges found in literature and websites, containing masks of the aforementioned tissues, are thoroughly discussed, highlighting the organizers original contributions, and those of other researchers. Also, the metrics that are used excessively in literature are mentioned in our review stressing their relevancy to the task at hand. Finally, critical challenges and future directions are emphasized for innovative researchers to tackle, exposing gaps that need addressing such as the scarcity of many studies on the vessels segmentation challenge, and why their absence needs to be dealt with in an accelerated manner.Comment: 41 pages, 4 figures, 13 equations, 1 table. A review paper on liver tissues segmentation based on automated ML-based technique

Human treelike tubular structure segmentation: A comprehensive review and future perspectives

Various structures in human physiology follow a treelike morphology, which often expresses complexity at very fine scales. Examples of such structures are intrathoracic airways, retinal blood vessels, and hepatic blood vessels. Large collections of 2D and 3D images have been made available by medical imaging modalities such as magnetic resonance imaging (MRI), computed tomography (CT), Optical coherence tomography (OCT) and ultrasound in which the spatial arrangement can be observed. Segmentation of these structures in medical imaging is of great importance since the analysis of the structure provides insights into disease diagnosis, treatment planning, and prognosis. Manually labelling extensive data by radiologists is often time-consuming and error-prone. As a result, automated or semi-automated computational models have become a popular research field of medical imaging in the past two decades, and many have been developed to date. In this survey, we aim to provide a comprehensive review of currently publicly available datasets, segmentation algorithms, and evaluation metrics. In addition, current challenges and future research directions are discussed

A Review on Computer Aided Diagnosis of Acute Brain Stroke.

Amongst the most common causes of death globally, stroke is one of top three affecting over 100 million people worldwide annually. There are two classes of stroke, namely ischemic stroke (due to impairment of blood supply, accounting for ~70% of all strokes) and hemorrhagic stroke (due to bleeding), both of which can result, if untreated, in permanently damaged brain tissue. The discovery that the affected brain tissue (i.e., 'ischemic penumbra') can be salvaged from permanent damage and the bourgeoning growth in computer aided diagnosis has led to major advances in stroke management. Abiding to the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines, we have surveyed a total of 177 research papers published between 2010 and 2021 to highlight the current status and challenges faced by computer aided diagnosis (CAD), machine learning (ML) and deep learning (DL) based techniques for CT and MRI as prime modalities for stroke detection and lesion region segmentation. This work concludes by showcasing the current requirement of this domain, the preferred modality, and prospective research areas