Secretory vesicles are preferentially targeted to areas of low molecular SNARE density

Intercellular communication is commonly mediated by the regulated fusion, or exocytosis, of vesicles with the cell surface. SNARE (soluble N-ethymaleimide sensitive factor attachment protein receptor) proteins are the catalytic core of the secretory machinery, driving vesicle and plasma membrane merger. Plasma membrane SNAREs (tSNAREs) are proposed to reside in dense clusters containing many molecules, thus providing a concentrated reservoir to promote membrane fusion. However, biophysical experiments suggest that a small number of SNAREs are sufficient to drive a single fusion event. Here we show, using molecular imaging, that the majority of tSNARE molecules are spatially separated from secretory vesicles. Furthermore, the motilities of the individual tSNAREs are constrained in membrane micro-domains, maintaining a non-random molecular distribution and limiting the maximum number of molecules encountered by secretory vesicles. Together our results provide a new model for the molecular mechanism of regulated exocytosis and demonstrate the exquisite organization of the plasma membrane at the level of individual molecular machines

A general strategy to develop fluorogenic polymethine dyes for bioimaging

Fluorescence imaging is an invaluable tool to study biological processes and further progress depends on the development of advanced fluorogenic probes that reach intracellular targets and label them with high specificity. Excellent fluorogenic rhodamine dyes have been reported, but they often require long and low-yielding syntheses, and are spectrally limited to the visible range. Here we present a general strategy to transform polymethine compounds into fluorogenic dyes using an intramolecular ring-closure approach. We illustrate the generality of this method by creating both spontaneously blinking and no-wash, turn-on polymethine dyes with emissions across the visible and near-infrared spectrum. These probes are compatible with self-labelling proteins and small-molecule targeting ligands, and can be combined with rhodamine-based dyes for multicolour and fluorescence lifetime multiplexing imaging. This strategy provides access to bright, fluorogenic dyes that emit at wavelengths that are more red-shifted compared with those of existing rhodamine-based dyes

Evidence for DNA-mediated nuclear compartmentalization distinct from phase separation.

RNA Polymerase II (Pol II) and transcription factors form concentrated hubs in cells via multivalent protein-protein interactions, often mediated by proteins with intrinsically disordered regions. During Herpes Simplex Virus infection, viral replication compartments (RCs) efficiently enrich host Pol II into membraneless domains, reminiscent of liquid-liquid phase separation. Despite sharing several properties with phase-separated condensates, we show that RCs operate via a distinct mechanism wherein unrestricted nonspecific protein-DNA interactions efficiently outcompete host chromatin, profoundly influencing the way DNA-binding proteins explore RCs. We find that the viral genome remains largely nucleosome-free, and this increase in accessibility allows Pol II and other DNA-binding proteins to repeatedly visit nearby DNA binding sites. This anisotropic behavior creates local accumulations of protein factors despite their unrestricted diffusion across RC boundaries. Our results reveal underappreciated consequences of nonspecific DNA binding in shaping gene activity, and suggest additional roles for chromatin in modulating nuclear function and organization

A Survey of Deep Learning-Based Object Detection

Object detection is one of the most important and challenging branches of computer vision, which has been widely applied in peoples life, such as monitoring security, autonomous driving and so on, with the purpose of locating instances of semantic objects of a certain class. With the rapid development of deep learning networks for detection tasks, the performance of object detectors has been greatly improved. In order to understand the main development status of object detection pipeline, thoroughly and deeply, in this survey, we first analyze the methods of existing typical detection models and describe the benchmark datasets. Afterwards and primarily, we provide a comprehensive overview of a variety of object detection methods in a systematic manner, covering the one-stage and two-stage detectors. Moreover, we list the traditional and new applications. Some representative branches of object detection are analyzed as well. Finally, we discuss the architecture of exploiting these object detection methods to build an effective and efficient system and point out a set of development trends to better follow the state-of-the-art algorithms and further research.Comment: 30 pages,12 figure

Appearance and Geometry Assisted Visual Navigation in Urban Areas

Navigation is a fundamental task for mobile robots in applications such as exploration, surveillance, and search and rescue. The task involves solving the simultaneous localization and mapping (SLAM) problem, where a map of the environment is constructed. In order for this map to be useful for a given application, a suitable scene representation needs to be defined that allows spatial information sharing between robots and also between humans and robots. High-level scene representations have the benefit of being more robust and having higher exchangeability for interpretation. With the aim of higher level scene representation, in this work we explore high-level landmarks and their usage using geometric and appearance information to assist mobile robot navigation in urban areas. In visual SLAM, image registration is a key problem. While feature-based methods such as scale-invariant feature transform (SIFT) matching are popular, they do not utilize appearance information as a whole and will suffer from low-resolution images. We study appearance-based methods and propose a scale-space integrated Lucas-Kanade’s method that can estimate geometric transformations and also take into account image appearance with different resolutions. We compare our method against state-of-the-art methods and show that our method can register images efficiently with high accuracy. In urban areas, planar building facades (PBFs) are basic components of the quasirectilinear environment. Hence, segmentation and mapping of PBFs can increase a robot’s abilities of scene understanding and localization. We propose a vision-based PBF segmentation and mapping technique that combines both appearance and geometric constraints to segment out planar regions. Then, geometric constraints such as reprojection errors, orientation constraints, and coplanarity constraints are used in an optimization process to improve the mapping of PBFs. A major issue in monocular visual SLAM is scale drift. While depth sensors, such as lidar, are free from scale drift, this type of sensors are usually more expensive compared to cameras. To enable low-cost mobile robots equipped with monocular cameras to obtain accurate position information, we use a 2D lidar map to rectify imprecise visual SLAM results using planar structures. We propose a two-step optimization approach assisted by a penalty function to improve on low-quality local minima results. Robot paths for navigation can be either automatically generated by a motion planning algorithm or provided by a human. In both cases, a scene representation of the environment, i.e., a map, is useful to specify meaningful tasks for the robot. However, SLAM results usually produce a sparse scene representation that consists of low-level landmarks, such as point clouds, which are neither convenient nor intuitive to use for task specification. We present a system that allows users to program mobile robots using high-level landmarks from appearance data

Light Microscopy Combined with Computational Image Analysis Uncovers Virus-Specific Infection Phenotypes and Host Cell State Variability

Abstract: The study of virus infection phenotypes and variability plays a critical role in understanding viral pathogenesis and host response. Virus-host interactions can be investigated by light and various label-free microscopy methods, which provide a powerful tool for the spatiotemporal analysis of patterns at the cellular and subcellular levels in live or fixed cells. Analysis of microscopy images is increasingly complemented by sophisticated statistical methods and leverages artificial intelligence (AI) to address the tasks of image denoising, segmentation, classification, and tracking. Work in this thesis demonstrates that combining microscopy with AI techniques enables models that accurately detect and quantify viral infection due to the virus-induced cytopathic effect (CPE). Furthermore, it shows that statistical analysis of microscopy image data can disentangle stochastic and deterministic factors that contribute to viral infection variability, such as the cellular state. In summary, the integration of microscopy and computational image analysis offers a powerful and flexible approach for studying virus infection phenotypes and variability, ultimately contributing to a more advanced understanding of infection processes and creating possibilities for the development of more effective antiviral strategies

객체검출 알고리즘을 위한 일관성과 보간법 기반의 준지도 학습

학위논문(박사) -- 서울대학교대학원 : 융합과학기술대학원 융합과학부(지능형융합시스템전공), 2021.8. 곽노준.Object detection, one of the main areas of computer vision researches, is a task that predicts where and what the objects are in an RGB image. While the object detection task requires a massive number of annotated samples to guarantee its performance, placing bounding boxes for every object in each sample is costly and time consuming. To alleviate this problem, Weakly-Supervised Learning and Semi-Supervised Learning methods have been proposed. However, they show large gaps from supervised learning in efficiency and require a lot of research. Especially in Semi-Supervised Learning, the deep learning-based learning methods are not yet applied to object detection. In this dissertation, we have applied the latest deep learning-based Semi-Supervised Learning methods to object detection, which considers and solves the problems caused by applying the established Semi-Supervised Learning algorithms. Specifically, we have adopted Consistency Regularization (CR) and Interpolation Regularization (IR) Semi-Supervised Learning methods to object detection individually and combined them together for performance improvement. It is the first attempt to extend CR and IR to object detection problem which was only used in conventional semi-supervised classification problems First, we propose a novel Consistency-based Semi-Supervised Learning method for object Detection (CSD), which is a way of using consistency constraints to enhance detection performance by making full use of available unlabeled data. To be specific, the consistency constraint is applied not only for object classification but also for localization. We also propose Background Elimination (BE) to avoid the negative effect of the predominant backgrounds on the detection performance. We evaluated the proposed CSD both in single-stage and two-stage detectors, and the results show the effectiveness of our method. Second, we present a novel Interpolation-based Semi-Supervised Learning method for object Detection (ISD), which considers and solves the problems caused by applying conventional Interpolation Regularization (IR) directly to object detection. We divide the output of the model into two types according to the objectness scores of both original patches that are mixed in IR. Then, we apply a separate loss suitable for each type in an unsupervised manner. The proposed losses dramatically improve the performance of Semi-Supervised Learning as well as supervised learning. Third, we introduce the method of combining CSD and ISD. In CSD, it requires an additional prediction for applying consistency regularization, and it allocates twice (x2) as much memory as conventional supervised learning. In ISD, in addition, two supplementary predictions are computed for applying interpolation regularization, and it takes three times (x3) as much memory as conventional training. Therefore, it requires three extra predictions to combine CSD and ISD. In our method, by applying shuffle the sample in mini-batch in CSD, we reduced the additional predictions from three to two, which can cut back the memory. Furthermore, combining two algorithms shows performance improvement.객체 검출 알고리즘은 RGB 이미지에서 어느 위치에 어떤 객체가 있는지를 검출하는 것으로, 컴퓨터 비전 분야에서 가장 중요한 연구분야 중 하나이다. 하지만, 이러한 객체 검출 알고리즘을 위해서는 잘 레이블링된 큰 데이터 셋을 필요로 하고, 이러한 레이블링은 매우 많은 비용과 시간을 필요로 한다. 위와 같은 문제를 해결하기 위하여 약한 지도학습 (Weakly Supervised Learning), 준지도 학습 (Semi Supervised Learning)의 방법들이 연구되고 있으나, 그 연구가 많지 않고, 준지도 학습의 경우, 최신 딥러닝 기반의 학습방법들이 적용되지 않고 있었다. 본 논문에서는 최신의 딥러닝 기반의 준지도 학습 방법들을 객체 검출 알고리즘에 적용하였고, 여기서 발생하는 문제들을 발견하고 해결하는 방법을 연구하였다. 구체적으로 일관성 정규화 (Consistency Regularization), 보간법 정규화 (Interpolation Regularization) 기반의 준지도 학습 방법을 제시하였고, 최종적으로 이 둘을 합치는 방법을 제시하였다. 이는 기존의 분류문제에서 사용되는 CR 과 IR 을 객체검출 알고리즘 문제에 처음으로 확장한 것이다. 첫 번째로, 우리는 객체 검출 알고리즘을 위한 일관성 정규화 기반의 준지도 학습방법 (CSD)을 제안하였다. 이는 정규화 제약을 사용하여 레이블링이 없는 모든 데이터를 활용하여 객체 검출 성능을 향상시키는 방법이다. 구체적으로 우리는 정규화 제약을 분류뿐만 아니라 회귀에 대해서도 적용하였다. 게다가, 우리는 한 이미지 내에서 대부분의 영역을 차지하는 배경 부분의 영향을 줄이기 위하여 배경 제거 (Background Elimination) 을 적용하였다. 우리는 제안한 CSD 를 싱글 단계 (Single-Stage)와 두 단계(Two-Stage) 검출기에 모두 적용하여 평가하였고, 결과들은 우리의 알고리즘의 효과를 보였다. 두번째로, 우리는 객체 검출 알고리즘을 위한 보간법 정규화 (IR) 기반의 준지도 학습방법 (ISD)을 제안하였다. 우리는 보간법 정규화를 객체 검출 알고리즘에 바로 적용시켰을 때 생기는 문제들을 고려하고 해결하였다. 우리는 두 원본 패치에서의 객체 확률에 따라 모델의 출력을 두개의 타입으로 나누었다. 그리고, 우리는 각각의 타입에 따라 각각에 맞는 손실 함수를 정의하였다. 제안한 알고리즘은 지도학습뿐만 아니라 준지도 학습에서도 매우 큰 성능향상을 보였다. 마지막으로, 우리는 위의 CSD 와 ISD 의 경합하는 방법을 소개하였다. CSD에서는 일관성 정규화를 적용하기 위하여 한번의 추가적인 연산을 필요로 하고, 이는 기존의 지도학습에 비해 2배의 메모리를 필요로 한다. ISD의 경우, 보간법 정규화를 적용하기 위하여 두번의 추가적인 연산을 필요로 하고, 이는 3배의 메모리를 필요로 한다. 그러므로, 두 알고리즘을 결합하기 위해서는 세번의 추가적인 결과값이 필요하다. 우리는 CSD 미니배치의 샘플들을 섞는 방법을 적용하였고, 이는 추가적인 연산을 세번에서 두번으로 줄여 메모리의 소모를 줄일 수 있었다. 또한, 이 두 알고리즘을 합쳐서 모델의 성능이 향상됨을 보였다.1 INTRODUCTION 1 2 Related works 12 3 Consistency-based Semi-supervised learning for object Detection (CSD) 42 4 Interpolation-based Semi-supervised learning for object Detection (ISD) 65 5 Combination of CSD and ISD 82 6 Conclusion 98 Abstract (In Korean) 116 감사의 글 118박

OV-VG: A Benchmark for Open-Vocabulary Visual Grounding

Open-vocabulary learning has emerged as a cutting-edge research area, particularly in light of the widespread adoption of vision-based foundational models. Its primary objective is to comprehend novel concepts that are not encompassed within a predefined vocabulary. One key facet of this endeavor is Visual Grounding, which entails locating a specific region within an image based on a corresponding language description. While current foundational models excel at various visual language tasks, there's a noticeable absence of models specifically tailored for open-vocabulary visual grounding. This research endeavor introduces novel and challenging OV tasks, namely Open-Vocabulary Visual Grounding and Open-Vocabulary Phrase Localization. The overarching aim is to establish connections between language descriptions and the localization of novel objects. To facilitate this, we have curated a comprehensive annotated benchmark, encompassing 7,272 OV-VG images and 1,000 OV-PL images. In our pursuit of addressing these challenges, we delved into various baseline methodologies rooted in existing open-vocabulary object detection, VG, and phrase localization frameworks. Surprisingly, we discovered that state-of-the-art methods often falter in diverse scenarios. Consequently, we developed a novel framework that integrates two critical components: Text-Image Query Selection and Language-Guided Feature Attention. These modules are designed to bolster the recognition of novel categories and enhance the alignment between visual and linguistic information. Extensive experiments demonstrate the efficacy of our proposed framework, which consistently attains SOTA performance across the OV-VG task. Additionally, ablation studies provide further evidence of the effectiveness of our innovative models. Codes and datasets will be made publicly available at https://github.com/cv516Buaa/OV-VG