136 research outputs found

    Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentation

    Get PDF
    Automatic surgical scene segmentation is fundamental for facilitating cognitive intelligence in the modern operating theatre. Previous works rely on conventional aggregation modules (e.g., dilated convolution, convolutional LSTM), which only make use of the local context. In this paper, we propose a novel framework STswinCL that explores the complementary intra- and inter-video relations to boost segmentation performance, by progressively capturing the global context. We firstly develop a hierarchy Transformer to capture intra-video relation that includes richer spatial and temporal cues from neighbor pixels and previous frames. A joint space-time window shift scheme is proposed to efficiently aggregate these two cues into each pixel embedding. Then, we explore inter-video relation via pixel-to-pixel contrastive learning, which well structures the global embedding space. A multi-source contrast training objective is developed to group the pixel embeddings across videos with the ground-truth guidance, which is crucial for learning the global property of the whole data. We extensively validate our approach on two public surgical video benchmarks, including EndoVis18 Challenge and CaDIS dataset. Experimental results demonstrate the promising performance of our method, which consistently exceeds previous state-of-the-art approaches. Code will be available at https://github.com/YuemingJin/STswinCL

    Helix-MO: Sample-Efficient Molecular Optimization on Scene-Sensitive Latent Space

    Full text link
    Efficient exploration of the chemical space to search the candidate drugs that satisfy various constraints is a fundamental task of drug discovery. Although many excellent deep molecular generative methods have been proposed to produce promising molecules, applying these methods in practice is still challenging since a great number of assessed molecules (samples) are required to provide the optimization direction, which is a considerable expense for drug discovery. To this end, we design a sample-efficient molecular generative method, namely Helix-MO, which can fast adapt to particular optimization scenes with only a small number of assessed samples. Helix-MO explores the chemical space in a scene-sensitive latent space, dynamically fine-tuned by multiple kinds of learning tasks from multiple perspectives. The learning tasks encourage the model to focus on modeling the more promising molecules during the optimization process to promote sample efficiency. Extensive experiments demonstrate that Helix-MO can achieve competitive performance with only a few assessed samples on four molecular optimization scenes. Ablation studies verify the impact of the learning tasks in the scene-specific latent space, efficiently identifying the critical characters of the satisfactory molecules. We also deployed Helix-MO on the website PaddleHelix (https://paddlehelix.baidu.com/app/drug/drugdesign/forecast) to provide drug design service and apply it to produce inhibitors of a kinase to demonstrate its practicability

    Toward Image-Guided Automated Suture Grasping Under Complex Environments: A Learning-Enabled and Optimization-Based Holistic Framework

    Get PDF
    To realize a higher-level autonomy of surgical knot tying in minimally invasive surgery (MIS), automated suture grasping, which bridges the suture stitching and looping procedures, is an important yet challenging task needs to be achieved. This paper presents a holistic framework with image-guided and automation techniques to robotize this operation even under complex environments. The whole task is initialized by suture segmentation, in which we propose a novel semi-supervised learning architecture featured with a suture-aware loss to pertinently learn its slender information using both annotated and unannotated data. With successful segmentation in stereo-camera, we develop a Sampling-based Sliding Pairing (SSP) algorithm to online optimize the suture's 3D shape. By jointly studying the robotic configuration and the suture's spatial characteristics, a target function is introduced to find the optimal grasping pose of the surgical tool with Remote Center of Motion (RCM) constraints. To compensate for inherent errors and practical uncertainties, a unified grasping strategy with a novel vision-based mechanism is introduced to autonomously accomplish this grasping task. Our framework is extensively evaluated from learning-based segmentation, 3D reconstruction, and image-guided grasping on the da Vinci Research Kit (dVRK) platform, where we achieve high performances and successful rates in perceptions and robotic manipulations. These results prove the feasibility of our approach in automating the suture grasping task, and this work fills the gap between automated surgical stitching and looping, stepping towards a higher-level of task autonomy in surgical knot tying

    BMAL1 but not CLOCK is associated with monochromatic green light-induced circadian rhythm of melatonin in chick pinealocytes

    Get PDF
    The avian pineal gland, an independent circadian oscillator, receives external photic cues and translates them for the rhythmical synthesis of melatonin. Our previous study found that monochromatic green light could increase the secretion of melatonin and expression of CLOCK and BMAL1 in chick pinealocytes. This study further investigated the role of BMAL1 and CLOCK in monochromatic green light-induced melatonin secretion in chick pinealocytes using siRNAs interference and overexpression techniques. The results showed that si-BMAL1 destroyed the circadian rhythms of AANAT and melatonin, along with the disruption of the expression of all the seven clock genes, except CRY1. Furthermore, overexpression of BMAL1 also disturbed the circadian rhythms of AANAT and melatonin, in addition to causing arrhythmic expression of BMAL1 and CRY1/2, but had no effect on the circadian rhythms of CLOCK, BMAL2 and PER2/3. The knockdown or overexpression of CLOCK had no impact on the circadian rhythms of AANAT, melatonin, BMAL1 and PER2, but it significantly deregulated the circadian rhythms of CLOCK, BMAL2, CRY1/2 and PER3. These results suggested that BMAL1 rather than CLOCK plays a critical role in the regulation of monochromatic green light-induced melatonin rhythm synthesis in chicken pinealocytes. Moreover, both knockdown and overexpression of BMAL1 could change the expression levels of CRY2, it indicated CRY2 may be involved in the BMAL1 pathway by modulating the circadian rhythms of AANAT and melatonin
    • …
    corecore