3D-BEVIS: Bird's-Eye-View Instance Segmentation

Recent deep learning models achieve impressive results on 3D scene analysis tasks by operating directly on unstructured point clouds. A lot of progress was made in the field of object classification and semantic segmentation. However, the task of instance segmentation is less explored. In this work, we present 3D-BEVIS, a deep learning framework for 3D semantic instance segmentation on point clouds. Following the idea of previous proposal-free instance segmentation approaches, our model learns a feature embedding and groups the obtained feature space into semantic instances. Current point-based methods scale linearly with the number of points by processing local sub-parts of a scene individually. However, to perform instance segmentation by clustering, globally consistent features are required. Therefore, we propose to combine local point geometry with global context information from an intermediate bird's-eye view representation.Comment: camera-ready version for GCPR '1

Deep semi-supervised segmentation with weight-averaged consistency targets

Recently proposed techniques for semi-supervised learning such as Temporal Ensembling and Mean Teacher have achieved state-of-the-art results in many important classification benchmarks. In this work, we expand the Mean Teacher approach to segmentation tasks and show that it can bring important improvements in a realistic small data regime using a publicly available multi-center dataset from the Magnetic Resonance Imaging (MRI) domain. We also devise a method to solve the problems that arise when using traditional data augmentation strategies for segmentation tasks on our new training scheme.Comment: 8 pages, 1 figure, accepted for DLMIA/MICCA

Geometry meets semantics for semi-supervised monocular depth estimation

Depth estimation from a single image represents a very exciting challenge in computer vision. While other image-based depth sensing techniques leverage on the geometry between different viewpoints (e.g., stereo or structure from motion), the lack of these cues within a single image renders ill-posed the monocular depth estimation task. For inference, state-of-the-art encoder-decoder architectures for monocular depth estimation rely on effective feature representations learned at training time. For unsupervised training of these models, geometry has been effectively exploited by suitable images warping losses computed from views acquired by a stereo rig or a moving camera. In this paper, we make a further step forward showing that learning semantic information from images enables to improve effectively monocular depth estimation as well. In particular, by leveraging on semantically labeled images together with unsupervised signals gained by geometry through an image warping loss, we propose a deep learning approach aimed at joint semantic segmentation and depth estimation. Our overall learning framework is semi-supervised, as we deploy groundtruth data only in the semantic domain. At training time, our network learns a common feature representation for both tasks and a novel cross-task loss function is proposed. The experimental findings show how, jointly tackling depth prediction and semantic segmentation, allows to improve depth estimation accuracy. In particular, on the KITTI dataset our network outperforms state-of-the-art methods for monocular depth estimation.Comment: 16 pages, Accepted to ACCV 201

Solid Ink Laser Patterning for High-Resolution Information Labels with Supervised Learning Readout

Tagging, tracking, or validation of products are often facilitated by inkjet-printed optical information labels. However, this requires thorough substrate pretreatment, ink optimization, and often lacks in printing precision/resolution. Herein, a printing method based on laser-driven deposition of solid polymer ink that allows for printing on various substrates without pretreatment is demonstrated. Since the deposition process has a precision of <1 µm, it can introduce the concept of sub-positions with overlapping spots. This enables high-resolution fluorescent labels with comparable spot-to-spot distance of down to 15 µm (444,444 spots cm−2) and rapid machine learning-supported readout based on low-resolution fluorescence imaging. Furthermore, the defined thickness of the printed polymer ink spots can be used to fabricate multi-channel information labels. Additional information can be stored in different fluorescence channels or in a hidden topography channel of the label that is independent of the fluorescence

TuNet: End-to-end Hierarchical Brain Tumor Segmentation using Cascaded Networks

Glioma is one of the most common types of brain tumors; it arises in the glial cells in the human brain and in the spinal cord. In addition to having a high mortality rate, glioma treatment is also very expensive. Hence, automatic and accurate segmentation and measurement from the early stages are critical in order to prolong the survival rates of the patients and to reduce the costs of the treatment. In the present work, we propose a novel end-to-end cascaded network for semantic segmentation that utilizes the hierarchical structure of the tumor sub-regions with ResNet-like blocks and Squeeze-and-Excitation modules after each convolution and concatenation block. By utilizing cross-validation, an average ensemble technique, and a simple post-processing technique, we obtained dice scores of 88.06, 80.84, and 80.29, and Hausdorff Distances (95th percentile) of 6.10, 5.17, and 2.21 for the whole tumor, tumor core, and enhancing tumor, respectively, on the online test set.Comment: Accepted at MICCAI BrainLes 201

ICNet for Real-Time Semantic Segmentation on High-Resolution Images

We focus on the challenging task of real-time semantic segmentation in this paper. It finds many practical applications and yet is with fundamental difficulty of reducing a large portion of computation for pixel-wise label inference. We propose an image cascade network (ICNet) that incorporates multi-resolution branches under proper label guidance to address this challenge. We provide in-depth analysis of our framework and introduce the cascade feature fusion unit to quickly achieve high-quality segmentation. Our system yields real-time inference on a single GPU card with decent quality results evaluated on challenging datasets like Cityscapes, CamVid and COCO-Stuff.Comment: ECCV 201

Nanolayer laser absorber for femtoliter chemistry in polymer reactors

Laser-induced forward transfer (LIFT) has the potential to be an alternative approach to atomic force microscopy based scanning probe lithography techniques, which have limitations in high-speed and large-scale patterning. However, traditional donor slides limit the resolution and chemical flexibility of LIFT. Here, we propose a hematite nanolayer absorber for donor slides to achieve high-resolution transfers down to sub-femtoliters. Being wettable by both aqueous and organic solvents, this new donor significantly increases the chemical scope for the LIFT process. For parallel amino acid coupling reactions, the patterning resolution can now be increased more than five times (>111,000 spots/cm2 for hematite donor versus 20,000 spots/cm2 for standard polyimide donor) with even faster scanning (2 versus 6 ms per spot). Due to the increased chemical flexibility, we could explore other types of reactions inside ultrasmall polymer reactors: copper (I) catalyzed click chemistry and laser-driven oxidation of a tetrahydroisoquinoline derivative, suggesting the potential of LIFT for both deposition of chemicals and laser-driven photochemical synthesis in femtoliters within milliseconds. Since the hematite shows no damage after typical laser transfer, donors can be regenerated by heat treatment. These findings will transform the LIFT process into an automatable, precise, and highly efficient technology for high-throughput femtoliter chemistry

Assessing polymer-surface adhesion with a polymer collection

Polymer modification plays an important role in the construction of devices, but the lack of fundamental understanding on polymer-surface adhesion limits the development of miniaturized devices. In this work, a thermoplastic polymer collection was established using the combinatorial laser-induced forward transfer technique as a research platform, to assess the adhesion of polymers to substrates of different wettability. Furthermore, it also revealed the influence of adhesion on dewetting phenomena during the laser transfer and relaxation process, resulting in polymer spots of various morphologies. This gives a general insight into polymer-surface adhesion and connects it with the generation of defined polymer microstructures, which can be a valuable reference for the rational use of polymers

An all-in-one nanoprinting approach for the synthesis of a nanofilm library for unclonable anti-counterfeiting applications

In addition to causing trillion-dollar economic losses every year, counterfeiting threatens human health, social equity and national security. Current materials for anti-counterfeiting labelling typically contain toxic inorganic quantum dots and the techniques to produce unclonable patterns require tedious fabrication or complex readout methods. Here we present a nanoprinting-assisted flash synthesis approach that generates fluorescent nanofilms with physical unclonable function micropatterns in milliseconds. This all-in-one approach yields quenching-resistant carbon dots in solid films, directly from simple monosaccharides. Moreover, we establish a nanofilm library comprising 1,920 experiments, offering conditions for various optical properties and microstructures. We produce 100 individual physical unclonable function patterns exhibiting near-ideal bit uniformity (0.492 ± 0.018), high uniqueness (0.498 ± 0.021) and excellent reliability (>93%). These unclonable patterns can be quickly and independently read out by fluorescence and topography scanning, greatly improving their security. An open-source deep-learning model guarantees precise authentication, even if patterns are challenged with different resolutions or devices