Real-time self-adaptive deep stereo

Deep convolutional neural networks trained end-to-end are the state-of-the-art methods to regress dense disparity maps from stereo pairs. These models, however, suffer from a notable decrease in accuracy when exposed to scenarios significantly different from the training set, e.g., real vs synthetic images, etc.). We argue that it is extremely unlikely to gather enough samples to achieve effective training/tuning in any target domain, thus making this setup impractical for many applications. Instead, we propose to perform unsupervised and continuous online adaptation of a deep stereo network, which allows for preserving its accuracy in any environment. However, this strategy is extremely computationally demanding and thus prevents real-time inference. We address this issue introducing a new lightweight, yet effective, deep stereo architecture, Modularly ADaptive Network (MADNet) and developing a Modular ADaptation (MAD) algorithm, which independently trains sub-portions of the network. By deploying MADNet together with MAD we introduce the first real-time self-adaptive deep stereo system enabling competitive performance on heterogeneous datasets.Comment: Accepted at CVPR2019 as oral presentation. Code Available https://github.com/CVLAB-Unibo/Real-time-self-adaptive-deep-stere

Monitoring social distancing with single image depth estimation

The recent pandemic emergency raised many challenges regarding the countermeasures aimed at containing the virus spread, and constraining the minimum distance between people resulted in one of the most effective strategies. Thus, the implementation of autonomous systems capable of monitoring the so-called social distance gained much interest. In this paper, we aim to address this task leveraging a single RGB frame without additional depth sensors. In contrast to existing single-image alternatives failing when ground localization is not available, we rely on single image depth estimation to perceive the 3D structure of the observed scene and estimate the distance between people. During the setup phase, a straightforward calibration procedure, leveraging a scale-aware SLAM algorithm available even on consumer smartphones, allows us to address the scale ambiguity affecting single image depth estimation. We validate our approach through indoor and outdoor images employing a calibrated LiDAR + RGB camera asset. Experimental results highlight that our proposal enables sufficiently reliable estimation of the inter-personal distance to monitor social distancing effectively. This fact confirms that despite its intrinsic ambiguity, if appropriately driven single image depth estimation can be a viable alternative to other depth perception techniques, more expensive and not always feasible in practical applications. Our evaluation also highlights that our framework can run reasonably fast and comparably to competitors, even on pure CPU systems. Moreover, its practical deployment on low-power systems is around the corner.Comment: Accepted for pubblication on IEEE Transactions on Emerging Topics in Computational Intelligence (TETCI

Continual Adaptation for Deep Stereo

Depth estimation from stereo images is carried out with unmatched results by convolutional neural networks trained end-to-end to regress dense disparities. Like for most tasks, this is possible if large amounts of labelled samples are available for training, possibly covering the whole data distribution encountered at deployment time. Being such an assumption systematically unmet in real applications, the capacity of adapting to any unseen setting becomes of paramount importance. Purposely, we propose a continual adaptation paradigm for deep stereo networks designed to deal with challenging and ever-changing environments. We design a lightweight and modular architecture, Modularly ADaptive Network (MADNet), and formulate Modular ADaptation algorithms (MAD, MAD++) which permit efficient optimization of independent sub-portions of the entire network. In our paradigm, the learning signals needed to continuously adapt models online can be sourced from self-supervision via right-to-left image warping or from traditional stereo algorithms. With both sources, no other data than the input images being gathered at deployment time are needed. Thus, our network architecture and adaptation algorithms realize the first real-time self-adaptive deep stereo system and pave the way for a new paradigm that can facilitate practical deployment of end-to-end architectures for dense disparity regression

Unsupervised Domain Adaptation for Depth Prediction from Images

State-of-the-art methods to infer dense and accurate depth measurements from images rely on deep CNN models trained in an end-to-end fashion on a significant amount of data. However, despite the outstanding performance achieved, these frameworks suffer a drastic drop in accuracy when dealing with unseen environments much different, concerning appearance (e.g., synthetic vs. real) or context (e.g., indoor vs. outdoor), from those observed during the training phase. Such domain shift issue is usually softened by fine-tuning on smaller sets of images with depth labels acquired in the target domain with active sensors (e.g., LiDAR). However, relying on such supervised labeled data is seldom feasible in practical applications. Therefore, we propose an effective unsupervised domain adaptation technique enabling to overcome the domain shift problem without requiring any groundtruth label. Our method, deploying much more accessible to obtain stereo pairs, leverages traditional and not learning-based stereo algorithms to produce disparity/depth labels and on confidence measures to assess their degree of reliability. With these cues, we can fine-tune deep models through a novel confidence-guided loss function, neglecting the effect of outliers gathered from the output of conventional stereo algorithms

Unsupervised Adaptation for Deep Stereo

Recent ground-breaking works have shown that deep neural networks can be trained end-to-end to regress dense disparity maps directly from image pairs. Computer generated imagery is deployed to gather the large data corpus required to train such networks, an additional fine-tuning allowing to adapt the model to work well also on real and possibly diverse environments. Yet, besides a few public datasets such as Kitti, the ground-truth needed to adapt the network to a new scenario is hardly available in practice. In this paper we propose a novel unsupervised adaptation approach that enables to fine-tune a deep learning stereo model without any ground-truth information. We rely on off-the-shelf stereo algorithms together with state-of-the-art confidence measures, the latter able to ascertain upon correctness of the measurements yielded by former. Thus, we train the network based on a novel loss-function that penalizes predictions disagreeing with the highly confident disparities provided by the algorithm and enforces a smoothness constraint. Experiments on popular datasets (KITTI 2012, KITTI 2015 and Middlebury 2014) and other challenging test images demonstrate the effectiveness of our proposal

Layered Double Hydroxides: A Toolbox for Chemistry and Biology

Layered double hydroxides (LDHs) are an emergent class of biocompatible inorganic lamellar nanomaterials that have attracted significant research interest owing to their high surface-to-volume ratio, the capability to accumulate specific molecules, and the timely release to targets. Their unique properties have been employed for applications in organic catalysis, photocatalysis, sensors, drug delivery, and cell biology. Given the widespread contemporary interest in these topics, time-to-time it urges to review the recent progresses. This review aims to summarize the most recent cutting-edge reports appearing in the last years. It firstly focuses on the application of LDHs as catalysts in relevant chemical reactions and as photocatalysts for organic molecule degradation, water splitting reaction, CO2 conversion, and reduction. Subsequently, the emerging role of these materials in biological applications is discussed, specifically focusing on their use as biosensors, DNA, RNA, and drug delivery, finally elucidating their suitability as contrast agents and for cellular differentiation. Concluding remarks and future prospects deal with future applications of LDHs, encouraging researches in better understanding the fundamental mechanisms involved in catalytic and photocatalytic processes, and the molecular pathways that are activated by the interaction of LDHs with cells in terms of both uptake mechanisms and nanotoxicology effects

Studio morfologico di nanostrutture di layered double hydroxides (LDH) depositate su film sottili di alluminio

We have synthesized nanoplatelets of crystalline (Zn,Al) Layered Double Hydroxide (LDH) by a single-step and room temperature hydrothermal process on aluminum thin layers sputtered on different substrates. The structure, morphology, dimensions and compositions of nanoplatelets have been investigated by Scanning Electron Microscopy (SEM), X-Rays diffraction (XRD), Energy Dispersion Spectroscopy (EDS) and Photoluminescence (PL). Different behaviours of the thickness of nanoplatelets have been obtained by varying the most important growth parameters (thickness of Al coatings, growth temperature and duration). The thickness of the observed nanoplatelets results to be clearly dependent on the aluminun content available in the coating. On the contrary, the stoichiometry and the Zn/Al ratio does not change appreciably. Furthermore, for the thinnest aluminum layer, the LDH nanostructures result to be not well-shaped, and the excess zinc, on the one hand does not cause changes in the composition, on the other hand has as a consequence the formation of insulated ZnO nanorods. These samples show the defect-related visible luminescence, approximately centered at 600nm, and due to the nanorods presence, while no significant luminescence was expected from LDH nanosheets. Results obtained show that a controlled and spatially localized synthesis of Zn/Al LDH nanoplatelets can be obtained even on substrates having large surface area provided that the sputtered aluminum coating results to be thicker than 10nm, thus making possible the integration of these nanostructures on substrates of different nature

Layered Double Hydroxides as a Drug Delivery Vehicle for S-Allyl-Mercapto-Cysteine (SAMC)

The intercalations of anionic molecules and drugs in layered double hydroxides (LDHs) have been intensively investigated in recent years. Due to their properties, such as versatility in chemical composition, good biocompatibility, high density and protection of loaded drugs, LDHs seem very promising nanosized systems for drug delivery. In this work, we report the intercalation of S-allyl-mercapto-cysteine (SAMC), which is a component of garlic that is well-known for its anti-tumor properties, inside ZnAl-LDH (hereafter LDH) nanostructured crystals. In order to investigate the efficacy of the intercalation and drug delivery of SAMC, the intercalated compounds were characterized using X-ray powder diffraction (XRD), Fourier-transform infrared spectroscopy (FT-IR) and scanning electron microscopy (SEM). The increase in the interlayer distance of LDH from 8.9 Å, typical of the nitrate phase, to 13.9 Å indicated the intercalation of SAMC, which was also confirmed using FT-IR spectra. Indeed, compared to that of the pristine LDH precursor, the spectrum of LDH-SAMC was richly structured in the fingerprint region below 1300 cm−1, whose peaks corresponded to those of the functional groups in the SAMC molecular anion. The LDH-SAMC empirical formula, obtained from UV-Vis spectrophotometry and thermogravimetric analysis, was [Zn0.67Al0.33(OH)2]SAMC0.15(NO3)0.18·0.6H2O. The morphology of the sample was investigated using SEM: LDH-SAMC exhibited a more irregular size and shape of the flake-like crystals in comparison with the pristine LDH, with a reduction in the average crystallite size from 3 µm to about 2 µm. In vitro drug release studies were performed in a phosphate buffer solution at pH 7.2 and 37 °C and were analyzed using UV-Vis spectrophotometry. The SAMC release from LDH-SAMC was initially characterized by a burst effect in the first four hours, during which, 32% of the SAMC is released. Subsequently, the release percentage increased at a slower rate until 42% after 48 h; then it stabilized at 43% and remained constant for the remaining period of the investigation. The LDH-SAMC complex that was developed in this study showed the improved efficacy of the action of SAMC in reducing the invasive capacity of a human hepatoma cell line