Attention Mechanisms for Object Recognition with Event-Based Cameras

Event-based cameras are neuromorphic sensors capable of efficiently encoding visual information in the form of sparse sequences of events. Being biologically inspired, they are commonly used to exploit some of the computational and power consumption benefits of biological vision. In this paper we focus on a specific feature of vision: visual attention. We propose two attentive models for event based vision: an algorithm that tracks events activity within the field of view to locate regions of interest and a fully-differentiable attention procedure based on DRAW neural model. We highlight the strengths and weaknesses of the proposed methods on four datasets, the Shifted N-MNIST, Shifted MNIST-DVS, CIFAR10-DVS and N-Caltech101 collections, using the Phased LSTM recognition network as a baseline reference model obtaining improvements in terms of both translation and scale invariance.Comment: WACV2019 camera-ready submissio

Multi-View Stereo with Single-View Semantic Mesh Refinement

While 3D reconstruction is a well-established and widely explored research topic, semantic 3D reconstruction has only recently witnessed an increasing share of attention from the Computer Vision community. Semantic annotations allow in fact to enforce strong class-dependent priors, as planarity for ground and walls, which can be exploited to refine the reconstruction often resulting in non-trivial performance improvements. State-of-the art methods propose volumetric approaches to fuse RGB image data with semantic labels; even if successful, they do not scale well and fail to output high resolution meshes. In this paper we propose a novel method to refine both the geometry and the semantic labeling of a given mesh. We refine the mesh geometry by applying a variational method that optimizes a composite energy made of a state-of-the-art pairwise photo-metric term and a single-view term that models the semantic consistency between the labels of the 3D mesh and those of the segmented images. We also update the semantic labeling through a novel Markov Random Field (MRF) formulation that, together with the classical data and smoothness terms, takes into account class-specific priors estimated directly from the annotated mesh. This is in contrast to state-of-the-art methods that are typically based on handcrafted or learned priors. We are the first, jointly with the very recent and seminal work of [M. Blaha et al arXiv:1706.08336, 2017], to propose the use of semantics inside a mesh refinement framework. Differently from [M. Blaha et al arXiv:1706.08336, 2017], which adopts a more classical pairwise comparison to estimate the flow of the mesh, we apply a single-view comparison between the semantically annotated image and the current 3D mesh labels; this improves the robustness in case of noisy segmentations.Comment: {\pounds}D Reconstruction Meets Semantic, ICCV worksho

ReConvNet: Video Object Segmentation with Spatio-Temporal Features Modulation

We introduce ReConvNet, a recurrent convolutional architecture for semi-supervised video object segmentation that is able to fast adapt its features to focus on any specific object of interest at inference time. Generalization to new objects never observed during training is known to be a hard task for supervised approaches that would need to be retrained. To tackle this problem, we propose a more efficient solution that learns spatio-temporal features self-adapting to the object of interest via conditional affine transformations. This approach is simple, can be trained end-to-end and does not necessarily require extra training steps at inference time. Our method shows competitive results on DAVIS2016 with respect to state-of-the art approaches that use online fine-tuning, and outperforms them on DAVIS2017. ReConvNet shows also promising results on the DAVIS-Challenge 2018 winning the $10$-th position.Comment: CVPR Workshop - DAVIS Challenge 201

Closing the loop of SIEM analysis to Secure Critical Infrastructures

Critical Infrastructure Protection is one of the main challenges of last years. Security Information and Event Management (SIEM) systems are widely used for coping with this challenge. However, they currently present several limitations that have to be overcome. In this paper we propose an enhanced SIEM system in which we have introduced novel components to i) enable multiple layer data analysis; ii) resolve conflicts among security policies, and discover unauthorized data paths in such a way to be able to reconfigure network devices. Furthermore, the system is enriched by a Resilient Event Storage that ensures integrity and unforgeability of events stored.Comment: EDCC-2014, BIG4CIP-2014, Security Information and Event Management, Decision Support System, Hydroelectric Da

Asynchronous Convolutional Networks for Object Detection in Neuromorphic Cameras

Event-based cameras, also known as neuromorphic cameras, are bioinspired sensors able to perceive changes in the scene at high frequency with low power consumption. Becoming available only very recently, a limited amount of work addresses object detection on these devices. In this paper we propose two neural networks architectures for object detection: YOLE, which integrates the events into surfaces and uses a frame-based model to process them, and fcYOLE, an asynchronous event-based fully convolutional network which uses a novel and general formalization of the convolutional and max pooling layers to exploit the sparsity of camera events. We evaluate the algorithm with different extensions of publicly available datasets and on a novel synthetic dataset.Comment: accepted at CVPR2019 Event-based Vision Worksho

Spatial Temporal Transformer Network for Skeleton-based Action Recognition

Skeleton-based human action recognition has achieved a great interest in recent years, as skeleton data has been demonstrated to be robust to illumination changes, body scales, dynamic camera views, and complex background. Nevertheless, an effective encoding of the latent information underlying the 3D skeleton is still an open problem. In this work, we propose a novel Spatial-Temporal Transformer network (ST-TR) which models dependencies between joints using the Transformer self-attention operator. In our ST-TR model, a Spatial Self-Attention module (SSA) is used to understand intra-frame interactions between different body parts, and a Temporal Self-Attention module (TSA) to model inter-frame correlations. The two are combined in a two-stream network which outperforms state-of-the-art models using the same input data on both NTU-RGB+D 60 and NTU-RGB+D 120.Comment: Accepted as ICPRW2020 (FBE2020, Workshop on Facial and Body Expressions, micro-expressions and behavior recognition) 8 pages, 2 figures. arXiv admin note: substantial text overlap with arXiv:2008.0740

Monitoring CO2 fluxes and partitioning of soil respiration in a Mediterranean forest ecosystem: an integrated approach to carbon cycle

The increase of greenhouse gases concentrations in the atmosphere is a major driver for global warming and climate change. Carbon dioxide is the primary anthropogenic greenhouse gas and is cumulatively responsible for approximately 55% of greenhouse-gas-related climate forcing, popularly known as "the greenhouse effect". Forests play an important role in the carbon cycle and carbon sequestration at both local and global scales. Trees remove CO2 from the atmosphere through photosynthesis and store carbon in different tree components and in the soil. Whether a forest acts as a carbon sink or source depends on the difference between photosynthetic uptake and respiratory release of CO2. The total flux of CO2 released from the ecosystem (Reco) originates form a range of sources, which can be broadly divided into those originating from aboveground plant tissues (RAa) and soil (RS). Soil CO2 efflux can furthermore be partitioned into belowground autotrophic respiration (roots plus associated microorganisms, RAb) and the decomposition of dead organic matter (heterotrophic respiration, RH). In order to predict likely changes in ecosystem carbon balance under changed environmental conditions, it is therefore necessary to identify and quantify the different sources of CO2 efflux and their dependence on environmental conditions. For this purpose a combined approach based on simultaneous eddy covariance and soil respiration measurements was applied in a maritime pine (Pinus pinaster) forest in Central Italy within the Regional Park of San Rossore – Migliarino – Massaciuccoli (Tuscany). In 2011 a girdling experiment has been developed to partition RS into RH and RAb. The experiment started in spring 2011 and the response has been followed till the end of 2012. Two weeks after the girdling treatment soil respiration in the girdled plots decreased by 30% and remained stable over the period of analysis with an average RH/RS that was estimated around 0.70, suggesting that at San Rossore site RH dominates RS. The anomalous low rainfall regimen recorded during 2011 growing season and 2012 summer offered the opportunity to investigate the decoupled response of respiration to water availability and temperature. RS and RH responded quite predictably to environmental controls. Nevertheless a dichotomous response was observed during the hot and dry season and during the wetter and colder winter. Soil water availability was the major control of RH and RS during the growing season. Severe drought masked the temperature response of respiration which was restored only during the wettest periods. At the ecosystem scale, it can be estimated that RH and total autotrophic respiration (RAt) contributed 40% and 60% of Reco, respectively. Our data suggested that photosynthesis was the major driver of RAt throughout the whole observation period, thus highlighting the important role exerted by newly assimilated carbon on plant tissue respiration. Large and consistent peaks of CO2 emissions were recorded from the soil after drying – rewetting cycles, as has been observed in many other water limited ecosystems (the so called “Birch effect”). These wet-days warm-soil respiration peaks contributed to 50 – 70% of Reco and released an amount of carbon to the atmosphere that was double the amount emitted during the whole dry season. Our data derived from natural field conditions unequivocally indicate that the soil CO2 pulses following rewetting of the dry soil derived from the rapid microbial oxidation of labile carbon compounds. A delayed effect of the water pulse on Rs was also observed, which was ascribed to the slower mobilization of more recalcitrant soil organic matter and to the time lag requested by recently assimilated photosynthates to be translocated from the canopy to the root system. Even though severe summer drought is common at this experimental site with a typical Mediterranean climate, the anomaly in the precipitation regimen experienced in 2011 and 2012 was unusual and significantly impacted the ecosystem carbon balance. Soil water content was clearly the main environmental driver for both gross primary productivity (GPP) and Reco. Nonetheless, GPP was less affected when trees could still access deep soil water reserves with roots. These observations highlight the vulnerability of the Mediterranean-type ecosystems not only in terms of their response to predicted increasing temperatures but also to changing precipitation patterns. This demands for an incorporation of these variables into models for prediction of ecosystems’ feedbacks to climate change. Since the capability of an ecosystem to be a net sink of CO2 depends on its ability to fix and retain more carbon than that is respired back to the atmosphere, carbon losses associated to an increased variability of rainfall events could dramatically impact the ecosystem carbon balance and determine the fate of this forest of being a net carbon sink or source

Samples and data accessibility in research biobanks. An explorative survey

Biobanks, which contain human biological samples and/or data, provide a crucial contribution to the progress of biomedical research. However, the effective and efficient use of biobank resources depends on their accessibility. In fact, making bio-resources promptly accessible to everybody may increase the benefits for society. Furthermore, optimizing their use and ensuring their quality will promote scientific creativity and, in general, contribute to the progress of bio-medical research. Although this has become a rather common belief, several laboratories are still secretive and continue to withhold samples and data. In this study, we conducted a questionnairebased survey in order to investigate sample and data accessibility in research biobanks operating all over the world. The survey involved a total of 46 biobanks. Most of them gave permission to access their samples (95.7%) and data (85.4%), but free and unconditioned accessibility seemed not to be common practice. The analysis of the guidelines regarding the accessibility to resources of the biobanks that responded to the survey highlights three issues: (i) the request for applicants to explain what they would like to do with the resources requested; (ii) the role of funding, public or private, in the establishment of fruitful collaborations between biobanks and research labs; (iii) the request of co-authorship in order to give access to their data. These results suggest that economic and academic aspects are involved in determining the extent of sample and data sharing stored in biobanks. As a second step of this study, we investigated the reasons behind the high diversity of requirements to access biobank resources. The analysis of informative answers suggested that the different modalities of resource accessibility seem to be largely influenced by both social context and legislation of the countries where the biobanks operate