A video object generation tool allowing friendly user interaction

In this paper we describe an interactive video object segmentation tool developed in the framework of the ACTS-AC098 MOMUSYS project. The Video Object Generator with User Environment (VOGUE) combines three different sets of automatic and semi-automatic-tool (spatial segmentation, object tracking and temporal segmentation) with general purpose tools for user interaction. The result is an integrated environment allowing the user-assisted segmentation of any sort of video sequences in a friendly and efficient manner.Peer ReviewedPostprint (published version

Lucid Data Dreaming for Video Object Segmentation

Convolutional networks reach top quality in pixel-level video object segmentation but require a large amount of training data (1k~100k) to deliver such results. We propose a new training strategy which achieves state-of-the-art results across three evaluation datasets while using 20x~1000x less annotated data than competing methods. Our approach is suitable for both single and multiple object segmentation. Instead of using large training sets hoping to generalize across domains, we generate in-domain training data using the provided annotation on the first frame of each video to synthesize ("lucid dream") plausible future video frames. In-domain per-video training data allows us to train high quality appearance- and motion-based models, as well as tune the post-processing stage. This approach allows to reach competitive results even when training from only a single annotated frame, without ImageNet pre-training. Our results indicate that using a larger training set is not automatically better, and that for the video object segmentation task a smaller training set that is closer to the target domain is more effective. This changes the mindset regarding how many training samples and general "objectness" knowledge are required for the video object segmentation task.Comment: Accepted in International Journal of Computer Vision (IJCV

Direct exoplanet detection and characterization using the ANDROMEDA method: Performance on VLT/NaCo data

Context. The direct detection of exoplanets with high-contrast imaging requires advanced data processing methods to disentangle potential planetary signals from bright quasi-static speckles. Among them, angular differential imaging (ADI) permits potential planetary signals with a known rotation rate to be separated from instrumental speckles that are either statics or slowly variable. The method presented in this paper, called ANDROMEDA for ANgular Differential OptiMal Exoplanet Detection Algorithm is based on a maximum likelihood approach to ADI and is used to estimate the position and the flux of any point source present in the field of view. Aims. In order to optimize and experimentally validate this previously proposed method, we applied ANDROMEDA to real VLT/NaCo data. In addition to its pure detection capability, we investigated the possibility of defining simple and efficient criteria for automatic point source extraction able to support the processing of large surveys. Methods. To assess the performance of the method, we applied ANDROMEDA on VLT/NaCo data of TYC-8979-1683-1 which is surrounded by numerous bright stars and on which we added synthetic planets of known position and flux in the field. In order to accommodate the real data properties, it was necessary to develop additional pre-processing and post-processing steps to the initially proposed algorithm. We then investigated its skill in the challenging case of a well-known target, $\beta$ Pictoris, whose companion is close to the detection limit and we compared our results to those obtained by another method based on principal component analysis (PCA). Results. Application on VLT/NaCo data demonstrates the ability of ANDROMEDA to automatically detect and characterize point sources present in the image field. We end up with a robust method bringing consistent results with a sensitivity similar to the recently published algorithms, with only two parameters to be fine tuned. Moreover, the companion flux estimates are not biased by the algorithm parameters and do not require a posteriori corrections. Conclusions. ANDROMEDA is an attractive alternative to current standard image processing methods that can be readily applied to on-sky data

Smart environment monitoring through micro unmanned aerial vehicles

In recent years, the improvements of small-scale Unmanned Aerial Vehicles (UAVs) in terms of flight time, automatic control, and remote transmission are promoting the development of a wide range of practical applications. In aerial video surveillance, the monitoring of broad areas still has many challenges due to the achievement of different tasks in real-time, including mosaicking, change detection, and object detection. In this thesis work, a small-scale UAV based vision system to maintain regular surveillance over target areas is proposed. The system works in two modes. The first mode allows to monitor an area of interest by performing several flights. During the first flight, it creates an incremental geo-referenced mosaic of an area of interest and classifies all the known elements (e.g., persons) found on the ground by an improved Faster R-CNN architecture previously trained. In subsequent reconnaissance flights, the system searches for any changes (e.g., disappearance of persons) that may occur in the mosaic by a histogram equalization and RGB-Local Binary Pattern (RGB-LBP) based algorithm. If present, the mosaic is updated. The second mode, allows to perform a real-time classification by using, again, our improved Faster R-CNN model, useful for time-critical operations. Thanks to different design features, the system works in real-time and performs mosaicking and change detection tasks at low-altitude, thus allowing the classification even of small objects. The proposed system was tested by using the whole set of challenging video sequences contained in the UAV Mosaicking and Change Detection (UMCD) dataset and other public datasets. The evaluation of the system by well-known performance metrics has shown remarkable results in terms of mosaic creation and updating, as well as in terms of change detection and object detection

The Iray Light Transport Simulation and Rendering System

While ray tracing has become increasingly common and path tracing is well understood by now, a major challenge lies in crafting an easy-to-use and efficient system implementing these technologies. Following a purely physically-based paradigm while still allowing for artistic workflows, the Iray light transport simulation and rendering system allows for rendering complex scenes by the push of a button and thus makes accurate light transport simulation widely available. In this document we discuss the challenges and implementation choices that follow from our primary design decisions, demonstrating that such a rendering system can be made a practical, scalable, and efficient real-world application that has been adopted by various companies across many fields and is in use by many industry professionals today