Smart environment monitoring through micro unmanned aerial vehicles

In recent years, the improvements of small-scale Unmanned Aerial Vehicles (UAVs) in terms of flight time, automatic control, and remote transmission are promoting the development of a wide range of practical applications. In aerial video surveillance, the monitoring of broad areas still has many challenges due to the achievement of different tasks in real-time, including mosaicking, change detection, and object detection. In this thesis work, a small-scale UAV based vision system to maintain regular surveillance over target areas is proposed. The system works in two modes. The first mode allows to monitor an area of interest by performing several flights. During the first flight, it creates an incremental geo-referenced mosaic of an area of interest and classifies all the known elements (e.g., persons) found on the ground by an improved Faster R-CNN architecture previously trained. In subsequent reconnaissance flights, the system searches for any changes (e.g., disappearance of persons) that may occur in the mosaic by a histogram equalization and RGB-Local Binary Pattern (RGB-LBP) based algorithm. If present, the mosaic is updated. The second mode, allows to perform a real-time classification by using, again, our improved Faster R-CNN model, useful for time-critical operations. Thanks to different design features, the system works in real-time and performs mosaicking and change detection tasks at low-altitude, thus allowing the classification even of small objects. The proposed system was tested by using the whole set of challenging video sequences contained in the UAV Mosaicking and Change Detection (UMCD) dataset and other public datasets. The evaluation of the system by well-known performance metrics has shown remarkable results in terms of mosaic creation and updating, as well as in terms of change detection and object detection

Applying Deep Learning to Fast Radio Burst Classification

Upcoming Fast Radio Burst (FRB) surveys will search $\sim$10\,$^3$ beams on sky with very high duty cycle, generating large numbers of single-pulse candidates. The abundance of false positives presents an intractable problem if candidates are to be inspected by eye, making it a good application for artificial intelligence (AI). We apply deep learning to single pulse classification and develop a hierarchical framework for ranking events by their probability of being true astrophysical transients. We construct a tree-like deep neural network (DNN) that takes multiple or individual data products as input (e.g. dynamic spectra and multi-beam detection information) and trains on them simultaneously. We have built training and test sets using false-positive triggers from real telescopes, along with simulated FRBs, and single pulses from pulsars. Training of the DNN was independently done for two radio telescopes: the CHIME Pathfinder, and Apertif on Westerbork. High accuracy and recall can be achieved with a labelled training set of a few thousand events. Even with high triggering rates, classification can be done very quickly on Graphical Processing Units (GPUs). That speed is essential for selective voltage dumps or issuing real-time VOEvents. Next, we investigate whether dedispersion back-ends could be completely replaced by a real-time DNN classifier. It is shown that a single forward propagation through a moderate convolutional network could be faster than brute-force dedispersion; but the low signal-to-noise per pixel makes such a classifier sub-optimal for this problem. Real-time automated classification may prove useful for bright, unexpected signals, both now and in the era of radio astronomy when data volumes and the searchable parameter spaces further outgrow our ability to manually inspect the data, such as for SKA and ngVLA

Deep Learning for Detecting Multiple Space-Time Action Tubes in Videos

In this work, we propose an approach to the spatiotemporal localisation (detection) and classification of multiple concurrent actions within temporally untrimmed videos. Our framework is composed of three stages. In stage 1, appearance and motion detection networks are employed to localise and score actions from colour images and optical flow. In stage 2, the appearance network detections are boosted by combining them with the motion detection scores, in proportion to their respective spatial overlap. In stage 3, sequences of detection boxes most likely to be associated with a single action instance, called action tubes, are constructed by solving two energy maximisation problems via dynamic programming. While in the first pass, action paths spanning the whole video are built by linking detection boxes over time using their class-specific scores and their spatial overlap, in the second pass, temporal trimming is performed by ensuring label consistency for all constituting detection boxes. We demonstrate the performance of our algorithm on the challenging UCF101, J-HMDB-21 and LIRIS-HARL datasets, achieving new state-of-the-art results across the board and significantly increasing detection speed at test time. We achieve a huge leap forward in action detection performance and report a 20% and 11% gain in mAP (mean average precision) on UCF-101 and J-HMDB-21 datasets respectively when compared to the state-of-the-art.Comment: Accepted by British Machine Vision Conference 201