IRON-MAN: An Approach To Perform Temporal Motionless Analysis of Video using CNN in MPSoC

This paper proposes a novel human-inspired methodology called IRON-MAN ( Integrated RatiONal prediction and Motionless ANalysis ) for mobile multi-processor systems-on-chips (MPSoCs). The methodology integrates analysis of the previous image frames of the video to represent the analysis of the current frame in order to perform Temporal Motionless Analysis of the Video ( TMAV ). This is the first work on TMAV using Convolutional Neural Network (CNN) for scene prediction in MPSoCs. Experimental results show that our methodology outperforms state-of-the-art. We also introduce a metric named, Energy Consumption per Training Image ( ECTI ) to assess the suitability of using a CNN model in mobile MPSoCs with a focus on energy consumption and lifespan reliability of the device

WIND TURBINE TOWER DETECTION USING FEATURE DESCRIPTORS AND DEEP LEARNING

Wind Turbine Towers (WTTs) are the main structures of wind farms. They are costly devices that must be thoroughly inspected according to maintenance plans. Today, existence of machine vision techniques along with unmanned aerial vehicles (UAVs) enable fast, easy, and intelligent visual inspection of the structures. Our work is aimed towards developing a visionbased system to perform Nondestructive tests (NDTs) for wind turbines using UAVs. In order to navigate the flying machine toward the wind turbine tower and reliably land on it, the exact position of the wind turbine and its tower must be detected. We employ several strong computer vision approaches such as Scale-Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), Features from Accelerated Segment Test (FAST), Brute-Force, Fast Library for Approximate Nearest Neighbors (FLANN) to detect the WTT. Then, in order to increase the reliability of the system, we apply the ResNet, MobileNet, ShuffleNet, EffNet, and SqueezeNet pre-trained classifiers in order to verify whether a detected object is indeed a turbine tower or not. This intelligent monitoring system has auto navigation ability and can be used for future goals including intelligent fault diagnosis and maintenance purposes. The simulation results show the accuracy of the proposed model are 89.4% in WTT detection and 97.74% in verification (classification) problems

Edge Artificial Intelligence for Real-Time Target Monitoring

The key enabling technology for the exponentially growing cellular communications sector is location-based services. The need for location-aware services has increased along with the number of wireless and mobile devices. Estimation problems, and particularly parameter estimation, have drawn a lot of interest because of its relevance and engineers' ongoing need for higher performance. As applications expanded, a lot of interest was generated in the accurate assessment of temporal and spatial properties. In the thesis, two different approaches to subject monitoring are thoroughly addressed. For military applications, medical tracking, industrial workers, and providing location-based services to the mobile user community, which is always growing, this kind of activity is crucial. In-depth consideration is given to the viability of applying the Angle of Arrival (AoA) and Receiver Signal Strength Indication (RSSI) localization algorithms in real-world situations. We presented two prospective systems, discussed them, and presented specific assessments and tests. These systems were put to the test in diverse contexts (e.g., indoor, outdoor, in water...). The findings showed the localization capability, but because of the low-cost antenna we employed, this method is only practical up to a distance of roughly 150 meters. Consequently, depending on the use-case, this method may or may not be advantageous. An estimation algorithm that enhances the performance of the AoA technique was implemented on an edge device. Another approach was also considered. Radar sensors have shown to be durable in inclement weather and bad lighting conditions. Frequency Modulated Continuous Wave (FMCW) radars are the most frequently employed among the several sorts of radar technologies for these kinds of applications. Actually, this is because they are low-cost and can simultaneously provide range and Doppler data. In comparison to pulse and Ultra Wide Band (UWB) radar sensors, they also need a lower sample rate and a lower peak to average ratio. The system employs a cutting-edge surveillance method based on widely available FMCW radar technology. The data processing approach is built on an ad hoc-chain of different blocks that transforms data, extract features, and make a classification decision before cancelling clutters and leakage using a frame subtraction technique, applying DL algorithms to Range-Doppler (RD) maps, and adding a peak to cluster assignment step before tracking targets. In conclusion, the FMCW radar and DL technique for the RD maps performed well together for indoor use-cases. The aforementioned tests used an edge device and Infineon Technologies' Position2Go FMCW radar tool-set

Radar-Based Multi-Target Classification Using Deep Learning

Real-time, radar-based human activity and target recognition has several applications in various fields. Examples include hand gesture recognition, border and home surveillance, pedestrian recognition for automotive safety and fall detection for assisted living. This dissertation sought to improve the speed and accuracy of a previously developed model classifying human activity and targets using radar data for outdoor surveillance purposes. An improvement in accuracy and speed of classification helps surveillance systems to provide reliable results on time. For example, the results can be used to intercept trespassers, poachers or smugglers. To achieve these objectives, radar data was collected using a C-band pulse-Doppler radar and converted to spectrograms using the Short-time Fourier transform (STFT) algorithm. Spectrograms of the following classes were utilised in classification: one human walking, two humans walking, one human running, moving vehicles, a swinging sphere and clutter/noise. A seven-layer residual network was proposed, which utilised batch normalisation (BN), global average pooling (GAP), and residual connections to achieve a classification accuracy of 92.90% and 87.72% on the validation and test data, respectively. Compared to the previously proposed model, this represented a 10% improvement in accuracy on the validation data and a 3% improvement on the test data. Applying model quantisation provided up to 3.8 times speedup in inference, with a less than 0.4% accuracy drop on both the validation and test data. The quantised model could support a range of up to 89.91 kilometres in real-time, allowing it to be used in radars that operate within this range

A New Form of Interlocking Developing Technology for Level Crossings and Depots with International Applications

There are multiple large rail infrastructure projects planned or currently being undertaken within the United Kingdom. Many of these projects aim to reduce the continual issue of limited or overcapacity service. These projects involve an expansion of Rail lines, introducing faster lines, improved stations in towns and cities and better communication networks. Some major projects like Control Period 6 (CP6) are being managed by Network Rail; where projects are initiated throughout Great Britain. Many projects are managed outside Great Britain e.g., Trans-European Transport Network Program, which is planning for expansion of Rail lines (almost double) for High-Speed Rails (category I and II). These projects will increase the number of junctions and Level Crossings. A Level Crossing is where a Rail Line is crossed by a road or a walkway without the use of a tunnel or bridge. The misuse from the road users account for nearly 90% of the fatalities and near misses at Level Crossings. During 2016/2017, the Rail Network recorded 6 fatalities, about 400 near-misses and more than 77 incidents of shock and trauma. Accidents at Level Crossings represent 8% of the total accidents from the whole Rail Network. Office of Rail and Road (ORR) suggested that among these accidents at Level Crossings 90% of them are pedestrians. Such high numbers of accidents, fatalities and high risk have alarmed authorities. These authorities found it necessary to invest time and utilise given resources to improve the safety system at a Level Crossing using the safer and reliable interlocking system. The interlocking system is a feature of a control system that makes the state of two functions mutually independent. The primary function of Interlocking is to ensure that trains are safe from collision and derailment. Considering the risk associated with the Level Crossing system, the new proposed interlocking system should utilise the sensing system available at a Level Crossing to significantly reduce implementation cost and comply with the given standards and Risk Assessments. The new proposed interlocking system is designed to meet the “Safety Integrity Level- SIL” and possibly use the “2oo2” approach for its application at a Level Crossing, where the operational cycle is automated or train driver is alarmed for risk situations. Importantly, the new proposed system should detect and classify small objects and provide a reasonable solution to the current risk associated with Level Crossing, which was impossible with the traditional sensing systems. The present work discusses the sensors and algorithms used and has the potential to detect and classify objects within a Level Crossing area. The review of existing solutions e.g Inductive Loops and other major sensors allows the reader to understand why RADAR and Video Cameras are preferable choices of a sensing system for a Level Crossing. Video data provides sufficient information for the proposed algorithm to detect and classify objects at Level Crossings without the need of a manual “operator”. The RADAR sensing system can provide information using micro-Doppler signatures, which are generated from small regular movements of an obstacle. The two sensors will make the system a two-layer resilient system. The processed information from these two sensing systems is used as the “2oo2” logic system for Interlocking for automating the operational cycle or alarm the train drive using effective communication e.g., GSM-R. These two sensors provide sufficient information for the proposed algorithm, which will allow the system to automatically make an “intelligent decision” and proceed with a safe Level Crossing operational cycle. Many existing traditional algorithms depend on pixels values, which are compared with background pixels. This approach cannot detect complex textures, adapt to a dynamic background or avoid detection of unnecessary harmless objects. To avoid these problems, the proposed work utilises “Deep Learning” technology integrated with the proposed Vision and RADAR system. The Deep Learning technology can learn representations from labelled pixels; hence it does not depend on background pixels. The Deep 3 | P a g e Learning technology can classify, detect and localise objects at a Level Crossing area. It can classify and differentiate between a child and a small inanimate object, which was impossible with traditional algorithms. The system can detect an object regardless of its position, orientation and scale without any additional training because it learns representation from the data and does not rely on background pixels. The proposed system e.g., Deep Learning technology is integrated with the existing Vision System and RADAR installed at a Level Crossing, hence implementation cost is significantly reduced as well. The proposed work address two main aspects of training a model using Deep Learning technology; training from scratch and training using Transfer Learning techniques. Results are demonstrated for Image Classification, Object Detection and micro-Doppler signals from RADAR. An architecture of Convolutional Neural Network from scratch is trained consisting of Input Layer, Convolution, Pooling and Dropout Layer. The model achieves an accuracy of about 66.78%. Different notable models are trained using Transfer Learning techniques and their results are mentioned along with the MobileNet model, which achieves the highest accuracy of 91.9%. The difference between Image Classification and Object Detection is discussed and results for Object Detection are mentioned as well, where the Loss metrics are used to evaluate the performance of the Object Detector. MobileNet achieves the smallest loss metric of about 0.092. These results clearly show the effectiveness and preferability of these models for their applicability at Level Crossings. Another Convolutional Neural Network is trained using micro-Doppler signatures from the Radar system. The model trained using the micro-Doppler signature achieved an accuracy of 92%. The present work also addresses the Risk Assessment associated with the installation and maintenance of the system using Deep Learning technology. RAMS (Reliability, Availability, Maintainability and Safety) management system is used to address the General and Specific Risks associated with the sensing system integrated with the Deep Learning technology. Finally, the work is concluded with the preferred choice, its application, results and associated Risk Assessment. Deep Learning is an evolving field with new improvements being introduced constantly. Any new challenges and problems should be monitored regularly. Some future work is discussed as well. To further improve the model's accuracy, the dataset from the same distribution should be gathered with the cooperation of relevant Railway authorities. Also, the RADAR dataset could be generated rather than simulated to further include diversity and avoid any biases in the dataset during the training process. Also, the proposed system can be implemented and used in different applications within the Rail Industry e.g., passenger census and classification of passengers at the platform as discussed in the work

Visual and Camera Sensors

This book includes 13 papers published in Special Issue ("Visual and Camera Sensors") of the journal Sensors. The goal of this Special Issue was to invite high-quality, state-of-the-art research papers dealing with challenging issues in visual and camera sensors