Pluggable drone imaging analysis framework for mob detection during open-air events

C

People detection using IR camera on a drone for more effective rescue operations

Bakalárska práca sa zaobera spojením disciplíny nazývanej temografia so softwarovými systémami na dekekciu objektov. Cieľom je pomocou analýzy a testovania nájsť vhodnú metódu, ktorá dokáže zautomatizovať analýzu dát z termokamier na dronoch. Využitie tejto práce spočíva napríklad v zefektívnení záchranných operácií. Pre dosiahnutie daných cieľov bolo potrebné implementovat aplikáciu v jazyku Python, ktorá realizuje detekciu pomocou dostupných systémov, ako je Darknet. Pomocou tejto aplikácie som experimentálne preukázal, že detekcia pomocou neurónových sietí predstavuje najlepšiu možnost a pomocou systému Darknet je možné detekovať objekty dostatočne rýchlo a presne.This bachelor's thesis investigates the usage of object detection algorithms on images captured by an infrared camera placed on a drone. The solution will help to automate the analysis of captured data, targeting to increase the effectiveness of rescue operations. During the completion of the task, I developed a Python desktop application, that realizes chosen detection methods. The methods selection was based on an analysis of current approaches and take advantage of the existing detection systems. The application was used to measure the accuracy and performance of these approaches on the dataset created as a part of the thesis. In the end, the conclusion evaluates the possibility to use image detection on a thermogram, in a real-world application. The single-stage Region Proposal Convolutional Network showed the best result and was chosen for future development

A Survey on Computer Vision based Human Analysis in the COVID-19 Era

The emergence of COVID-19 has had a global and profound impact, not only on society as a whole, but also on the lives of individuals. Various prevention measures were introduced around the world to limit the transmission of the disease, including face masks, mandates for social distancing and regular disinfection in public spaces, and the use of screening applications. These developments also triggered the need for novel and improved computer vision techniques capable of (i) providing support to the prevention measures through an automated analysis of visual data, on the one hand, and (ii) facilitating normal operation of existing vision-based services, such as biometric authentication schemes, on the other. Especially important here, are computer vision techniques that focus on the analysis of people and faces in visual data and have been affected the most by the partial occlusions introduced by the mandates for facial masks. Such computer vision based human analysis techniques include face and face-mask detection approaches, face recognition techniques, crowd counting solutions, age and expression estimation procedures, models for detecting face-hand interactions and many others, and have seen considerable attention over recent years. The goal of this survey is to provide an introduction to the problems induced by COVID-19 into such research and to present a comprehensive review of the work done in the computer vision based human analysis field. Particular attention is paid to the impact of facial masks on the performance of various methods and recent solutions to mitigate this problem. Additionally, a detailed review of existing datasets useful for the development and evaluation of methods for COVID-19 related applications is also provided. Finally, to help advance the field further, a discussion on the main open challenges and future research direction is given.Comment: Submitted to Image and Vision Computing, 44 pages, 7 figure

Image-based Deep Learning for Smart Digital Twins: a Review

Smart Digital twins (SDTs) are being increasingly used to virtually replicate and predict the behaviors of complex physical systems through continual data assimilation enabling the optimization of the performance of these systems by controlling the actions of systems. Recently, deep learning (DL) models have significantly enhanced the capabilities of SDTs, particularly for tasks such as predictive maintenance, anomaly detection, and optimization. In many domains, including medicine, engineering, and education, SDTs use image data (image-based SDTs) to observe and learn system behaviors and control their behaviors. This paper focuses on various approaches and associated challenges in developing image-based SDTs by continually assimilating image data from physical systems. The paper also discusses the challenges involved in designing and implementing DL models for SDTs, including data acquisition, processing, and interpretation. In addition, insights into the future directions and opportunities for developing new image-based DL approaches to develop robust SDTs are provided. This includes the potential for using generative models for data augmentation, developing multi-modal DL models, and exploring the integration of DL with other technologies, including 5G, edge computing, and IoT. In this paper, we describe the image-based SDTs, which enable broader adoption of the digital twin DT paradigms across a broad spectrum of areas and the development of new methods to improve the abilities of SDTs in replicating, predicting, and optimizing the behavior of complex systems.Comment: 12 pages, 2 figures, and 3 table

ORGAN LOCALIZATION AND DETECTION IN SOW’S USING MACHINE LEARNING AND DEEP LEARNING IN COMPUTER VISION

The objective of computer vision research is to endow computers with human-like perception to enable the capability to detect their surroundings, interpret the data they sense, take appropriate actions, and learn from their experiences to improve future performance. The area has progressed from using traditional pattern recognition and image processing technologies to advanced techniques in image understanding such as model-based and knowledge-based vision. In the past few years there has been a surge of interest in machine learning algorithms for computer vision-based applications. Machine learning technology has the potential to significantly contribute to the development of flexible and robust vision algorithms that will improve the performance of practical vision systems with a higher level of competence and greater generality. Additionally, the development of machine learning-based architectures has the potential to reduce system development time while simultaneously achieving the above-stated performance improvements. This work proposes the utilization of a computer vision-based approach that leverages machine and deep learning systems to aid the detection and identification of sow reproduction cycles by segmentation and object detection techniques. A lightweight machine learning system is proposed for object detection to address dataset collection issues in one of the most crucial and potentially lucrative farming applications. This technique was designed to detect the vulvae region in pre-estrous sows using a single thermal image. In the first experiment, the support vector machine (SVM) classifier was used after extracting features determined by 12 Gabor filters. The features are then concatenated with the features obtained from the Histogram of oriented gradients (HOG) to produce the results of the first experiment. In the second experiment, the number of distinct Gabor filters used was increased from 12 to 96. The system is trained on cropped image windows and uses the Gaussian pyramid technique to look for the vulva in the input image. The resulting process is shown to be lightweight, simple, and robust when applied to and evaluated on a large number of images. The results from extensive qualitative and quantitative testing experiments are included. The experimental results include false detection, missing detection and favorable detection rates. The results indicate state-of-the-art accuracy. Additionally, the project was expanded by utilizing the You Only Look Once (YOLO) deep learning Object Detection models for fast object detection. The results from object detection have been used to label images for segmentation. The bounding box from the detected area was systematically colored to achieve the segmented and labeled images. Then these segmented images are used as custom data to train U-Net segmentation. The first step involves building a machine learning model using Gabor filters and HOG for feature extraction and SVM for classification. The results discovered the deficiency of the model, therefore a second stage was suggested in which the dataset was trained using YOLOv3-dependent deep learning object detection. The resulting segmentation model is found to be the best choice to aid the process of vulva localization. Since the model depends on the original gray-scale image and the mask of the region of interest (ROI), a custom dataset containing these features was obtained, augmented, and used to train a U-Net segmentation model. The results of the final approach shows that the proposed system can segment sow\u27s vulva region even in low rank images and has an excellent performance efficiency. Furthermore, the resulting algorithm can be used to improve the automation of estrous detection by providing reliable ROI identification and segmentation and enabling beneficial temporal change detection and tracking in future efforts

Human detection in real time with thermal camera using drones

We currently live in a world where technology predominates and advances very quickly, developing applications and devices that help us in daily life to improve our lives and solve everyday problems. One of these technologies that are in full development and evolution are the Drones, or also called RPAS (Remotely Piloted Aircraft System). These systems are unmanned aerial vehicles that offer us infinity applications to cover these needs and problems that we have on a daily basis. On the other hand, another technology that is in full development and that has been seen to have great potential for developing applications is Artificial Intelligence (AI). To quickly define AI, we say that it is the intelligence that humans develop with body, brain and mind but expressed by a machine, processor and software. Thanks to the fact that Drones can fly over places where humans cannot reach, through their cameras we can see what is in those areas, so one of the most useful applications in Drones is Search and Rescue operations. The main focus of this project is to merge RPAS with AI to develope a Search and Rescue application. For this, an executable software has been created for any computer, which will allow to detect people lost in the forest or other places, through a video, either through a Streaming on Youtube or a video saved locally, allowing detection both in real time as well as in deferred time, making the detection done by the machine and the human being able to do other functions while the search is done with the Drone. The objective is to use both, the thermal and visual cameras of a Drone, to record a video or stream the image and send it to the software so that, through Artificial Intelligence, if it finds a person, it detects the human and an alarm sounds. The software has been developed in Python, an open source, cross-platform programming language that can be used for web development, software creation, and data processing. This language is truly useful since it is one of the most used in the world of programming, thus there are multiple libraries, as OpenCV, created by open source users that have allowed the development of this human detection software. To develop the program, it has been necessary to train the machine with images of people. These images have been obtained with real flights in the area of Collserola, Barcelona. This area is close to the airport of Barcelona El Prat, so permissions and coordination are needed to be able to fly completely legally. For this reason, this project has also included all the documentation and legal part necessary to be able to fly in the Barcelona area