RGB-D datasets using microsoft kinect or similar sensors: a survey

RGB-D data has turned out to be a very useful representation of an indoor scene for solving fundamental computer vision problems. It takes the advantages of the color image that provides appearance information of an object and also the depth image that is immune to the variations in color, illumination, rotation angle and scale. With the invention of the low-cost Microsoft Kinect sensor, which was initially used for gaming and later became a popular device for computer vision, high quality RGB-D data can be acquired easily. In recent years, more and more RGB-D image/video datasets dedicated to various applications have become available, which are of great importance to benchmark the state-of-the-art. In this paper, we systematically survey popular RGB-D datasets for different applications including object recognition, scene classification, hand gesture recognition, 3D-simultaneous localization and mapping, and pose estimation. We provide the insights into the characteristics of each important dataset, and compare the popularity and the difficulty of those datasets. Overall, the main goal of this survey is to give a comprehensive description about the available RGB-D datasets and thus to guide researchers in the selection of suitable datasets for evaluating their algorithms

Weed/Plant Classification Using Evolutionary Optimised Ensemble Based On Local Binary Patterns

This thesis presents a novel pixel-level weed classification through rotation-invariant uniform local binary pattern (LBP) features for precision weed control. Based on two-level optimisation structure; First, Genetic Algorithm (GA) optimisation to select the best rotation-invariant uniform LBP configurations; Second, Covariance Matrix Adaptation Evolution Strategy (CMA-ES) in the Neural Network (NN) ensemble to select the best combinations of voting weights of the predicted outcome for each classifier. The model obtained 87.9% accuracy in CWFID public benchmark

Autonomous Vehicles an overview on system, cyber security, risks, issues, and a way forward

This chapter explores the complex realm of autonomous cars, analyzing their fundamental components and operational characteristics. The initial phase of the discussion is elucidating the internal mechanics of these automobiles, encompassing the crucial involvement of sensors, artificial intelligence (AI) identification systems, control mechanisms, and their integration with cloud-based servers within the framework of the Internet of Things (IoT). It delves into practical implementations of autonomous cars, emphasizing their utilization in forecasting traffic patterns and transforming the dynamics of transportation. The text also explores the topic of Robotic Process Automation (RPA), illustrating the impact of autonomous cars on different businesses through the automation of tasks. The primary focus of this investigation lies in the realm of cybersecurity, specifically in the context of autonomous vehicles. A comprehensive analysis will be conducted to explore various risk management solutions aimed at protecting these vehicles from potential threats including ethical, environmental, legal, professional, and social dimensions, offering a comprehensive perspective on their societal implications. A strategic plan for addressing the challenges and proposing strategies for effectively traversing the complex terrain of autonomous car systems, cybersecurity, hazards, and other concerns are some resources for acquiring an understanding of the intricate realm of autonomous cars and their ramifications in contemporary society, supported by a comprehensive compilation of resources for additional investigation. Keywords: RPA, Cyber Security, AV, Risk, Smart Car

Precision Poultry Farming

This book presents the latest advances in applications of continuous, objective, and automated sensing technologies and computer tools for sustainable and efficient poultry production, and it offers solutions to the poultry industry to address challenges in terms of poultry management, the environment, nutrition, automation and robotics, health, welfare assessment, behavior monitoring, waste management, etc. The reader will find original research papers that address, on a global scale, the sustainability and efficiency of the poultry industry and explore the above-mentioned areas through applications of PPF solutions in poultry meat and egg productio

Soft computing applied to optimization, computer vision and medicine

Artificial intelligence has permeated almost every area of life in modern society, and its significance continues to grow. As a result, in recent years, Soft Computing has emerged as a powerful set of methodologies that propose innovative and robust solutions to a variety of complex problems. Soft Computing methods, because of their broad range of application, have the potential to significantly improve human living conditions. The motivation for the present research emerged from this background and possibility. This research aims to accomplish two main objectives: On the one hand, it endeavors to bridge the gap between Soft Computing techniques and their application to intricate problems. On the other hand, it explores the hypothetical benefits of Soft Computing methodologies as novel effective tools for such problems. This thesis synthesizes the results of extensive research on Soft Computing methods and their applications to optimization, Computer Vision, and medicine. This work is composed of several individual projects, which employ classical and new optimization algorithms. The manuscript presented here intends to provide an overview of the different aspects of Soft Computing methods in order to enable the reader to reach a global understanding of the field. Therefore, this document is assembled as a monograph that summarizes the outcomes of these projects across 12 chapters. The chapters are structured so that they can be read independently. The key focus of this work is the application and design of Soft Computing approaches for solving problems in the following: Block Matching, Pattern Detection, Thresholding, Corner Detection, Template Matching, Circle Detection, Color Segmentation, Leukocyte Detection, and Breast Thermogram Analysis. One of the outcomes presented in this thesis involves the development of two evolutionary approaches for global optimization. These were tested over complex benchmark datasets and showed promising results, thus opening the debate for future applications. Moreover, the applications for Computer Vision and medicine presented in this work have highlighted the utility of different Soft Computing methodologies in the solution of problems in such subjects. A milestone in this area is the translation of the Computer Vision and medical issues into optimization problems. Additionally, this work also strives to provide tools for combating public health issues by expanding the concepts to automated detection and diagnosis aid for pathologies such as Leukemia and breast cancer. The application of Soft Computing techniques in this field has attracted great interest worldwide due to the exponential growth of these diseases. Lastly, the use of Fuzzy Logic, Artificial Neural Networks, and Expert Systems in many everyday domestic appliances, such as washing machines, cookers, and refrigerators is now a reality. Many other industrial and commercial applications of Soft Computing have also been integrated into everyday use, and this is expected to increase within the next decade. Therefore, the research conducted here contributes an important piece for expanding these developments. The applications presented in this work are intended to serve as technological tools that can then be used in the development of new devices

Pixel-Level Deep Multi-Dimensional Embeddings for Homogeneous Multiple Object Tracking

The goal of Multiple Object Tracking (MOT) is to locate multiple objects and keep track of their individual identities and trajectories given a sequence of (video) frames. A popular approach to MOT is tracking by detection consisting of two processing components: detection (identification of objects of interest in individual frames) and data association (connecting data from multiple frames). This work addresses the detection component by introducing a method based on semantic instance segmentation, i.e., assigning labels to all visible pixels such that they are unique among different instances. Modern tracking methods often built around Convolutional Neural Networks (CNNs) and additional, explicitly-defined post-processing steps. This work introduces two detection methods that incorporate multi-dimensional embeddings. We train deep CNNs to produce easily-clusterable embeddings for semantic instance segmentation and to enable object detection through pose estimation. The use of embeddings allows the method to identify per-pixel instance membership for both tasks. Our method specifically targets applications that require long-term tracking of homogeneous targets using a stationary camera. Furthermore, this method was developed and evaluated on a livestock tracking application which presents exceptional challenges that generalized tracking methods are not equipped to solve. This is largely because contemporary datasets for multiple object tracking lack properties that are specific to livestock environments. These include a high degree of visual similarity between targets, complex physical interactions, long-term inter-object occlusions, and a fixed-cardinality set of targets. For the reasons stated above, our method is developed and tested with the livestock application in mind and, specifically, group-housed pigs are evaluated in this work. Our method reliably detects pigs in a group housed environment based on the publicly available dataset with 99% precision and 95% using pose estimation and achieves 80% accuracy when using semantic instance segmentation at 50% IoU threshold. Results demonstrate our method\u27s ability to achieve consistent identification and tracking of group-housed livestock, even in cases where the targets are occluded and despite the fact that they lack uniquely identifying features. The pixel-level embeddings used by the proposed method are thoroughly evaluated in order to demonstrate their properties and behaviors when applied to real data. Adivser: Lance C. Pére

Pedestrian detection in far infrared images

Detection of people in images is a relatively new field of research, but has been widely accepted. The applications are multiple, such as self-labeling of large databases, security systems and pedestrian detection in intelligent transportation systems. Within the latter, the purpose of a pedestrian detector from a moving vehicle is to detect the presence of people in the path of the vehicle. The ultimate goal is to avoid a collision between the two. This thesis is framed with the advanced driver assistance systems, passive safety systems that warn the driver of conditions that may be adverse. An advanced driving assistance system module, aimed to warn the driver about the presence of pedestrians, using computer vision in thermal images, is presented in this thesis. Such sensors are particularly useful under conditions of low illumination.The document is divided following the usual parts of a pedestrian detection system: development of descriptors that define the appearance of people in these kind of images, the application of these descriptors to full-sized images and temporal tracking of pedestrians found. As part of the work developed in this thesis, database of pedestrians in the far infrared spectrum is presented. This database has been used in developing an evaluation of pedestrian detection systems as well as for the development of new descriptors. These descriptors use techniques for the systematic description of the shape of the pedestrian as well as methods to achieve invariance to contrast, illumination or ambient temperature. The descriptors are analyzed and modified to improve their performance in a detection problem, where potential candidates are searched for in full size images. Finally, a method for tracking the detected pedestrians is proposed to reduce the number of miss-detections that occurred at earlier stages of the algorithm. --La detección de personas en imágenes es un campo de investigación relativamente nuevo, pero que ha tenido una amplia acogida. Las aplicaciones son múltiples, tales como auto-etiquetado de grandes bases de datos, sistemas de seguridad y detección de peatones en sistemas inteligentes de transporte. Dentro de este último, la detección de peatones desde un vehículo móvil tiene como objetivo detectar la presencia de personas en la trayectoria del vehículo. EL fin último es evitar una colisión entre ambos. Esta tesis se enmarca en los sistemas avanzados de ayuda a la conducción; sistemas de seguridad pasivos, que advierten al conductor de condiciones que pueden ser adversas. En esta tesis se presenta un módulo de ayuda a la conducción destinado a advertir de la presencia de peatones, mediante el uso de visión por computador en imágenes térmicas. Este tipo de sensores resultan especialmente útiles en condiciones de baja iluminación. El documento se divide siguiendo las partes habituales de una sistema de detección de peatones: desarrollo de descriptores que defina la apariencia de las personas en este tipo de imágenes, la aplicación de estos en imágenes de tamano completo y el seguimiento temporal de los peatones encontrados. Como parte del trabajo desarrollado en esta tesis se presenta una base de datos de peatones en el espectro infrarrojo lejano. Esta base de datos ha sido utilizada para desarrollar una evaluación de sistemas de detección de peatones, así como para el desarrollo de nuevos descriptores. Estos integran técnicas para la descripción sistemática de la forma del peatón, así como métodos para la invariancia al contraste, la iluminación o la temperatura externa. Los descriptores son analizados y modificados para mejorar su rendimiento en un problema de detección, donde se buscan posibles candidatos en una imagen de tamano completo. Finalmente, se propone una método de seguimiento de los peatones detectados para reducir el número de fallos que se hayan producido etapas anteriores del algoritmo

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available

Advancing heat stress detection in dairy cows through machine learning and computer vision

Heat stress detection in dairy cows has long been connected with production loss. However, the reduction in milk yield lags behind the exposure to heat stress events for about two days. Other stress responses, such as physiological and behavioural changes, are well documented to be activated by dairy cows in the earlier stage of heat stress compared with production loss. Among all candidate indicators, body surface temperatures (BST), respiration rate (RR), and relevant behaviours have been concluded to be the most appropriate indicators due to their high feasibility of acquisition and early response. Vision-based methods are promising for accurate measurements while adhering to animal welfare principles. Meanwhile, predictive models show a non-invasive alternative to obtain these data and can provide useful insights with their interpretations. Thus, this thesis aimed to provide non-invasive solutions to the detection of heat stress in dairy cows by using artificial intelligence techniques. The detailed research content and relevant conclusions are as follows: An automated tool based on improved UNet was proposed to collect facial BST from five facial landmarks (i.e., eyes, muzzle, nostrils, ears, and horns) on cattle infrared images. The baseline UNet model was improved by replacing the traditional convolutional layers in the decoder with Ghost modules and adding efficient channel attention modules. The improved UNet outperformed other comparable models with the highest mean Intersection of Union of 80.76% and a slightly slower but still good inference speed of 32.7 frames per second (FPS). Agreement analysis reveals small to negligible differences between the temperatures obtained automatically in the area of eyes and ears and the ground truth. A vision-based method was proposed to measure RR for multiple dairy cows lying on free stalls. The proposed method involved various computer vision tasks (i.e., instance segmentation, object detection, object tracking, video stabilisation, and optical flow) to obtain respiration-related signals and finally utilised Fast Fourier Transform to extract RR. The results show that the measured RR had a Pearson correlation coefficient of 0.945, a root mean square error (RMSE) of 5.24 breaths per minute (bpm), and an intraclass correlation coefficient of 0.98 compared with visual observation. The average processing time and FPS on 55 test video clips (mean ± standard deviation duration of 16 ± 4 s) was 8.2 s and 64, respectively. A deep learning-based model was proposed to recognise cow behaviours (i.e., drinking, eating, lying, standing-in, and standing-out) that are known to be influenced by heat stress. The YOLOv5s model was selected due to its ability to compress the weight size while maintaining accuracy. It had a mean average precision of 0.985 and an inference speed of 73 FPS. Further validation demonstrates the excellent capacity of the proposed model in measuring herd-level behavioural indicators, with an intraclass correlation coefficient of 0.97 compared with manual observation. Critical thresholds were determined by using piecewise regression models with environmental indicators as the predictors and animal-based indicators as the outcomes. An ambient temperature (Ta) threshold was determined at 26.1 °C when the automated measured mean eye temperature reached 35.3 °C. A Ta threshold of 23.6 °C and a temperature-humidity index (THI) threshold of 72 were determined when the automated measured RR reached 61.1 and 60.4 bpm, respectively. In addition, the test dairy herd began to change their standing and lying behaviour at the earliest Ta of 23.8 ℃ or THI of 68.5. Four machine learning algorithms were used to predict RR, vaginal temperature (VT), and eye temperature (ET) from 13 predictor variables from three dimensions: production, cow-related, and environmental factors. The artificial neural networks yielded the lowest RMSE for predicting RR (13.24 bpm), VT (0.30 ℃), and ET (0.29 ℃). The results interpreted with partial dependence plots and Local Interpretable Model-agnostic Explanations show that P.M. measurements and winter calving contributed most to high RR and VT predictions, whereas lying posture, high Ta, and low wind speed contributed most to high ET predictions. Based on these results, an integrative application of all the proposed measurement, prediction, and assessment methods has been suggested, wherein RGB and infrared cameras are used to measure animal-based indicators, and critical thresholds, along with model interpretation, are used to assess the heat stress state of dairy cows. This strategy ensures timely and thorough cooling of cows in all areas of the dairy farm, thereby minimising the negative impact of heat stress to the greatest extent

Human-Centric Machine Vision

Recently, the algorithms for the processing of the visual information have greatly evolved, providing efficient and effective solutions to cope with the variability and the complexity of real-world environments. These achievements yield to the development of Machine Vision systems that overcome the typical industrial applications, where the environments are controlled and the tasks are very specific, towards the use of innovative solutions to face with everyday needs of people. The Human-Centric Machine Vision can help to solve the problems raised by the needs of our society, e.g. security and safety, health care, medical imaging, and human machine interface. In such applications it is necessary to handle changing, unpredictable and complex situations, and to take care of the presence of humans