Human Body Posture Recognition Approaches: A Review

Human body posture recognition has become the focus of many researchers in recent years. Recognition of body posture is used in various applications, including surveillance, security, and health monitoring. However, these systems that determine the body’s posture through video clips, images, or data from sensors have many challenges when used in the real world. This paper provides an important review of how most essential ‎ hardware technologies are ‎used in posture recognition systems‎. These systems capture and collect datasets through ‎accelerometer sensors or computer vision. In addition, this paper presents a comparison ‎study with state-of-the-art in terms of accuracy. We also present the advantages and ‎limitations of each system and suggest promising future ideas that can increase the ‎efficiency of the existing posture recognition system. Finally, the most common datasets ‎applied in these systems are described in detail. It aims to be a resource to help choose one of the methods in recognizing the posture of the human body and the techniques that suit each method. It analyzes more than 80 papers between 2015 and 202

Classifiers and machine learning techniques for image processing and computer vision

Orientador: Siome Klein GoldensteinTese (doutorado) - Universidade Estadual de Campinas, Instituto da ComputaçãoResumo: Neste trabalho de doutorado, propomos a utilizaçãoo de classificadores e técnicas de aprendizado de maquina para extrair informações relevantes de um conjunto de dados (e.g., imagens) para solução de alguns problemas em Processamento de Imagens e Visão Computacional. Os problemas de nosso interesse são: categorização de imagens em duas ou mais classes, detecçãao de mensagens escondidas, distinção entre imagens digitalmente adulteradas e imagens naturais, autenticação, multi-classificação, entre outros. Inicialmente, apresentamos uma revisão comparativa e crítica do estado da arte em análise forense de imagens e detecção de mensagens escondidas em imagens. Nosso objetivo é mostrar as potencialidades das técnicas existentes e, mais importante, apontar suas limitações. Com esse estudo, mostramos que boa parte dos problemas nessa área apontam para dois pontos em comum: a seleção de características e as técnicas de aprendizado a serem utilizadas. Nesse estudo, também discutimos questões legais associadas a análise forense de imagens como, por exemplo, o uso de fotografias digitais por criminosos. Em seguida, introduzimos uma técnica para análise forense de imagens testada no contexto de detecção de mensagens escondidas e de classificação geral de imagens em categorias como indoors, outdoors, geradas em computador e obras de arte. Ao estudarmos esse problema de multi-classificação, surgem algumas questões: como resolver um problema multi-classe de modo a poder combinar, por exemplo, caracteríisticas de classificação de imagens baseadas em cor, textura, forma e silhueta, sem nos preocuparmos demasiadamente em como normalizar o vetor-comum de caracteristicas gerado? Como utilizar diversos classificadores diferentes, cada um, especializado e melhor configurado para um conjunto de caracteristicas ou classes em confusão? Nesse sentido, apresentamos, uma tecnica para fusão de classificadores e caracteristicas no cenário multi-classe através da combinação de classificadores binários. Nós validamos nossa abordagem numa aplicação real para classificação automática de frutas e legumes. Finalmente, nos deparamos com mais um problema interessante: como tornar a utilização de poderosos classificadores binarios no contexto multi-classe mais eficiente e eficaz? Assim, introduzimos uma tecnica para combinação de classificadores binarios (chamados classificadores base) para a resolução de problemas no contexto geral de multi-classificação.Abstract: In this work, we propose the use of classifiers and machine learning techniques to extract useful information from data sets (e.g., images) to solve important problems in Image Processing and Computer Vision. We are particularly interested in: two and multi-class image categorization, hidden messages detection, discrimination among natural and forged images, authentication, and multiclassification. To start with, we present a comparative survey of the state-of-the-art in digital image forensics as well as hidden messages detection. Our objective is to show the importance of the existing solutions and discuss their limitations. In this study, we show that most of these techniques strive to solve two common problems in Machine Learning: the feature selection and the classification techniques to be used. Furthermore, we discuss the legal and ethical aspects of image forensics analysis, such as, the use of digital images by criminals. We introduce a technique for image forensics analysis in the context of hidden messages detection and image classification in categories such as indoors, outdoors, computer generated, and art works. From this multi-class classification, we found some important questions: how to solve a multi-class problem in order to combine, for instance, several different features such as color, texture, shape, and silhouette without worrying about the pre-processing and normalization of the combined feature vector? How to take advantage of different classifiers, each one custom tailored to a specific set of classes in confusion? To cope with most of these problems, we present a feature and classifier fusion technique based on combinations of binary classifiers. We validate our solution with a real application for automatic produce classification. Finally, we address another interesting problem: how to combine powerful binary classifiers in the multi-class scenario more effectively? How to boost their efficiency? In this context, we present a solution that boosts the efficiency and effectiveness of multi-class from binary techniques.DoutoradoEngenharia de ComputaçãoDoutor em Ciência da Computaçã

Nondestructive Multivariate Classification of Codling Moth Infested Apples Using Machine Learning and Sensor Fusion

Apple is the number one on the list of the most consumed fruits in the United States. The increasing market demand for high quality apples and the need for fast, and effective quality evaluation techniques have prompted research into the development of nondestructive evaluation methods. Codling moth (CM), Cydia pomonella L. (Lepidoptera: Tortricidae), is the most devastating pest of apples. Therefore, this dissertation is focused on the development of nondestructive methods for the detection and classification of CM-infested apples. The objective one in this study was aimed to identify and characterize the source of detectable vibro-acoustic signals coming from CM-infested apples. A novel approach was developed to correlate the larval activities to low-frequency vibro-acoustic signals, by capturing the larval activities using a digital camera while simultaneously registering the signal patterns observed in the contact piezoelectric sensors on apple surface. While the larva crawling was characterized by the low amplitude and higher frequency (around 4 Hz) signals, the chewing signals had greater amplitude and lower frequency (around 1 Hz). In objective two and three, vibro-acoustic and acoustic impulse methods were developed to classify CM-infested and healthy apples. In the first approach, the identified vibro-acoustic patterns from the infested apples were used for the classification of the CM-infested and healthy signal data. The classification accuracy was as high as 95.94% for 5 s signaling time. For the acoustic impulse method, a knocking test was performed to measure the vibration/acoustic response of the infested apple fruit to a pre-defined impulse in comparison to that of a healthy sample. The classification rate obtained was 99% for a short signaling time of 60-80 ms. In objective four, shortwave near infrared hyperspectral imaging (SWNIR HSI) in the wavelength range of 900-1700 nm was applied to detect CM infestation at the pixel level for the three apple cultivars reaching an accuracy of up to 97.4%. In objective five, the physicochemical characteristics of apples were predicted using HSI method. The results showed the correlation coefficients of prediction (Rp) up to 0.90, 0.93, 0.97, and 0.91 for SSC, firmness, pH and moisture content, respectively. Furthermore, the effect of long-term storage (20 weeks) at three different storage conditions (0 °C, 4 °C, and 10 °C) on CM infestation and the detectability of the infested apples was studied. At a constant storage temperature the detectability of infested samples remained the same for the first three months then improved in the fourth month followed by a decrease until the end of the storage. Finally, a sensor data fusion method was developed which showed an improvement in the classification performance compared to the individual methods. These findings indicated there is a high potential of acoustic and NIR HSI methods for detecting and classifying CM infestation in different apple cultivars

Multimodal Analytics for Healthcare

The ailing healthcare system demands effective autonomous solutions to improve service and provide individualize care, while reducing the burden on the scarce healthcare workforce. Most of these solutions require a multidisciplinary approach that combines healthcare with computational abilities. The work presented in this thesis introduces a multimodal multiview network along with methods and solutions that leverage inexpensive visual sensors and computers to monitor healthcare. One of the most prominent outcomes of this work includes enabling the medical analysis of ICU conditions such as sleep disorders, decubitus ulcerations, and hospital acquired infections, which are preventable and negatively affect patients' health population. The problems tackled include patient pose classification, pose motion analysis and summarization, role representation and identification, and activity and event logging in natural hospital settings. These problems are addressed via a non-intrusive non-disruptive multimodal multiview sensor network (Medical Internet-of-Things). The multimodal data is combined with coupled-optimization to estimate source weights and accurately classify patient poses. Pose patterns such as pose transitions are represented using deep convolutional features and pose duration is modelled via segments. The proposed techniques serve to differentiate between poses and pseudo-poses (transitory poses) and create effective motion summaries. The role representation is tackled using novel appearance and semantic interaction maps to assign generic labels to individuals (doctor, nurse, visitor, etc) without using identifiable information (e.g., facetracking or badges), which is prohibited in healthcare applications. Finally, activity and event analysis is tackled using a new contextual aspect frames where aspect bases and weights are learned and then used to reconstruct activities. The objective of this thesis is to enable the development, evaluation, and optimization of individualized therapies, standards-of-care, room infrastructural designs, and clinical workflows and procedures

Gesture recognition using a depth sensor and machine learning techniques

Advances in depth sensing provide great opportunities for the development of new methods for human computer interactivity. With the launch of the Microsoft Kinect sensor, high-resolution depth and visual sensing has become available for widespread use. As it is suitable for measuring distances to objects at high frame rate, such kind of sensors are increasingly used for 3D acquisitions, and more generally for applications in robotics or computer vision. The aim of this survey is to implement a gesture recognition system using the Kinect version 2 of Microsoft in order to interact with a virtual TV weather studio. The Kinect sensor was used to build up a dataset, which contains motion recordings of 8 different gestures and was build up by two different gesture training machine learning algorithms. Then, the system is evaluated in a user study, which allows a direct comparison and reveals benefits and limits of using such technique. Finally, it is given an overview of the challenges in this field and future work trends.Los avances en los sensores de profundidad ofrecen grandes oportunidades para el desarrollo de nuevos métodos para la interactividad computadora-humano. Con el lanzamiento del sensor Kinect de Microsoft, la detección de profundidad de alta resolución se ha convertido en un componente disponible para el uso generalizado. Como es adecuado para medir distancias a objetos a alta velocidad, este tipo de sensores se utilizan cada vez más para adquirir información 3D, y más en general para aplicaciones en robótica o en visión artificial. El objetivo de este estudio es implementar un sistema de reconocimiento de gestos utilizando la Kinect versión 2 de Microsoft con el fin de interactuar con un estudio virtual de TV. El sensor Kinect se utilizó para construir una base de datos, que contiene grabaciones de movimientos para 8 gestos distintos y fue entrenado por dos algoritmos diferentes de aprendizaje de máquinas. A continuación, el sistema se evaluó con un conjunto de usuarios en un estudio virtual, lo que permite una comparación directa y revela los beneficios y los límites de la utilización de tal técnica. Por último, se da una visión general de los retos en este campo y futuras líneas de trabajoEls avenços en els sensors de profunditat ofereixen grans oportunitats per al desenvolupament de nous mètodes per a la interactivitat ordinador-humà. Amb el llançament del sensor Kinect de Microsoft, la detecció de profunditat d’alta resolució s’ha convertit en un component disponible per a l’ús generalitzat. Com és adequat per mesurar distàncies a objectes a alta velocitat, aquest tipus de sensors s’utilitzen cada vegada més per adquirir informació 3D, i més en general per a aplicacions en robòtica i en visió artificial. L’objectiu d’aquest estudi és implementar un sistema de reconeixement de gestos utilitzant la Kinect versió 2 de Microsoft per tal d’interactuar amb un estudi virtual de TV. El sensor Kinect es va utilitzar per construir una base de dades, que conté gravacions de moviments per a 8 gestos diferents i va ser entrenat per dos algoritmes diferents d’aprenentatge de màquines. A continuació, el sistema es va avaluar amb un conjunt d’usuaris en un estudi virtual, el que permet una comparació directa i revela els beneficis i els límits de la utilització de tal tècnica. Finalment, es dóna una visió general dels reptes en aquest camp i futures línies de treball.Ballester Ripoll, M. (2016). Gesture recognition using a depth sensor and machine learning techniques. Universitat Politècnica de València. http://hdl.handle.net/10251/77950TFG

Computer vision based classification of fruits and vegetables for self-checkout at supermarkets

The field of machine learning, and, in particular, methods to improve the capability of machines to perform a wider variety of generalised tasks are among the most rapidly growing research areas in today’s world. The current applications of machine learning and artificial intelligence can be divided into many significant fields namely computer vision, data sciences, real time analytics and Natural Language Processing (NLP). All these applications are being used to help computer based systems to operate more usefully in everyday contexts. Computer vision research is currently active in a wide range of areas such as the development of autonomous vehicles, object recognition, Content Based Image Retrieval (CBIR), image segmentation and terrestrial analysis from space (i.e. crop estimation). Despite significant prior research, the area of object recognition still has many topics to be explored. This PhD thesis focuses on using advanced machine learning approaches to enable the automated recognition of fresh produce (i.e. fruits and vegetables) at supermarket self-checkouts. This type of complex classification task is one of the most recently emerging applications of advanced computer vision approaches and is a productive research topic in this field due to the limited means of representing the features and machine learning techniques for classification. Fruits and vegetables offer significant inter and intra class variance in weight, shape, size, colour and texture which makes the classification challenging. The applications of effective fruit and vegetable classification have significant importance in daily life e.g. crop estimation, fruit classification, robotic harvesting, fruit quality assessment, etc. One potential application for this fruit and vegetable classification capability is for supermarket self-checkouts. Increasingly, supermarkets are introducing self-checkouts in stores to make the checkout process easier and faster. However, there are a number of challenges with this as all goods cannot readily be sold with packaging and barcodes, for instance loose fresh items (e.g. fruits and vegetables). Adding barcodes to these types of items individually is impractical and pre-packaging limits the freedom of choice when selecting fruits and vegetables and creates additional waste, hence reducing customer satisfaction. The current situation, which relies on customers correctly identifying produce themselves leaves open the potential for incorrect billing either due to inadvertent error, or due to intentional fraudulent misclassification resulting in financial losses for the store. To address this identified problem, the main goals of this PhD work are: (a) exploring the types of visual and non-visual sensors that could be incorporated into a self-checkout system for classification of fruits and vegetables, (b) determining a suitable feature representation method for fresh produce items available at supermarkets, (c) identifying optimal machine learning techniques for classification within this context and (d) evaluating our work relative to the state-of-the-art object classification results presented in the literature. An in-depth analysis of related computer vision literature and techniques is performed to identify and implement the possible solutions. A progressive process distribution approach is used for this project where the task of computer vision based fruit and vegetables classification is divided into pre-processing and classification techniques. Different classification techniques have been implemented and evaluated as possible solution for this problem. Both visual and non-visual features of fruit and vegetables are exploited to perform the classification. Novel classification techniques have been carefully developed to deal with the complex and highly variant physical features of fruit and vegetables while taking advantages of both visual and non-visual features. The capability of classification techniques is tested in individual and ensemble manner to achieved the higher effectiveness. Significant results have been obtained where it can be concluded that the fruit and vegetables classification is complex task with many challenges involved. It is also observed that a larger dataset can better comprehend the complex variant features of fruit and vegetables. Complex multidimensional features can be extracted from the larger datasets to generalise on higher number of classes. However, development of a larger multiclass dataset is an expensive and time consuming process. The effectiveness of classification techniques can be significantly improved by subtracting the background occlusions and complexities. It is also worth mentioning that ensemble of simple and less complicated classification techniques can achieve effective results even if applied to less number of features for smaller number of classes. The combination of visual and nonvisual features can reduce the struggle of a classification technique to deal with higher number of classes with similar physical features. Classification of fruit and vegetables with similar physical features (i.e. colour and texture) needs careful estimation and hyper-dimensional embedding of visual features. Implementing rigorous classification penalties as loss function can achieve this goal at the cost of time and computational requirements. There is a significant need to develop larger datasets for different fruit and vegetables related computer vision applications. Considering more sophisticated loss function penalties and discriminative hyper-dimensional features embedding techniques can significantly improve the effectiveness of the classification techniques for the fruit and vegetables applications

Forum Bildverarbeitung 2022

Bildverarbeitung verknüpft das Fachgebiet die Sensorik von Kameras – bildgebender Sensorik – mit der Verarbeitung der Sensordaten – den Bildern. Daraus resultiert der besondere Reiz dieser Disziplin. Der vorliegende Tagungsband des „Forums Bildverarbeitung“, das am 24. und 25.11.2022 in Karlsruhe als Veranstaltung des Karlsruher Instituts für Technologie und des Fraunhofer-Instituts für Optronik, Systemtechnik und Bildauswertung stattfand, enthält die Aufsätze der eingegangenen Beiträge

Human Body Posture Recognition Approaches

Human body posture recognition has become the focus of many researchers in recent years. Recognition of body posture is used in various applications, including surveillance, security, and health monitoring. However, these systems that determine the body’s posture through video clips, images, or data from sensors have many challenges when used in the real world. This paper provides an important review of how most essential ‎ hardware technologies are ‎used in posture recognition systems‎. These systems capture and collect datasets through ‎accelerometer sensors or computer vision. In addition, this paper presents a comparison ‎study with state-of-the-art in terms of accuracy. We also present the advantages and ‎limitations of each system and suggest promising future ideas that can increase the ‎efficiency of the existing posture recognition system. Finally, the most common datasets ‎applied in these systems are described in detail. It aims to be a resource to help choose one of the methods in recognizing the posture of the human body and the techniques that suit each method. It analyzes more than 80 papers between 2015 and 202