A Tutorial on Learning Human Welder\u27s Behavior: Sensing, Modeling, and Control

Human welder\u27s experiences and skills are critical for producing quality welds in manual GTAW process. Learning human welder\u27s behavior can help develop next generation intelligent welding machines and train welders faster. In this tutorial paper, various aspects of mechanizing the welder\u27s intelligence are surveyed, including sensing of the weld pool, modeling of the welder\u27s adjustments and this model-based control approach. Specifically, different sensing methods of the weld pool are reviewed and a novel 3D vision-based sensing system developed at University of Kentucky is introduced. Characterization of the weld pool is performed and human intelligent model is constructed, including an extensive survey on modeling human dynamics and neuro-fuzzy techniques. Closed-loop control experiment results are presented to illustrate the robustness of the model-based intelligent controller despite welding speed disturbance. A foundation is thus established to explore the mechanism and transformation of human welder\u27s intelligence into robotic welding system. Finally future research directions in this field are presented

Design and Implementation of an Artificial Neural Network Controller for Quadrotor Flight in Confined Environment

Quadrotors offer practical solutions for many applications, such as emergency rescue, surveillance, military operations, videography and many more. For this reason, they have recently attracted the attention of research and industry. Even though they have been intensively studied, quadrotors still suffer from some challenges that limit their use, such as trajectory measurement, attitude estimation, obstacle avoidance, safety precautions, and land cybersecurity. One major problem is flying in a confined environment, such as closed buildings and tunnels, where the aerodynamics around the quadrotor are affected by close proximity objects, which result in tracking performance deterioration, and sometimes instability. To address this problem, researchers followed three different approaches; the Modeling approach, which focuses on the development of a precise dynamical model that accounts for the different aerodynamic effects, the Sensor Integration approach, which focuses on the addition of multiple sensors to the quadrotor and applying algorithms to stabilize the quadrotor based on their measurements, and the Controller Design approach, which focuses on the development of an adaptive and robust controller. In this research, a learning controller is proposed as a solution for the issue of quadrotor trajectory control in confined environments. This controller utilizes Artificial Neural Networks to adjust for the unknown aerodynamics on-line. A systematic approach for controller design is developed, so that, the approach could be followed for the development of controllers for other nonlinear systems of similar form. One goal for this research is to develop a global controller that could be applied to any quadrotor with minimal adjustment. A novel Artificial Neural Network structure is presented that increases learning efficiency and speed. In addition, a new learning algorithm is developed for the Artificial Neural Network, when utilized with the developed controller. Simulation results for the designed controller when applied to the Qball-X4 quadrotor are presented that show the effectiveness of the proposed Artificial Neural Network structure and the developed learning algorithm in the presence of variety of different unknown aerodynamics. These results are confirmed with real time experimentation, as the developed controller was successfully applied to Quanser’s Qball-X4 quadrotor for the flight control in confined environment. The practical challenges associated with the application of such a controller for quadrotor flight in confined environment are analyzed and adequately resolved to achieve an acceptable tracking performance

State of the art of control schemes for smart systems featuring magneto-rheological materials

This review presents various control strategies for application systems utilizing smart magneto-rheological fluid (MRF) and magneto-rheological elastomers (MRE). It is well known that both MRF and MRE are actively studied and applied to many practical systems such as vehicle dampers. The mandatory requirements for successful applications of MRF and MRE include several factors: advanced material properties, optimal mechanisms, suitable modeling, and appropriate control schemes. Among these requirements, the use of an appropriate control scheme is a crucial factor since it is the final action stage of the application systems to achieve the desired output responses. There are numerous different control strategies which have been applied to many different application systems of MRF and MRE, summarized in this review. In the literature review, advantages and disadvantages of each control scheme are discussed so that potential researchers can develop more effective strategies to achieve higher control performance of many application systems utilizing magneto-rheological materials

Techniques Based on Adaptive Neuro-Fuzzy Inference Systems (ANFIS) for Estimating and Evaluating Physical Demands at Work Using Heart Rate

RÉSUMÉ : Malgré l'évolution rapide de la mécanisation dans les industries lourdes, les emplois physiquement exigeants qui nécessitent un effort humain excessif représentent encore une part importante dans de nombreuses industries (foresterie, construction, mines, etc.). Des études ont montré que les charges de travail excessives imposées aux travailleurs sont la principale cause de fatigue physique, ce qui a des effets négatifs sur les travailleurs, leur performance et la qualité du travail. Par conséquent, les chercheurs ont souligné l'importance de la conception optimale des tâches (à l'intérieur des compétences des travailleurs) afin de maintenir la sécurité, la santé et la productivité des travailleurs. Toutefois, cela ne peut être atteint sans comprendre (c'est-à-dire mesurer et évaluer) les exigences physiologiques du travail. À cet égard, les trois études comprises dans cette thèse présentent des approches pratiques pour estimer et évaluer la dépense énergétique (DE), exprimée en termes de consommation d'oxygène (VO2), au cours du travail réel. La première étude présente de nouvelles approches basées sur le système d'inférence neuro-flou adaptatif (ANFIS) pour l'estimation de la VO2 à partir des mesures de la fréquence cardiaque (FC). Cette étude comprend deux étapes auxquelles ont participé 35 individus en bonne santé. Dans un premier temps, deux modèles novateurs individuels ont été développés en se basant sur l’ANFIS et les méthodes analytiques. Ces modèles s'attaquent au problème de l'incertitude et de la non-linéarité entre la FC et la VO2. Dans un deuxième temps, un modèle général ANFIS qui ne requiert pas d'étalonnage individuel a été développé. Les trois modèles ont été testés en laboratoire et sur le terrain. La performance de chaque modèle a été évaluée et comparée aux VO2 mesurées et à deux méthodes d'estimation individuelles et traditionnelles de VO2 (étalonnage linéaire et Flex-HR). Les résultats ont indiqué la précision supérieure obtenue avec la modélisation ANFIS individualisée (EMQ = 1,0 à 2,8 ml/kg.min en laboratoire et sur le terrain, respectivement). Le modèle analytique a surpassé l'étalonnage linéaire traditionnel et les méthodes Flex-HR avec des données terrain. Les estimations du modèle général ANFIS de la VO2 ne différaient pas significativement des mesures réelles terrain VO2 (EMQ = 3,5 ml/kg.min). Avec sa facilité d'utilisation et son faible coût de mise en œuvre, le modèle général ANFIS montre du potentiel pour remplacer n'importe laquelle des méthodes traditionnelles individualisées pour l’estimation de la VO2 à partir de données recueillies sur le terrain. La deuxième étude présente un modèle de prédiction de la VO2 basé sur ANFIS qui est inspiré de la méthode Flex-HR. Des études ont montré que la méthode Flex-HR est une des méthodes les plus précises pour l'estimation de la VO2. Toutefois, cette méthode est basée sur quatre paramètres qui sont déterminés individuellement et par conséquent ceci est considéré comme coûteux, chronophage et souvent peu pratique, surtout lorsque le nombre de travailleurs augmente. Le modèle prédictif proposé se compose de trois modules ANFIS pour estimer les paramètres de Flex-HR. Pour chaque module ANFIS, la sélection de variables d'entrée et le modèle d'évaluation ont été simultanément réalisés à l'aide de la combinaison de la technique de division des données en trois parties et la technique de validation croisée. La performance de chaque module ANFIS a été testée et comparée avec les paramètres observés ainsi qu'avec les modèles de Rennie et coll. (2001) à l'aide de données de test indépendant. En outre, les performances du modèle global de prédiction ANFIS dans l'estimation de la VO2 a été testé et comparé avec les valeurs mesurées de la VO2, la méthode de Flex-HR standard ainsi qu'avec les autres modèles généraux (c.-à-d., les modèles de Rennie et coll. (2001) et de Keytel et coll. (2005)). Les résultats n'ont indiqué aucune différence significative entre les paramètres observés et estimés de Flex-HR et entre la VO2 mesurée et estimée dans la plage de fréquence cardiaque globale et séparément dans différentes gammes de FC. Le modèle de prédiction ANFIS (EMA = 3 ml/kg.min) a montré de meilleures performances que les modèles de Rennie et coll. (EMA = 7 ml/kg.min) et les modèles de Keytel et coll. (EMA = 6 ml/kg.min) et des performances comparables avec la méthode standard de Flex-HR (EMA = 2,3 ml/kg.min) tout au long de la plage de fréquence cardiaque. Le modèle ANFIS fournit ainsi aux praticiens une méthode pratique, économique et rapide pour l'estimation de la VO2 sans besoin d'étalonnage individuel. La troisième étude présente une nouvelle approche basée sur l'ANFIS pour classer les travaux en quatre classes d'intensité (c'est-à-dire, très léger, léger, modéré et lourd) à l'aide du monitorage du rythme cardiaque. La variabilité intra-individuelle (différences physiologiques et physiques) a été examinée. Vingt-huit participants ont effectué le test de la montée des marches Meyer et Flenghi (1995) et le test maximal sur le tapis roulant pendant lesquels la fréquence cardiaque et la consommation d'oxygène ont été mesurées. Les résultats ont indiqué que le monitorage du rythme cardiaque (FC, FC max et FC repos) et du poids corporel sont des variables significatives pour classer le rythme de travail. Le classificateur ANFIS a montré une sensibilité, une spécificité et une exactitude supérieures par rapport à la pratique courante à l'aide de catégories de rythme de travail basées sur le pourcentage de fréquence cardiaque de réserve (% FCR), avec une différence globale de 29,6 % dans la précision de classification entre les deux méthodes et un bon équilibre entre la sensibilité (90,7 %, en moyenne) et la spécificité (95,2 %, en moyenne). Avec sa facilité de mise en œuvre et sa mesure variable, le classificateur ANFIS montre un potentiel pour une utilisation généralisée par les praticiens pour évaluation du rythme de travail.----------ABSTRACT : Despite the rapid evolution of mechanization in heavy industries, physically demanding jobs that require excessive human effort still represent a significant part of many industries (e.g., forestry, construction, mining etc.). Studies have shown that excessive workloads placed on workers are the main cause of physical fatigue, which has negative effects on the workers, their performance and quality of work. Therefore, researchers have emphasized on the importance of the optimal job design (within workers’ capacity) in order to maintain workers’ safety, health and productivity. However, this cannot be achieved without understanding (i.e., measuring and evaluating) the physiological demands of work. In this respect, the three studies comprising this dissertation present practical approaches for estimating and evaluating energy expenditure (EE), expressed in terms of oxygen consumption (VO2), during actual work. The first study presents new approaches based on adaptive neuro-fuzzy inference system (ANFIS) for the estimation of VO2 from heart rate (HR) measurements. This study comprises two stages in which 35 healthy individuals participated. In the first stage, two novel individual models were developed based on the ANFIS and the analytical methods. These models tackle the problem of uncertainty and nonlinearity between HR and VO2. In the second stage, a General ANFIS model was developed which does not require individual calibration. The three models were tested under laboratory and field conditions. Performance of each model was evaluated and compared to the measured VO2 and two traditional individual VO2 estimation methods (linear calibration and Flex-HR). Results indicated the superior precision achieved with individualized ANFIS modeling (RMSE= 1.0 and 2.8 ml/kg.min in laboratory and field, respectively). The analytical model outperformed the traditional linear calibration and Flex-HR methods with field data. The General ANFIS model’s estimates of VO2 were not significantly different from actual field VO2 measurements (RMSE= 3.5 ml/kg.min). With its ease of use and low implementation cost, the General ANFIS model shows potential to replace any of the traditional individualized methods for VO2 estimation from HR data collected in the field. The second study presents an ANFIS-based VO2 prediction model that is inspired by the Flex-HR method. Studies have shown that the Flex-HR method is one of the most accurate methods for VO2 estimation. However, this method is based on four parameters that are determined individually and therefore it is considered costly, time consuming and often impractical, especially when the number of workers increases. The proposed prediction model consists of three ANFIS modules for estimating the Flex-HR parameters. For each ANFIS module, input variables selection and model assessment were simultaneously performed using the combination of three-way data split and cross-validation techniques. The performance of each ANFIS module was tested and compared with the observed parameters as well as with Rennie et al.’s (2001) models using independent test data. In addition, the performance of the overall ANFIS prediction model in estimating VO2 was tested and compared with the measured VO2 values, the standard Flex-HR method as well as with other general models (i.e., Rennie et al.’s (2001) and Keytel et al.’s (2005) models). Results indicated no significant difference between observed and estimated Flex-HR parameters and between measured and estimated VO2 in the overall HR range, and separately in different HR ranges. The ANFIS prediction model (MAE = 3 ml/kg.min) demonstrated better performance than Rennie et al.’s (MAE = 7 ml/kg.min) and Keytel et al.’s (MAE = 6 ml/kg.min) models, and comparable performance with the standard Flex-HR method (MAE = 2.3 ml/kg.min) throughout the HR range. The ANFIS model thus provides practitioners with a practical, cost- and time-efficient method for VO2 estimation without the need for individual calibration. The third study presents a new approach based ANFIS for classifying work intensity into four classes (i.e., very light, light, moderate and heavy) by using heart rate monitoring. Intersubject variability (physiological and physical differences) was considered. Twenty-eight participants performed Meyer and Flenghi (1995) step-test and a maximal treadmill test, during which heart rate and oxygen consumption were measured. Results indicated that heart rate monitoring (HR, HRmax, and HRrest) and body weight are significant variables for classifying work rate. The ANFIS classifier showed superior sensitivity, specificity, and accuracy compared to current practice using established work rate categories based on percent heart rate reserve (%HRR), with an overall 29.6% difference in classification accuracy between the two methods, and good balance between sensitivity (90.7%, on average) and specificity (95.2%, on average). With its ease of implementation and variable measurement, the ANFIS classifier shows potential for widespread use by practitioners for work rate assessment

Optimized state feedback regulation of 3DOF helicopter system via extremum seeking

In this paper, an optimized state feedback regulation of a 3 degree of freedom (DOF) helicopter is designed via extremum seeking (ES) technique. Multi-parameter ES is applied to optimize the tracking performance via tuning State Vector Feedback with Integration of the Control Error (SVFBICE). Discrete multivariable version of ES is developed to minimize a cost function that measures the performance of the controller. The cost function is a function of the error between the actual and desired axis positions. The controller parameters are updated online as the optimization takes place. This method significantly decreases the time in obtaining optimal controller parameters. Simulations were conducted for the online optimization under both fixed and varying operating conditions. The results demonstrate the usefulness of using ES for preserving the maximum attainable performance

Advances in Reinforcement Learning

Reinforcement Learning (RL) is a very dynamic area in terms of theory and application. This book brings together many different aspects of the current research on several fields associated to RL which has been growing rapidly, producing a wide variety of learning algorithms for different applications. Based on 24 Chapters, it covers a very broad variety of topics in RL and their application in autonomous systems. A set of chapters in this book provide a general overview of RL while other chapters focus mostly on the applications of RL paradigms: Game Theory, Multi-Agent Theory, Robotic, Networking Technologies, Vehicular Navigation, Medicine and Industrial Logistic

Failure Prognosis of Wind Turbine Components

Wind energy is playing an increasingly significant role in the World\u27s energy supply mix. In North America, many utility-scale wind turbines are approaching, or are beyond the half-way point of their originally anticipated lifespan. Accurate estimation of the times to failure of major turbine components can provide wind farm owners insight into how to optimize the life and value of their farm assets. This dissertation deals with fault detection and failure prognosis of critical wind turbine sub-assemblies, including generators, blades, and bearings based on data-driven approaches. The main aim of the data-driven methods is to utilize measurement data from the system and forecast the Remaining Useful Life (RUL) of faulty components accurately and efficiently. The main contributions of this dissertation are in the application of ALTA lifetime analysis to help illustrate a possible relationship between varying loads and generators reliability, a wavelet-based Probability Density Function (PDF) to effectively detecting incipient wind turbine blade failure, an adaptive Bayesian algorithm for modeling the uncertainty inherent in the bearings RUL prediction horizon, and a Hidden Markov Model (HMM) for characterizing the bearing damage progression based on varying operating states to mimic a real condition in which wind turbines operate and to recognize that the damage progression is a function of the stress applied to each component using data from historical failures across three different Canadian wind farms