CONCEPTS FOR DEVELOPMENT OF SHUTTLE CAR AUTONOMOUS DOCKING WITH CONTINUOUS MINER USING 3-D DEPTH CAMERA

In recent years, a great deal of work has been conducted in automating mining equipment with the goals of increasing worker health and safety and increasing mine productivity. Automating vehicles such as load-haul-dumps been successful even in underground environments where the use of global positioning systems are unavailable. This thesis addresses automating the operation of a shuttle car, specifically focusing on positioning the shuttle car under the continuous miner coal-discharge conveyor during cutting and loading operations. This task requires recognition of the target and precise control of the tramming operation because a specific orientation and distance from the coal discharge conveyor is needed to avoid coal spillage. The proposed approach uses a stereo depth camera mounted on a small-scale mockup of a shuttle car. Machine learning algorithms are applied to the camera output to identify the continuous miner coal-discharge conveyor and segment the scene into various regions such as roof, ribs, and personnel. This information is used to plan the shuttle car path to the continuous miner coal-discharge conveyor. These methods are currently applied on 1/6th scale continuous miner and shuttle car in an appropriately scaled mock mine

Actuators for Intelligent Electric Vehicles

This book details the advanced actuators for IEVs and the control algorithm design. In the actuator design, the configuration four-wheel independent drive/steering electric vehicles is reviewed. An in-wheel two-speed AMT with selectable one-way clutch is designed for IEV. Considering uncertainties, the optimization design for the planetary gear train of IEV is conducted. An electric power steering system is designed for IEV. In addition, advanced control algorithms are proposed in favour of active safety improvement. A supervision mechanism is applied to the segment drift control of autonomous driving. Double super-resolution network is used to design the intelligent driving algorithm. Torque distribution control technology and four-wheel steering technology are utilized for path tracking and adaptive cruise control. To advance the control accuracy, advanced estimation algorithms are studied in this book. The tyre-road peak friction coefficient under full slip rate range is identified based on the normalized tyre model. The pressure of the electro-hydraulic brake system is estimated based on signal fusion. Besides, a multi-semantic driver behaviour recognition model of autonomous vehicles is designed using confidence fusion mechanism. Moreover, a mono-vision based lateral localization system of low-cost autonomous vehicles is proposed with deep learning curb detection. To sum up, the discussed advanced actuators, control and estimation algorithms are beneficial to the active safety improvement of IEVs

Automatic Identification of Algae using Low-cost Multispectral Fluorescence Digital Microscopy, Hierarchical Classification & Deep Learning

Harmful algae blooms (HABs) can produce lethal toxins and are a rising global concern. In response to this threat, many organizations are monitoring algae populations to determine if a water body might be contaminated. However, identifying algae types in a water sample requires a human expert, a taxonomist, to manually identify organisms using an optical microscope. This is a tedious, time-consuming process that is prone to human error and bias. Since many facilities lack on-site taxonomists, they must ship their water samples off site, further adding to the analysis time. Given the urgency of this problem, this thesis hypothesizes that multispectral fluorescence microscopy with a deep learning hierarchical classification structure is the optimal method to automatically identify algae in water on-site. To test this hypothesis, a low-cost system was designed and built which was able generate one brightfield image and four fluorescence images. Each of the four fluorescence images was designed to target a different pigment in algae, resulting in a unique autofluorescence spectral fingerprint for different phyla groups. To complement this hardware system, a software framework was designed and developed. This framework used the prior taxonomic structure of algae to create a hierarchical classification structure. This hierarchical classifier divided the classification task into three steps which were phylum, genus, and species level classification. Deep learning models were used at each branch of this hierarchical classifier allowing the optimal set of features to be implicitly learned from the input data. In order to test the efficacy of the proposed hardware system and corresponding software framework, a dataset of nine algae from 4 different phyla groups was created. A number of preprocessing steps were required to prepare the data for analysis. These steps were flat field correction, thresholding and cropping. With this multispectral imaging data, a number of spatial and spectral features were extracted for use in the feature-extraction-based models. This dataset was used to determine the relative performance of 12 different model architectures, and the proposed multispectral hierarchical deep learning approach achieved the top classification accuracy of 97% to the species level. Further inspection revealed that a traditional feature extraction method was able to achieve 95% to the phyla level when only using the multispectral fluorescence data. These observations strongly support that: (1) the proposed low-cost multispectral fluorescence imaging system, and (2) the proposed hierarchical structure based on the taxonomy prior, in combination with (3) deep learning methods for feature learning, is an effective method to automatically classify algae

Multimodal machine learning for intelligent mobility

Scientific problems are solved by finding the optimal solution for a specific task. Some problems can be solved analytically while other problems are solved using data driven methods. The use of digital technologies to improve the transportation of people and goods, which is referred to as intelligent mobility, is one of the principal beneficiaries of data driven solutions. Autonomous vehicles are at the heart of the developments that propel Intelligent Mobility. Due to the high dimensionality and complexities involved in real-world environments, it needs to become commonplace for intelligent mobility to use data-driven solutions. As it is near impossible to program decision making logic for every eventuality manually. While recent developments of data-driven solutions such as deep learning facilitate machines to learn effectively from large datasets, the application of techniques within safety-critical systems such as driverless cars remain scarce.Autonomous vehicles need to be able to make context-driven decisions autonomously in different environments in which they operate. The recent literature on driverless vehicle research is heavily focused only on road or highway environments but have discounted pedestrianized areas and indoor environments. These unstructured environments tend to have more clutter and change rapidly over time. Therefore, for intelligent mobility to make a significant impact on human life, it is vital to extend the application beyond the structured environments. To further advance intelligent mobility, researchers need to take cues from multiple sensor streams, and multiple machine learning algorithms so that decisions can be robust and reliable. Only then will machines indeed be able to operate in unstructured and dynamic environments safely. Towards addressing these limitations, this thesis investigates data driven solutions towards crucial building blocks in intelligent mobility. Specifically, the thesis investigates multimodal sensor data fusion, machine learning, multimodal deep representation learning and its application of intelligent mobility. This work demonstrates that mobile robots can use multimodal machine learning to derive driver policy and therefore make autonomous decisions.To facilitate autonomous decisions necessary to derive safe driving algorithms, we present an algorithm for free space detection and human activity recognition. Driving these decision-making algorithms are specific datasets collected throughout this study. They include the Loughborough London Autonomous Vehicle dataset, and the Loughborough London Human Activity Recognition dataset. The datasets were collected using an autonomous platform design and developed in house as part of this research activity. The proposed framework for Free-Space Detection is based on an active learning paradigm that leverages the relative uncertainty of multimodal sensor data streams (ultrasound and camera). It utilizes an online learning methodology to continuously update the learnt model whenever the vehicle experiences new environments. The proposed Free Space Detection algorithm enables an autonomous vehicle to self-learn, evolve and adapt to new environments never encountered before. The results illustrate that online learning mechanism is superior to one-off training of deep neural networks that require large datasets to generalize to unfamiliar surroundings. The thesis takes the view that human should be at the centre of any technological development related to artificial intelligence. It is imperative within the spectrum of intelligent mobility where an autonomous vehicle should be aware of what humans are doing in its vicinity. Towards improving the robustness of human activity recognition, this thesis proposes a novel algorithm that classifies point-cloud data originated from Light Detection and Ranging sensors. The proposed algorithm leverages multimodality by using the camera data to identify humans and segment the region of interest in point cloud data. The corresponding 3-dimensional data was converted to a Fisher Vector Representation before being classified by a deep Convolutional Neural Network. The proposed algorithm classifies the indoor activities performed by a human subject with an average precision of 90.3%. When compared to an alternative point cloud classifier, PointNet[1], [2], the proposed framework out preformed on all classes. The developed autonomous testbed for data collection and algorithm validation, as well as the multimodal data-driven solutions for driverless cars, is the major contributions of this thesis. It is anticipated that these results and the testbed will have significant implications on the future of intelligent mobility by amplifying the developments of intelligent driverless vehicles.</div

Design and Implementation of the Kinect Controlled Electro-Mechanical Skeleton (K.C.E.M.S)

Mimicking real-time human motion with a low cost solution has been an extremely difficult task in the past but with the release of the Microsoft Kinect motion capture system, this problem has been simplified. This thesis discusses the feasibility and design behind a simple robotic skeleton that utilizes the Kinect to mimic human movements in near real-time. The goal of this project is to construct a 1/3-scale model of a robotically enhanced skeleton and demonstrate the abilities of the Kinect as a tool for human movement mimicry. The resulting robot was able to mimic many human movements but was mechanically limited in the shoulders. Its movements were slower then real-time due to the inability for the controller to handle real-time motions. This research was presented and published at the 2012 SouthEastCon. Along with this, research papers about the formula hybrid accumulator design and the 2010 autonomous surface vehicle were presented and published

Advances in Intelligent Vehicle Control

This book is a printed edition of the Special Issue Advances in Intelligent Vehicle Control that was published in the journal Sensors. It presents a collection of eleven papers that covers a range of topics, such as the development of intelligent control algorithms for active safety systems, smart sensors, and intelligent and efficient driving. The contributions presented in these papers can serve as useful tools for researchers who are interested in new vehicle technology and in the improvement of vehicle control systems

Trustworthy Wireless Personal Area Networks

In the Internet of Things (IoT), everyday objects are equipped with the ability to compute and communicate. These smart things have invaded the lives of everyday people, being constantly carried or worn on our bodies, and entering into our homes, our healthcare, and beyond. This has given rise to wireless networks of smart, connected, always-on, personal things that are constantly around us, and have unfettered access to our most personal data as well as all of the other devices that we own and encounter throughout our day. It should, therefore, come as no surprise that our personal devices and data are frequent targets of ever-present threats. Securing these devices and networks, however, is challenging. In this dissertation, we outline three critical problems in the context of Wireless Personal Area Networks (WPANs) and present our solutions to these problems. First, I present our Trusted I/O solution (BASTION-SGX) for protecting sensitive user data transferred between wirelessly connected (Bluetooth) devices. This work shows how in-transit data can be protected from privileged threats, such as a compromised OS, on commodity systems. I present insights into the Bluetooth architecture, Intel’s Software Guard Extensions (SGX), and how a Trusted I/O solution can be engineered on commodity devices equipped with SGX. Second, I present our work on AMULET and how we successfully built a wearable health hub that can run multiple health applications, provide strong security properties, and operate on a single charge for weeks or even months at a time. I present the design and evaluation of our highly efficient event-driven programming model, the design of our low-power operating system, and developer tools for profiling ultra-low-power applications at compile time. Third, I present a new approach (VIA) that helps devices at the center of WPANs (e.g., smartphones) to verify the authenticity of interactions with other devices. This work builds on past work in anomaly detection techniques and shows how these techniques can be applied to Bluetooth network traffic. Specifically, we show how to create normality models based on fine- and course-grained insights from network traffic, which can be used to verify the authenticity of future interactions

Computational intelligent impact force modeling and monitoring in HISLO conditions for maximizing surface mining efficiency, safety, and health

Shovel-truck systems are the most widely employed excavation and material handling systems for surface mining operations. During this process, a high-impact shovel loading operation (HISLO) produces large forces that cause extreme whole body vibrations (WBV) that can severely affect the safety and health of haul truck operators. Previously developed solutions have failed to produce satisfactory results as the vibrations at the truck operator seat still exceed the “Extremely Uncomfortable Limits”. This study was a novel effort in developing deep learning-based solution to the HISLO problem. This research study developed a rigorous mathematical model and a 3D virtual simulation model to capture the dynamic impact force for a multi-pass shovel loading operation. The research further involved the application of artificial intelligence and machine learning for implementing the impact force detection in real time. Experimental results showed the impact force magnitudes of 571 kN and 422 kN, for the first and second shovel pass, respectively, through an accurate representation of HISLO with continuous flow modelling using FEA-DEM coupled methodology. The novel ‘DeepImpact’ model, showed an exceptional performance, giving an R2, RMSE, and MAE values of 0.9948, 10.750, and 6.33, respectively, during the model validation. This research was a pioneering effort for advancing knowledge and frontiers in addressing the WBV challenges in deploying heavy mining machinery in safe and healthy large surface mining environments. The smart and intelligent real-time monitoring system from this study, along with process optimization, minimizes the impact force on truck surface, which in turn reduces the level of vibration on the operator, thus leading to a safer and healthier working mining environments --Abstract, page iii

Online synchronous inspection and system optimization of flexible food packaging bags by using machine vision and sensing technique

Flexible food packaging in the market is increasingly favored, and its quality is essential and indispensable for safety and convenience.  However, quality inspection still stays in the manual stage, or partially manual inspection remains, in production, leading low efficiency, lack and even false inspection, hardly meeting the requirements of the modern output.  This paper proposes and optimizes the design of an automatic detection system with intelligence for flexible food packaging bag, which can effectively be adopted to check the quality of packaging trademark patterns, fillers, and sealing quality.  The inspection system runs with two-stage structure, machine vision, pressure sensing and synchronization to improve efficiency and ensure the normal production beat. Simplex Method is adopted to determine the best synchronous speeds online to achieve the best expectation. Comparison has been made between the manual inspection and our automatic operation, the sample of 10000 was statistically analyzed and results have shown that two workers were saved and the correctness rate of inspection raised up to 999.8‰

Machine Learning Methods for Product Quality Monitoring in Electric Resistance Welding

Elektrisches Widerstandsschweißen (Englisch: Electric Resistance Welding, ERW) ist eine Gruppe von vollautomatisierten Fertigungsprozessen, bei denen metallische Werkstoffe durch Wärme verbunden werden, die von elektrischem Strom und Widerstand erzeugt wird. Eine genaue Qualitätsüberwachung von ERW kann oft nur teilweise mit destruktiven Methoden durchgeführt werden. Es besteht ein großes industrielles und wirtschaftliches Potenzial, datengetriebene Ansätze für die Qualitätsüberwachung in ERW zu entwickeln, um die Wartungskosten zu senken und die Qualitätskontrolle zu verbessern. Datengetriebene Ansätze wie maschinelles Lernen (ML) haben aufgrund der enormen Menge verfügbarer Daten, die von Technologien der Industrie 4.0 bereitgestellt werden, viel Aufmerksamkeit auf sich gezogen. Datengetriebene Ansätze ermöglichen eine zerstörungsfreie, umfassende und präzise Qualitätsüberwachung, wenn eine bestimmte Menge präziser Daten verfügbar ist. Dies kann eine umfassende Online-Qualitätsüberwachung ermöglichen, die ansonsten mit herkömmlichen empirischen Methoden äußerst schwierig ist. Es gibt jedoch noch viele Herausforderungen bei der Adoption solcher Ansätze in der Fertigungsindustrie. Zu diesen Herausforderungen gehören: effiziente Datensammlung, die dasWissen von erforderlichen Datenmengen und relevanten Sensoren für erfolgreiches maschinelles Lernen verlangt; das anspruchsvolle Verstehen von komplexen Prozessen und facettenreichen Daten; eine geschickte Selektion geeigneter ML-Methoden und die Integration von Domänenwissen für die prädiktive Qualitätsüberwachung mit inhomogenen Datenstrukturen, usw. Bestehende ML-Lösungen für ERW liefern keine systematische Vorgehensweise für die Methodenauswahl. Jeder Prozess der ML-Entwicklung erfordert ein umfassendes Prozess- und Datenverständnis und ist auf ein bestimmtes Szenario zugeschnitten, das schwer zu verallgemeinern ist. Es existieren semantische Lösungen für das Prozess- und Datenverständnis und Datenmanagement. Diese betrachten die Datenanalyse als eine isolierte Phase. Sie liefern keine Systemlösungen für das Prozess- und Datenverständnis, die Datenaufbereitung und die ML-Verbesserung, die konfigurierbare und verallgemeinerbare Lösungen für maschinelles Lernen ermöglichen. Diese Arbeit versucht, die obengenannten Herausforderungen zu adressieren, indem ein Framework für maschinelles Lernen für ERW vorgeschlagen wird, und demonstriert fünf industrielle Anwendungsfälle, die das Framework anwenden und validieren. Das Framework überprüft die Fragen und Datenspezifitäten, schlägt eine simulationsunterstützte Datenerfassung vor und erörtert Methoden des maschinellen Lernens, die in zwei Gruppen unterteilt sind: Feature Engineering und Feature Learning. Das Framework basiert auf semantischen Technologien, die eine standardisierte Prozess- und Datenbeschreibung, eine Ontologie-bewusste Datenaufbereitung sowie halbautomatisierte und Nutzer-konfigurierbare ML-Lösungen ermöglichen. Diese Arbeit demonstriert außerdem die Übertragbarkeit des Frameworks auf einen hochpräzisen Laserprozess. Diese Arbeit ist ein Beginn des Wegs zur intelligenten Fertigung von ERW, der mit dem Trend der vierten industriellen Revolution korrespondiert