Navigation for automatic guided vehicles using omnidirectional optical sensing

Thesis (M. Tech. (Engineering: Electrical)) -- Central University of technology, Free State, 2013Automatic Guided Vehicles (AGVs) are being used more frequently in a manufacturing environment. These AGVs are navigated in many different ways, utilising multiple types of sensors for detecting the environment like distance, obstacles, and a set route. Different algorithms or methods are then used to utilise this environmental information for navigation purposes applied onto the AGV for control purposes. Developing a platform that could be easily reconfigured in alternative route applications utilising vision was one of the aims of the research. In this research such sensors detecting the environment was replaced and/or minimised by the use of a single, omnidirectional Webcam picture stream utilising an own developed mirror and Perspex tube setup. The area of interest in each frame was extracted saving on computational recourses and time. By utilising image processing, the vehicle was navigated on a predetermined route. Different edge detection methods and segmentation methods were investigated on this vision signal for route and sign navigation. Prewitt edge detection was eventually implemented, Hough transfers used for border detection and Kalman filtering for minimising border detected noise for staying on the navigated route. Reconfigurability was added to the route layout by coloured signs incorporated in the navigation process. The result was the manipulation of a number of AGV’s, each on its own designated coloured signed route. This route could be reconfigured by the operator with no programming alteration or intervention. The YCbCr colour space signal was implemented in detecting specific control signs for alternative colour route navigation. The result was used generating commands to control the AGV through serial commands sent on a laptop’s Universal Serial Bus (USB) port with a PIC microcontroller interface board controlling the motors by means of pulse width modulation (PWM). A total MATLAB® software development platform was utilised by implementing written M-files, Simulink® models, masked function blocks and .mat files for sourcing the workspace variables and generating executable files. This continuous development system lends itself to speedy evaluation and implementation of image processing options on the AGV. All the work done in the thesis was validated by simulations using actual data and by physical experimentation

Study on Object Detection using Computer Vision by Artificial Neural Network（論文要旨/審査結果要旨）

芝浦工業大学2018年

Scheduling Optimization And Coordination With Target Tracking Under Heterogeneous Networks In Automated Guided Vehicles (AGVs)

Throughout the development of the multi-AGV systems, prevailing research directions contain improving the performance of individual AGV, optimizing the coordination of multiple AGVs, and enhancing the efficiency of communication among AGVs. Current researchers tend to pay attention to one research direction at a time. There is a lack of research on the overall AGV system design that tackles multiple critical design aspects of the system. This PhD research addresses four key factors of the AGV system which are AGV prototypes, target tracking algorithms, AGVs scheduling optimization and the communication of a multi-AGV system. Extensive field experiments and algorithm optimization are implemented. Comprehensive literature review is conducted to identify the research gap. The proposed solutions cover the following three aspects of the AGV system design including communication between AGVs, AGVs scheduling and computer vision in AGVs.        For AGV communication, a network selection optimization algorithm is presented. An improved method for preventing convolutional neural network (CNN) immune from backdoor attack to ensure a multi-AGV system's communication security is presented. Meanwhile, a transmission framework for a multi-AGV system is presented. Those methods are used to establish a safe and efficient multi-AGV system's communication environment. For AGV scheduling, a multi-robot planning algorithm with quadtree map division for obstacles of irregular shape is presented. In addition, a scheduling optimization platform is presented. These methods are used to make a multi-AGV system have a shorter time delay and decrease the possibility of collision in a multi-robot system.Meanwhile, a scheduling optimization method based on the combination of a handover rule and the A* algorithm is proposed. The system properties that may affect the scheduling performance are also discussed. Finally, the overall performance of the newly integrated scheduling system is compared with other scheduling systems to validate its superiority and shortcomings in different corresponding work scenarios. Computer vision in AGV is investigated in detail. To improve an individual AGV's performance, an improved Camshift Algorithm has been proposed and applied to AGV prototypes. Furthermore, three deep learning models are tested under specific environments. In addition, based on the designed algorithm, the AGV prototype is able to make a convergent prediction of the pixels in the target area after the first detection of the object. Relative coordinates of the target can be located more accurately in less time. As tested in the experiments, the system architecture and new algorithm lead to reduced hardware cost, shorter time delay, improved robustness, and higher accuracy in tracking.        With the three design aspects in mind, a novel method for real-time visual tracking and distance measurement is proposed. Tracking and collision avoidance functions are tested in the designed multi-AGV prototype system. Detailed design procedure, numerical analysis of the measurement data and recommendations for further improvement of the system design are presented

Robots learn to behave: improving human-robot collaboration in flexible manufacturing applications

L'abstract è presente nell'allegato / the abstract is in the attachmen

Voting Classifier for The Interactive Design with Deep Learning for Scene Theory

Tool products play a pivotal role in assisting individuals in various domains, ranging from professional work to everyday tasks. The success of these tools is not solely determined by their functionality but also by the quality of user experience they offer. Designing tool products that effectively engage users, enhance their productivity, and provide a seamless interaction experience has become a critical focus for researchers and practitioners in the field of interaction design. Scene theory proposes that individuals perceive and interpret their surroundings as dynamic "scenes," wherein environmental and situational factors influence their cognitive processes and behavior. This research paper presented a novel approach to the interaction design of tool products by integrating scene theory, flow experience, the Moth Flame optimization (MFO), cooperative game theory (CGT), and voting deep learning. Tool products play a vital role in various domains, and their interaction design significantly influences user satisfaction and task performance. Building upon the principles of scene theory and flow experience, this study proposes an innovative framework that considers the contextual factors and aims to create a seamless and enjoyable user experience. The MFO algorithm, inspired by the behavior of moth flame, is employed to optimize the design parameters and enhance the efficiency of the interaction design process. Furthermore, CGT is integrated to model cooperative relationships between users and tool products, fostering collaborative and engaging experiences. Voting deep learning is employed to analyze user feedback and preferences, enabling personalized and adaptive design recommendations. With the proposed CGT, this paper investigates the impact of the proposed approach on user engagement, task efficiency, and overall satisfaction. The findings contribute to the field of interaction design by providing practical insights for creating tool products that align with users' cognitive processes, environmental constraints, flow-inducing experiences, and cooperative dynamics

Human-Robot Collaborations in Industrial Automation

Technology is changing the manufacturing world. For example, sensors are being used to track inventories from the manufacturing floor up to a retail shelf or a customer’s door. These types of interconnected systems have been called the fourth industrial revolution, also known as Industry 4.0, and are projected to lower manufacturing costs. As industry moves toward these integrated technologies and lower costs, engineers will need to connect these systems via the Internet of Things (IoT). These engineers will also need to design how these connected systems interact with humans. The focus of this Special Issue is the smart sensors used in these human–robot collaborations

SLAM research for port AGV based on 2D LIDAR

With the increase in international trade, the transshipment of goods at international container ports is very busy. The AGV (Automated Guided Vehicle) has been used as a new generation of automated container horizontal transport equipment. The AGV is an automated unmanned vehicle that can work 24 hours a day, increasing productivity and reducing labor costs compared to using container trucks. The ability to obtain information about the surrounding environment is a prerequisite for the AGV to automatically complete tasks in the port area. At present, the method of AGV based on RFID tag positioning and navigation has a problem of excessive cost. This dissertation has carried out a research on applying light detection and ranging (LIDAR) simultaneous localization and mapping (SLAM) technology to port AGV. In this master's thesis, a mobile test platform based on a laser range finder is developed to scan 360-degree environmental information (distance and angle) centered on the LIDAR and upload the information to a real-time database to generate surrounding environmental maps, and the obstacle avoidance strategy was developed based on the acquired information. The effectiveness of the platform was verified by the experiments from multiple scenarios. Then based on the first platform, another experimental platform with encoder and IMU sensor was developed. In this platform, the functionality of SLAM is enabled by the GMapping algorithm and the installation of the encoder and IMU sensor. Based on the established environment SLAM map, the path planning and obstacle avoidance functions of the platform were realized.Com o aumento do comércio internacional, o transbordo de mercadorias em portos internacionais de contentores é muito movimentado. O AGV (“Automated Guided Vehicle”) foi usado como uma nova geração de equipamentos para transporte horizontal de contentores de forma automatizada. O AGV é um veículo não tripulado automatizado que pode funcionar 24 horas por dia, aumentando a produtividade e reduzindo os custos de mão-de-obra em comparação com o uso de camiões porta-contentores. A capacidade de obter informações sobre o ambiente circundante é um pré-requisito para o AGV concluir automaticamente tarefas na área portuária. Atualmente, o método de AGV baseado no posicionamento e navegação de etiquetas RFID apresenta um problema de custo excessivo. Nesta dissertação foi realizada uma pesquisa sobre a aplicação da tecnologia LIDAR de localização e mapeamento simultâneo (SLAM) num AGV. Uma plataforma de teste móvel baseada num telémetro a laser é desenvolvida para examinar o ambiente em redor em 360 graus (distância e ângulo), centrado no LIDAR, e fazer upload da informação para uma base de dados em tempo real para gerar um mapa do ambiente em redor. Uma estratégia de prevenção de obstáculos foi também desenvolvida com base nas informações adquiridas. A eficácia da plataforma foi verificada através da realização de testes com vários cenários e obstáculos. Por fim, com base na primeira plataforma, uma outra plataforma experimental com codificador e sensor IMU foi também desenvolvida. Nesta plataforma, a funcionalidade do SLAM é ativada pelo algoritmo GMapping e pela instalação do codificador e do sensor IMU. Com base no estabelecimento do ambiente circundante SLAM, foram realizadas as funções de planeamento de trajetória e prevenção de obstáculos pela plataforma

Intrusion Detection for Cyber-Physical Attacks in Cyber-Manufacturing System

In the vision of Cyber-Manufacturing System (CMS) , the physical components such as products, machines, and tools are connected, identifiable and can communicate via the industrial network and the Internet. This integration of connectivity enables manufacturing systems access to computational resources, such as cloud computing, digital twin, and blockchain. The connected manufacturing systems are expected to be more efficient, sustainable and cost-effective. However, the extensive connectivity also increases the vulnerability of physical components. The attack surface of a connected manufacturing environment is greatly enlarged. Machines, products and tools could be targeted by cyber-physical attacks via the network. Among many emerging security concerns, this research focuses on the intrusion detection of cyber-physical attacks. The Intrusion Detection System (IDS) is used to monitor cyber-attacks in the computer security domain. For cyber-physical attacks, however, there is limited work. Currently, the IDS cannot effectively address cyber-physical attacks in manufacturing system: (i) the IDS takes time to reveal true alarms, sometimes over months; (ii) manufacturing production life-cycle is shorter than the detection period, which can cause physical consequences such as defective products and equipment damage; (iii) the increasing complexity of network will also make the detection period even longer. This gap leaves the cyber-physical attacks in manufacturing to cause issues like over-wearing, breakage, defects or any other changes that the original design didn’t intend. A review on the history of cyber-physical attacks, and available detection methods are presented. The detection methods are reviewed in terms of intrusion detection algorithms, and alert correlation methods. The attacks are further broken down into a taxonomy covering four dimensions with over thirty attack scenarios to comprehensively study and simulate cyber-physical attacks. A new intrusion detection and correlation method was proposed to address the cyber-physical attacks in CMS. The detection method incorporates IDS software in cyber domain and machine learning analysis in physical domain. The correlation relies on a new similarity-based cyber-physical alert correlation method. Four experimental case studies were used to validate the proposed method. Each case study focused on different aspects of correlation method performance. The experiments were conducted on a security-oriented manufacturing testbed established for this research at Syracuse University. The results showed the proposed intrusion detection and alert correlation method can effectively disclose unknown attack, known attack and attack interference that causes false alarms. In case study one, the alarm reduction rate reached 99.1%, with improvement of detection accuracy from 49.6% to 100%. The case studies also proved the proposed method can mitigate false alarms, detect attacks on multiple machines, and attacks from the supply chain. This work contributes to the security domain in cyber-physical manufacturing systems, with the focus on intrusion detection. The dataset collected during the experiments has been shared with the research community. The alert correlation methodology also contributes to cyber-physical systems, such as smart grid and connected vehicles, which requires enhanced security protection in today’s connected world

Safe navigation and human-robot interaction in assistant robotic applications

L'abstract è presente nell'allegato / the abstract is in the attachmen