Ageneric predictive information system for resource planning and optimisation

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel UniversityThe purpose of this research work is to demonstrate the feasibility of creating a quick response decision platform for middle management in industry. It utilises the strengths of current, but more importantly creates a leap forward in the theory and practice of Supervisory and Data Acquisition (SCADA) systems and Discrete Event Simulation and Modelling (DESM). The proposed research platform uses real-time data and creates an automatic platform for real-time and predictive system analysis, giving current and ahead of time information on the performance of the system in an efficient manner. Data acquisition as the backend connection of data integration system to the shop floor faces both hardware and software challenges for coping with large scale real-time data collection. Limited scope of SCADA systems does not make them suitable candidates for this. Cost effectiveness, complexity, and efficiency-orientation of proprietary solutions leave space for more challenge. A Flexible Data Input Layer Architecture (FDILA) is proposed to address generic data integration platform so a multitude of data sources can be connected to the data processing unit. The efficiency of the proposed integration architecture lies in decentralising and distributing services between different layers. A novel Sensitivity Analysis (SA) method called EvenTracker is proposed as an effective tool to measure the importance and priority of inputs to the system. The EvenTracker method is introduced to deal with the complexity systems in real-time. The approach takes advantage of event-based definition of data involved in process flow. The underpinning logic behind EvenTracker SA method is capturing the cause-effect relationships between triggers (input variables) and events (output variables) at a specified period of time determined by an expert. The approach does not require estimating data distribution of any kind. Neither the performance model requires execution beyond the real-time. The proposed EvenTracker sensitivity analysis method has the lowest computational complexity compared with other popular sensitivity analysis methods. For proof of concept, a three tier data integration system was designed and developed by using National Instruments’ LabVIEW programming language, Rockwell Automation’s Arena simulation and modelling software, and OPC data communication software. A laboratory-based conveyor system with 29 sensors was installed to simulate a typical shop floor production line. In addition, EvenTracker SA method has been implemented on the data extracted from 28 sensors of one manufacturing line in a real factory. The experiment has resulted 14% of the input variables to be unimportant for evaluation of model outputs. The method proved a time efficiency gain of 52% on the analysis of filtered system when unimportant input variables were not sampled anymore. The EvenTracker SA method compared to Entropy-based SA technique, as the only other method that can be used for real-time purposes, is quicker, more accurate and less computationally burdensome. Additionally, theoretic estimation of computational complexity of SA methods based on both structural complexity and energy-time analysis resulted in favour of the efficiency of the proposed EvenTracker SA method. Both laboratory and factory-based experiments demonstrated flexibility and efficiency of the proposed solution.The Engineering and Physical Sciences Research Council

Industry/University Collaboration at the University of Michigan-Dearborn: A Focus on Relevant Technology

https://deepblue.lib.umich.edu/bitstream/2027.42/154106/1/kampfner1998.pd

CNN YOLO를 이용한 열화상기반 영상과 점구름 기반 엔드밀 감시 시스템

학위논문(석사) -- 서울대학교대학원 : 공과대학 기계공학부, 2022. 8. 안성훈.As adoption of smart-factory system in manufacturing becoming inevitable, autonomous monitoring system in the field of machining has become viral nowadays. Among various methods in autonomous monitoring, vision-based monitoring is the most sought-after. This system uses vision sensors integrated with detection models developed through deep learning. However, the disadvantage of being greatly affected by optical conditions, such as ambient lighting or reflective materials, critically affects the performance in terms of monitoring. Instead of vision sensors, LiDAR, which provides depth map by measuring light returning time using infrared radiation (IR) directly to the object, can be complementary method. The study presents a LiDAR ((Light Detection and Ranging)-based end mill state monitoring system, which renders strengths of both vision and LiDAR detecting. This system uses point cloud and IR intensity data acquired by the LiDAR while object detection algorithm developed based on deep learning is engaged during the detection stage. The point cloud data is used to detect and determine the length of the endmill while the IR intensity is used to detect the wear present on the endmill. Convolutional neural network based You Only Look Once (YOLO) algorithm is selected as an object detection algorithm for real-time monitoring. Also, the quality of point cloud has been improved using data prep-processing method. Finally, it is verified that end mill state has been monitored with high accuracy at the actual machining environment.제조 분야에서 스마트 팩토리 시스템의 도입으로 인해 가공 과정의 무인 모니터링 시스템이 필연적으로 도입되고 있다. 무인 모니터링의 다양한 방법 중 비전 기반 모니터링이 가장 많이 쓰이고 있다. 해당 비전 기반 시스템의 경우 딥 러닝을 통해 개발된 감지 모델과 통합된 비전 센서를 사용한다. 하지만 주변 조명이나 반사 물질과 같은 광학적 조건에 크게 영향을 받는 단점은 모니터링 측면에서 성능에 치명적인 영향을 미치기에 이를 보완하는 대안이 필요하다. 이 연구에서는 비전 센서 대신 적외선(IR)을 물체에 직접 조사하여 빛의 왕복 시간을 측정하여 깊이 정보를 측정하는 LiDAR를 이용하여 비전 센서의 한계를 보완하는 시스템을 소개한다. 또한 비전과 LiDAR 감지의 장점을 모두 제공하는 LiDAR 기반 엔드밀 상태 모니터링 시스템을 제시한다. 이 시스템은 LiDAR에서 획득한 점 구름 정보 및 IR 강도 데이터를 사용하며, 딥 러닝을 기반으로 개발된 객체 감지 알고리즘은 감지 단계와 엔드밀의 길이를 감지하고 측정하는 데 사용되며 IR 강도는 엔드밀에 존재하는 마모 혹은 파손 정보를 감지하는 데 사용된다. 실시간 모니터링을 위한 객체 감지 알고리즘으로 YOLO(You Only Look Once) 알고리즘을 기반으로 하는 컨볼루션 신경망이 선택되었으며 데이터 전처리를 통해 포인트 클라우드의 품질을 향상시켰다. 마지막으로 실제 가공 환경에서 엔드밀 상태를 높은 정확도로 모니터링하는 과정을 진행하였다.1. Introduction . 1 1.1 Tool monitoring in CNC machines 1 1.2 LiDAR and point cloud map. 5 1.3 IR intensity application 7 2. System modelling 9 2.1 End mill monitoring system overview 9 2.2 Hardware setup . 11 2.3 End mill failure monitoring 15 2.4 YOLO setup 18 3. Data processing . 19 3.1 Confidence score. 19 3.2 Noise removal 20 3.3 Point cloud accumulation. 22 3.4 IR intensity monitoring 26 4. Experiments and results . 28 4.1 Data gathering 28 4.2 Training 30 4.3 Results . 32 5. Conclusion . 39 Reference 41 Abstract (In Korean) 43석

30th International Conference on Condition Monitoring and Diagnostic Engineering Management (COMADEM 2017)

Proceedings of COMADEM 201

Examining the Supply Chain Management Models for Agricultural Products Under the Context of E-Commerce

Agricultural products market changes constantly along with the thriving of e-commerce and agricultural products e-commerce keeps growing as an innovative industry; however, there are still many loopholes in the management of the supply chain from beginning to end. In order to effectively address these issues, this paper utilizes the dynamic requirement forecasting method based on SVM (support vector machine) to identify and fit the secular trend in and potential cyclical fluctuation factors for the market requirements for agricultural products. The supply chain coordination decision center is established by integrating the collaborative supply management component and other components. XML technology and CORBA technology are adopted to construct the integrated management model of agricultural products supply chain in e-commerce environment. For its relatively high management level, the model established can promote both agricultural consumption and agricultural economic output, strengthen the competitiveness of enterprises in agricultural products market and realize maximization of profit targets

END-TO-END PREDICTION OF WELD PENETRATION IN REAL TIME BASED ON DEEP LEARNING

Welding is an important joining technique that has been automated/robotized. In automated/robotic welding applications, however, the parameters are preset and are not adaptively adjusted to overcome unpredicted disturbances, which cause these applications to not be able to meet the standards from welding/manufacturing industry in terms of quality, efficiency, and individuality. Combining information sensing and processing with traditional welding techniques is a significant step toward revolutionizing the welding industry. In practical welding, the weld penetration as measured by the back-side bead width is a critical factor when determining the integrity of the weld produced. However, the back-side bead width is difficult to be directly monitored during manufacturing because it occurs underneath the surface of the welded workpiece. Therefore, predicting back-side bead width based on conveniently sensible information from the welding process is a fundamental issue in intelligent welding. Traditional research methods involve an indirect process that includes defining and extracting key characteristic information from the sensed data and building a model to predict the target information from the characteristic information. Due to a lack of feature information, the cumulative error of the extracted information and the complex sensing process directly affect prediction accuracy and real-time performance. An end-to-end, data-driven prediction system is proposed to predict the weld penetration status from top-side images during welding. In this method, a passive-vision sensing system with two cameras to simultaneously monitor the top-side and back-bead information is developed. Then the weld joints are classified into three classes (i.e., under penetration, desirable penetration, and excessive penetration) according to the back-bead width. Taking the weld pool-arc images as inputs and corresponding penetration statuses as labels, an end-to-end convolutional neural network (CNN) is designed and trained so the features are automatically defined and extracted. In order to increase accuracy and training speed, a transfer learning approach based on a residual neural network (ResNet) is developed. This ResNet-based model is pre-trained on an ImageNet dataset to process a better feature-extracting ability, and its fully connected layers are modified based on our own dataset. Our experiments show that this transfer learning approach can decrease training time and improve performance. Furthermore, this study proposes that the present weld pool-arc image is fused with two previous images that were acquired 1/6s and 2/6s earlier. The fused single image thus reflects the dynamic welding phenomena, and prediction accuracy is significantly improved with the image-sequence data by fusing temporal information to the input layer of the CNN (early fusion). Due to the critical role of weld penetration and the negligible impact on system implementation, this method represents major progress in the field of weld-penetration monitoring and is expected to provide more significant improvements during welding using pulsed current where the process becomes highly dynamic

Machine Learning Based Defect Detection in Robotic Wire Arc Additive Manufacturing

In the last ten years, research interests in various aspects of the Wire Arc Additive Manufacturing (WAAM) processes have grown exponentially. More recently, efforts to integrate an automatic quality assurance system for the WAAM process are increasing. No reliable online monitoring system for the WAAM process is a key gap to be filled for the commercial application of the technology, as it will enable the components produced by the process to be qualified for the relevant standards and hence be fit for use in critical applications in the aerospace or naval sectors. However, most of the existing monitoring methods only detect or solve issues from a specific sensor, no monitoring system integrated with different sensors or data sources is developed in WAAM in the last three years. In addition, complex principles and calculations of conventional algorithms make it hard to be applied in the manufacturing of WAAM as the character of a long manufacturing cycle. Intelligent algorithms provide in-built advantages in processing and analysing data, especially for large datasets generated during the long manufacturing cycles. In this research, in order to establish an intelligent WAAM defect detection system, two intelligent WAAM defect detection modules are developed successfully. The first module takes welding arc current / voltage signals during the deposition process as inputs and uses algorithms such as support vector machine (SVM) and incremental SVM to identify disturbances and continuously learn new defects. The incremental learning module achieved more than a 90% f1-score on new defects. The second module takes CCD images as inputs and uses object detection algorithms to predict the unfused defect during the WAAM manufacturing process with above 72% mAP. This research paves the path for developing an intelligent WAAM online monitoring system in the future. Together with process modelling, simulation and feedback control, it reveals the future opportunity for a digital twin system

Sustainable supply chain management trends in world regions: A data-driven analysis

This study proposes a data-driven analysis that describes the overall situation and reveals the factors hindering improvement in the sustainable supply chain management field. The literature has presented a summary of the evolution of sustainable supply chain management across attributes. Prior studies have evaluated different parts of the supply chain as independent entities. An integrated systematic assessment is absent in the extant literature and makes it necessary to identify potential opportunities for research direction. A hybrid of data-driven analysis, the fuzzy Delphi method, the entropy weight method and fuzzy decision-making trial and evaluation laboratory is adopted to address uncertainty and complexity. This study contributes to locating the boundary of fundamental knowledge to advance future research and support practical execution. Valuable direction is provided by reviewing the existing literature to identify the critical indicators that need further examination. The results show that big data, closed-loop supply chains, industry 4.0, policy, remanufacturing, and supply chain network design are the most important indicators of future trends and disputes. The challenges and gaps among different geographical regions is offered that provides both a local viewpoint and a state-of-the-art advanced sustainable supply chain management assessment

TOWARD INTELLIGENT WELDING BY BUILDING ITS DIGITAL TWIN

To meet the increasing requirements for production on individualization, efficiency and quality, traditional manufacturing processes are evolving to smart manufacturing with the support from the information technology advancements including cyber-physical systems (CPS), Internet of Things (IoT), big industrial data, and artificial intelligence (AI). The pre-requirement for integrating with these advanced information technologies is to digitalize manufacturing processes such that they can be analyzed, controlled, and interacted with other digitalized components. Digital twin is developed as a general framework to do that by building the digital replicas for the physical entities. This work takes welding manufacturing as the case study to accelerate its transition to intelligent welding by building its digital twin and contributes to digital twin in the following two aspects (1) increasing the information analysis and reasoning ability by integrating deep learning; (2) enhancing the human user operative ability to physical welding manufacturing via digital twins by integrating human-robot interaction (HRI). Firstly, a digital twin of pulsed gas tungsten arc welding (GTAW-P) is developed by integrating deep learning to offer the strong feature extraction and analysis ability. In such a system, the direct information including weld pool images, arc images, welding current and arc voltage is collected by cameras and arc sensors. The undirect information determining the welding quality, i.e., weld joint top-side bead width (TSBW) and back-side bead width (BSBW), is computed by a traditional image processing method and a deep convolutional neural network (CNN) respectively. Based on that, the weld joint geometrical size is controlled to meet the quality requirement in various welding conditions. In the meantime, this developed digital twin is visualized to offer a graphical user interface (GUI) to human users for their effective and intuitive perception to physical welding processes. Secondly, in order to enhance the human operative ability to the physical welding processes via digital twins, HRI is integrated taking virtual reality (VR) as the interface which could transmit the information bidirectionally i.e., transmitting the human commends to welding robots and visualizing the digital twin to human users. Six welders, skilled and unskilled, tested this system by completing the same welding job but demonstrate different patterns and resulted welding qualities. To differentiate their skill levels (skilled or unskilled) from their demonstrated operations, a data-driven approach, FFT-PCA-SVM as a combination of fast Fourier transform (FFT), principal component analysis (PCA), and support vector machine (SVM) is developed and demonstrates the 94.44% classification accuracy. The robots can also work as an assistant to help the human welders to complete the welding tasks by recognizing and executing the intended welding operations. This is done by a developed human intention recognition algorithm based on hidden Markov model (HMM) and the welding experiments show that developed robot-assisted welding can help to improve welding quality. To further take the advantages of the robots i.e., movement accuracy and stability, the role of the robot upgrades to be a collaborator from an assistant to complete a subtask independently i.e., torch weaving and automatic seam tracking in weaving GTAW. The other subtask i.e., welding torch moving along the weld seam is completed by the human users who can adjust the travel speed to control the heat input and ensure the good welding quality. By doing that, the advantages of humans (intelligence) and robots (accuracy and stability) are combined together under this human-robot collaboration framework. The developed digital twin for welding manufacturing helps to promote the next-generation intelligent welding and can be applied in other similar manufacturing processes easily after small modifications including painting, spraying and additive manufacturing