Joint Manufacturing and Onsite Microgrid System Control using Markov Decision Process and Neural Network Integrated Reinforcement Learning

Onsite microgrid generation systems with renewable sources are considered a promising complementary energy supply system for manufacturing plant, especially when outage occurs during which the energy supplied from the grid is not available. Compared to the widely recognized benefits in terms of the resilience improvement when it is used as a backup energy system, the operation along with the electricity grid to support the manufacturing operations in non-emergent mode has been less investigated. In this paper, we propose a joint dynamic decision-making model for the optimal control for both manufacturing system and onsite generation system. Markov Decision Process (MDP) is used to formulate the decision-making model. A neural network integrated reinforcement learning algorithm is proposed to approximately estimate the value function given policy of MDP. A case study based on a manufacturing system as well as a typical onsite microgrid generation system is conducted to validate the proposed MDP model as well as the solution strategy

Energy efficiency in discrete-manufacturing systems: insights, trends, and control strategies

Since the depletion of fossil energy sources, rising energy prices, and governmental regulation restrictions, the current manufacturing industry is shifting towards more efficient and sustainable systems. This transformation has promoted the identification of energy saving opportunities and the development of new technologies and strategies oriented to improve the energy efficiency of such systems. This paper outlines and discusses most of the research reported during the last decade regarding energy efficiency in manufacturing systems, the current technologies and strategies to improve that efficiency, identifying and remarking those related to the design of management/control strategies. Based on this fact, this paper aims to provide a review of strategies for reducing energy consumption and optimizing the use of resources within a plant into the context of discrete manufacturing. The review performed concerning the current context of manufacturing systems, control systems implemented, and their transformation towards Industry 4.0 might be useful in both the academic and industrial dimension to identify trends and critical points and suggest further research lines.Peer ReviewedPreprin

An optimization-based control strategy for energy efficiency of discrete manufacturing systems

In order to reduce the global energy consumption and avoid highest power peaks during operation of manufacturing systems, an optimization-based controller for selective switching on/off of peripheral devices in a test bench that emulates the energy consumption of a periodic system is proposed. First, energy consumption models for the test-bench devices are obtained based on data and subspace identification methods. Next, a control strategy is designed based on both optimization and receding horizon approach, considering the energy consumption models, operating constraints, and the real processes performed by peripheral devices. Thus, a control policy based on dynamical models of peripheral devices is proposed to reduce the energy consumption of the manufacturing systems without sacrificing the productivity. Afterward, the proposed strategy is validated in the test bench and comparing to a typical rule-based control scheme commonly used for these manufacturing systems. Based on the obtained results, reductions near 7% could be achieved allowing improvements in energy efficiency via minimization of the energy costs related to nominal power purchased.Peer ReviewedPostprint (author's final draft

Joint Control of Manufacturing and Onsite Microgrid System Via Novel Neural-Network Integrated Reinforcement Learning Algorithms

Microgrid is a promising technology of distributed energy supply system, which consists of storage devices, generation capacities including renewable sources, and controllable loads. It has been widely investigated and applied for residential and commercial end-use customers as well as critical facilities. In this paper, we propose a joint state-based dynamic control model on microgrids and manufacturing systems where optimal controls for both sides are implemented to coordinate the energy demand and supply so that the overall production cost can be minimized considering the constraint of production target. Markov Decision Process (MDP) is used to formulate the decision-making procedure. The main computing challenge to solve the formulated MDP lies in the co-existence of both discrete and continuous parts of the high-dimensional state/action space that are intertwined with constraints. A novel reinforcement learning algorithm that leverages both Temporal Difference (TD) and Deterministic Policy Gradient (DPG) algorithms is proposed to address the computation challenge. Experiments for a manufacturing system with an onsite microgrid system with renewable sources have been implemented to justify the effectiveness of the proposed method

Advanced Techniques for Assets Maintenance Management

16th IFAC Symposium on Information Control Problems in Manufacturing INCOM 2018 Bergamo, Italy, 11–13 June 2018. Edited by Marco Macchi, László Monostori, Roberto PintoThe aim of this paper is to remark the importance of new and advanced techniques supporting decision making in different business processes for maintenance and assets management, as well as the basic need of adopting a certain management framework with a clear processes map and the corresponding IT supporting systems. Framework processes and systems will be the key fundamental enablers for success and for continuous improvement. The suggested framework will help to define and improve business policies and work procedures for the assets operation and maintenance along their life cycle. The following sections present some achievements on this focus, proposing finally possible future lines for a research agenda within this field of assets management

Enhancing cutting tool sustainability based on remaining useful life prediction

As a critical part of machining, cutting tools are of great importance to sustainability enhancement. Normally, they are underused, resulting in huge waste. However, the lack of reliable support leads to a high risk on improving the cutting tool utilization. Aiming at this problem, this paper proposes an approach to enhance the cutting tool sustainability. A non-linear cutting tool remaining useful life prediction model is developed based on tool wear historical data. Probability distribution function and cumulative distribution function are used to quantize the uncertainty of the prediction. Under a constant machining condition, a cutting tool life is extended according to its specific remaining useful life prediction, rather than a unified one. Under various machining conditions, machining parameters are optimized to improve efficiency or capability. Cutting tool sustainability is assessed in economic, environmental and social dimensions. Experimental study verifies that both material removal rate and material removal volume are improved. Carbon emission and cutting tool cost are also reduced. The balance between benefit and risk is achieved by assigning a reasonable confidence level. Cutting tool sustainability can be enhanced by improving cutting tool utilization at controllable risk.©2020 Elsevier. This manuscript version is made available under the Creative Commons Attribution–NonCommercial–NoDerivatives 4.0 International (CC BY–NC–ND 4.0) license, https://creativecommons.org/licenses/by-nc-nd/4.0/fi=vertaisarvioitu|en=peerReviewed

Design and Management of Manufacturing Systems

Although the design and management of manufacturing systems have been explored in the literature for many years now, they still remain topical problems in the current scientific research. The changing market trends, globalization, the constant pressure to reduce production costs, and technical and technological progress make it necessary to search for new manufacturing methods and ways of organizing them, and to modify manufacturing system design paradigms. This book presents current research in different areas connected with the design and management of manufacturing systems and covers such subject areas as: methods supporting the design of manufacturing systems, methods of improving maintenance processes in companies, the design and improvement of manufacturing processes, the control of production processes in modern manufacturing systems production methods and techniques used in modern manufacturing systems and environmental aspects of production and their impact on the design and management of manufacturing systems. The wide range of research findings reported in this book confirms that the design of manufacturing systems is a complex problem and that the achievement of goals set for modern manufacturing systems requires interdisciplinary knowledge and the simultaneous design of the product, process and system, as well as the knowledge of modern manufacturing and organizational methods and techniques

Review of Markov models for maintenance optimization in the context of offshore wind

The offshore environment poses a number of challenges to wind farm operators. Harsher climatic conditions typically result in lower reliability while challenges in accessibility make maintenance difficult. One of the ways to improve availability is to optimize the Operation and Maintenance (O&M) actions such as scheduled, corrective and proactive maintenance. Many authors have attempted to model or optimize O&M through the use of Markov models. Two examples of Markov models, Hidden Markov Models (HMMs) and Partially Observable Markov Decision Processes (POMDPs) are investigated in this paper. In general, Markov models are a powerful statistical tool, which has been successfully applied for component diagnostics, prognostics and maintenance optimization across a range of industries. This paper discusses the suitability of these models to the offshore wind industry. Existing models which have been created for the wind industry are critically reviewed and discussed. As there is little evidence of widespread application of these models, this paper aims to highlight the key factors required for successful application of Markov models to practical problems. From this, the paper identifies the necessary theoretical and practical gaps that must be resolved in order to gain broad acceptance of Markov models to support O&M decision making in the offshore wind industry

Economic and environmental impact assessment through system dynamics of technology-enhanced maintenance services

This work presents an economic and environmental impact assessment of maintenance services in order to evaluate how they contribute to sustainable value creation through field service delivery supported by advanced technologies. To this end, systems dynamics is used to assist the prediction of economic and environmental impacts of maintenance services supported by the use of an e-maintenance platform implementing prognosis and health management. A special concern is given to the energy use and related carbon footprint as environmental impacts