Design of process control scheme is critical for quality assurance to reduce
variations in manufacturing systems. Taking semiconductor manufacturing as an
example, extensive literature focuses on control optimization based on certain
process models (usually linear models), which are obtained by experiments
before a manufacturing process starts. However, in real applications,
pre-defined models may not be accurate, especially for a complex manufacturing
system. To tackle model inaccuracy, we propose a model-free reinforcement
learning (MFRL) approach to conduct experiments and optimize control
simultaneously according to real-time data. Specifically, we design a novel
MFRL control scheme by updating the distribution of disturbances using Bayesian
inference to reduce their large variations during manufacturing processes. As a
result, the proposed MFRL controller is demonstrated to perform well in a
nonlinear chemical mechanical planarization (CMP) process when the process
model is unknown. Theoretical properties are also guaranteed when disturbances
are additive. The numerical studies also demonstrate the effectiveness and
efficiency of our methodology.Comment: 31 pages, 7 figures, and 3 table