Reinforcement Learning Based Minimum State-flipped Control for the Reachability of Boolean Control Networks

To realize reachability as well as reduce control costs of Boolean Control Networks (BCNs) with state-flipped control, a reinforcement learning based method is proposed to obtain flip kernels and the optimal policy with minimal flipping actions to realize reachability. The method proposed is model-free and of low computational complexity. In particular, Q-learning (QL), fast QL, and small memory QL are proposed to find flip kernels. Fast QL and small memory QL are two novel algorithms. Specifically, fast QL, namely, QL combined with transfer-learning and special initial states, is of higher efficiency, and small memory QL is applicable to large-scale systems. Meanwhile, we present a novel reward setting, under which the optimal policy with minimal flipping actions to realize reachability is the one of the highest returns. Then, to obtain the optimal policy, we propose QL, and fast small memory QL for large-scale systems. Specifically, on the basis of the small memory QL mentioned before, the fast small memory QL uses a changeable reward setting to speed up the learning efficiency while ensuring the optimality of the policy. For parameter settings, we give some system properties for reference. Finally, two examples, which are a small-scale system and a large-scale one, are considered to verify the proposed method

Pinning controllability of autonomous Boolean control networks

Autonomous Boolean networks (ABNs), which are developed to model the Boolean networks (BNs) with regulatory delays, are well known for their advantages of characterizing the intrinsic evolution rules of biological systems such as the gene regulatory networks. As a special type of ABNs with binary inputs, the autonomous Boolean control networks (ABCNs) are introduced for designing and analyzing the therapeutic intervention strategies where the binary inputs represent whether a certain medicine is dominated or not. An important problem in the therapeutic intervention is to design a control sequence steering an ABCN from an undesirable location (implying a diseased state) to a desirable one (corresponding to a healthy state). Motivated by such background, this paper aims to investigate the reachability and controllability of ABCNs with pinning controllers. Several necessary and sufficient criteria are provided by resorting to the semi-tensor product techniques of matrices. Moreover, an effective pinning control algorithm is presented for steering an ABCN from any given states to the desired state in the shortest time period. Numerical examples are also presented to demonstrate the results obtained.This work was supported in part by National Natural Science Foundation of China (Grant Nos. 61329301, 61273156), Natural Science Foundation of Jiangsu Province of China (Grant No. BK20130017), Six Talent Peaks Project for the High Level Personnel from the Jiangsu Province of China (Grant No. 2015- DZXX-003), Scientific Research Foundations of Graduate School of Southeast University (Grant No. YBJJ1560), and Innovation Program of Jiangsu Province (Grant No. KYZZ15 0050)

Towards a new methodology for design, modelling, and verification of reconfigurable distributed control systems based on a new extension to the IEC 61499 standard

In order to meet user requirements and system environment changes, reconfigurable control systems must dynamically adapt their structure and behaviour without disrupting system operation. IEC 61499 standard provides limited support for the design and verification of such systems. In fact, handling different reconfiguration scenarios at runtime is difficult since function blocks in IEC 61499 cannot be changed at run-time. Hence, this thesis promotes an IEC 61499 extension called reconfigurable function block (RFB) that increases design readability and smoothly switches to the most appropriate behaviour when a reconfiguration event occurs. To ensure system feasibility after reconfiguration, in addition to the qualitative verification, quantitative verification based on probabilistic model checking is addressed in a new RFBA approach. The latter aims to transform the designed RFB model automatically into a generalised reconfigurable timed net condition/event system model (GRTNCES) using a newly developed environment called RFBTool. The GR-TNCES fits well with RFB and preserves its semantic. Using the probabilistic model checker PRISM, the generated GR-TNCES model is checked using defined properties specified in computation tree logic. As a result, an evaluation of system performance and an estimation of reconfiguration risks are obtained. The RFBA methodology is applied on a distributed power system case study.Dynamische Anforderungen und Umgebungen erfordern rekonfigurierbare Anlagen und Steuerungssysteme. Rekonfiguration ermöglicht es einem System, seine Struktur und sein Verhalten an interne oder externe Änderungen anzupassen. Die Norm IEC 61499 wurde entwickelt, um (verteilte) Steuerungssysteme auf Basis von Funktionsbausteinen zu entwickeln. Sie bietet jedoch wenig Unterstützung für Entwurf und Verifikation. Die Tatsache, dass eine Rekonfiguration das System-Ausführungsmodell verändert, erschwert die Entwicklung in IEC 61499 zusätzlich. Daher schlägt diese Dissertation rekonfigurierbare Funktionsbausteine (RFBs) als Erweiterung der Norm vor. Ein RFB verarbeitet über einen Master-Slave-Automaten Rekonfigurationsereignisse und löst das entsprechende Verhalten aus. Diese Hierarchie trennt das Rekonfigurationsmodell vom Steuerungsmodell und vereinfacht so den Entwurf. Die Funktionalität des Entwurfs muss verifiziert werden, damit die Ausführbarkeit des Systems nach einer Rekonfiguration gewährleistet ist. Hierzu wird das entworfene RFB-Modell automatisch in ein generalised reconfigurable timed net condition/event system übersetzt. Dieses wird mit dem Model-Checker PRISM auf qualitative und quantitative Eigenschaften überprüft. Somit wird eine Bewertung der Systemperformanz und eine Einschätzung der Rekonfigurationsrisiken erreicht. Die RFB-Methodik wurde in einem Softwarewerkzeug umgesetzt und in einer Fallstudie auf ein dezentrales Stromnetz angewendet