Improved learning automata applied to routing in multi-service networks

Multi-service communications networks are generally designed, provisioned and configured, based on source-destination user demands expected to occur over a recurring time period. However due to network users' actions being non-deterministic, actual user demands will vary from those expected, potentially causing some network resources to be under- provisioned, with others possibly over-provisioned. As actual user demands vary over the recurring time period from those expected, so the status of the various shared network resources may also vary. This high degree of uncertainty necessitates using adaptive resource allocation mechanisms to share the finite network resources more efficiently so that more of actual user demands may be accommodated onto the network. The overhead for these adaptive resource allocation mechanisms must be low in order to scale for use in large networks carrying many source-destination user demands. This thesis examines the use of stochastic learning automata for the adaptive routing problem (these being adaptive, distributed and simple in implementation and operation) and seeks to improve their weakness of slow convergence whilst maintaining their strength of subsequent near optimal performance. Firstly, current reinforcement algorithms (the part causing the automaton to learn) are examined for applicability, and contrary to the literature the discretised schemes are found in general to be unsuitable. Two algorithms are chosen (one with fast convergence, the other with good subsequent performance) and are improved through automatically adapting the learning rates and automatically switching between the two algorithms. Both novel methods use local entropy of action probabilities for determining convergence state. However when the convergence speed and blocking probability is compared to a bandwidth-based dynamic link-state shortest-path algorithm, the latter is found to be superior. A novel re-application of learning automata to the routing problem is therefore proposed: using link utilisation levels instead of call acceptance or packet delay. Learning automata now return a lower blocking probability than the dynamic shortest-path based scheme under realistic loading levels, but still suffer from a significant number of convergence iterations. Therefore the final improvement is to combine both learning automata and shortest-path concepts to form a hybrid algorithm. The resulting blocking probability of this novel routing algorithm is superior to either algorithm, even when using trend user demands

Adaptive Approximation-Based Control for Nonlinear Systems: A Unified Solution with Accurate and Inaccurate Measurements

A unified solution to adaptive approximation-based control for nonlinear systems with accurate and inaccurate state measurement is synthesized in this study. Starting from the standard adaptive approximation-based controller with accurate state measurement, its corresponding physical interpretation, stability conclusion, and learning ability are rigorously addressed when facing additive measurement inaccuracy, and explicit answers are obtained in the framework of both controller matching and system matching. Finally, it proves that, with a certain condition, the standard adaptive approximation-based controller works as a unified solution for the cases with accurate and inaccurate measurement, and the solution can be extended to the nonlinear system control problems with extra unknown dynamics or faults in actuator and/or process dynamics. A single-link robot arm example is used for the simulation demonstration of the unified solution

Lifelong Learning-Based Multilayer Neural Network Control of Nonlinear Continuous-Time Strict-Feedback Systems

In This Paper, We Investigate Lifelong Learning (LL)-Based Tracking Control for Partially Uncertain Strict Feedback Nonlinear Systems with State Constraints, employing a Singular Value Decomposition (SVD) of the Multilayer Neural Networks (MNNs) Activation Function based Weight Tuning Scheme. the Novel SVD-Based Approach Extends the MNN Weight Tuning to (Formula Presented.) Layers. a Unique Online LL Method, based on Tracking Error, is Integrated into the MNN Weight Update Laws to Counteract Catastrophic Forgetting. to Adeptly Address Constraints for Safety Assurances, Taking into Account the Effects Caused by Disturbances, We Utilize a Time-Varying Barrier Lyapunov Function (TBLF) that Ensures a Uniformly Ultimately Bounded Closed-Loop System. the Effectiveness of the Proposed Safe LL MNN Approach is Demonstrated through a Leader-Follower Formation Scenario Involving Unknown Kinematics and Dynamics. Supporting Simulation Results of Mobile Robot Formation Control Are Provided, Confirming the Theoretical Findings

Adaptive Backstepping Control for Fractional-Order Nonlinear Systems with External Disturbance and Uncertain Parameters Using Smooth Control

In this paper, we consider controlling a class of single-input-single-output (SISO) commensurate fractional-order nonlinear systems with parametric uncertainty and external disturbance. Based on backstepping approach, an adaptive controller is proposed with adaptive laws that are used to estimate the unknown system parameters and the bound of unknown disturbance. Instead of using discontinuous functions such as the $\mathrm{sign}$ function, an auxiliary function is employed to obtain a smooth control input that is still able to achieve perfect tracking in the presence of bounded disturbances. Indeed, global boundedness of all closed-loop signals and asymptotic perfect tracking of fractional-order system output to a given reference trajectory are proved by using fractional directed Lyapunov method. To verify the effectiveness of the proposed control method, simulation examples are presented.Comment: Accepted by the IEEE Transactions on Systems, Man and Cybernetics: Systems with Minor Revision