PAC: A Novel Self-Adaptive Neuro-Fuzzy Controller for Micro Aerial Vehicles

There exists an increasing demand for a flexible and computationally efficient controller for micro aerial vehicles (MAVs) due to a high degree of environmental perturbations. In this work, an evolving neuro-fuzzy controller, namely Parsimonious Controller (PAC) is proposed. It features fewer network parameters than conventional approaches due to the absence of rule premise parameters. PAC is built upon a recently developed evolving neuro-fuzzy system known as parsimonious learning machine (PALM) and adopts new rule growing and pruning modules derived from the approximation of bias and variance. These rule adaptation methods have no reliance on user-defined thresholds, thereby increasing the PAC's autonomy for real-time deployment. PAC adapts the consequent parameters with the sliding mode control (SMC) theory in the single-pass fashion. The boundedness and convergence of the closed-loop control system's tracking error and the controller's consequent parameters are confirmed by utilizing the LaSalle-Yoshizawa theorem. Lastly, the controller's efficacy is evaluated by observing various trajectory tracking performance from a bio-inspired flapping-wing micro aerial vehicle (BI-FWMAV) and a rotary wing micro aerial vehicle called hexacopter. Furthermore, it is compared to three distinctive controllers. Our PAC outperforms the linear PID controller and feed-forward neural network (FFNN) based nonlinear adaptive controller. Compared to its predecessor, G-controller, the tracking accuracy is comparable, but the PAC incurs significantly fewer parameters to attain similar or better performance than the G-controller.Comment: This paper has been accepted for publication in Information Science Journal 201

External Support Vector Machine Clustering

The external-Support Vector Machine (SVM) clustering algorithm clusters data vectors with no a priori knowledge of each vector\u27s class. The algorithm works by first running a binary SVM against a data set, with each vector in the set randomly labeled, until the SVM converges. It then relabels data points that are mislabeled and a large distance from the SVM hyperplane. The SVM is then iteratively rerun followed by more label swapping until no more progress can be made. After this process, a high percentage of the previously unknown class labels of the data set will be known. With sub-cluster identification upon iterating the overall algorithm on the positive and negative clusters identified (until the clusters are no longer separable into sub-clusters), this method provides a way to cluster data sets without prior knowledge of the data\u27s clustering characteristics, or the number of clusters

External Support Vector Machine Clustering

The external-Support Vector Machine (SVM) clustering algorithm clusters data vectors with no a priori knowledge of each vector\u27s class. The algorithm works by first running a binary SVM against a data set, with each vector in the set randomly labeled, until the SVM converges. It then relabels data points that are mislabeled and a large distance from the SVM hyperplane. The SVM is then iteratively rerun followed by more label swapping until no more progress can be made. After this process, a high percentage of the previously unknown class labels of the data set will be known. With sub-cluster identification upon iterating the overall algorithm on the positive and negative clusters identified (until the clusters are no longer separable into sub-clusters), this method provides a way to cluster data sets without prior knowledge of the data\u27s clustering characteristics, or the number of clusters

Development of advanced autonomous learning algorithms for nonlinear system identification and control

Identification of nonlinear dynamical systems, data stream analysis, etc. is usually handled by autonomous learning algorithms like evolving fuzzy and evolving neuro-fuzzy systems (ENFSs). They are characterized by the single-pass learning mode and open structure-property. Such features enable their effective handling of fast and rapidly changing natures of data streams. The underlying bottleneck of ENFSs lies in its design principle, which involves a high number of free parameters (rule premise and rule consequent) to be adapted in the training process. This figure can even double in the case of the type-2 fuzzy system. From this literature gap, a novel ENFS, namely Parsimonious Learning Machine (PALM) is proposed in this thesis. To reduce the number of network parameters significantly, PALM features utilization of a new type of fuzzy rule based on the concept of hyperplane clustering, where it has no rule premise parameters. PALM is proposed in both type-1 and type-2 fuzzy systems where all of them characterize a fully dynamic rule-based system. Thus, it is capable of automatically generating, merging, and tuning the hyperplane-based fuzzy rule in a single-pass manner. Moreover, an extension of PALM, namely recurrent PALM (rPALM), is proposed and adopts the concept of teacher-forcing mechanism in the deep learning literature. The efficacy of both PALM and rPALM have been evaluated through numerical study with data streams and to identify nonlinear unmanned aerial vehicle system. The proposed models showcase significant improvements in terms of computational complexity and the number of required parameters against several renowned ENFSs while attaining comparable and often better predictive accuracy. The ENFSs have also been utilized to develop three autonomous intelligent controllers (AICons) in this thesis. They are namely Generic (G) controller, Parsimonious controller (PAC), and Reduced Parsimonious Controller (RedPAC). All these controllers start operating from scratch with an empty set of fuzzy rules, and no offline training is required. To cope with the dynamic behavior of the plant, these controllers can add, merge or prune the rules on demand. Among three AICons, the G-controller is built by utilizing an advanced incremental learning machine, namely Generic Evolving Neuro-Fuzzy Inference System. The integration of generalized adaptive resonance theory provides a compact structure of the G-controller. Consequently, the faster evolution of structure is witnessed, which lowers its computational cost. Another AICon namely, PAC is rooted with PALM's architecture. Since PALM has a dependency on user-defined thresholds to adapt the structure, these thresholds are replaced with the concept of bias- variance trade-off in PAC. In RedPAC, the network parameters have further reduced in contrast with PALM-based PAC, where the number of consequent parameters has reduced to one parameter per rule. These AICons work with very minor expert domain knowledge and developed by incorporating the sliding mode control technique. In G-controller and RedPAC, the control law and adaptation laws for the consequent parameters are derived from the SMC algorithm to establish a stable closed-loop system, where the stability of these controllers are guaranteed by using the Lyapunov function and the uniform asymptotic convergence of tracking error to zero is witnessed through the implication of an auxiliary robustifying control term. While using PAC, the boundedness and convergence of the closed-loop control system's tracking error and the controller's consequent parameters are confirmed by utilizing the LaSalle-Yoshizawa theorem. Their efficacy is evaluated by observing various trajectory tracking performance of unmanned aerial vehicles. The accuracy of these controllers is comparable or better than the benchmark controllers where the proposed controllers incur significantly fewer parameters to attain similar or better tracking performance

Fuzzy System Identification Based Upon a Novel Approach to Nonlinear Optimization

Fuzzy systems are often used to model the behavior of nonlinear dynamical systems in process control industries because the model is linguistic in nature, uses a natural-language rule set, and because they can be included in control laws that meet the design goals. However, because the rigorous study of fuzzy logic is relatively recent, there is a shortage of well-defined and understood mechanisms for the design of a fuzzy system. One of the greatest challenges in fuzzy modeling is to determine a suitable structure, parameters, and rules that minimize an appropriately chosen error between the fuzzy system, a mathematical model, and the target system. Numerous methods for establishing a suitable fuzzy system have been proposed, however, none are able to demonstrate the existence of a structure, parameters, or rule base that will minimize the error between the fuzzy and the target system. The piecewise linear approximator (PLA) is a mathematical construct that can be used to approximate an input-output data set with a series of connected line segments. The number of segments in the PLA is generally selected by the designer to meet a given error criteria. Increasing the number of segments will generally improve the approximation. If the location of the breakpoints between segments is known, it is a straightforward process to select the PLA parameters to minimize the error. However, if the location of the breakpoints is not known, a mechanism is required to determine their locations. While algorithms exist that will determine the location of the breakpoints, they do not minimize the error between data and the model. This work will develop theory that shows that an optimal solution to this nonlinear optimization problem exists and demonstrates how it can be applied to fuzzy modeling. This work also demonstrates that a fuzzy system restricted to a particular class of input membership functions, output membership functions, conjunction operator, and defuzzification technique is equivalent to a piecewise linear approximator (PLA). Furthermore, this work develops a new nonlinear optimization technique that minimizes the error between a PLA and an arbitrary one-dimensional set of input-output data and solves the optimal breakpoint problem. This nonlinear optimization technique minimizes the approximation error of several classes of nonlinear functions leading up to the generalized PLA. While direct application of this technique is computationally intensive, several paths are available for investigation that may ease this limitation. An algorithm is developed based on this optimization theory that is significantly more computationally tractable. Several potential applications of this work are discussed including the ability to model the nonlinear portions of Hammerstein and Wiener systems

Switching control systems and their design automation via genetic algorithms

The objective of this work is to provide a simple and effective nonlinear controller. Our strategy involves switching the underlying strategies in order to maintain a robust control. If a disturbance moves the system outside the region of stability or the domain of attraction, it will be guided back onto the desired course by the application of a different control strategy. In the context of switching control, the common types of controller present in the literature are based either on fuzzy logic or sliding mode. Both of them are easy to implement and provide efficient control for non-linear systems, their actions being based on the observed input/output behaviour of the system. In the field of fuzzy logic control (FLC) using error feedback variables there are two main problems. The first is the poor transient response (jerking) encountered by the conventional 2-dimensional rule-base fuzzy PI controller. Secondly, conventional 3-D rule-base fuzzy PID control design is both computationally intensive and suffers from prolonged design times caused by a large dimensional rule-base. The size of the rule base will increase exponentially with the increase of the number of fuzzy sets used for each input decision variable. Hence, a reduced rule-base is needed for the 3-term fuzzy controller. In this thesis a direct implementation method is developed that allows the size of the rule-base to be reduced exponentially without losing the features of the PID structure. This direct implementation method, when applied to the reduced rule-base fuzzy PI controller, gives a good transient response with no jerking

Multiple instance fuzzy inference.

A novel fuzzy learning framework that employs fuzzy inference to solve the problem of multiple instance learning (MIL) is presented. The framework introduces a new class of fuzzy inference systems called Multiple Instance Fuzzy Inference Systems (MI-FIS). Fuzzy inference is a powerful modeling framework that can handle computing with knowledge uncertainty and measurement imprecision effectively. Fuzzy Inference performs a non-linear mapping from an input space to an output space by deriving conclusions from a set of fuzzy if-then rules and known facts. Rules can be identified from expert knowledge, or learned from data. In multiple instance problems, the training data is ambiguously labeled. Instances are grouped into bags, labels of bags are known but not those of individual instances. MIL deals with learning a classifier at the bag level. Over the years, many solutions to this problem have been proposed. However, no MIL formulation employing fuzzy inference exists in the literature. In this dissertation, we introduce multiple instance fuzzy logic that enables fuzzy reasoning with bags of instances. Accordingly, different multiple instance fuzzy inference styles are proposed. The Multiple Instance Mamdani style fuzzy inference (MI-Mamdani) extends the standard Mamdani style inference to compute with multiple instances. The Multiple Instance Sugeno style fuzzy inference (MI-Sugeno) is an extension of the standard Sugeno style inference to handle reasoning with multiple instances. In addition to the MI-FIS inference styles, one of the main contributions of this work is an adaptive neuro-fuzzy architecture designed to handle bags of instances as input and capable of learning from ambiguously labeled data. The proposed architecture, called Multiple Instance-ANFIS (MI-ANFIS), extends the standard Adaptive Neuro Fuzzy Inference System (ANFIS). We also propose different methods to identify and learn fuzzy if-then rules in the context of MIL. In particular, a novel learning algorithm for MI-ANFIS is derived. The learning is achieved by using the backpropagation algorithm to identify the premise parameters and consequent parameters of the network. The proposed framework is tested and validated using synthetic and benchmark datasets suitable for MIL problems. Additionally, we apply the proposed Multiple Instance Inference to the problem of region-based image categorization as well as to fuse the output of multiple discrimination algorithms for the purpose of landmine detection using Ground Penetrating Radar