On Sensorless Collision Detection and Measurement of External Forces in Presence of Modeling Inaccuracies

The field of human-robot interaction has garnered significant interest in the last decade. Every form of human-robot coexistence must guarantee the safety of the user. Safety in human-robot interaction is being vigorously studied, in areas such as collision avoidance, soft actuators, light-weight robots, computer vision techniques, soft tissue modeling, collision detection, etc. Despite the safety provisions, unwanted collisions can occur in case of system faults. In such cases, before post-collision strategies are triggered, it is imperative to effectively detect the collisions. Implementation of tactile sensors, vision systems, sonar and Lidar sensors, etc., allows for detection of collisions. However, due to the cost of such methods, more practical approaches are being investigated. A general goal remains to develop methods for fast detection of external contacts using minimal sensory information. Availability of position data and command torques in manipulators permits development of observer-based techniques to measure external forces/torques. The presence of disturbances and inaccuracies in the model of the robot presents challenges in the efficacy of observers in the context of collision detection. The purpose of this thesis is to develop methods that reduce the effects of modeling inaccuracies in external force/torque estimation and increase the efficacy of collision detection. It is comprised of the following four parts: 1. The KUKA Light-Weight Robot IV+ is commonly employed for research purposes. The regressor matrix, minimal inertial parameters and the friction model of this robot are identified and presented in detail. To develop the model, relative weight analysis is employed for identification. 2. Modeling inaccuracies and robot state approximation errors are considered simultaneously to develop model-based time-varying thresholds for collision detection. A metric is formulated to compare trajectories realizing the same task in terms of their collision detection and external force/torque estimation capabilities. A method for determining optimal trajectories with regards to accurate external force/torque estimation is also developed. 3. The effects of velocity on external force/torque estimation errors are studied with and without the use of joint force/torque sensors. Velocity-based thresholds are developed and implemented to improve collision detection. The results are compared with the collision detection module integrated in the KUKA Light-Weight Robot IV+. 4. An alternative joint-by-joint heuristic method is proposed to identify the effects of modeling inaccuracies on external force/torque estimation. Time-varying collision detection thresholds associated with the heuristic method are developed and compared with constant thresholds. In this work, the KUKA Light-Weight Robot IV+ is used for obtaining the experimental results. This robot is controlled via the Fast Research Interface and Visual C++ 2008. The experimental results confirm the efficacy of the proposed methodologies

A Dependable Actuator and Sensor Health Monitoring System

Impressive progress in vehicle control technologies has equipped modern vehicles with advanced driver assistance and stability control systems to help the driver handle unfavorable driving conditions. These control systems rely on sensor measurements and/or information estimated based on these measurements to generate control commands for vehicle actuators. Therefore, any failure in sensors and actuators can degrade the performance of these control systems and cause instability in vehicle operation. Sensor failures make the control systems generate undesired control commands and actuator failures prevent desired control commands from being applied. To achieve safe and satisfactory vehicle operation, a real-time health monitoring system is crucial for vehicle sensors and actuators. The reliability of health monitoring is determined by its sensitivity to faults and robustness to disturbances. A reliable health monitoring system is responsible for timely detecting the occurrence of a fault in the target vehicle, accurately identifying the source of the fault, and properly determining the type and magnitude of the fault. This information is crucial for reconstructing sensor data or scaling actuator output, which ultimately could result in a fault-tolerant vehicle system. This thesis proposes a hybrid model/data fault detection and diagnosis system to monitor the health status of any sensor or actuator in a vehicle. The proposed approach works based on residuals generated by comparing sensor measurements or control inputs with their estimations. The estimations are obtained by a hybrid estimator that is developed based on the integration of model-based and data-driven estimators to leverage their strengths. Due to the poor performance of data-driven estimators in unknown conditions, a self-updating dataset is proposed to learn new cases. Estimation based on updated datasets necessitates the use of data-driven estimators that do not require any pre-training. As case studies, the proposed hybrid fault detection and diagnosis system is applied to a vehicle’s lateral acceleration sensor and traction motor. The experimental results show that the hybrid estimator outperforms the model-based and data-driven estimators used individually. These results also confirm that the proposed hybrid fault detection and diagnosis system can detect the faults in the target sensor or actuator, and then reconstruct the healthy value of the faulty sensor or find the failure level of the faulty actuator. For cases where a set of sensors and actuators should be monitored to evaluate their health status, this thesis develops a general data-driven health monitoring system to detect, isolate, and quantify faults in these components. This method checks the coherency among the target vehicle's variables by using the vehicle's data. The coherency among the vehicle variables means that the variables reflect the physical principles governing the target vehicle's motion and the causality between its states. Each variable corresponds to a component in the vehicle. When a fault occurs in one of the vehicle's components, the coherency among the variables is no longer valid. This idea is incorporated to detect faults. After fault detection, to isolate the faulty component, the developed system explores the coherency in the subsets of the vehicle's variables to find which variable is not coherent with others. Once the faulty component is determined, the health monitoring system uses the remaining healthy components to reconstruct the true value of the faulty sensor or find the failure level of the faulty actuator. The experimental results show that the developed health monitoring system appropriately detects, isolates, and quantifies faults in the test vehicle's sensors and actuators. The developed data-driven health monitoring system requires a pre-collected dataset for each vehicle to monitor the health status of its sensors and actuators. To relax this requirement, this thesis proposes a universal health monitoring system for the vehicle IMU sensor (measuring the longitudinal/lateral accelerations and yaw rate) by involving vehicle parameters in the health monitoring process. The proposed universal health monitoring system is able to monitor the health status of the target vehicle's IMU sensor using the other vehicles' data. The performance of the universal health monitoring system is evaluated by simulations. The simulation results are in line with what is expected

Practical Moving Target Detection in Maritime Environments Using Fuzzy Multi-sensor Data Fusion

As autonomous ships become the future trend for maritime transportation, it is of importance to develop intelligent autonomous navigation systems to ensure the navigation safety of ships. Among the three core components (sensing, planning and control modules) of the system, an accurate detection of target ships’ navigation information is critical. Within a typical maritime environment, the existence of sensor noises as well as the influences generated by varying environment conditions largely limit the reliability of using a single sensor for environment awareness. It is therefore vital to use multiple sensors together with a multi-sensor data fusion technology to improve the detection performance. In this paper, a fuzzy logic-based multi-sensor data fusion algorithm for moving target ships detection has been proposed and designed using both AIS and radar information. A two-stage fuzzy logic association method has been particularly developed and integrated with Kalman filtering to achieve a computationally efficient performance. The effectiveness of the proposed algorithm has been tested and validated in simulations where multiple target ships are transiting with complex movements

Online model estimation and haptic characterization for robotic-assisted minimally invasive surgery

Online soft tissue characterization is important for robotic-assisted minimally invasive surgery (RAMIS) to achieve a precise and stable robotic control with haptic feedback. The traditional linear regression method (i.e. the recursive least square (RLS) method) is inappropriate to handle nonlinear Hunt-Crossley (H-C) model since its linearization process involves unacceptable errors. This thesis presents a new nonlinear estimation method for online soft tissue characterization. To deal with nonlinear and dynamic conditions involved in soft tissue characterization, the approach expands the nonlinearity and dynamics of the H-C model by treating parameter p as an independent variable. Based on this, an unscented Kalman filter (UKF) was adapted for online nonlinear soft tissue characterization. A comparison analysis of the UKF and RLS methods was conducted to validate the performance of the UKF-based method. The UKF-based method suffers from two major problems. The first one is that it requires prior noise statistics of the corresponding system to be precisely known. However, due to uncertainties in the dynamic environment of RAMIS, it is difficult to accurately describe noise characteristics. This leads to biased or even divergent UKF solutions. Therefore, in order to attain accurate estimation results from the UKF-based approach, it is necessary to estimate noise statistics online to restrain the disturbance of noise uncertainty. Secondly, the UKF performance depends on the pre-defined system and measurement models. If the models involve stochastic errors, the UKF-based solution will be unstable. In fact, the measurement model’s accuracy can be guaranteed by using high-precision measurement equipment together with a high volume of available measurement data. On the other hand, the system model is more often involved with the inaccuracy problem. In RAMIS, the system model is a theoretical approximation of the physical contact between robotic tool and biological soft tissue. The approximation is intended to fulfil the requirement of real-time performance in RAMIS. Therefore, it is essential to improve the UKF performance in the presence of system model (the contact model) uncertainty. To address the UKF problem for inaccurate noise statistics, this thesis further presents a new recursive adaptive UKF (RAUKF) method for online nonlinear soft tissue characterization. It was developed, based on the H-C model, to estimate system noise statistics in real-time with windowing approximation. The method was developed under the condition that system noises are of small variation. In order to account for the inherent relationship between the current and previous states of soft tissue deformation involved in RAMIS, a recursive formulation was further constructed by introducing a fading scaling factor. This factor was further modified to accommodate noise statistics of a large variation, which may be caused by rupture events or geometric discontinuities in RAMIS. Simulations and comparison analyses verified the performance of the proposed RAUKF. The second UKF limitation regarding the requirement of the accurate system model was also addressed. A random weighting strong tracking unscented Kalman filter (RWSTUKF) was developed based on the Hunt-Crossley model for online nonlinear soft tissue characterization. This RWSTUKF overcomes the problem of performance degradation in the UKF due to system model errors. It adopts a scaling factor in the predicted state covariance to compensate the inaccuracy of the system model. This scaling factor was derived by combining the orthogonality principle with the random weighting concept to prevent the cumbersome computation from Jacobian matrix and offer the reliable estimation for innovation covariances. Simulation and comparison analyses demonstrated that the proposed RWSTUKF can characterise soft tissue parameters in the presence of system model error for RAMIS in on online mode. Using the proposed methods, a master-slave robotic system has been developed with a nonlinear state observer for soft tissue characterization. Robotic indentation and needle insertion tests conducted to evaluate performances of the proposed methods. Further, a rupture detection approach was established based on the RWSTUKF. It was also integrated into the master-slave robotic system to detect rupture events occurred during needle insertion. The experiment results demonstrated that the RWSTUKF outperforms RLS, UKF and RAUKF for soft tissue characterization

Industrial applications of the Kalman filter:a review

International audienc

Transmissibility operators for state and output estimation in nonlinear systems

Transmissibility operators are mathematical objects that characterize the relationship between two subsets of responses of an underlying system. The importance of transmissiblity operators comes from the fact that these operators are independent on the system inputs. This work develops the transmissibility theory for nonlinear systems for the first time. The system nonlinearities are assumed to be unknown, which gives a wide range of possible engineering applications in different disciplines. Four different methods are developed to deal with these nonlinearities. The first method is by re-constructing the system nonlinearities as independent excitations on the system. This method handles the inherent unmodeled nonlinearities within the system. The second method is by designing a transmissibility-based sliding mode control. This method rejects unwanted nonlinearities such as system faults. The third method is by constructing the system as time-variant linear system, and use recursive least squares to solve it. This method can handle nonlinear systems with time-variant dynamics. The fourth method is by designing a new robust estimation technique called high-gain transmissibility (HGT) that is inspired by high-gain observers. This estimator has the ability to robustly estimate the system states in a high-gain form. The majority of modern fault detection, control systems, and robots localization depend on mathematically estimating the system states and outputs. Transmissibility-based estimation is incorporated in this work with these three theoretical applications. For fault detection, transmissibility operators are used along a set of outputs to estimate the measurements of another set of outputs. Then faults are detected by comparing the estimated and measured outputs with each other. Control approaches use the transmissibility-based estimation to construct the control signal, in which is injected back to the original system. Robots localization fuses the transmissibility-based estimation with real-time sensor measurements to minimize the error in determining the robot displacements. These three theoretical applications are applied on four different systems. The first system is Connected Autonomous Vehicles (CAV) platoons. A CAV platoon is a network of connected autonomous vehicles that communicate together to move in a specific path with the desired velocity. Transmissibilities are proposed along with the measurements from sensors available in CAV platoons to identify transmissibility operators. This will be then developed to mixed autonomous and human-driven vehicle platoons. Besides the wide range of physical and cyber faults in such systems, this is also motivated by the fact that on-road human-drivers’ behaviour is unknown and difficult to be predicted. Transmissibility operators are used here to handle both cyber-physical faults as well as the human-drivers’ behaviour. The platoon faults are then proposed to be mitigated using a transmissibility-based sliding mode controller. Moreover, transmissibilities are integrated with Kalman filter to localize CAV platoons while operating under non-Gaussian environment as unknown nonlinearities. The second system is a multi-actuator micro positioning system that is used in the semi-conductors industry. Transmissibility operators are applied on this system for fault detection and fault-tolerant control. Fault detection is represented in applying the proposed developments to actuator fault detection. Some of the most common actuator faults such as actuator loss of effectiveness and fatigue crack in the connection hinges will be considered. Transmissibilities then will be used for fault detection without knowledge of the dynamics of the system or the excitation that acts on the system. Next, a transmissibility-based sliding mode control will be implemented to mitigate common actuator faults in multi-actuator systems. The third system is flexible structures subjected to unknown and random excitations. Structures used in applications subjected to turbulent fluid flow such as aerospace and underwater applications are subjected to random excitations distributed along the structure. Transmissibility operators are used for the purpose of structural fault detection and localization during the system operation. The fourth system is robotic manipulators with bounded nonlinearities and time-variant parameters. Both parameter variation and system nonlinearities are considered to be unknown. Transmissibility operators are integrated with Recursive Least Squares (RLS) to overcome the unknown variant parameters. RLS identifies transmissibilities used in the structure of noncausal FIR (Finite Impulse Response) models. While parameter variation can be treated as system nonlinearities, the RLS algorithm is used to optimize what time-variant dynamics to include in the transmissibility operator and what dynamics to push to the system nonlinearities over time. The identified transmissibilities are then used for the purpose of fault detection in an experimental robotic arm with variant picked mass

Fault-tolerant control strategies for a class of Euler-Lagrange nonlinear systems subject to simultaneous sensor and actuator faults

The problem of Fault Detection, Isolation, and Estimation (FDIE) as well as Fault-Tolerant Control (FTC) for a class of nonlinear systems modeled with Euler-Lagrange (EL) equations subjected to simultaneous sensor and actuator faults are considered in this study. To tackle this problem, first state and output linear transformations are introduced to decouple the effects of sensor and actuator faults. These transformations do not depend on the system nonlinearities. An analytical procedure based on two Linear Matrix Inequality (LMI) feasibility conditions is proposed to obtain these transformations. Once, the effects of faults are decoupled, two Sliding Mode Observers (SMO) are designed to reconstruct each type of fault, separately. Subsequently, the results of fault estimations are fed back to the controller and the effects of faults are compensated for. In this study, the mathematical stability proof of the coupled controller, observers, and the nonlinear system is provided. Unlike previous methodologies in the literature, no limiting assumptions such as Lipschitz conditions are imposed on the system. Next, a novel fault tolerant control scheme is proposed in which a single SMO is used to reconstruct sensor faults and provide a compensation term to rectify the effects of faults. However, to deal with actuator faults, a Sliding Mode Controller (SMC) is employed. Using this robust FTC technique, zero tracking error in the presence of uncertainties, measurement noise, disturbances, and faults as well as estimation of the actuator faults are possible. The stability proof for the coupled nonlinear controller, observer and plant is provided by using the properties of Euler-Lagrange equations and sliding mode techniques. Finally, to evaluate the performance of the proposed FDIE and FTC approaches, extensive sets of simulations are performed on a 3 Degrees Of Freedom (DOF) Autonomous Underwater Vehicle (AUV). Simulation studies show the promising results obtained as a result of the presented approaches as compared to those obtained by using the existing methodologies

Anti-disturbance fault tolerant initial alignment for inertial navigation system subjected to multiple disturbances

Modeling error, stochastic error of inertial sensor, measurement noise and environmental disturbance affect the accuracy of an inertial navigation system (INS). In addition, some unpredictable factors, such as system fault, directly affect the reliability of INSs. This paper proposes a new anti-disturbance fault tolerant alignment approach for a class of INSs sub- jected to multiple disturbances and system faults. Based on modeling and error analysis, stochastic error of inertial sensor, measurement noise, modeling error and environmental disturbance are formulated into different types of disturbances described by a Markov stochastic process, Gaussian noise and a norm-bounded variable, respectively. In order to improve the accuracy and reliability of an INS, an anti-disturbance fault tolerant filter is designed. Then, a mixed dissipative/guarantee cost performance is applied to attenuate the norm-bounded disturbance and to optimize the estimation error. Slack variables and dissipativeness are introduced to reduce the conservatism of the proposed approach. Finally, compared with the unscented Kalman filter (UKF), simulation results for self-alignment of an INS are provided based on experimental data. It can be shown that the proposed method has an enhanced disturbance rejection and attenuation performance with high reliability

Robust Nonlinear Control Design with Proportional-Integral-Observer Technique

In this thesis, the motivation is explained in the introduction part based on a review of the development of observer technique especially the Proportional-Integral-Observer (PI-Observer) technique and an analysis of robust control for nonlinear systems. After that, the goals of the work are set as: improvement of the high gain PI-Observer design in the way that the high observer gains of PI-Observer can be automatically chosen and fit the current situation; and the application of PI-Observer in robust nonlinear control for nonlinear systems with unknown effects. To the first point, the high-gain PI-Observer design is systematically discussed and the online adjustment of the PI-Observer gains proposed as an advanced PI-Observer (API-Observer) is detailed. At the same time, the implementation of the addressed adjustment algorithm is included showing the adaption of the PI-Observer gains to the current situation of system dynamics and disturbances on a practical system with simulation results and real measurements respectively. To the other aspect, a high gain PI-Observer-based nonlinear control method is proposed as a combination of the PI-Observer and the classical exact feedback linearization method (EFL-PIO) and directly after it the applicability on mechanical systems is surveyed. Numerical examples of mechanical systems are given to illustrate the application of the EFL-PIO approach. At last, the implementation of the proposed PI-Observer-based nonlinear control method is illustrated in detail on a hydraulic cylinder system for its position control. Besides that, the position control for the cylinder system without direct position measurement is realized with the PI-Observer applied as virtual sensor. Here a discrete-time control algorithm is also designed and programmed to satisfy the normal industrial requirement. At last, a broader application of the robust EFL-PIO approach, a more efficient and compact numerical realization of the adaption algorithm for the API-Observer, and an API-Observer-based robust control or fault detection are suggested as future work