Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available

Exploring space situational awareness using neuromorphic event-based cameras

The orbits around earth are a limited natural resource and one that hosts a vast range of vital space-based systems that support international systems use by both commercial industries, civil organisations, and national defence. The availability of this space resource is rapidly depleting due to the ever-growing presence of space debris and rampant overcrowding, especially in the limited and highly desirable slots in geosynchronous orbit. The field of Space Situational Awareness encompasses tasks aimed at mitigating these hazards to on-orbit systems through the monitoring of satellite traffic. Essential to this task is the collection of accurate and timely observation data. This thesis explores the use of a novel sensor paradigm to optically collect and process sensor data to enhance and improve space situational awareness tasks. Solving this issue is critical to ensure that we can continue to utilise the space environment in a sustainable way. However, these tasks pose significant engineering challenges that involve the detection and characterisation of faint, highly distant, and high-speed targets. Recent advances in neuromorphic engineering have led to the availability of high-quality neuromorphic event-based cameras that provide a promising alternative to the conventional cameras used in space imaging. These cameras offer the potential to improve the capabilities of existing space tracking systems and have been shown to detect and track satellites or ‘Resident Space Objects’ at low data rates, high temporal resolutions, and in conditions typically unsuitable for conventional optical cameras. This thesis presents a thorough exploration of neuromorphic event-based cameras for space situational awareness tasks and establishes a rigorous foundation for event-based space imaging. The work conducted in this project demonstrates how to enable event-based space imaging systems that serve the goals of space situational awareness by providing accurate and timely information on the space domain. By developing and implementing event-based processing techniques, the asynchronous operation, high temporal resolution, and dynamic range of these novel sensors are leveraged to provide low latency target acquisition and rapid reaction to challenging satellite tracking scenarios. The algorithms and experiments developed in this thesis successfully study the properties and trade-offs of event-based space imaging and provide comparisons with traditional observing methods and conventional frame-based sensors. The outcomes of this thesis demonstrate the viability of event-based cameras for use in tracking and space imaging tasks and therefore contribute to the growing efforts of the international space situational awareness community and the development of the event-based technology in astronomy and space science applications

Point process modeling and estimation: advances in the analysis of dynamic neural spiking data

A common interest of scientists in many fields is to understand the relationship between the dynamics of a physical system and the occurrences of discrete events within such physical system. Seismologists study the connection between mechanical vibrations of the Earth and the occurrences of earthquakes so that future earthquakes can be better predicted. Astrophysicists study the association between the oscillating energy of celestial regions and the emission of photons to learn the Universe's various objects and their interactions. Neuroscientists study the link between behavior and the millisecond-timescale spike patterns of neurons to understand higher brain functions. Such relationships can often be formulated within the framework of state-space models with point process observations. The basic idea is that the dynamics of the physical systems are driven by the dynamics of some stochastic state variables and the discrete events we observe in an interval are noisy observations with distributions determined by the state variables. This thesis proposes several new methodological developments that advance the framework of state-space models with point process observations at the intersection of statistics and neuroscience. In particular, we develop new methods 1) to characterize the rhythmic spiking activity using history-dependent structure, 2) to model population spike activity using marked point process models, 3) to allow for real-time decision making, and 4) to take into account the need for dimensionality reduction for high-dimensional state and observation processes. We applied these methods to a novel problem of tracking rhythmic dynamics in the spiking of neurons in the subthalamic nucleus of Parkinson's patients with the goal of optimizing placement of deep brain stimulation electrodes. We developed a decoding algorithm that can make decision in real-time (for example, to stimulate the neurons or not) based on various sources of information present in population spiking data. Lastly, we proposed a general three-step paradigm that allows us to relate behavioral outcomes of various tasks to simultaneously recorded neural activity across multiple brain areas, which is a step towards closed-loop therapies for psychological diseases using real-time neural stimulation. These methods are suitable for real-time implementation for content-based feedback experiments

Tracking times in temporal patterns embodied in intra-cortical data for controling neural prosthesis an animal simulation study

Brain-machines capture brain signals in order to restore communication and movement to disabled people who suffer from brain palsy or motor disorders. In brain regions, the ensemble firing of populations of neurons represents spatio-temporal patterns that are transformed into outgoing spatio-temporal patterns which encode complex cognitive task. This transformation is dynamic, non-stationary (time-varying) and highly nonlinear. Hence, modeling such complex biological patterns requires specific model structures to uncover the underlying physiological mechanisms and their influences on system behavior. In this study, a recent multi-electrode technology allows the record of the simultaneous neuron activities in behaving animals. Intra-cortical data are processed according to these steps: spike detection and sorting, than desired action extraction from the rate of the obtained signal. We focus on the following important questions about (i) the possibility of linking the brain signal time events with some time-delayed mapping tools; (ii) the use of some suitable inputs than others for the decoder; (iii) a consideration of separated data or a special representation founded on multi-dimensional statistics. This paper concentrates mostly on the analysis of parallel spike train when certain critical hypotheses are ignored by the data for the working method. We have made efforts to define explicitly whether the underlying hypotheses are actually achieved. In this paper, we propose an algorithm to define the embedded memory order of NARX recurrent neural networks to the hand trajectory tracking process. We also demonstrate that this algorithm can improve performance on inference tasks

Deep Learning Assisted Intelligent Visual and Vehicle Tracking Systems

Sensor fusion and tracking is the ability to bring together measurements from multiple sensors of the current and past time to estimate the current state of a system. The resulting state estimate is more accurate compared with the direct sensor measurement because it balances between the state prediction based on the assumed motion model and the noisy sensor measurement. Systems can then use the information provided by the sensor fusion and tracking process to support more-intelligent actions and achieve autonomy in a system like an autonomous vehicle. In the past, widely used sensor data are structured, which can be directly used in the tracking system, e.g., distance, temperature, acceleration, and force. The measurements\u27 uncertainty can be estimated from experiments. However, currently, a large number of unstructured data sources can be generated from sensors such as cameras and LiDAR sensors, which bring new challenges to the fusion and tracking system. The traditional algorithm cannot directly use these unstructured data, and it needs another method or process to “understand” them first. For example, if a system tries to track a particular person in a video sequence, it needs to understand where the person is in the first place. However, the traditional tracking method cannot finish such a task. The measurement model for unstructured data is usually difficult to construct. Deep learning techniques provide promising solutions to this type of problem. A deep learning method can learn and understand the unstructured data to accomplish tasks such as object detection in images, object localization in LiDAR point clouds, and driver behavior prediction from the current traffic conditions. Deep-learning architectures such as deep neural networks, deep belief networks, recurrent neural networks, and convolutional neural networks have been applied to fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, and machine translation, where they have produced results comparable with human expert performance. How to incorporate information obtained via deep learning into our tracking system is one of the topics of this dissertation. Another challenging task is using learning methods to improve a tracking filter\u27s performance. In a tracking system, many manually tuned system parameters affect the tracking performance, e.g., the process noise covariance and measurement noise covariance in a Kalman Filter (KF). These parameters used to be estimated by running the tracking algorithm several times and selecting the one that gives the optimal performance. How to learn the system parameters automatically from data, and how to use machine learning techniques directly to provide useful information to the tracking systems are critical to the proposed tracking system. The proposed research on the intelligent tracking system has two objectives. The first objective is to make a visual tracking filter smart enough to understand unstructured data sources. The second objective is to apply learning algorithms to improve a tracking filter\u27s performance. The goal is to develop an intelligent tracking system that can understand the unstructured data and use the data to improve itself

Real-Time, Multiple Pan/Tilt/Zoom Computer Vision Tracking and 3D Positioning System for Unmanned Aerial System Metrology

The study of structural characteristics of Unmanned Aerial Systems (UASs) continues to be an important field of research for developing state of the art nano/micro systems. Development of a metrology system using computer vision (CV) tracking and 3D point extraction would provide an avenue for making these theoretical developments. This work provides a portable, scalable system capable of real-time tracking, zooming, and 3D position estimation of a UAS using multiple cameras. Current state-of-the-art photogrammetry systems use retro-reflective markers or single point lasers to obtain object poses and/or positions over time. Using a CV pan/tilt/zoom (PTZ) system has the potential to circumvent their limitations. The system developed in this paper exploits parallel-processing and the GPU for CV-tracking, using optical flow and known camera motion, in order to capture a moving object using two PTU cameras. The parallel-processing technique developed in this work is versatile, allowing the ability to test other CV methods with a PTZ system using known camera motion. Utilizing known camera poses, the object\u27s 3D position is estimated and focal lengths are estimated for filling the image to a desired amount. This system is tested against truth data obtained using an industrial system

On adaptive control and particle filtering in the automatic administration of medicinal drugs

Automatic feedback methodologies for the administration of medicinal drugs offer undisputed potential benefits in terms of cost reduction and improved clinical outcomes. However, despite several decades of research, the ultimate safety of many--it would be fair to say most--closed-loop drug delivery approaches remains under question and manual methods based on clinicians' expertise are still dominant in clinical practice. Key challenges to the design of control systems for these applications include uncertainty in pharmacological models, as well as intra- and interpatient variability in the response to drug administration. Pharmacological systems may feature nonlinearities, time delays, time-varying parameters and non-Gaussian stochastic processes. This dissertation investigates a novel multi-controller adaptive control strategy capable of delivering safe control for closed-loop drug delivery applications without impairing clinicians' ability to make an expert assessment of a clinical situation. Our new feedback control approach, which we have named Robust Adaptive Control with Particle Filtering (RAC-PF), estimates a patient's individual response characteristic in real-time through particle filtering and uses the Bayesian inference result to select the most suitable controller for closed-loop operation from a bank of candidate controllers designed using the robust methodology of mu-synthesis. The work is presented as four distinct pieces of research. We first apply the existing approach of Robust Multiple-Model Adaptive Control (RMMAC), which features robust controllers and Kalman filter estimators, to the case-study of administration of the vasodepressor drug sodium nitroprusside and examine benefits and drawbacks. We then consider particle filtering as an alternative to Kalman filter-based methods for the real-time estimation of pharmacological dose-response, and apply this to the nonlinear pharmacokinetic-pharmacodynamic model of the anaesthetic drug propofol. We ultimately combine particle filters and robust controllers to create RAC-PF, and test our novel approach first in a proof-of-concept design and finally in the case of sodium nitroprusside. The results presented in the dissertation are based on computational studies, including extensive Monte-Carlo simulation campaigns. Our findings of improved parameter estimates from noisy observations support the use of particle filtering as a viable tool for real-time Bayesian inference in pharmacological system identification. The potential of the RAC-PF approach as an extension of RMMAC for closed-loop control of a broader class of systems is also clearly highlighted, with the proposed new approach delivering safe control of acute hypertension through sodium nitroprusside infusion when applied to a very general population response model. All approaches presented are generalisable and may be readily adapted to other drug delivery instances