Sequential Modelling and Inference of High-frequency Limit Order Book with State-space Models and Monte Carlo Algorithms

The high-frequency limit order book (LOB) market has recently attracted increasing research attention from both the industry and the academia as a result of expanding algorithmic trading. However, the massive data throughput and the inherent complexity of high-frequency market dynamics also present challenges to some classic statistical modelling approaches. By adopting powerful state-space models from the field of signal processing as well as a number of Bayesian inference algorithms such as particle filtering, Markov chain Monte Carlo and variational inference algorithms, this thesis presents my extensive research into the high-frequency limit order book covering a wide scope of topics. Chapter 2 presents a novel construction of the non-homogeneous Poisson process to allow online intensity inference of limit order transactions arriving at a central exchange as point data. Chapter 3 extends a baseline jump diffusion model for market fair-price process to include three additional model features taken from real-world market intuitions. In Chapter 4, another price model is developed to account for both long-term and short-term diffusion behaviours of the price process. This is achieved by incorporating multiple jump-diffusion processes each exhibiting a unique characteristic. Chapter 5 observes the multi-regime nature of price diffusion processes as well as the non-Markovian switching behaviour between regimes. As such, a novel model is proposed which combines the continuous-time state-space model, the hidden semi-Markov switching model and the non-parametric Dirichlet process model. Additionally, building upon the general structure of the particle Markov chain Monte Carlo algorithm, I further propose an algorithm which achieves sequential state inference, regime identification and regime parameters learning requiring minimal prior assumptions. Chapter 6 focuses on the development of efficient parameter-learning algorithms for state-space models and presents three algorithms each demonstrating promising results in comparison to some well-established methods. The models and algorithms proposed in this thesis not only are practical tools for analysing high-frequency LOB markets, but can also be applied in various areas and disciplines beyond finance

Retrospective Cost Adaptive Unknown Input Observers with Application to State and Driver Estimation in the Ionosphere-Thermosphere.

The classical Kalman filter is the optimal state estimator for linear systems under white process and sensor noise with zero mean and finite second moments. In addition, the Kalman filter accommodates the presence of a known, deterministic input. In practice, however, the deterministic input may not be known exactly, and this error can be viewed as a component of the process noise. However, this approach may be too conservative and can lead to bias when the unknown input has a nonzero ``mean'' value. Consequently, a more direct approach is to extend the estimator to include an estimate of the unknown input. In this work, we consider an unknown input observer based on retrospective cost optimization, where the unknown input is estimated by first minimizing a retrospective cost function, and then updating an adaptive feedback system using recursive least squares. The retrospective cost method is a minimal modeling approach that is applicable to both minimum- and nonminimum-phase systems. Since the retrospective cost observer relies on recursive least squares to update an adaptive feedback system, a novel sliding window, variable regularization recursive least squares algorithm is developed and investigated. In contrast to classical recursive least squares algorithms, the sliding window recursive least squares algorithm does not lose its ability to adapt, and does not become unstable when the data lose persistency.PhDAerospace EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/99995/1/asadali_1.pd

A Fault Tolerant System for an Integrated Avionics Sensor Configuration

An aircraft sensor fault tolerant system methodology for the Transport Systems Research Vehicle in a Microwave Landing System (MLS) environment is described. The fault tolerant system provides reliable estimates in the presence of possible failures both in ground-based navigation aids, and in on-board flight control and inertial sensors. Sensor failures are identified by utilizing the analytic relationships between the various sensors arising from the aircraft point mass equations of motion. The estimation and failure detection performance of the software implementation (called FINDS) of the developed system was analyzed on a nonlinear digital simulation of the research aircraft. Simulation results showing the detection performance of FINDS, using a dual redundant sensor compliment, are presented for bias, hardover, null, ramp, increased noise and scale factor failures. In general, the results show that FINDS can distinguish between normal operating sensor errors and failures while providing an excellent detection speed for bias failures in the MLS, indicated airspeed, attitude and radar altimeter sensors

Multi-Robot FastSLAM for Large Domains

For a robot to build a map of its surrounding area, it must have accurate position information within the area, and to obtain accurate position information within the area, the robot needs to have an accurate map of the area. This circular problem is the Simultaneous Localization and Mapping (SLAM) problem. An efficient algorithm to solve it is FastSLAM, which is based on the Rao-Blackwellized particle filter. FastSLAM solves the SLAM problem for single-robot mapping using particles to represent the posterior of the robot pose and the map. Each particle of the filter possesses its own global map which is likely to be a grid map. The memory space required for these maps poses a serious limitation to the algorithm\u27s capability when the problem space is large. The problem will only get worse if the algorithm is adapted to multi-robot mapping. This thesis presents an alternate mapping algorithm that extends the single-robot FastSLAM algorithm to a multi-robot mapping algorithm that uses Absolute Space Representations (ASR) to represent the world. But each particle still maintains a local grid to map its vicinity and periodically this grid map is converted into an ASR. An ASR expresses a world in polygons requiring only a minimal amount of memory space. By using this altered mapping strategy, the problem faced in FastSLAM when mapping a large domain can be alleviated. In this algorithm, each robot maps separately, and when two robots encounter each other they exchange range and odometry readings from their last encounter to this encounter. Each robot then sets up another filter for the other robot\u27s data and incrementally updates its own map, incorporating the passed data and its own data at the same time. The passed data is processed in reverse by the receiving robot as if a virtual robot is back-tracking the path of the other robot. The algorithm is demonstrated using three data sets collected using a single robot equipped with odometry and laser-range finder sensors

Decoding Electrophysiological Correlates of Selective Attention by Means of Circular Data

Sustaining our attention to a relevant sensory input in a complex listening environment, is of great importance for a successful auditory communication. To avoid the overload of the auditory system, the importance of the stimuli is estimated in the higher levels of the auditory system. Based on these information, the attention is drifted away from the irrelevant and unimportant stimuli. Long-term habituation, a gradual process independent from sensory adaptation, plays a major role in drifting away our attention from irrelevant stimuli. A better understanding of attention-modulated neural activity is important for shedding light on the encoding process of auditory streams. For instance, these information can have a direct impact on developing smarter hearing aid devices in which more accurate objective measures can be used to re ect the hearing capabilities of patients with hearing pathologies. As an example, an objective measures of long-term habituation with respect to di erent level of sound stimuli can be used more accurately for adjustment of hearing aid devices in comparison to verbal reports. The main goal of this thesis is to analyze the neural decoding signatures of long-term habituation and neural modulations of selective attention by exploiting circular regularities in electrophysiological (EEG) data, in which we can objectively measure the level of attentional-binding to di erent stimuli. We study, in particular, the modulations of the instantaneous phase (IP) in event related potentials (ERPs) over trials for di erent experimental settings. This is in contrast to the common approach where the ERP component of interest is computed through averaging a su ciently large number of ERP trials. It is hypothesized that a high attentional binding to a stimulus is related to a high level of IP cluster. As the attention binding reduces, IP is spread more uniformly on a unit circle. This work is divided into three main parts. In the initial part, we investigate the dynamics of long-term habituation with di erent acoustical stimuli (soft vs. loud) over ERP trials. The underlying temporal dynamics in IP and the level of phase cluster of the ERPs are assessed by tting circular probability functions (pdf) over data segments. To increase the temporal resolution of detecting times at which a signi cant change in IP occurs, an abrupt change point model at di erent pure-tone stimulations is used. In a second study, we improve upon the results and methodology by relaxing some of the constrains in order to integrate the gradual process of long-term habituation into the model. For this means, a Bayesian state-space model is proposed. In all of the aforementioned studies, we successfully classi ed between di erent stimulation levels, using solely the IP of ERPs over trials. In the second part of the thesis, the experimental setting is expanded to contain longer and more complex auditory stimuli as in real-world scenarios. Thereby, we study the neural-correlates of attention in spontaneous modulations of EEG (ongoing activity) which uses the complete temporal resolution of the signal. We show a mapping between the ERP results and the ongoing EEG activity based on IP. A Markov-based model is developed for removing spurious variations that can occur in ongoing signals. We believe the proposed method can be incorporated as an important preprocessing step for a more reliable estimation of objective measures of the level of selective attention. The proposed model is used to pre-process and classify between attending and un-attending states in a seminal dichotic tone detection experiment. In the last part of this thesis, we investigate the possibility of measuring a mapping between the neural activities of the cortical laminae with the auditory evoked potentials (AEP) in vitro. We show a strong correlation between the IP of AEPs and the neural activities at the granular layer, using mutual information.Die Aufmerksamkeit auf ein relevantes auditorisches Signal in einer komplexen H orumgebung zu lenken ist von gro er Bedeutung f ur eine erfolgreiche akustische Kommunikation. Um eine Uberlastung des H orsystems zu vermeiden, wird die Bedeutung der Reize in den h oheren Ebenen des auditorischen Systems bewertet. Basierend auf diesen Informationen wird die Aufmerksamkeit von den irrelevanten und unwichtigen Reizen abgelenkt. Dabei spielt die sog. Langzeit- Habituation, die einen graduellen Prozess darstellt der unabh angig von der sensorischen Adaptierung ist, eine wichtige Rolle. Ein besseres Verst andnis der aufmerksamkeits-modulierten neuronalen Aktivit at ist wichtig, um den Kodierungsprozess von sog. auditory streams zu beleuchten. Zum Beispiel k onnen diese Informationen einen direkten Ein uss auf die Entwicklung intelligenter H orsysteme haben bei denen genauere, objektive Messungen verwendet werden k onnen, um die H orf ahigkeiten von Patienten mit H orpathologien widerzuspiegeln. So kann beispielsweise ein objektives Ma f ur die Langzeit- Habituation an unterschiedliche Schallreize genutzt werden um - im Vergleich zu subjektiven Selbsteinsch atzungen - eine genauere Anpassung der H orsysteme zu erreichen. Das Hauptziel dieser Dissertation ist die Analyse neuronaler Dekodierungssignaturen der Langzeit- Habituation und neuronaler Modulationen der selektiver Aufmerksamkeit durch Nutzung zirkul arer Regularit aten in elektroenzephalogra schen Daten, in denen wir objektiv den Grad der Aufmerksamkeitsbindung an verschiedene Reize messen k onnen. Wir untersuchen insbesondere die Modulation der Momentanphase (engl. Instantaneous phase, IP) in ereigniskorrelierten Potenzialen (EKPs) in verschiedenen experimentellen Settings. Dies steht im Gegensatz zu dem traditionellen Ansatz, bei dem die interessierenden EKP-Komponenten durch Mittelung einer ausreichend gro en Anzahl von Einzelantworten im Zeitbereich ermittelt werden. Es wird vermutet, dass eine hohe Aufmerksamkeitsbindung an einen Stimulus mit einem hohen Grad an IP-Clustern verbunden ist. Nimmt die Aufmerksamkeitsbindung hingegen ab, so ist die Momentanphase uniform auf dem Einheitskreis verteilt. Diese Arbeit gliedert sich in drei Teile. Im ersten Teil untersuchen wir die Dynamik der Langzeit-Habituation mit verschiedenen akustischen Reizen (leise vs. laut) in EKP-Studien. Die zugrundeliegende zeitliche Dynamik der Momentanphase und die Ebene des Phasenclusters der EKPs werden durch die Anpassung von zirkul aren Wahrscheinlichkeitsfunktionen (engl. probability density function, pdf) uber Datensegmente bewertet. Mithilfe eines sog. abrupt change-point Modells wurde die zeitliche Au osung der Daten erh oht, sodass signi kante Anderungen in der Momentanphase bei verschiedenen Reintonstimulationen detektierbar sind. In einer zweiten Studie verbessern wir die Ergebnisse und die Methodik, indem wir einige der Einschr ankungen lockern, um den gradualen Prozess der Langzeit-Habituation in das abrupt changepoint Modell zu integrieren. Dazu wird ein bayes`sches Zustands-Raum-Modell vorgeschlagen. In den zuvor genannten Studien konnte erfolgreich mithilfe der Momentanphase zwischen verschiedenen Stimulationspegeln unterschieden werden. Im zweiten Teil der Arbeit wird der experimentelle Rahmen erweitert, um komplexere auditorische Reize wie in realen H orsituationen untersuchen zu k onnen. Dabei analysieren wir die neuronalen Korrelate der Aufmerksamkeit anhand spontaner Modulationen der kontinuierlichen EEG-Aktivit at, die eine zeitliche Au osung erm oglicht. Wir zeigen eine Abbildung zwischen den EKP-Ergebnissen und der kontinuierlichen EEG-Aktivit at auf Basis der Momentanphase. Ein Markov-basiertes Modell wird entwickelt, um st orende Variationen zu entfernen, die in kontinuierlichen EEG-Signalen auftreten k onnen. Wir glauben, dass die vorgeschlagene Methode als wichtiger Vorverarbeitungsschritt zur soliden objektiven Absch atzung des Aufmerksamkeitsgrades mithilfe von EEG-Daten verwendet werden kann. In einem dichotischen Tonerkennungsexperiment wird das vorgeschlagene Modell zur Vorverarbeitung der EEG-Daten und zur Klassi zierung zwischen gerichteten und ungerichteten Aufmerksamkeitszust anden erfolgreich verwendet. Im letzten Teil dieser Arbeit untersuchen wir den Zusammenhang zwischen den neuronalen Aktivit aten der kortikalen Laminae und auditorisch evozierten Potentialen (AEP) in vitro im Tiermodell. Wir zeigen eine starke Korrelation zwischen der Momentanphase der AEPs und den neuronalen Aktivit aten in der Granularschicht unter Verwendung der Transinformation

Developing models for the data-based mechanistic approach to systems analysis:Increasing objectivity and reducing assumptions

Stochastic State-Space Time-Varying Random Walk models have been developed, allowing the existing Stochastic State Space models to operate directly on irregularly sampled time-series. These TVRW models have been successfully applied to two different classes of models benefiting each class in different ways. The first class of models - State Dependent Parameter (SDP) models and used to investigate the dominant dynamic modes of nonlinear dynamic systems and the non-linearities in these models affected by arbitrary State Variables. In SDP locally linearised models it is assumed that the parameters that describe system’s behaviour changes are dependent upon some aspect of the system (it’s ‘state’). Each parameter can be dependent on one or more states. To estimate the parameters that are changing at a rate related to that of it’s states, the estimation procedure is conducted in the state-space along the potentially multivariate trajectory of the states which drive the parameters. The introduction of the newly developed TVRW models significantly improves parameter estimation, particularly in data rich neighbourhoods of the state-space when the parameter is dependent on more than one state, and the ends of the data-series when the parameter is dependent on one state with few data points. The second class of models are known as Dynamic Harmonic Regression (DHR) models and are used to identify the dominant cycles and trends of time-series. DHR models the assumption is that a signal (such as a time-series) can be broken down into four (unobserved) components occupying different parts of the spectrum: trend, seasonal cycle, other cycles, and a high frequency irregular component. DHR is confined to uniformly sampled time-series. The introduction of the TVRW models allows DHR to operate on irregularly sampled time-series, with the added benefit of forecasting origin no longer being confined to starting at the end of the time-series but can now begin at any point in the future. Additionally, the forecasting sampling rate is no longer limited to the sampling rate of the time-series. Importantly, both classes of model were designed to follow the Data-Based Mechanistic (DBM) approach to modelling environmental systems, where the model structure and parameters are to be determined by the data (Data-Based) and then the subsequent models are to be validated based on their physical interpretation (Mechanistic). The aim is to remove the researcher’s preconceptions from model development in order to eliminate any bias, and then use the researcher’s knowledge to validate the models presented to them. Both classes of model lacked model structure identification procedures and so model structure was determined by the researcher, against the DBM approach. Two different model structure identification procedures, one for SDP and the other for DHR, were developed to bring both classes of models back within the DBM framework. These developments have been presented and tested here on both simulated data and real environmental data, demonstrating their importance, benefits and role in environmental modelling and exploratory data analysis

Information fusion architectures for security and resource management in cyber physical systems

Data acquisition through sensors is very crucial in determining the operability of the observed physical entity. Cyber Physical Systems (CPSs) are an example of distributed systems where sensors embedded into the physical system are used in sensing and data acquisition. CPSs are a collaboration between the physical and the computational cyber components. The control decisions sent back to the actuators on the physical components from the computational cyber components closes the feedback loop of the CPS. Since, this feedback is solely based on the data collected through the embedded sensors, information acquisition from the data plays an extremely vital role in determining the operational stability of the CPS. Data collection process may be hindered by disturbances such as system faults, noise and security attacks. Hence, simple data acquisition techniques will not suffice as accurate system representation cannot be obtained. Therefore, more powerful methods of inferring information from collected data such as Information Fusion have to be used. Information fusion is analogous to the cognitive process used by humans to integrate data continuously from their senses to make inferences about their environment. Data from the sensors is combined using techniques drawn from several disciplines such as Adaptive Filtering, Machine Learning and Pattern Recognition. Decisions made from such combination of data form the crux of information fusion and differentiates it from a flat structured data aggregation. In this dissertation, multi-layered information fusion models are used to develop automated decision making architectures to service security and resource management requirements in Cyber Physical Systems --Abstract, page iv