Dynamic network loading: a stochastic differentiable model that derives link state distributions

We present a dynamic network loading model that yields queue length distributions, accounts for spillbacks, and maintains a differentiable mapping from the dynamic demand on the dynamic queue lengths. The model also captures the spatial correlation of all queues adjacent to a node, and derives their joint distribution. The approach builds upon an existing stationary queueing network model that is based on finite capacity queueing theory. The original model is specified in terms of a set of differentiable equations, which in the new model are carried over to a set of equally smooth difference equations. The physical correctness of the new model is experimentally confirmed in several congestion regimes. A comparison with results predicted by the kinematic wave model (KWM) shows that the new model correctly represents the dynamic build-up, spillback, and dissipation of queues. It goes beyond the KWM in that it captures queue lengths and spillbacks probabilistically, which allows for a richer analysis than the deterministic predictions of the KWM. The new model also generates a plausible fundamental diagram, which demonstrates that it captures well the stationary flow/density relationships in both congested and uncongested conditions

Statistical physics and information theory perspectives on complex systems and networks

Complex physical, biological, and sociotehnical systems often display various phenomena that can't be understood using traditional tools of single disciplines. We describe work on developing and applying theoretical methods to understand phenomena of this type, using statistical physics, networks, spectral graph theory, information theory, and geometry. Financial systems--being highly stochastic, with agents in a complex environment--offer a unique arena to develop and test new ways of thinking about complexity. We develop a framework for analyzing market dynamics motivated by linear response theory, and propose a model based on agent behavior that naturally incorporates external influences. We investigate central issues such as price dynamics, processing and incorporation of information, and how agent behavior influences stability. We find that the mean field behavior of our model captures important aspects of return dynamics, and identify a stable-unstable regime transition depending on easily measurable model parameters. Our methods naturally connect external factors to internal market features and behaviors, and therefore address the crucial question of how system stability relates to agent behavior and external forces. Complex systems are often interconnected heterogeneously, with subunits influencing others counterintuitively due to specific details of their connections. Correlations are insufficient to characterize this due to, e.g., being symmetric and unable to discern directional relationships. We synthesize ideas from information and network theory to introduce a general tool for studying such relations in networks. Based on transfer entropy, we propose a measure--Effective Transfer Entropy Dependency--that measures influence by considering precisely how much of a source node's influence on targets is due to intermediates. We apply this to indices of the world's major markets, finding that our measure anticipates same-day correlation structure from lagged time-series data, and identifies influencers not found using standard correlations. Graphs are essential for understanding complex systems and datasets. We present new methods for identifying important structure in graphs, based on ideas from quantum information theory and statistical mechanics, and the renormalization group. We apply information geometry and spectral geometry to study the geometric structures that arise from graphs and random graph models, and suggest future extensions and applications to important problems like graph partitioning and machine learning.2020-04-22T00:00:00

Finding patterns in timed data with spike timing dependent plasticity

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.Cataloged from PDF version of thesis.My research focuses on finding patterns in events - in sequences of data that happen over time. It takes inspiration from a neuroscience phenomena believed to be deeply involved in learning. I propose a machine learning algorithm that finds patterns in timed data and is highly robust to noise and missing data. It can find both coincident relationships, where two events tend to happen together; as well as causal relationships, where one event appears to be caused by another. I analyze stock price information using this algorithm and strong relationships are found between companies within the same industry. In particular, I worked with 12 stocks taken from the banking, information technology, healthcare, and oil industries. The relationships are almost exclusively coincidental, rather than causal.by Alexandre Oliveira.M.Eng

A Hybrid Framework for Sequential Data Prediction with End-to-End Optimization

We investigate nonlinear prediction in an online setting and introduce a hybrid model that effectively mitigates, via an end-to-end architecture, the need for hand-designed features and manual model selection issues of conventional nonlinear prediction/regression methods. In particular, we use recursive structures to extract features from sequential signals, while preserving the state information, i.e., the history, and boosted decision trees to produce the final output. The connection is in an end-to-end fashion and we jointly optimize the whole architecture using stochastic gradient descent, for which we also provide the backward pass update equations. In particular, we employ a recurrent neural network (LSTM) for adaptive feature extraction from sequential data and a gradient boosting machinery (soft GBDT) for effective supervised regression. Our framework is generic so that one can use other deep learning architectures for feature extraction (such as RNNs and GRUs) and machine learning algorithms for decision making as long as they are differentiable. We demonstrate the learning behavior of our algorithm on synthetic data and the significant performance improvements over the conventional methods over various real life datasets. Furthermore, we openly share the source code of the proposed method to facilitate further research

The effects of interbank networks on efficiency and stability in a macroeconomic agent-based model

We develop a macroeconomic agent-based model that consists of firms, banks, unions and households who interact on labour, goods, credit and interbank markets. The model endogenises pricing decisions by firms, wage setting by unions and interest rate setting by banks on both firm and interbank lending. Banks also set leverage targets and precautionary liquidity buffers on the basis of internal risk models. Our model produces endogenous fluctuations driven by the pricing behaviour of firms and the wage setting behaviour of unions. Fluctuations lead to loan defaults which are exacerbated as lenders reduce lending and charge higher interest rates, inducing a credit crunch. We also study how making the inter-banking network more connected affects the key outcomes of the economy and find that while the flow of funds from surplus banks to firms can be increased, the latter effect is soon dominated by increasing instability in the real sector as firms default at higher rates. While the banking sector experiences fewer defaults as a whole, losses on the interbank market increase as a source of bank defaults

Dynamic network loading: a differentiable model that derives link state distributions

We present a dynamic network loading model that yields queue length distributions, accounts for spillbacks, and maintains a differentiable mapping from the dynamic demand on the dynamic queue lengths. The approach builds upon an existing stationary queueing network model that is based on finite capacity queueing theory. The original model is specified in terms of a set of differentiable equations, which in the new model are carried over to a set of equally smooth difference equations. The physical correctness of the new model is experimentally confirmed in several congestion regimes. A comparison with results predicted by the kinematic wave model (KWM) shows that the new model correctly represents the dynamic build-up, spillback, and dissipation of queues. It goes beyond the KWM in that it captures queue lengths and spillbacks probabilistically, which allows for a richer analysis than the deterministic predictions of the KWM. The new model also generates a plausible fundamental diagram, which demonstrates that it captures well the stationary flow/density relationships in both congested and uncongested conditions

Information fusion architectures for security and resource management in cyber physical systems

Data acquisition through sensors is very crucial in determining the operability of the observed physical entity. Cyber Physical Systems (CPSs) are an example of distributed systems where sensors embedded into the physical system are used in sensing and data acquisition. CPSs are a collaboration between the physical and the computational cyber components. The control decisions sent back to the actuators on the physical components from the computational cyber components closes the feedback loop of the CPS. Since, this feedback is solely based on the data collected through the embedded sensors, information acquisition from the data plays an extremely vital role in determining the operational stability of the CPS. Data collection process may be hindered by disturbances such as system faults, noise and security attacks. Hence, simple data acquisition techniques will not suffice as accurate system representation cannot be obtained. Therefore, more powerful methods of inferring information from collected data such as Information Fusion have to be used. Information fusion is analogous to the cognitive process used by humans to integrate data continuously from their senses to make inferences about their environment. Data from the sensors is combined using techniques drawn from several disciplines such as Adaptive Filtering, Machine Learning and Pattern Recognition. Decisions made from such combination of data form the crux of information fusion and differentiates it from a flat structured data aggregation. In this dissertation, multi-layered information fusion models are used to develop automated decision making architectures to service security and resource management requirements in Cyber Physical Systems --Abstract, page iv

Machine Learning Approaches for Traffic Flow Forecasting

Intelligent Transport Systems (ITS) as a field has emerged quite rapidly in the recent years. A competitive solution coupled with big data gathered for ITS applications needs the latest AI to drive the ITS for the smart and effective public transport planning and management. Although there is a strong need for ITS applications like Advanced Route Planning (ARP) and Traffic Control Systems (TCS) to take the charge and require the minimum of possible human interventions. This thesis develops the models that can predict the traffic link flows on a junction level such as road traffic flows for a freeway or highway road for all traffic conditions. The research first reviews the state-of-the-art time series data prediction techniques with a deep focus in the field of transport Engineering along with the existing statistical and machine leaning methods and their applications for the freeway traffic flow prediction. This review setup a firm work focussed on the view point to look for the superiority in term of prediction performance of individual statistical or machine learning models over another. A detailed theoretical attention has been given, to learn the structure and working of individual chosen prediction models, in relation to the traffic flow data. In modelling the traffic flows from the real-world Highway England (HE) gathered dataset, a traffic flow objective function for highway road prediction models is proposed in a 3-stage framework including the topological breakdown of traffic network into virtual patches, further into nodes and to the basic links flow profiles behaviour estimations. The proposed objective function is tested with ten different prediction models including the statistical, shallow and deep learning constructed hybrid models for bi-directional links flow prediction methods. The effectiveness of the proposed objective function greatly enhances the accuracy of traffic flow prediction, regardless of the machine learning model used. The proposed prediction objective function base framework gives a new approach to model the traffic network to better understand the unknown traffic flow waves and the resulting congestions caused on a junction level. In addition, the results of applied Machine Learning models indicate that RNN variant LSTMs based models in conjunction with neural networks and Deep CNNs, when applied through the proposed objective function, outperforms other chosen machine learning methods for link flow predictions. The experimentation based practical findings reveal that to arrive at an efficient, robust, offline and accurate prediction model apart from feeding the ML mode with the correct representation of the network data, attention should be paid to the deep learning model structure, data pre-processing (i.e. normalisation) and the error matrices used for data behavioural learning. The proposed framework, in future can be utilised to address one of the main aims of the smart transport systems i.e. to reduce the error rates in network wide congestion predictions and the inflicted general traffic travel time delays in real-time

Response times in healthcare systems

It is a goal universally acknowledged that a healthcare system should treat its patients – and especially those in need of critical care – in a timely manner. However, this is often not achieved in practice, particularly in state-run public healthcare systems that suffer from high patient demand and limited resources. In particular, Accident and Emergency (A&E) departments in England have been placed under increasing pressure, with attendances rising year on year, and a national government target whereby 98% of patients should spend 4 hours or less in an A&E department from arrival to admission, transfer or discharge. This thesis presents techniques and tools to characterise and forecast patient arrivals, to model patient flow and to assess the response-time impact of different resource allocations, patient treatment schemes and workload scenarios. Having obtained ethical approval to access five years of pseudonymised patient timing data from a large case study A&E department, we present a number of time series models that characterise and forecast daily A&E patient arrivals. Patient arrivals are classified as one of two arrival streams (walk-in and ambulance) by mode of arrival. Using power spectrum analysis, we find the two arrival streams exhibit different statistical properties and hence require separate time series models. We find that structural time series models best characterise and forecast walk-in arrivals, but that time series analysis may not be appropriate for ambulance arrivals; this prompts us to investigate characterisation by a non-homogeneous Poisson process. Next we present a hierarchical multiclass queueing network model of patient flow in our case study A&E department. We investigate via a discrete-event simulation the impact of class and time-based priority treatment of patients, and compare the resulting service-time densities and moments with actual data. Then, by performing bottleneck analysis and investigating various workload and resource scenarios, we pinpoint the resources that have the greatest impact on mean service times. Finally we describe an approximate generating function analysis technique which efficiently approximates the first two moments of customer response time in class-dependent priority queueing networks with population constraints. This technique is applied to the model of A&E and the results compared with those from simulation. We find good agreement for mean service times especially when minors patients are given priority

From statistical- to machine learning-based network traffic prediction

Nowadays, due to the exponential and continuous expansion of new paradigms such as Internet of Things (IoT), Internet of Vehicles (IoV) and 6G, the world is witnessing a tremendous and sharp increase of network traffic. In such large-scale, heterogeneous, and complex networks, the volume of transferred data, as big data, is considered a challenge causing different networking inefficiencies. To overcome these challenges, various techniques are introduced to monitor the performance of networks, called Network Traffic Monitoring and Analysis (NTMA). Network Traffic Prediction (NTP) is a significant subfield of NTMA which is mainly focused on predicting the future of network load and its behavior. NTP techniques can generally be realized in two ways, that is, statistical- and Machine Learning (ML)-based. In this paper, we provide a study on existing NTP techniques through reviewing, investigating, and classifying the recent relevant works conducted in this field. Additionally, we discuss the challenges and future directions of NTP showing that how ML and statistical techniques can be used to solve challenges of NTP.publishedVersio