Estimation, diagnostics, and extensions of nonparametric Hawkes processes

ISM Online Open House, 2020.10.27統計数理研究所オープンハウス（オンライン開催）、R2.10.27ポスター発

Point Process Models for Heterogeneous Event Time Data

Interaction event times observed on a social network provide valuable information for social scientists to gain insight into complex social dynamics that are challenging to understand. However, it can be difficult to accurately represent the heterogeneity in the data and to model the dependence structure in the network system. This requires flexible models that can capture the complicated dynamics and complex patterns. Point process models offer an elegant framework for modeling event time data. This dissertation concentrates on developing point process models and related diagnostic tools, with a real data application involving an animal behavior network. In this dissertation, we first propose a Markov-modulated Hawkes process (MMHP) model to capture the sporadic and bursty patterns often observed in event time data. A Bayesian inference procedure is developed to evaluate the likelihood by using a variational approximation and the forward-backward algorithm. The validity of the proposed model and associated estimation algorithms is demonstrated using synthetic data and the animal behavior data. Facilitated by the power of the MMHP model, we construct network point process models that can capture a social hierarchy structure by embedding nodes in a latent space that can represent the underlying social ranks. Our model provides a ranking method for social hierarchy studies and describes the dynamics of social hierarchy formation from a novel perspective – taking advantage of the detailed information available in event time data. We show that the network point process models appropriately captures the temporal dynamics and heterogeneity in the network event time data, by providing meaningful inferred rankings and by calibrating the accuracy of predictions with relevant measures of uncertainty. In addition to developing a sensible and flexible model for network event time data, the last part of this dissertation provides essential tools for diagnosing lack of fit issues for such models. We develop a systematic set of diagnostic tools and visualizations for point process models fitted to data in the dynamic network setting. By inspecting the structure of the residual process and Pearson residual on the network, we can validate whether a model adequately captures the temporal and network dependence structures in the observed data

Flexible estimation of temporal point processes and graphs

Handling complex data types with spatial structures, temporal dependencies, or discrete values, is generally a challenge in statistics and machine learning. In the recent years, there has been an increasing need of methodological and theoretical work to analyse non-standard data types, for instance, data collected on protein structures, genes interactions, social networks or physical sensors. In this thesis, I will propose a methodology and provide theoretical guarantees for analysing two general types of discrete data emerging from interactive phenomena, namely temporal point processes and graphs. On the one hand, temporal point processes are stochastic processes used to model event data, i.e., data that comes as discrete points in time or space where some phenomenon occurs. Some of the most successful applications of these discrete processes include online messages, financial transactions, earthquake strikes, and neuronal spikes. The popularity of these processes notably comes from their ability to model unobserved interactions and dependencies between temporally and spatially distant events. However, statistical methods for point processes generally rely on estimating a latent, unobserved, stochastic intensity process. In this context, designing flexible models and consistent estimation methods is often a challenging task. On the other hand, graphs are structures made of nodes (or agents) and edges (or links), where an edge represents an interaction or relationship between two nodes. Graphs are ubiquitous to model real-world social, transport, and mobility networks, where edges can correspond to virtual exchanges, physical connections between places, or migrations across geographical areas. Besides, graphs are used to represent correlations and lead-lag relationships between time series, and local dependence between random objects. Graphs are typical examples of non-Euclidean data, where adequate distance measures, similarity functions, and generative models need to be formalised. In the deep learning community, graphs have become particularly popular within the field of geometric deep learning. Structure and dependence can both be modelled by temporal point processes and graphs, although predominantly, the former act on the temporal domain while the latter conceptualise spatial interactions. Nonetheless, some statistical models combine graphs and point processes in order to account for both spatial and temporal dependencies. For instance, temporal point processes have been used to model the birth times of edges and nodes in temporal graphs. Moreover, some multivariate point processes models have a latent graph parameter governing the pairwise causal relationships between the components of the process. In this thesis, I will notably study such a model, called the Hawkes model, as well as graphs evolving in time. This thesis aims at designing inference methods that provide flexibility in the contexts of temporal point processes and graphs. This manuscript is presented in an integrated format, with four main chapters and two appendices. Chapters 2 and 3 are dedicated to the study of Bayesian nonparametric inference methods in the generalised Hawkes point process model. While Chapter 2 provides theoretical guarantees for existing methods, Chapter 3 also proposes, analyses, and evaluates a novel variational Bayes methodology. The other main chapters introduce and study model-free inference approaches for two estimation problems on graphs, namely spectral methods for the signed graph clustering problem in Chapter 4, and a deep learning algorithm for the network change point detection task on temporal graphs in Chapter 5. Additionally, Chapter 1 provides an introduction and background preliminaries on point processes and graphs. Chapter 6 concludes this thesis with a summary and critical thinking on the works in this manuscript, and proposals for future research. Finally, the appendices contain two supplementary papers. The first one, in Appendix A, initiated after the COVID-19 outbreak in March 2020, is an application of a discrete-time Hawkes model to COVID-related deaths counts during the first wave of the pandemic. The second work, in Appendix B, was conducted during an internship at Amazon Research in 2021, and proposes an explainability method for anomaly detection models acting on multivariate time series

Bayesian point processes models with applications in the COVID-19 pandemic

A point process is a set of points randomly located in a space, such as time or abstract spaces. Point process models have found numerous applications in epidemiology, ecology, geophysics, social networks and many other areas. The Poisson process is the most widely known point process. Poisson intensity estimation is a vital task in various applications including medical imaging, astrophysics and network traffic analysis. A Bayesian Additive Regression Trees (BART) scheme for estimating the intensity of inhomogeneous Poisson processes is introduced. The new approach enables full posterior inference of the intensity in a non-parametric regression setting. The performance of the novel scheme is demonstrated through simulation studies on synthetic and real datasets up to five dimensions, and the new scheme is compared with alternative approaches. A drawback of the proposed algorithm is its axis-alignment nature. We discuss this problem and suggest alternative approaches to remedy the drawback. The novel coronavirus disease (COVID-19) has been declared a Global Health Emergency of International Concern with over 557 million cases and 6.36 million deaths as of 3 August 2022 according to the World Health Organization. Understanding the spread of COVID-19 has been the subject of numerous studies, highlighting the significance of reliable epidemic models. We introduce a novel epidemic model using a latent Hawkes process with temporal covariates for modelling the infections. Unlike other Hawkes models, we model the reported cases via a probability distribution driven by the underlying Hawkes process. Modelling the infections via a Hawkes process allows us to estimate by whom an infected individual was infected. We propose a Kernel Density Particle Filter (KDPF) for inference of both latent cases and reproduction number and for predicting new cases in the near future. The computational effort is proportional to the number of infections making it possible to use particle filter-type algorithms, such as the KDPF. We demonstrate the performance of the proposed algorithm on synthetic data sets and COVID-19 reported cases in various local authorities in the UK, and benchmark our model to alternative approaches. We extend the unstructured homogeneously mixing epidemic model considering a finite population stratified by age bands. We model the actual unobserved infections using a latent marked Hawkes process and the reported aggregated infections as random quantities driven by the underlying Hawkes process. We apply a Kernel Density Particle Filter (KDPF) to infer the marked counting process, the instantaneous reproduction number for each age group and forecast the epidemic’s future trajectory in the near future. We demonstrate the performance of the proposed inference algorithm on synthetic data sets and COVID-19 reported cases in various local authorities in the UK. Taking into account the individual heterogeneity in age provides a real-time measurement of interventions and behavioural changes.Open Acces

Applications of Bayesian mixture models and self-exciting processes to retail analytics

Retail analytics has been transformed by big data, which has led to many retailers using detailed analytics to improve performance at a range of operational levels. This is the case with the collaborator of this research, dunnhumby, who have large amounts of retailer data derived from the numerous activities that retailers operate at. This thesis focuses on two challenges retailers face; the analysis of products through their price elasticity coefficients and demand forecasting of products known as slow-moving inventory. The analysis of products in terms of their price elasticity coefficients is well studied. Existing approaches are hampered by the challenging nature of cross-elasticity data, as cross-elasticity coefficients typically vary in dimension and exhibit an inherent censoring. We address these problems by developing a systematic model-based approach by reinterpreting the cross-elasticity coefficients as realisations of variable length order statistics sequences, and develop a nonparametric Bayesian methodology to cluster these sequences. Our approach uses the Dirichlet process mixture model that allows data to dictate the appropriate number of clusters and provides interpretable parameters characterising the decay of the leading entries. Slow-moving inventory are characterised by having intermittent demand, in that the demand is populated with an abundance of zero sales and that, when a sale does a occur, it is often followed by a quick succession of sales. This demand intermittency inhibits the use of traditional analytics which crucially affects optimal inventory management. To combat this, we represent intermittent demand as a structured multivariate point process which allows for auto- and cross- correlation frequently observed in sparse sales data. Our approach uses a hurdle component to cope with zero sales inflation, the Hawkes process to capture the temporal clustering and a hierarchal structure to pool information across products. We illustrate our methods on real retailer data, from access granted by dunnhumby

Spatial marked point processes: Models and inferences

A spatial marked point process describes the locations of randomly distributed events in a region, with a mark attached to each observed point. Nowadays, the availability of spatiotemporal data is increasing and many spatiotemporal models are studied with applications in a wide range of disciplines. Spatial marked point processes are then extended to spatiotemporal marked point processes if time component is taken into account. In general, the marks can be quantitative or categorical variables. Independence between points and marks is a convenient assumption, but may not be true in practice. Tests for independence between points and marks are proposed previously, though only a few models have been developed to describe dependence between points and marks. In this dissertation, I focus on quantitative marks and the objective is to provide flexible models for both spatial and spatiotemporal marked point processes when points and marks are dependent.^ Three approaches to describe dependence between points and marks are studied in this dissertation, while the first two approaches are for spatial marked point processes and the last is for spatiotemporal marked point processes. First, we derive a covariance function of additive models for marked point processes. This covariance function carries information of dependence between points and marks, which can be used in kriging to make predictions of marks at unknown locations. We expect to obtain better prediction results by using this covariance function when the points and marks are dependent.^ The second approach is to consider intensity-dependent models. We study both univariate and bivariate intensity marked Log Gaussian Cox processes and apply an empirical Bayesian estimation procedure with implementation of Markov Chain Monte Carlo methodology for statistical inference. We allow dependence between marks after conditioning on the intensity which is more flexible than conditional independence assumption. The influence of adding cross covariance in modeling bivariate marks is also explored. The first two approaches are applied to model the dependence between points and marks of a white oak data.^ The last approach is to consider the partially stationary spatiotemporal marked point process, where the distribution of the spatiotemporal marked process is invariant under parallel shift of time, but may not be invariant under parallel shift of points or marks. It can be classified as a location-dependent model. To determine the potential usefulness of this approach, we illustrate through two typical examples in natural hazards: a forest wild fire study and an earthquake study. The results show that the distribution of marks and points is significantly different at local scale. It is expected that the proposed approach will have wide applications in the study of natural hazards