44 research outputs found
SMURF-THP: Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process
Transformer Hawkes process models have shown to be successful in modeling
event sequence data. However, most of the existing training methods rely on
maximizing the likelihood of event sequences, which involves calculating some
intractable integral. Moreover, the existing methods fail to provide
uncertainty quantification for model predictions, e.g., confidence intervals
for the predicted event's arrival time. To address these issues, we propose
SMURF-THP, a score-based method for learning Transformer Hawkes process and
quantifying prediction uncertainty. Specifically, SMURF-THP learns the score
function of events' arrival time based on a score-matching objective that
avoids the intractable computation. With such a learned score function, we can
sample arrival time of events from the predictive distribution. This naturally
allows for the quantification of uncertainty by computing confidence intervals
over the generated samples. We conduct extensive experiments in both event type
prediction and uncertainty quantification of arrival time. In all the
experiments, SMURF-THP outperforms existing likelihood-based methods in
confidence calibration while exhibiting comparable prediction accuracy
Modeling Multivariate Hopfield-Transformer Hawkes Process: Application to Sovereign Credit Default Swaps
Hawkes process was evolved so that the past events contribute to the occurrence time of future events by self-exciting or mutually exciting. However, many real-world data do not follow the Hawkes process\u27s assumptions (i.e., positivity, additivity, and exponential decay) and become more complex to be modeled by the traditional Hawkes processes, so the neural Hawkes process was developed to tackle the challenges. However, Recurrent Neural Networks (RNN) fail to capture long-term dependencies among multiple point processes, and Transformer Hawkes processes only address temporal characteristics of Hawkes processes. In this thesis, we proposed a combination of neural networks and Hawkes processes to tackle the aforementioned challenges and to capture contagious effects among different points processes. First, we made substantial modifications to the Transformer Hawkes process by utilizing two encoders, which include two Multi-Head attention modules: 1) event significance attention and 2) temporal attention. Then, to improve this model, the Modern Hopfield Neural Network was incorporated to better assign the attention to the test set by appending the decoder layer to the previous modified encoder layers. Credit Default Swap data for ten European countries were tested, and the results revealed that modeling the contagious effect ameliorates the prediction performance
Intensity-free Integral-based Learning of Marked Temporal Point Processes
In the marked temporal point processes (MTPP), a core problem is to
parameterize the conditional joint PDF (probability distribution function)
for inter-event time and mark , conditioned on the history.
The majority of existing studies predefine intensity functions. Their utility
is challenged by specifying the intensity function's proper form, which is
critical to balance expressiveness and processing efficiency. Recently, there
are studies moving away from predefining the intensity function -- one models
and separately, while the other focuses on temporal point
processes (TPPs), which do not consider marks. This study aims to develop
high-fidelity for discrete events where the event marks are either
categorical or numeric in a multi-dimensional continuous space. We propose a
solution framework IFIB (\underline{I}ntensity-\underline{f}ree
\underline{I}ntegral-\underline{b}ased process) that models conditional joint
PDF directly without intensity functions. It remarkably simplifies
the process to compel the essential mathematical restrictions. We show the
desired properties of IFIB and the superior experimental results of IFIB on
real-world and synthetic datasets. The code is available at
\url{https://github.com/StepinSilence/IFIB}
Score Matching-based Pseudolikelihood Estimation of Neural Marked Spatio-Temporal Point Process with Uncertainty Quantification
Spatio-temporal point processes (STPPs) are potent mathematical tools for
modeling and predicting events with both temporal and spatial features. Despite
their versatility, most existing methods for learning STPPs either assume a
restricted form of the spatio-temporal distribution, or suffer from inaccurate
approximations of the intractable integral in the likelihood training
objective. These issues typically arise from the normalization term of the
probability density function. Moreover, current techniques fail to provide
uncertainty quantification for model predictions, such as confidence intervals
for the predicted event's arrival time and confidence regions for the event's
location, which is crucial given the considerable randomness of the data. To
tackle these challenges, we introduce SMASH: a Score MAtching-based
pSeudolikeliHood estimator for learning marked STPPs with uncertainty
quantification. Specifically, our framework adopts a normalization-free
objective by estimating the pseudolikelihood of marked STPPs through
score-matching and offers uncertainty quantification for the predicted event
time, location and mark by computing confidence regions over the generated
samples. The superior performance of our proposed framework is demonstrated
through extensive experiments in both event prediction and uncertainty
quantification
Learning Neural Point Processes with Latent Graphs
Neural point processes (NPPs) employ neural networks to capture complicated dynamics of asynchronous event sequences. Existing NPPs feed all history events into neural networks, assuming that all event types contribute to the prediction of the target type. How- ever, this assumption can be problematic because in reality some event types do not contribute to the predictions of another type. To correct this defect, we learn to omit those types of events that do not contribute to the prediction of one target type during the formulation of NPPs. Towards this end, we simultaneously consider the tasks of (1) finding event types that contribute to predictions of the target types and (2) learning a NPP model from event se- quences. For the former, we formulate a latent graph, with event types being vertices and non-zero contributing relationships being directed edges; then we propose a probabilistic graph generator, from which we sample a latent graph. For the latter, the sampled graph can be readily used as a plug-in to modify an existing NPP model. Because these two tasks are nested, we propose to optimize the model parameters through bilevel programming, and develop an efficient solution based on truncated gradient back-propagation. Experimental results on both synthetic and real-world datasets show the improved performance against state-of-the-art baselines. This work removes disturbance of non-contributing event types with the aid of a validation procedure, similar to the practice to mitigate overfitting used when training machine learning models
EasyTPP: Towards Open Benchmarking Temporal Point Processes
Continuous-time event sequences play a vital role in real-world domains such
as healthcare, finance, online shopping, social networks, and so on. To model
such data, temporal point processes (TPPs) have emerged as the most natural and
competitive models, making a significant impact in both academic and
application communities. Despite the emergence of many powerful models in
recent years, there hasn't been a central benchmark for these models and future
research endeavors. This lack of standardization impedes researchers and
practitioners from comparing methods and reproducing results, potentially
slowing down progress in this field. In this paper, we present EasyTPP, the
first central repository of research assets (e.g., data, models, evaluation
programs, documentations) in the area of event sequence modeling. Our EasyTPP
makes several unique contributions to this area: a unified interface of using
existing datasets and adding new datasets; a wide range of evaluation programs
that are easy to use and extend as well as facilitate reproducible research;
implementations of popular neural TPPs, together with a rich library of modules
by composing which one could quickly build complex models. All the data and
implementation can be found at
https://github.com/ant-research/EasyTemporalPointProcess. We will actively
maintain this benchmark and welcome contributions from other researchers and
practitioners. Our benchmark will help promote reproducible research in this
field, thus accelerating research progress as well as making more significant
real-world impacts.Comment: ICLR 2024 camera read
Enhancing Event Sequence Modeling with Contrastive Relational Inference
Neural temporal point processes(TPPs) have shown promise for modeling
continuous-time event sequences. However, capturing the interactions between
events is challenging yet critical for performing inference tasks like
forecasting on event sequence data. Existing TPP models have focused on
parameterizing the conditional distribution of future events but struggle to
model event interactions. In this paper, we propose a novel approach that
leverages Neural Relational Inference (NRI) to learn a relation graph that
infers interactions while simultaneously learning the dynamics patterns from
observational data. Our approach, the Contrastive Relational Inference-based
Hawkes Process (CRIHP), reasons about event interactions under a variational
inference framework. It utilizes intensity-based learning to search for
prototype paths to contrast relationship constraints. Extensive experiments on
three real-world datasets demonstrate the effectiveness of our model in
capturing event interactions for event sequence modeling tasks.Comment: 6 pages, 2 figure
A Graph Regularized Point Process Model For Event Propagation Sequence
Point process is the dominant paradigm for modeling event sequences occurring
at irregular intervals. In this paper we aim at modeling latent dynamics of
event propagation in graph, where the event sequence propagates in a directed
weighted graph whose nodes represent event marks (e.g., event types). Most
existing works have only considered encoding sequential event history into
event representation and ignored the information from the latent graph
structure. Besides they also suffer from poor model explainability, i.e.,
failing to uncover causal influence across a wide variety of nodes. To address
these problems, we propose a Graph Regularized Point Process (GRPP) that can be
decomposed into: 1) a graph propagation model that characterizes the event
interactions across nodes with neighbors and inductively learns node
representations; 2) a temporal attentive intensity model, whose excitation and
time decay factors of past events on the current event are constructed via the
contextualization of the node embedding. Moreover, by applying a graph
regularization method, GRPP provides model interpretability by uncovering
influence strengths between nodes. Numerical experiments on various datasets
show that GRPP outperforms existing models on both the propagation time and
node prediction by notable margins.Comment: IJCNN 202