1,398 research outputs found
Temporal Spatial Decomposition and Fusion Network for Time Series Forecasting
Feature engineering is required to obtain better results for time series
forecasting, and decomposition is a crucial one. One decomposition approach
often cannot be used for numerous forecasting tasks since the standard time
series decomposition lacks flexibility and robustness. Traditional feature
selection relies heavily on preexisting domain knowledge, has no generic
methodology, and requires a lot of labor. However, most time series prediction
models based on deep learning typically suffer from interpretability issue, so
the "black box" results lead to a lack of confidence. To deal with the above
issues forms the motivation of the thesis. In the paper we propose TSDFNet as a
neural network with self-decomposition mechanism and an attentive feature
fusion mechanism, It abandons feature engineering as a preprocessing convention
and creatively integrates it as an internal module with the deep model. The
self-decomposition mechanism empowers TSDFNet with extensible and adaptive
decomposition capabilities for any time series, users can choose their own
basis functions to decompose the sequence into temporal and generalized spatial
dimensions. Attentive feature fusion mechanism has the ability to capture the
importance of external variables and the causality with target variables. It
can automatically suppress the unimportant features while enhancing the
effective ones, so that users do not have to struggle with feature selection.
Moreover, TSDFNet is easy to look into the "black box" of the deep neural
network by feature visualization and analyze the prediction results. We
demonstrate performance improvements over existing widely accepted models on
more than a dozen datasets, and three experiments showcase the interpretability
of TSDFNet.Comment: 10 page
New Levels of Language Processing Complexity and Organization Revealed by Granger Causation
Granger causation analysis of high spatiotemporal resolution reconstructions of brain activation offers a new window on the dynamic interactions between brain areas that support language processing. Premised on the observation that causes both precede and uniquely predict their effects, this approach provides an intuitive, model-free means of identifying directed causal interactions in the brain. It requires the analysis of all non-redundant potentially interacting signals, and has shown that even “early” processes such as speech perception involve interactions of many areas in a strikingly large network that extends well beyond traditional left hemisphere perisylvian cortex that play out over hundreds of milliseconds. In this paper we describe this technique and review several general findings that reframe the way we think about language processing and brain function in general. These include the extent and complexity of language processing networks, the central role of interactive processing dynamics, the role of processing hubs where the input from many distinct brain regions are integrated, and the degree to which task requirements and stimulus properties influence processing dynamics and inform our understanding of “language-specific” localized processes
Lightweight, Pre-trained Transformers for Remote Sensing Timeseries
Machine learning algorithms for parsing remote sensing data have a wide range
of societally relevant applications, but labels used to train these algorithms
can be difficult or impossible to acquire. This challenge has spurred research
into self-supervised learning for remote sensing data aiming to unlock the use
of machine learning in geographies or application domains where labelled
datasets are small. Current self-supervised learning approaches for remote
sensing data draw significant inspiration from techniques applied to natural
images. However, remote sensing data has important differences from natural
images -- for example, the temporal dimension is critical for many tasks and
data is collected from many complementary sensors. We show that designing
models and self-supervised training techniques specifically for remote sensing
data results in both smaller and more performant models. We introduce the
Pretrained Remote Sensing Transformer (Presto), a transformer-based model
pre-trained on remote sensing pixel-timeseries data. Presto excels at a wide
variety of globally distributed remote sensing tasks and outperforms much
larger models. Presto can be used for transfer learning or as a feature
extractor for simple models, enabling efficient deployment at scale
DDMT: Denoising Diffusion Mask Transformer Models for Multivariate Time Series Anomaly Detection
Anomaly detection in multivariate time series has emerged as a crucial
challenge in time series research, with significant research implications in
various fields such as fraud detection, fault diagnosis, and system state
estimation. Reconstruction-based models have shown promising potential in
recent years for detecting anomalies in time series data. However, due to the
rapid increase in data scale and dimensionality, the issues of noise and Weak
Identity Mapping (WIM) during time series reconstruction have become
increasingly pronounced. To address this, we introduce a novel Adaptive Dynamic
Neighbor Mask (ADNM) mechanism and integrate it with the Transformer and
Denoising Diffusion Model, creating a new framework for multivariate time
series anomaly detection, named Denoising Diffusion Mask Transformer (DDMT).
The ADNM module is introduced to mitigate information leakage between input and
output features during data reconstruction, thereby alleviating the problem of
WIM during reconstruction. The Denoising Diffusion Transformer (DDT) employs
the Transformer as an internal neural network structure for Denoising Diffusion
Model. It learns the stepwise generation process of time series data to model
the probability distribution of the data, capturing normal data patterns and
progressively restoring time series data by removing noise, resulting in a
clear recovery of anomalies. To the best of our knowledge, this is the first
model that combines Denoising Diffusion Model and the Transformer for
multivariate time series anomaly detection. Experimental evaluations were
conducted on five publicly available multivariate time series anomaly detection
datasets. The results demonstrate that the model effectively identifies
anomalies in time series data, achieving state-of-the-art performance in
anomaly detection.Comment: 16 pages, 9 figure
Hierarchical Bayesian Modeling of Manipulation Sequences from Bimodal Input
Barchunova A, Moringen J, Haschke R, Ritter H. Hierarchical Bayesian Modeling of Manipulation Sequences from Bimodal Input. Presented at the Proceedings of the 11th International Conference on Cognitive Modeling, Berlin
MedLens: Improve mortality prediction via medical signs selecting and regression interpolation
Monitoring the health status of patients and predicting mortality in advance
is vital for providing patients with timely care and treatment. Massive medical
signs in electronic health records (EHR) are fitted into advanced machine
learning models to make predictions. However, the data-quality problem of
original clinical signs is less discussed in the literature. Based on an
in-depth measurement of the missing rate and correlation score across various
medical signs and a large amount of patient hospital admission records, we
discovered the comprehensive missing rate is extremely high, and a large number
of useless signs could hurt the performance of prediction models. Then we
concluded that only improving data-quality could improve the baseline accuracy
of different prediction algorithms. We designed MEDLENS, with an automatic
vital medical signs selection approach via statistics and a flexible
interpolation approach for high missing rate time series. After augmenting the
data-quality of original medical signs, MEDLENS applies ensemble classifiers to
boost the accuracy and reduce the computation overhead at the same time. It
achieves a very high accuracy performance of 0.96% AUC-ROC and 0.81% AUC-PR,
which exceeds the previous benchmark
Towards On-Board Panoptic Segmentation of Multispectral Satellite Images
With tremendous advancements in low-power embedded computing devices and
remote sensing instruments, the traditional satellite image processing pipeline
which includes an expensive data transfer step prior to processing data on the
ground is being replaced by on-board processing of captured data. This paradigm
shift enables critical and time-sensitive analytic intelligence to be acquired
in a timely manner on-board the satellite itself. However, at present, the
on-board processing of multi-spectral satellite images is limited to
classification and segmentation tasks. Extending this processing to its next
logical level, in this paper we propose a lightweight pipeline for on-board
panoptic segmentation of multi-spectral satellite images. Panoptic segmentation
offers major economic and environmental insights, ranging from yield estimation
from agricultural lands to intelligence for complex military applications.
Nevertheless, the on-board intelligence extraction raises several challenges
due to the loss of temporal observations and the need to generate predictions
from a single image sample. To address this challenge, we propose a multimodal
teacher network based on a cross-modality attention-based fusion strategy to
improve the segmentation accuracy by exploiting data from multiple modes. We
also propose an online knowledge distillation framework to transfer the
knowledge learned by this multi-modal teacher network to a uni-modal student
which receives only a single frame input, and is more appropriate for an
on-board environment. We benchmark our approach against existing
state-of-the-art panoptic segmentation models using the PASTIS multi-spectral
panoptic segmentation dataset considering an on-board processing setting. Our
evaluations demonstrate a substantial increase in accuracy metrics compared to
the existing state-of-the-art models
Learning the Joint Representation of Heterogeneous Temporal Events for Clinical Endpoint Prediction
The availability of a large amount of electronic health records (EHR)
provides huge opportunities to improve health care service by mining these
data. One important application is clinical endpoint prediction, which aims to
predict whether a disease, a symptom or an abnormal lab test will happen in the
future according to patients' history records. This paper develops deep
learning techniques for clinical endpoint prediction, which are effective in
many practical applications. However, the problem is very challenging since
patients' history records contain multiple heterogeneous temporal events such
as lab tests, diagnosis, and drug administrations. The visiting patterns of
different types of events vary significantly, and there exist complex nonlinear
relationships between different events. In this paper, we propose a novel model
for learning the joint representation of heterogeneous temporal events. The
model adds a new gate to control the visiting rates of different events which
effectively models the irregular patterns of different events and their
nonlinear correlations. Experiment results with real-world clinical data on the
tasks of predicting death and abnormal lab tests prove the effectiveness of our
proposed approach over competitive baselines.Comment: 8 pages, this paper has been accepted by AAAI 201
Forecasting Stock Time-Series using Data Approximation and Pattern Sequence Similarity
Time series analysis is the process of building a model using statistical
techniques to represent characteristics of time series data. Processing and
forecasting huge time series data is a challenging task. This paper presents
Approximation and Prediction of Stock Time-series data (APST), which is a two
step approach to predict the direction of change of stock price indices. First,
performs data approximation by using the technique called Multilevel Segment
Mean (MSM). In second phase, prediction is performed for the approximated data
using Euclidian distance and Nearest-Neighbour technique. The computational
cost of data approximation is O(n ni) and computational cost of prediction task
is O(m |NN|). Thus, the accuracy and the time required for prediction in the
proposed method is comparatively efficient than the existing Label Based
Forecasting (LBF) method [1].Comment: 11 page
- …