An Experimental Review on Deep Learning Architectures for Time Series Forecasting

In recent years, deep learning techniques have outperformed traditional models in many machine learning tasks. Deep neural networks have successfully been applied to address time series forecasting problems, which is a very important topic in data mining. They have proved to be an effective solution given their capacity to automatically learn the temporal dependencies present in time series. However, selecting the most convenient type of deep neural network and its parametrization is a complex task that requires considerable expertise. Therefore, there is a need for deeper studies on the suitability of all existing architectures for different forecasting tasks. In this work, we face two main challenges: a comprehensive review of the latest works using deep learning for time series forecasting; and an experimental study comparing the performance of the most popular architectures. The comparison involves a thorough analysis of seven types of deep learning models in terms of accuracy and efficiency. We evaluate the rankings and distribution of results obtained with the proposed models under many different architecture configurations and training hyperparameters. The datasets used comprise more than 50000 time series divided into 12 different forecasting problems. By training more than 38000 models on these data, we provide the most extensive deep learning study for time series forecasting. Among all studied models, the results show that long short-term memory (LSTM) and convolutional networks (CNN) are the best alternatives, with LSTMs obtaining the most accurate forecasts. CNNs achieve comparable performance with less variability of results under different parameter configurations, while also being more efficient

An Experimental Review on Deep Learning Architectures for Time Series Forecasting

In recent years, deep learning techniques have outperformed traditional models in many machine learning tasks. Deep neural networks have successfully been applied to address time series forecasting problems, which is a very important topic in data mining. They have proved to be an effective solution given their capacity to automatically learn the temporal dependencies present in time series. However, selecting the most convenient type of deep neural network and its parametrization is a complex task that requires considerable expertise. Therefore, there is a need for deeper studies on the suitability of all existing architectures for different forecasting tasks. In this work, we face two main challenges: a comprehensive review of the latest works using deep learning for time series forecasting and an experimental study comparing the performance of the most popular architectures. The comparison involves a thorough analysis of seven types of deep learning models in terms of accuracy and efficiency. We evaluate the rankings and distribution of results obtained with the proposed models under many different architecture configurations and training hyperparameters. The datasets used comprise more than 50,000 time series divided into 12 different forecasting problems. By training more than 38,000 models on these data, we provide the most extensive deep learning study for time series forecasting. Among all studied models, the results show that long short-term memory (LSTM) and convolutional networks (CNN) are the best alternatives, with LSTMs obtaining the most accurate forecasts. CNNs achieve comparable performance with less variability of results under different parameter configurations, while also being more efficient.Ministerio de Ciencia, Innovación y Universidades TIN2017-88209-C2Junta de Andalucía US-1263341Junta de Andalucía P18-RT-277

On the Performance of One-Stage and Two-Stage Object Detectors in Autonomous Vehicles Using Camera Data

Object detection using remote sensing data is a key task of the perception systems of self-driving vehicles. While many generic deep learning architectures have been proposed for this problem, there is little guidance on their suitability when using them in a particular scenario such as autonomous driving. In this work, we aim to assess the performance of existing 2D detection systems on a multi-class problem (vehicles, pedestrians, and cyclists) with images obtained from the on-board camera sensors of a car. We evaluate several one-stage (RetinaNet, FCOS, and YOLOv3) and two-stage (Faster R-CNN) deep learning meta-architectures under different image resolutions and feature extractors (ResNet, ResNeXt, Res2Net, DarkNet, and MobileNet). These models are trained using transfer learning and compared in terms of both precision and efficiency, with special attention to the real-time requirements of this context. For the experimental study, we use theWaymo Open Dataset, which is the largest existing benchmark. Despite the rising popularity of one-stage detectors, our findings show that two-stage detectors still provide the most robust performance. Faster R-CNN models outperform one-stage detectors in accuracy, being also more reliable in the detection of minority classes. Faster R-CNN Res2Net-101 achieves the best speed/accuracy tradeoff but needs lower resolution images to reach real-time speed. Furthermore, the anchor-free FCOS detector is a slightly faster alternative to RetinaNet, with similar precision and lower memory usage.Ministerio de Economía y Competitividad TIN2017-88209-C2-2-RJunta de Andalucía US-1263341Junta de Andalucía P18-RT-277

Evaluation of the transformer architecture for univariate time series forecasting

The attention-based Transformer architecture is earning in- creasing popularity for many machine learning tasks. In this study, we aim to explore the suitability of Transformers for time series forecasting, which is a crucial problem in di erent domains. We perform an extensive experimental study of the Transformer with di erent architecture and hyper-parameter con gurations over 12 datasets with more than 50,000 time series. The forecasting accuracy and computational e ciency of Transformers are compared with state-of-the-art deep learning networks such as LSTM and CNN. The obtained results demonstrate that Trans- formers can outperform traditional recurrent or convolutional models due to their capacity to capture long-term dependencies, obtaining the most accurate forecasts in ve out of twelve datasets. However, Transformers are generally more di cult to parametrize and show higher variability of results. In terms of e ciency, Transformer models proved to be less competitive in inference time and similar to the LSTM in training time.Ministerio de Ciencia, Innovación y Universidades TIN2017-88209-C2Junta de Andalucía US-1263341Junta de Andalucía P18-RT-277

On the performance of deep learning models for time series classification in streaming

Processing data streams arriving at high speed requires the development of models that can provide fast and accurate predictions. Although deep neural networks are the state-of-the-art for many machine learning tasks, their performance in real-time data streaming scenarios is a research area that has not yet been fully addressed. Nevertheless, there have been recent efforts to adapt complex deep learning models for streaming tasks by reducing their processing rate. The design of the asynchronous dual-pipeline deep learning framework allows to predict over incoming instances and update the model simultaneously using two separate layers. The aim of this work is to assess the performance of different types of deep architectures for data streaming classification using this framework. We evaluate models such as multi-layer perceptrons, recurrent, convolutional and temporal convolutional neural networks over several time-series datasets that are simulated as streams. The obtained results indicate that convolutional architectures achieve a higher performance in terms of accuracy and efficiency.Ministerio de Economía y Competitividad TIN2017-88209-C2-2-RJunta de Andalucía US-1263341Junta de Andalucía P18-RT-277

Data streams classification using deep learning under different speeds and drifts

Processing data streams arriving at high speed requires the development of models that can provide fast and accurate predictions. Although deep neural networks are the state-of-the-art for many machine learning tasks, their performance in real-time data streaming scenarios is a research area that has not yet been fully addressed. Nevertheless, much effort has been put into the adaption of complex deep learning (DL) models to streaming tasks by reducing the processing time. The design of the asynchronous dual-pipeline DL framework allows making predictions of incoming instances and updating the model simultaneously, using two separate layers. The aim of this work is to assess the performance of different types of DL architectures for data streaming classification using this framework. We evaluate models such as multi-layer perceptrons, recurrent, convolutional and temporal convolutional neural networks over several time series datasets that are simulated as streams at different speeds. In addition, we evaluate how the different architectures react to concept drifts typically found in evolving data streams. The obtained results indicate that convolutional architectures achieve a higher performance in terms of accuracy and efficiency, but are also the most sensitive to concept drifts.Ministerio de Ciencia, Innovación y Universidades PID2020-117954RB-C22Junta de Andalucía US-1263341Junta de Andalucía P18-RT-277

Asynchronous dual-pipeline deep learning framework for online data stream classification

Data streaming classification has become an essential task in many fields where real-time decisions have to be made based on incoming information. Neural networks are a particularly suitable technique for the streaming scenario due to their incremental learning nature. However, the high computation cost of deep architectures limits their applicability to high-velocity streams, hence they have not yet been fully explored in the literature. Therefore, in this work, we aim to evaluate the effectiveness of complex deep neural networks for supervised classification in the streaming context. We propose an asynchronous deep learning framework in which training and testing are performed simultaneously in two different processes. The data stream entering the system is dual fed into both layers in order to concurrently provide quick predictions and update the deep learning model. This separation reduces processing time while obtaining high accuracy on classification. Several time-series datasets from the UCR repository have been simulated as streams to evaluate our proposal, which has been compared to other methods such as Hoeffding trees, drift detectors, and ensemble models. The statistical analysis carried out verifies the improvement in performance achieved with our dual-pipeline deep learning framework, that is also competitive in terms of computation time.Ministerio de Economía y Competitividad TIN2017-88209-C2-2-

Temporal convolutional networks applied to energy-related time series forecasting

Modern energy systems collect high volumes of data that can provide valuable information about energy consumption. Electric companies can now use historical data to make informed decisions on energy production by forecasting the expected demand. Many deep learning models have been proposed to deal with these types of time series forecasting problems. Deep neural networks, such as recurrent or convolutional, can automatically capture complex patterns in time series data and provide accurate predictions. In particular, Temporal Convolutional Networks (TCN) are a specialised architecture that has advantages over recurrent networks for forecasting tasks. TCNs are able to extract long-term patterns using dilated causal convolutions and residual blocks, and can also be more efficient in terms of computation time. In this work, we propose a TCN-based deep learning model to improve the predictive performance in energy demand forecasting. Two energy-related time series with data from Spain have been studied: the national electric demand and the power demand at charging stations for electric vehicles. An extensive experimental study has been conducted, involving more than 1900 models with different architectures and parametrisations. The TCN proposal outperforms the forecasting accuracy of Long Short-Term Memory (LSTM) recurrent networks, which are considered the state-of-the-art in the field.Ministerio de Economía y Competitividad TIN2017-88209-C2-2-RJunta de Andalucía US-1263341Junta de Andalucía P18-RT-277

Multi‑omic alterations of the SWI/SNF complex define a clinical subgroup in lung adenocarcinoma

PPM's lab is funded by the Ministry of Economy of Spain (SAF2015-67919-R), Junta de Andalucia (P20-00688, PI-0135-2020, PIGE-0213-2020, PIGE-04402019, PI-0245-2017), University of Granada (B-CTS-480-UGR20), International Association for the Study of Lung Cancer (IASLC), and Spanish Association for Cancer Research (LAB-AECC-2018). PP is supported by a PhD "La Caixa Foundation"LCF/BQ/DE15/10360019 Fellowship. AA is supported by an FPU17/00067 fellowship. IFC was supported by a PhD FPI-fellowship (BES-2013-064596). DJG was supported by a "Fundacion Benefica Anticancer Santa Candida y San Francisco Javier"predoctoral fellowship. MSBC and CC's work is supported by the project DPI2017-84439-R Ministry of Economy of Spain and FEDER and by the fellowship "Beca de Iniciacion a la Investigacion del Plan Propio de Investigacion 2019" by University of Granada. MSBC is supported by an FPU19/00576 predoctoral fellowship. CNIO Proteomics Unit is a member of Proteored PRB3 and is supported by grant PT17/0019, of the PE I + D + i 2013-2016, funded by ISCIII and ERDF.SWI/SNF complexes are major targets of mutations in cancer. Here, we combined multiple “-omics” methods to assess SWI/SNF composition and aberrations in LUAD. Mutations in lung SWI/SNF subunits were highly recurrent in our LUAD cohort (41.4%), and over 70% of the mutations were predicted to have functional impact. Furthermore, SWI/ SNF expression in LUAD suffered an overall repression that could not be explained exclusively by genetic alterations. Finally, SWI/SNF mutations were associated with poorer overall survival in TCGA-LUAD. We propose SWI/SNF-mutant LUAD as a separate clinical subgroup with practical implications.Spanish Government SAF2015-67919-R DPI2017-84439-RJunta de Andalucia P20-00688 PI-0135-2020 PIGE-0213-2020 PIGE-0440-2019 PI-0245-2017University of Granada B-CTS-480-UGR20International Association for the Study of Lung Cancer (IASLC)Spanish Association for Cancer Research LAB-AECC-2018La Caixa Foundation LCF/BQ/DE15/10360019PhD FPI-fellowship BES-2013-064596"Fundacion Benefica Anticancer Santa Candida y San Francisco Javier" predoctoral fellowshipEuropean Commissionfellowship "Beca de Iniciacion a la Investigacion del Plan Propio de Investigacion 2019" by University of Granada Instituto de Salud Carlos III PT17/0019European Commission PT17/0019 FPU17/00067 FPU19/0057

Substrate-induced dimerization of engineered monomeric variants of triosephosphate isomerase from Trichomonas vaginalis

"The dimeric nature of triosephosphate isomerases (TIMs) is maintained by an extensive surface area interface of more than 1600 angstrom 2. TIMs from Trichomonas vaginalis (TvTIM) are held in their dimeric state by two mechanisms: a ball and socket interaction of residue 45 of one subunit that fits into the hydrophobic pocket of the complementary subunit and by swapping of loop 3 between subunits. TvTIMs differ from other TIMs in their unfolding energetics. In TvTIMs the energy necessary to unfold a monomer is greater than the energy necessary to dissociate the dimer. Herein we found that the character of residue I45 controls the dimer-monomer equilibrium in TvTIMs. Unfolding experiments employing monomeric and dimeric mutants led us to conclude that dimeric TvTIMs unfold following a four state model denaturation process whereas monomeric TvTIMs follow a three state model. In contrast to other monomeric TIMs, monomeric variants of TvTIM1 are stable and unexpectedly one of them (I45A) is only 29-fold less active than wild-type TvTIM1. The high enzymatic activity of monomeric TvTIMs contrast with the marginal catalytic activity of diverse monomeric TIMs variants. The stability of the monomeric variants of TvTIM1 and the use of cross-linking and analytical ultracentrifugation experiments permit us to understand the differences between the catalytic activities of TvTIMs and other marginally active monomeric TIMs. As TvTIMs do not unfold upon dimer dissociation, herein we found that the high enzymatic activity of monomeric TvTIM variants is explained by the formation of catalytic dimeric competent species assisted by substrate binding.