Deep Learning: Our Miraculous Year 1990-1991

In 2020, we will celebrate that many of the basic ideas behind the deep learning revolution were published three decades ago within fewer than 12 months in our "Annus Mirabilis" or "Miraculous Year" 1990-1991 at TU Munich. Back then, few people were interested, but a quarter century later, neural networks based on these ideas were on over 3 billion devices such as smartphones, and used many billions of times per day, consuming a significant fraction of the world's compute.Comment: 37 pages, 188 references, based on work of 4 Oct 201

Data-driven Soft Sensors in the Process Industry

In the last two decades Soft Sensors established themselves as a valuable alternative to the traditional means for the acquisition of critical process variables, process monitoring and other tasks which are related to process control. This paper discusses characteristics of the process industry data which are critical for the development of data-driven Soft Sensors. These characteristics are common to a large number of process industry fields, like the chemical industry, bioprocess industry, steel industry, etc. The focus of this work is put on the data-driven Soft Sensors because of their growing popularity, already demonstrated usefulness and huge, though yet not completely realised, potential. A comprehensive selection of case studies covering the three most important Soft Sensor application fields, a general introduction to the most popular Soft Sensor modelling techniques as well as a discussion of some open issues in the Soft Sensor development and maintenance and their possible solutions are the main contributions of this work

Period and chemical evolution of SC stars

The SC and CS stars are thermal-pulsing AGB stars with C/O ratio close to unity. Within this small group, the Mira variable BH Cru recently evolved from spectral type SC (showing ZrO bands) to CS (showing weak C2). Wavelet analysis shows that the spectral evolution was accompanied by a dramatic period increase, from 420 to 540 days, indicating an expanding radius. The pulsation amplitude also increased. Old photographic plates are used to establish that the period before 1940 was around 490 days. Chemical models indicate that the spectral changes were caused by a decrease in stellar temperature, related to the increasing radius. There is no evidence for a change in C/O ratio. The evolution in BH Cru is unlikely to be related to an on-going thermal pulse. Periods of the other SC and CS stars, including nine new periods, are determined. A second SC star, LX Cyg, also shows evidence for a large increase in period, and one further star shows a period inconsistent with a previous determination. Mira periods may be intrinsically unstable for C/O ~ 1; possibly because of a feedback between the molecular opacities, pulsation amplitude, and period. LRS spectra of 6 SC stars suggest a feature at wavelength > 15 micron, which resembles one recently attributed to the iron-sulfide troilite. Chemical models predict a large abundance of FeS in SC stars, in agreement with the proposed association.Comment: 14 pages, 20 figures. MNRAS, 2004, accepted for publication. Janet Mattei, one of the authors, died on 22 March, 2004. This paper is dedicated to her memor

Iterative Temporal Learning and Prediction with the Sparse Online Echo State Gaussian Process

Abstract—In this work, we contribute the online echo state gaussian process (OESGP), a novel Bayesian-based online method that is capable of iteratively learning complex temporal dy-namics and producing predictive distributions (instead of point predictions). Our method can be seen as a combination of the echo state network with a sparse approximation of Gaussian processes (GPs). Extensive experiments on the one-step prediction task on well-known benchmark problems show that OESGP produced statistically superior results to current online ESNs and state-of-the-art regression methods. In addition, we characterise the benefits (and drawbacks) associated with the considered online methods, specifically with regards to the trade-off between computational cost and accuracy. For a high-dimensional action recognition task, we demonstrate that OESGP produces high accuracies comparable to a recently published graphical model, while being fast enough for real-time interactive scenarios. I