Ensemble weather forecast post-processing with a flexible probabilistic neural network approach

Ensemble forecast post-processing is a necessary step in producing accurate probabilistic forecasts. Conventional post-processing methods operate by estimating the parameters of a parametric distribution, frequently on a per-location or per-lead-time basis. We propose a novel, neural network-based method, which produces forecasts for all locations and lead times, jointly. To relax the distributional assumption of many post-processing methods, our approach incorporates normalizing flows as flexible parametric distribution estimators. This enables us to model varying forecast distributions in a mathematically exact way. We demonstrate the effectiveness of our method in the context of the EUPPBench benchmark, where we conduct temperature forecast post-processing for stations in a sub-region of western Europe. We show that our novel method exhibits state-of-the-art performance on the benchmark, outclassing our previous, well-performing entry. Additionally, by providing a detailed comparison of three variants of our novel post-processing method, we elucidate the reasons why our method outperforms per-lead-time-based approaches and approaches with distributional assumptions

SMIXS: Novel efficient algorithm for non-parametric mixture regression-based clustering

We investigate a novel non-parametric regression-based clustering algorithm for longitudinal data analysis. Combining natural cubic splines with Gaussian mixture models (GMM), the algorithm can produce smooth cluster means that describe the underlying data well. However, there are some shortcomings in the algorithm: high computational complexity in the parameter estimation procedure and a numerically unstable variance estimator. Therefore, to further increase the usability of the method, we incorporated approaches to reduce its computational complexity, we developed a new, more stable variance estimator, and we developed a new smoothing parameter estimation procedure. We show that the developed algorithm, SMIXS, performs better than GMM on a synthetic dataset in terms of clustering and regression performance. We demonstrate the impact of the computational speed-ups, which we formally prove in the new framework. Finally, we perform a case study by using SMIXS to cluster vertical atmospheric measurements to determine different weather regimes

Automatic razlikovanje of pathologic and non-pathologic changes in ECG signals

Snemanje in analiza posnetkov EKG sta pomembna postopka v diagnozi srčnih bolezni. Postopek je še posebej priljubljen, ker ni zahteven za izvedbo, je neinvaziven, poceni in dostopen v primerjavi z drugimi postopki, za diagnozo srčnih bolezni. Poleg kratkih EKG posnetkov v kontroliranem okolju (ambulanti), pa pogosto snemajo tudi dolge (24-urne) EKG posnetke. Taki posnetki so uporabni predvsem za diagnozo različnih aritmij, lahko pa tudi za zgodnje odkrivanje srčne ishemije. Naše delo obravnava razpoznavanje različnih patoloških stanj (anomalij) v dolgih EKG posnetkih. Prva anomalija, ki jo obravnavamo, je srčna ishemija, ki se v EKG posnetkih odraža kot deviacija segmenta ST. V preteklih letih je bilo objavljenih veliko število člankov na temo odkrivanja prehodnih epizod segmenta ST. Težava opisanih pristopov je, da ne znajo razlikovati med ishemičnimi epizodami in nepatološkimi epizodami, ki nastanejo zaradi spremembe srčne frekvence. V našem delu predstavimo dva načina za razlikovanje med tipoma epizod. Prvi način je s pomočjo izbora atributovatributi, ki opisujejo spremembo srčne frekvence, atributi, ki opisujejo deviacije in spremembe morfologije segmenta ST in spremembo kompleksa QRS. S pomočjo izbranih atributov in različnih klasifikatorjev pokažemo, da je razlikovanje med tipoma epizod mogoče. Po zgledu metod uporabljenih v obremenilnih testih izpeljemo ST(HR) diagram za dolge EKG posnetke in iz njega odčitamo množico atributov, ki nam pomagajo razlikovati med tipoma epizod. Opazujemo diagram v dveh odsekih (podobno kot v obremenilnih testih) od začetka epizode do njenega ekstrema, ter od ekstrema epizode do njenega konca in definiramo maksimalen naklon diagrama, celoten naklon diagrama in kot ob ekstremu diagrama. Nakloni, dva celotna in dva maksimalna, se pokažejo kot relativno dobri atributi (po kriterijih za ocenjevanje kvalitete atributov), medtem ko kot ob ekstremu diagrama ni relevanten atribut. Tem novim atributom dodamo še vrednosti srčne frekvence ob začetku, ekstremu in koncu epizode. S pomočjo teh atributov klasificiramo epizode nekoliko slabše kot z atributi opisanimi v prejšnjem odstavku. Velika prednost te metode je, da je uporablja manj atributov, ki so lažje razumljivi in jih bolj enostavno izračunamo. Druga vrsta anomalij, ki jih obravnavamo, so aritmični utripi. V ta namen razvijemo postopek za detekcijo kompleksov QRS, ki temelji na diskretni Morsejevi teoriji in postopek za detekcijo aritmičnih utripov. Razviti postopek za detekcijo kompleksov QRS temelji na krajšanju parov sosednjih minimumov in maksimumov po postopku krajšanja v diskretni Morsejevi teoriji. Z dodatkom pravil o obliki kompleksa QRS, ki temeljijo na predznanju o EKG posnetkih, razvijemo uspešen in učinkovit postopek (v primerjavi z objavljenimi postopki) za detekcijo kompleksov QRS. Na podoben način razvijemo tudi postopek, ki hkrati poišče komplekse QRS in do 4 valove, značilne za utrip EKG ( P, QRS, T in U) med zaporednima kompleksoma QRS. S pomočjo originalne mere podobnosti med dvema krivuljama ocenjujemo podobnost med sosednjima utripoma. Ker se aritmije pojavljajo kot nenadna spremembe oblike in frekvence srčnega utripa, lahko kot atributa pri klasifikaciji utripov (aritmični ali ne) uporabimo podobnost med dvema zaporednima utripoma in razliko v dolžinah RR intervalov. Rezultati metode za odkrivanje kompleksov QRS so primerljivi z najboljšimi rezultati objavljenimi v literaturi. Zmogljivost postopka za odkrivanje anomalij je tudi visoka, predvsem na posnetkih, ki vsebujejo veliko število patoloških utripov (npr. PVC).Heart disease is the leading cause of death in the developed world. ECG signal recording and analysis is the easiest is the prime diagnostic procedure for early diagnosis of heart conditions. It is non-invasive, inexpensive and accessible when compared to other clinical procedures used in the diagnosis of heart disease. In clinical practice, short ECG recordings are usually used, which are recorded in a controlled environment, but long (24-hour) ECG recording (AECG) are also gaining popularity. AECG recordings are used mostly in the diagnosis of different arrhythmias, sometimes even in the diagnosis oh heart ischemia. Our work deals with the automatic detection of pathologic events in long ECG recordings. The first heart pathology we research is heart ischemia. It manifests as ST segment deviation in ECG signals. In the past years a large number of research papers have been published, dealing with the detection of transient ST segment episodes in AECG. The described ST segment episode detectors fail to differentiate between transient ischemic and transient non-ischemic heart rate related episodes. In our work we describe two methods to differentiate transient ST segment episodes. The first method uses a set of featuresfeatures that describe heart rate changes, ST segment deviation and morphology changes and QRS complex morphology changes. Using a set of features and different classifiers we show that automatic classification of the two types of episodes can be successful. Following the example of the methods used in exercise ECG (EECG) we define a ST(HR) diagram for AECG. The diagram is used to calculate a subset of features that help us differentiate between ischemic and heart rate related episodes. Similarly as in EECG we observe the diagram in two partsfrom the beginning to the extreme of the episode and from the extreme to the end of the episode. We define two overall slopes, two maximal slope and the angle at the extrema of the episode. The slopes are good features (ranked with feature evaluation techniques) while the angle at the extrema is not a good feature. We also observe the heart rate at the beginning, extrema and end of the episodes. The performance of the classification with this set of features is worse than the classification with the set of features described in the last paragraph. The advantage of the classification with the ST(HR) diagram is that it uses a smaller number of features, the features are more comprehensive and easier to calculate. The second anomaly we study in our work is the detection of arrhythmic beats. Here we first develop a QRS detector based on the discrete Morse theory and then an arrhythmia detector. The QRS detector is based on cancelling neighbouring minima and maxima as in discrete Morse theory. With the addition of knowledge from ECG signal theory the performance of our QRS detector is very similar to the best performances published in the literature. Using a similar method as in QRS detection, we develop a procedure that simultaneously finds QRS complexes and up to four of the most significant waves (typical for ECG signalsP, QRS, T and U) between consecutive R waves. With the help of a newly developed algorithm, that evaluates the similarity between two trajectories, we assessed the similarity between consecutive heart beats. Then we classify pairs of heart beats as normal of arrhythmic with the help of two featuresthe similarity between the pair of beats and the difference between their RR intervals. The performance of the arrhythmia detector is high, especially on records containing a large number of very pathologic heart beats (e.g. PVC)

Automatic razlikovanje of pathologic and non-pathologic changes in ECG signals

Heart disease is the leading cause of death in the developed world. ECG signal recording and analysis is the easiest is the prime diagnostic procedure for early diagnosis of heart conditions. It is non-invasive, inexpensive and accessible when compared to other clinical procedures used in the diagnosis of heart disease. In clinical practice, short ECG recordings are usually used, which are recorded in a controlled environment, but long (24-hour) ECG recording (AECG) are also gaining popularity. AECG recordings are used mostly in the diagnosis of different arrhythmias, sometimes even in the diagnosis oh heart ischemia. Our work deals with the automatic detection of pathologic events in long ECG recordings. The first heart pathology we research is heart ischemia. It manifests as ST segment deviation in ECG signals. In the past years a large number of research papers have been published, dealing with the detection of transient ST segment episodes in AECG. The described ST segment episode detectors fail to differentiate between transient ischemic and transient non-ischemic heart rate related episodes. In our work we describe two methods to differentiate transient ST segment episodes. The first method uses a set of features; features that describe heart rate changes, ST segment deviation and morphology changes and QRS complex morphology changes. Using a set of features and different classifiers we show that automatic classification of the two types of episodes can be successful. Following the example of the methods used in exercise ECG (EECG) we define a ST(HR) diagram for AECG. The diagram is used to calculate a subset of features that help us differentiate between ischemic and heart rate related episodes. Similarly as in EECG we observe the diagram in two parts; from the beginning to the extreme of the episode and from the extreme to the end of the episode. We define two overall slopes, two maximal slope and the angle at the extrema of the episode. The slopes are good features (ranked with feature evaluation techniques) while the angle at the extrema is not a good feature. We also observe the heart rate at the beginning, extrema and end of the episodes. The performance of the classification with this set of features is worse than the classification with the set of features described in the last paragraph. The advantage of the classification with the ST(HR) diagram is that it uses a smaller number of features, the features are more comprehensive and easier to calculate. The second anomaly we study in our work is the detection of arrhythmic beats. Here we first develop a QRS detector based on the discrete Morse theory and then an arrhythmia detector. The QRS detector is based on cancelling neighbouring minima and maxima as in discrete Morse theory. With the addition of knowledge from ECG signal theory the performance of our QRS detector is very similar to the best performances published in the literature. Using a similar method as in QRS detection, we develop a procedure that simultaneously finds QRS complexes and up to four of the most significant waves (typical for ECG signals; P, QRS, T and U) between consecutive R waves. With the help of a newly developed algorithm, that evaluates the similarity between two trajectories, we assessed the similarity between consecutive heart beats. Then we classify pairs of heart beats as normal of arrhythmic with the help of two features; the similarity between the pair of beats and the difference between their RR intervals. The performance of the arrhythmia detector is high, especially on records containing a large number of very pathologic heart beats (e.g. PVC)

SuperFormer

One of the biggest challenges in continual learning domains is the tendency of machine learning models to forget previously learned information over time. While overcoming this issue, the existing approaches often exploit large amounts of additional memory and apply model forgetting mitigation mechanisms which substantially prolong the training process. Therefore, we propose a novel SuperFormer method that alleviates model forgetting, while spending negligible additional memory and time. We tackle the continual learning challenges in a learning scenario, where we learn different tasks in a sequential order. We compare our method against several prominent continual learning methods, i.e., EWC, SI, MAS, GEM, PSP, etc. on a set of text classification tasks. We achieve the best average performance in terms of AUROC and AUPRC (0.7% and 0.9% gain on average, respectively) and the lowest training time among all the methods of comparison. On average, our method reduces the total training time by a factor of 5.4-8.5 in comparison to similarly performing methods. In terms of the additional memory, our method is on par with the most memory-efficient approaches

Comparison of in-situ chlorophyll-a time series and Sentinel-3 Ocean and Land Color Instrument data in Slovenian national waters (Gulf of Trieste, Adriatic Sea)

While satellite remote sensing of ocean color is a viable tool for estimating large-scale patterns of chlorophyll-a (Chl-a) and global ocean primary production, its application in coastal waters is limited by the complex optical properties. An exploratory study was conducted in the Gulf of Trieste (Adriatic Sea) to assess the usefulness of Sentinel-3 satellite data in the Slovenian national waters. OLCI (Ocean and Land Colour Instrument) Chl-a level 2 products (OC4Me and NN) were compared to monthly Chl-a in-situ measurements at fixed sites from 2017 to 2019. In addition, eight other methods for estimating Chl-a concentration based on reflectance in different spectral bands were tested (OC3M, OC4E, MedOC4, ADOC4, AD4, 3B-OLCI, 2B-OLCI and G2B). For some of these methods, calibration was performed on in-situ data to achieve a better agreement. Finally, L1-regularized regression and random forest were trained on the available dataset to test the capabilities of the machine learning approach. The results show rather poor performance of the two originally available products. The same is true for the other eight methods and the fits to the measured values also show only marginal improvement. The best results are obtained with the blue-green methods (OC3, OC4 and AD4), especially the AD4SI (a designated fit of AD4) with R = 0.56 and RMSE = 0.4 mg/m$^3$, while the near infrared (NIR) methods show underwhelming performance. The machine learning approach can only explain 30% of the variability and the RMSE is of the same order as for the blue-green methods. We conclude that due to the low Chl-a concentration and the moderate turbidity of the seawater, the reflectance provided by the Sentinel-3 OLCI spectrometer carries little information about Chl-a in the Slovenian national waters within the Gulf of Trieste and is therefore of limited use for our purposes. This requires that we continue to improve satellite products for use in those marine waters that have not yet proven suitable. In this way, satellite data could be effectively integrated into a comprehensive network that would allow a reliable assessment of ecological status, taking into account environmental regulations