HPC-oriented Canonical Workflows for Machine Learning Applications in Climate and Weather Prediction

Machine learning (ML) applications in weather and climate are gaining momentum as big data and the immense increase in High-performance computing (HPC) power are paving the way. Ensuring FAIR data and reproducible ML practices are significant challenges for Earth system researchers. Even though the FAIR principle is well known to many scientists, research communities are slow to adopt them. Canonical Workflow Framework for Research (CWFR) provides a platform to ensure the FAIRness and reproducibility of these practices without overwhelming researchers. This conceptual paper envisions a holistic CWFR approach towards ML applications in weather and climate, focusing on HPC and big data. Specifically, we discuss Fair Digital Object (FDO) and Research Object (RO) in the DeepRain project to achieve granular reproducibility. DeepRain is a project that aims to improve precipitation forecast in Germany by using ML. Our concept envisages the raster datacube to provide data harmonization and fast and scalable data access. We suggest the Juypter notebook as a single reproducible experiment. In addition, we envision JuypterHub as a scalable and distributed central platform that connects all these elements and the HPC resources to the researchers via an easy-to-use graphical interface

Disease Evolution and Response to Rapamycin in Activated Phosphoinositide 3-Kinase δ Syndrome: The European Society for Immunodeficiencies-Activated Phosphoinositide 3-Kinase δ Syndrome Registry

Activated phosphoinositide 3-kinase (PI3K) δ Syndrome (APDS), caused by autosomal dominant mutations in PIK3CD (APDS1) or PIK3R1 (APDS2), is a heterogeneous primary immunodeficiency. While initial cohort-descriptions summarized the spectrum of clinical and immunological manifestations, questions about long-term disease evolution and response to therapy remain. The prospective European Society for Immunodeficiencies (ESID)-APDS registry aims to characterize the disease course, identify outcome predictors, and evaluate treatment responses. So far, 77 patients have been recruited (51 APDS1, 26 APDS2). Analysis of disease evolution in the first 68 patients pinpoints the early occurrence of recurrent respiratory infections followed by chronic lymphoproliferation, gastrointestinal manifestations, and cytopenias. Although most manifestations occur by age 15, adult-onset and asymptomatic courses were documented. Bronchiectasis was observed in 24/40 APDS1 patients who received a CT-scan compared with 4/15 APDS2 patients. By age 20, half of the patients had received at least one immunosuppressant, but 2–3 lines of immunosuppressive therapy were not unusual before age 10. Response to rapamycin was rated by physician visual analog scale as good in 10, moderate in 9, and poor in 7. Lymphoproliferation showed the best response (8 complete, 11 partial, 6 no remission), while bowel inflammation (3 complete, 3 partial, 9 no remission) and cytopenia (3 complete, 2 partial, 9 no remission) responded less well. Hence, non-lymphoproliferative manifestations should be a key target for novel therapies. This report from the ESID-APDS registry provides comprehensive baseline documentation for a growing cohort that will be followed prospectively to establish prognostic factors and identify patients for treatment studies

Neual Computation and Time

Time is not only the fundamental organizing principle of the universe, it is also the primary organizer of information about the world we perceive. Our brain encodes these perceptions in sequential patterns of spiking activity. But different stimuli lead to different information encoded on different timescales; sometimes the same stimulus carries information pertaining to different perceptions on different timescales. The orders of time are many and the computational circuits of the brain must disentangle these interwoven threads to decode the underlying structure. This thesis deals with solutions to this disentanglement problem implemented not at the network level, but in smaller systems and single neurons that represent the past by clever use of internal mechanisms. Often, these solutions involve the intricate tools of the neural dendrite or other peculiar aspects of neural circuits that are well known to physiologists and biologists but disregarded in favor of more homogeneous models by many theoreticians. It is at the intersection of the diverse biological reality of the brain and the difficulty of the computational problem to disentangle the threads of temporal order that we find new and powerful computational principles: Symbolic computation on the level of single neurons via dendritic plateau potentials, embedding history in delayed feedback dynamics or consecutive filter responses, or the idea that learning a generalized differential description of a systems can largely forgo the need to remember the past – instead, patterns can freely be generated. Together, the different challenges that information ordered in different, asynchronous times present require a diverse palette of solutions. At the same time, computation and the structure imposed by time are deeply connected

Dendritic plateau potentials can process spike sequences across multiple time-scales

The brain constantly processes information encoded in temporal sequences of spiking activity. This sequential activity emerges from sensory inputs as well as from the brain's own recurrent connectivity and spans multiple dynamically changing timescales. Decoding the temporal order of spiking activity across these varying timescales is a critical function of the brain, but we do not yet understand its neural implementation. The problem is, that the passive dynamics of neural membrane potentials occur on a short millisecond timescale, whereas many cognitive tasks require the integration of information across much slower behavioral timescales. However, actively generated dendritic plateau potentials do occur on such longer timescales, and their essential role for many aspects of cognition has been firmly established by recent experiments. Here, we build on these discoveries and propose a new model of neural computation that emerges from the interaction of localized plateau potentials across a functionally compartmentalized dendritic tree. We show how this interaction offers a robust solution to the timing invariant detection and processing of sequential spike patterns in single neurons. Stochastic synaptic transmission complements the deterministic all-or-none plateau process and improves information transmission by allowing ensembles of neurons to produce graded responses to continuous combinations of features. We found that networks of such neurons can solve highly complex sequence detection tasks by breaking down long inputs into sequences of shorter, random features that can be classified reliably. These results suggest that active dendritic processes are fundamental to neural computation

A trajectory-based loss function to learn missing terms in bifurcating dynamical systems

Missing terms in dynamical systems are a challenging problem for modeling. Recent developments in the combination of machine learning and dynamical system theory open possibilities for a solution. We show how physics-informed differential equations and machine learning—combined in the Universal Differential Equation (UDE) framework by Rackauckas et al.—can be modified to discover missing terms in systems that undergo sudden fundamental changes in their dynamical behavior called bifurcations. With this we enable the application of the UDE approach to a wider class of problems which are common in many real world applications. The choice of the loss function, which compares the training data trajectory in state space and the current estimated solution trajectory of the UDE to optimize the solution, plays a crucial role within this approach. The Mean Square Error as loss function contains the risk of a reconstruction which completely misses the dynamical behavior of the training data. By contrast, our suggested trajectory-based loss function which optimizes two largely independent components, the length and angle of state space vectors of the training data, performs reliable well in examples of systems from neuroscience, chemistry and biology showing Saddle-Node, Pitchfork, Hopf and Period-doubling bifurcations

HPC-oriented Canonical Workflows for Machine Learning Applications in Climate and Weather Prediction

Machine learning (ML) applications in weather and climate are gaining momentum as big data and the immense increase in High-performance computing (HPC) power are paving the way. Ensuring FAIR data and reproducible ML practices are significant challenges for Earth system researchers. Even though the FAIR principle is well known to many scientists, research communities are slow to adopt them. Canonical Workflow Framework for Research (CWFR) provides a platform to ensure the FAIRness and reproducibility of these practices without overwhelming researchers. This conceptual paper envisions a holistic CWFR approach towards ML applications in weather and climate, focusing on HPC and big data. Specifically, we discuss Fair Digital Object (FDO) and Research Object (RO) in the DeepRain project to achieve granular reproducibility. DeepRain is a project that aims to improve precipitation forecast in Germany by using ML. Our concept envisages the raster datacube to provide data harmonization and fast and scalable data access. We suggest the Juypter notebook as a single reproducible experiment. In addition, we envision JuypterHub as a scalable and distributed central platform that connects all these elements and the HPC resources to the researchers via an easy-to-use graphical interface

Disease evolution and response to rapamycin in activated phosphoinositide 3-kinase δ syndrome : The European society for immunodeficiencies-activated phosphoinositide 3-kinase δ syndrome registry

Activated phosphoinositide 3-kinase (PI3K) δ Syndrome (APDS), caused by autosomal dominant mutations in PIK3CD (APDS1) or PIK3R1 (APDS2), is a heterogeneous primary immunodeficiency. While initial cohort-descriptions summarized the spectrum of clinical and immunological manifestations, questions about long-term disease evolution and response to therapy remain. The prospective European Society for Immunodeficiencies (ESID)-APDS registry aims to characterize the disease course, identify outcome predictors, and evaluate treatment responses. So far, 77 patients have been recruited (51 APDS1, 26 APDS2). Analysis of disease evolution in the first 68 patients pinpoints the early occurrence of recurrent respiratory infections followed by chronic lymphoproliferation, gastrointestinal manifestations, and cytopenias. Although most manifestations occur by age 15, adult-onset and asymptomatic courses were documented. Bronchiectasis was observed in 24/40 APDS1 patients who received a CT-scan compared with 4/15 APDS2 patients. By age 20, half of the patients had received at least one immunosuppressant, but 2-3 lines of immunosuppressive therapy were not unusual before age 10. Response to rapamycin was rated by physician visual analog scale as good in 10, moderate in 9, and poor in 7. Lymphoproliferation showed the best response (8 complete, 11 partial, 6 no remission), while bowel inflammation (3 complete, 3 partial, 9 no remission) and cytopenia (3 complete, 2 partial, 9 no remission) responded less well. Hence, non-lymphoproliferative manifestations should be a key target for novel therapies. This report from the ESID-APDS registry provides comprehensive baseline documentation for a growing cohort that will be followed prospectively to establish prognostic factors and identify patients for treatment studies

Activated Phosphoinositide 3-Kinase δ Syndrome: Update from the ESID Registry and comparison with other autoimmune-lymphoproliferative inborn errors of immunity

BACKGROUND: Activated phosphoinositide-3-kinase (PI3K) δ Syndrome (APDS) is an inborn error of immunity (IEI) with infection susceptibility and immune dysregulation, clinically overlapping with other conditions. Management depends on disease evolution, but predictors of severe disease are lacking. OBJECTIVES: Report the extended spectrum of disease manifestations in APDS1 versus APDS2, compare these to CTLA-4 deficiency, NFκB1 deficiency, and STAT3 gain-of-function (GOF) disease; identify predictors of severity in APDS. METHODS: Data collection with the European Society for Immunodeficiencies (ESID)-APDS registry. Comparison with published cohorts of the other IEIs. RESULTS: The analysis of 170 APDS patients outlines high penetrance and early-onset of APDS compared to the other IEIs. The large clinical heterogeneity even in individuals with the same PIK3CD variant E1021K illustrates how poorly the genotype predicts the disease phenotype and course. The high clinical overlap between APDS and the other investigated IEIs suggests relevant pathophysiological convergence of the affected pathways. Preferentially affected organ systems indicate specific pathophysiology: bronchiectasis is typical of APDS1; interstitial lung disease and enteropathy are more common in STAT3 GOF and CTLA-4 deficiency. Endocrinopathies are most frequent in STAT3 GOF, but growth impairment is also common particularly in APDS2. Early clinical presentation is a risk factor for severe disease in APDS. CONCLUSION: APDS illustrates how a single genetic variant can result in a diverse autoimmune-lymphoproliferative phenotype. Overlap with other IEI is substantial. Some specific features distinguish APDS1 from APDS2. Early-onset is a risk factor for severe disease course calling for specific treatment studies in younger patients