MAPPING on UPMEM

This paper presents the implementation of a mapping algorithm on the UPMEM architecture. The mapping is a basic bioinformatics application that consists in finding the best location of millions of short DNA sequences on a full genome. The mapping can be constrained by a maximum number of differences between the DNA sequence and the region of the genome where a high similarity has been found. UPMEM’s Processing-In-Memory (PIM) solution consist of adding processing units into the DRAM, to minimize data access time and maximize bandwidth, in order to drastically accelerate data-consuming algorithms. A 16 GBytes UPMEM-DIMM module comes then with 256 UPMEM DRAM Processing Units (named DPU). The mapping algorithm implemented on the UPMEM architecture dispatches a huge index across the DPU memories. DNA sequences are assigned to a specific DPU according to k-mers features, allowing to massively map in parallel million of them. Experimentation on Human genome dataset shows that speed-up of 25 can be obtained with PIM, compared to fast mapping software such as BWA, Bowtie2 or NextGenMap running 16 Intel threads. Experimentation also highlight that data transfer from storage device limits the performances of the implementation. The use of SSD drives can boost the speed-up to 80

Biomanycores, open-source parallel code for many-core bioinformatics

International audienceBiomanycores is a collection of bioinformatics tools, designed to bridge the gap between researches in OpenCL/CUDA high-performance computing on GPU and other "manycore processors" and usual bioinformaticians and biologists

In-situ visualization using Damaris: the Code Saturne use case

PRACE White PaperInternational audienceAs the exascale era approaches, maintaining scalable performance in data management tasks (storage, visualization, analysis, etc.) remains a key challenge in sustaining high performance for the application execution. To address this challenge, the Damaris middleware leverages dedicated computational resources in multicore nodes to offload data management tasks, including I/O, data compression, scheduling of data movements, in-situ analysis, and visualization. In this study we evaluate the benefits of Damaris to improve the efficiency of in-situ visualization for Code_Saturne, a fluid dynamics modeling environment. The experiments show Damaris to adequately hide the I/O processing of various Paraview processing pipelines in Code_Saturne. In all cases the Damaris enabled version of Code_Saturne was found to be more efficient than the identical non-Damaris capable version when running the same Paraview pipeline

Speeding up NGS software development

International audienceThe analysis of NGS data remains a time and space-consuming task. Many efforts have been made toprovide efficient data structures for indexing the terabytes of data generated by the fast sequencingmachines (Suffix Array, Burrows-Wheeler transform, Bloom Filter, etc.). Mapper tools, genomeassemblers, SNP callers, etc., make an intensive use of these data structures to keep their memoryfootprint as lower as possible.The overall efficiency of NGS software is brought by a smart combination of how data are representedinside the computer memory and how they are processed through the available processing units insidea processor. Developing such software is thus a real challenge, as it requires a large spectrum ofcompetences from high-level data structure and algorithm concepts to tiny details of implementation.We have developed a C++ library, called GATB (Genomic Assembly and Analysis Tool Box) tospeed up the design of NGS algorithms. This library offers a panel of high-level optimized buildingblocks. The underlying data structure is the de Bruijn graph, and the general parallelism model ismultithreading. The GATB library targets standard computing resources such as current multicoreprocessor (laptop computer, small server) with a few GB of memory. Hence, from high-level C++API, NGS programing designers can rapidly elaborate their own software based on state-of-the-artalgorithms and data structures of the domain.To demonstrate the efficiency of the GATB library, several NGS software have been designed such ascontiger (Minia), read corrector (Bloocoo) or SNP discovery (DiscoSNP). The GATB library iswritten in C++ and is available at the following web site http://gatb.inria.fr under the GNU AfferoGPL license

Biomanycores, open-source parallel code for many-core bioinformatics

International audienceBiomanycores is a collection of bioinformatics tools, designed to bridge the gap between researches in OpenCL/CUDA high-performance computing on GPU and other "manycore processors" and usual bioinformaticians and biologists

Critical Assessment of Metagenome Interpretation:A benchmark of metagenomics software

International audienceIn metagenome analysis, computational methods for assembly, taxonomic profilingand binning are key components facilitating downstream biological datainterpretation. However, a lack of consensus about benchmarking datasets andevaluation metrics complicates proper performance assessment. The CriticalAssessment of Metagenome Interpretation (CAMI) challenge has engaged the globaldeveloper community to benchmark their programs on datasets of unprecedentedcomplexity and realism. Benchmark metagenomes were generated from newlysequenced ~700 microorganisms and ~600 novel viruses and plasmids, includinggenomes with varying degrees of relatedness to each other and to publicly availableones and representing common experimental setups. Across all datasets, assemblyand genome binning programs performed well for species represented by individualgenomes, while performance was substantially affected by the presence of relatedstrains. Taxonomic profiling and binning programs were proficient at high taxonomicranks, with a notable performance decrease below the family level. Parametersettings substantially impacted performances, underscoring the importance ofprogram reproducibility. While highlighting current challenges in computationalmetagenomics, the CAMI results provide a roadmap for software selection to answerspecific research questions

Estimation de la circulation dans l'Océan Atlantique Sud par assimilation variationnelle de données in situ (Impact du contrôle optimal des forçages et de l'hydrologie aux frontières ouvertes)

Les observations montrent que les modèles d'océans actuels ne parviennent souvent pas à représenter la réalité avec précision. Cela s'explique en partie par un manque de précision dans les conditions initiales (CI) et aux limites. Mais les flux air-mer sont notoirement mal connus, et leur estimation au moyen de méthodes avancées reste peu explorée. Ici, des expériences jumelles systématiques ont été effectuées pour évaluer dans quelle mesure des tels forçages, CI et conditions aux frontières ouvertes (FO) peuvent être améliorés par assimilation 4D-Var de données de profils dans un modèle d'océan. Le système est basé sur le code OPA8.1 et son adjoint. Le modèle résoud le cycle saisonnier de l'Atlantique Sud. Les données assimilées sont l'hydrologie de l'atlas saisonnier climatologique de Reynaud et al. [1998] ou, lors des expériences idéalisées, un jeu de données simulées équivalent. Dans la partie I, la configuration Atlantique Sud est décrite. Une mise en régime de plusieurs années est validée, et fournit les données idéalisées nécessaires pour la suite. Dans la partie II, nous décrivons l'implémentation du contrôle des FO et des forçages, en supplément des CI déjà estimées. Dans la partie III, des expériences idéalisées durant l'été austral montrent que nos données sont appropriées à identifier des flux de chaleur et d'eau douce saisonniers moyens, mais pas leur variabilité intrasaisonnière. De plus, l'estimation des tensions de vent est un problème mal posé, en particulier dans la région des vents d'ouest. Un résultat important est que le sytème peut dégrader un vent parfait pour équilibrer une erreur dans les flux de chaleur. Dans la partie IV, les données réelles sont assimilées sur 4 saisons cyclées, et seul le vent n'est pas estimé. Les trajectoires optimales sont compatibles avec les données, mais des biais systématiques apparaissent, pour lesquels des solutions sont discutées. les paramètres contrôlées ont tous un impact positif sur l'écart modèle-données.Observations show that state-of-the-art numerical ocean models often fail to accurately represent reality. This is partly due to a lack of accuracy in initial and boundary conditions. Also, air-sea fluxes are known to have large uncertainties, and their estimation via advanced estimation methods remains poorly explored. Here, systematic twin data experiments were performed to assess how well such forcings, initial conditions (IC) and open boundary conditions (OB) may be improved via 4D-Var assimilation of profile data in an ocean model. The system is based on the OPA8.1 model and its adjoint code. The model resolves the seasonal circulation of the South Atlantic Ocean. The assimilated data are either the Reynaud et al. [1998] climatological seasonal hydrological data. In part I, the South Atlantic configuration is described. A spin-up simulation of several years is validated and gives the idealized data needed later. In part II, we describe how the estimation of the OBs anf forcings in the assimilation scheme is implemented, in addition to the previously existing ICs estimation. In part III, idealized experiments during austral summer show that our data are appropriate in identifying the seasonal mean heat and freshwater fluxes, but not their intraseasonal variability. In addition, the wind stress estimation proves an ill-posed problem, in particular in the westerly winds region. A stronger result is that the system can erroneously modify perfect wind data to balance a heat flux error. Limitations of the results due to our configuration are discussed. In part IV, we move to real data and cycle four climatological seasons, estimating all unknowns but the wind. Optimal trajectories prove compatible with the data, but exhibit systematic biases for which solutions are discussed. The parameters controlled in this study show a positive impact on observational fit.BREST-BU Droit-Sciences-Sports (290192103) / SudocPLOUZANE-Bibl.La Pérouse (290195209) / SudocSudocFranceF

NEMO Tangent & Adjoint Models (NemoTam) Reference Manual & User's Guide

The development of the tangent linear and adjoint models (TAM in the following) of the dynamical core of the NEMO ocean engine (NEMOTAM) is a key objective of the VODA project. TAM are widely used for variational assimilation applications, but they are also powerful tools for the analysis of physical processes, since they can be used for sensitivity analysis, parameter identification and for the computation of characteristic vectors (singular vectors, Liapunov vectors, etc.). In the framework of VODA, a work package has been set-up in order to develop a comprehensive NEMOTAM package, and to define an effective long-term development strategy for ensuring synchronisation of NEMOTAM with future NEMO releases. This is a heavy task, but it is worth the effort since NEMOTAM will benefit all NEMO users for the wide range of applications described above. Ideally, this strategy should be defined to allow NEMOTAM to adapt to future NEMO developments as quickly and as efficiently as possible, so that new releases of NEMOTAM can be made soon after new releases of NEMO. This will require careful coordination between the main development teams of NEMO, NEMOTAM and possibly NEMOVAR (INRIA, NEMO Team, CERFACS, ECMWF)

The Mediterranean response to different space-time resolution atmospheric forcings using perpetual mode sensitivity simulations

International audienceThe Mediterranean basin features a semi-enclosed sea, where interactions and feedbacks between the atmosphere and the Sea at various temporal and spatial scales play a predominant role in the regional climate. This study analyzes the Mediterranean Sea response in sensitivity experiments conducted by driving the NEMO-MED12 oceanic model in perpetual mode with various atmospheric forcings, all produced by the WRF non-hydrostatic mesoscale atmospheric model, but differing by their resolutions : two horizontal resolutions (20 km at basin scale and 6.7 km in the North-Western [NWE] area) and two temporal resolutions (daily and three-hourly). The atmospheric fields available from August 1998 to July 1999 are in good agreement with estimates derived from satellite data. The heat budget of the Mediterranean Sea represents an heat loss of 5 W/m2 and the annual freshwater budget is -1.04 m, in agreement with climatologies. An increase in the spatial resolution in the NWE area modifies the modeled circulation from -10% to +15% for the SST, from -30% to +50% for the SSS, from -10% to +30% for the MLD and from -10% to +30% for the EKE in surface. The increase in the wind speed with a better chanelling by the land orography enhances in particular the oceanic convection process in the NWE area. On the other hand, the increase in the temporal resolution reduces the convection process, because of the diurnal restratification of the oceanic upper layer. It also reduces the surface parameters high-frequency variability, whereas it increases the EKE values in surface, due to the rapid response to the wind

BLAST on UPMEM

This paper presents the implementation of the BLAST software on the UPMEM architecture. BLAST is a well-known molecular biology software to rapidly scan DNA or protein genomic banks. It is daily used by thousands of biologists. Given a query sequence, BLAST reports all the sequences of the bank where similarities are found. The result is a list of alignments that detail the similarities. UPMEM’s Processing-In-Memory (PIM) solution consist of adding processing units into the DRAM, to minimize data access time and maximize bandwidth, in order to drastically accelerate data-consuming algorithms. A 16 GBytes UPMEM-DIMM module comes then with 256 UPMEM DRAM Processing Units (named DPU). To find similarities, BLAST proceeds in 3 steps: (1) search of common words between the query sequence and the bank sequences; (2) evaluation of local similarity on the neighbourhood of these words; and (3) computation of the final alignment. As the 2 first steps are limited by memory bandwidth, and represent the majority of time of the overall computation, they have been massively parallelized on UPMEM DPUs. The 3rd step is performed on the host processor and is overlapped with the UPMEM processing. Experimentation on real datasets shows a speed-up of 25 when using UPMEM configuration, versus a standard server running 20 Intel cores