The LCG PI project: using interfaces for physics data analysis

In the context of the LHC Computing Grid (LCG) project, the Applications Area develops and maintains that part of the physics applications software and associated infrastructure that is shared among the LHC experiments. The "Physicist Interface" (PI) project of the LCG Application Area encompasses the interfaces and tools by which physicists will directly use the software, providing implementations based on agreed standards like the AIDA interfaces for data analysis. In collaboration with users from the experiments, work has started with implementing the AIDA interfaces for (binned and unbinned) histogramming, fitting and minimization as well as manipulation of tuples. These implementations have been developed by re-using existing packages either directly or by using a (thin) layer of wrappers. In addition, bindings of these interfaces to the Python interpreted language have been done using the dictionary subsystem of the LCG-AA/SEAL project. The actual status and the future planning of the project will be presented

Increased Productivity for Emerging Grid Applications: the Application Support System

Recently a growing number of various applications have been quickly and successfully enabled on the Grid by the CERN Grid application support team. This allowed the applications to achieve and publish large-scale results in a short time which otherwise would not be possible. We present the general infrastructure, support procedures and tools that have been developed. We discuss the general patterns observed in supporting new applications and porting them to the EGEE environment. The CERN Grid application support team has been working with the following real-life applications: medical and particle physics simulation (Geant4, Garfield), satellite imaging and geographic information for humanitarian relief operations (UNOSAT), telecommunications (ITU), theoretical physics (Lattice QCD, Feynman-loop evaluation), Bio-informatics (Avian Flu Data Challenge), commercial imaging processing and classification (Imense Ltd.) and physics experiments (ATLAS, LHCb, HARP). Using the EGEE Grid we created a standard infrastructure - set of services and tools - customized for the emerging applications. This includes creation of a generic Virtual Organization easily accessible by small communities and adding resources and services to it. We provide the consultancy service to help the porting of the applications to the Grid using the Ganga and DIANE tools. The system may be operated with only small maintenance and support overhead and is easily accessible by new applications. The various parts of the application support system developed by the CERN Grid application team were used by more than 1000 individual users in the year 2007. More than 10 new applications have been successfully enabled and produced large scale results. We consider that the efficient application support is the key point for further development of the Grid as it allows to continuously attract new application communities, strengthen the Grid infrastructure and enhance the productivity of the users. We plan to further consolidate the application support system in order to minimize the maintenance overhead and further increase the autonomy of the application communities in the efficient Grid usage

Biomedical applications on the GRID: efficient management of parallel jobs

Distributed computing based on the Master-Worker and PULL interaction model is applicable to a number of applications in high energy physics, medical physics and bio-informatics. We demonstrate a realistic medical physics use-case of a dosimetric system for brachytherapy using distributed Grid resources. We present the efficient techniques for running parallel jobs in a case of the BLAST, a gene sequencing application, as well as for the Monte Carlo simulation based on Geant4. We present a strategy for improving the runtime performance and robustness of the jobs as well as for the minimization of the development time needed to migrate the applications to a distributed environment

GANGA: powerful job submission and management tool

The computational and storage capability of the Grid are attracting several research communities, also beyond HEP. Ganga is a lightweight Grid job management tool developed at CERN. It is a key component in the distributed Data Analysis for ATLAS and LHCb. Ganga`s open and general framework allows to plug-in applications, which has attracted users from other domains outside HEP. In addition, Ganga interfaces to a variety of Grid and non-Grid backends using the same, simple end-user interface Ganga has already gained widespread use, the incomplete list of applications using Ganga include: Imaging processing and classification (developed by Cambridge Ontology Ltd.), Theoretical physics (Lattice QCD, Feynman-loop evaluation), Bio-informatics (Avian Flu Data Challenge), Geant4 (Monte Carlo package), HEP data analysis (ATLAS, LHCb). All these communities have different goals and requirements and the main challenge is the creation of a standard and general software infrastructure for the immersion of these communities onto the Grid. This general infrastructure is effectively "shielding" the applications from the details of the Grid. Finally, it is flexible and general enough to match the requirements of the different productions without including mayor changes in the design of the tool. Ganga supports a large number of backends without the underlying knowledge of each one: EGEE gLite and NorduGrid ARC middlewares, Condor and Cronus (Condor/G), various batch systems, etc From January to end of 2007 Ganga has been used by around 1000 users and has been installed locally in more than 50 sites around the world. Recently also the educative aspect of Ganga has been recognized and Ganga has become a part of the official EGEE tutorials. Contrary to other portals or tools, Ganga is not limited to specific VOs or infrastructures allowing new users to quickly exploit the EGEE infrastructure. It also allows for the interoperability of various Grid backends. Ganga has demonstrated to be a powerful job submission tool able to allow a fast merge of any new community onto the Grid. Its value has been demonstrated also by HEP communities that have adopted Ganga as the submission tool for their productions. In the new phase of the EGEE project the fast immersion of new communities will continue being a central goal and we will continue working for the confirmation of Ganga as the Gridification tool for new communities

Predictive Clinical Neuroscience Portal (PCNportal):instant online access to research-grade normative models for clinical neuroscientists [version 1; peer review: awaiting peer review]

Background: The neurobiology of mental disorders remains poorly understood despite substantial scientific efforts, due to large clinical heterogeneity and to a lack of tools suitable to map individual variability. Normative modeling is one recently successful framework that can address these problems by comparing individuals to a reference population. The methodological underpinnings of normative modelling are, however, relatively complex and computationally expensive. Our research group has developed the python-based normative modelling package Predictive Clinical Neuroscience toolkit (PCNtoolkit) which provides access to many validated algorithms for normative modelling. PCNtoolkit has since proven to be a strong foundation for large scale normative modelling, but still requires significant computation power, time and technical expertise to develop.Methods: To address these problems, we introduce PCNportal. PCNportal is an online platform integrated with PCNtoolkit that offers access to pre-trained research-grade normative models estimated on tens of thousands of participants, without the need for computation power or programming abilities. PCNportal is an easy-to-use web interface that is highly scalable to large user bases as necessary. Finally, we demonstrate how the resulting normalized deviation scores can be used in a clinical application through a schizophrenia classification task applied to cortical thickness and volumetric data from the longitudinal Northwestern University Schizophrenia Data and Software Tool (NUSDAST) dataset.Results: At each longitudinal timepoint, the transferred normative models achieved a mean[std. dev.] explained variance of 9.4[8.8]%, 9.2[9.2]%, 5.6[7.4]% respectively in the control group and 4.7[5.5]%, 6.0[6.2]%, 4.2[6.9]% in the schizophrenia group. Diagnostic classifiers achieved AUC of 0.78, 0.76 and 0.71 respectively.Conclusions: This replicates the utility of normative models for diagnostic classification of schizophrenia and showcases the use of PCNportal for clinical neuroimaging. By facilitating and speeding up research with high-quality normative models, this work contributes to research in inter-individual variability, clinical heterogeneity and precision medicine

Methods A Complexity Reduction Algorithm for Analysis and Annotation of Large Genomic Sequences

DNA is a universal language encrypted with biological instruction for life.In higher organisms, the genetic information is preserved predominantly in an organized exon/intron structure.When a gene is expressed, the exons are spliced together to form the transcript for protein synthesis.We have developed a complexity reduction algorithm for sequence analysis (CRASA) that enables direct alignment of cDNA sequences to the genome.This method features a progressive data structure in hierarchical orders to facilitate a fast and efficient search mechanism.CRASA implementation was tested with already annotated genomic sequences in two benchmark data sets and compared with 15 annotation programs (10 ab initio and 5 homology-based approaches) against the EST database.By the use of layered noise filters, the complexity of CRASA-matched data was reduced exponentially.The results from the benchmark tests showed that CRASA annotation excelled in both the sensitivity and specificity categories.When CRASA was applied to the analysis of human Chromosomes 21 and 22, an additional 83 potential genes were identified.With its large-scale processing capability, CRASA can be used as a robust tool for genome annotation with high accuracy by matching the EST sequences precisely to the genomic sequences. [Supplementary material is available online a

Ganga - an Optimiser and Front-End for Grid Job Submission (Demo)

The presentation will introduce the Ganga job-management system (http://cern.ch/ganga), developed as an ATLAS/LHCb common project. The main goal of Ganga is to provide a simple and consistent way of preparing, organising and executing analysis tasks, allowing physicists to concentrate on the algorithmic part without having to worry about technical details. Ganga provides a clean Python API that reduces and simplifies the work involved in preparing an application, organizing the submission, and gathering results. Technical details of submitting a job to the Grid, for example the preparation of a job-description file, are factored out and taken care of transparently by the system. By changing the parameter that identifies the execution backend, a user can trivially switch between running an application on a portable PC, running higherstatistics tests on a local batch system, and analysing all available statistics on the Grid. Although Ganga is being developed for LHCb and ATLAS, it is not limited to use with HEP applications, and already has several non-HEP users. These include users on projects in biomedicine, engineering, and (Grid) software testing. Ganga is a higher-level Grid tool and therefore tries to circumvent typical problems when submitting jobs to the Grid, easing the user experience. Ganga has a plug-in mechanism, so that it can be highly customised to suit the needs of a given user community

Deploying scientific applications to the PRAGMA grid testbed: strategies and lessons

Recent advances in grid infrastructure and middleware development have enabled various types of applications in science and engineering to be deployed on the grid. The characteristics of these applications and the diverse infrastructure and middleware solutions developed, utilized or adapted by PRAGMA member institutes are summarized. The applications include those for climate modeling, computational chemistry, bioinformatics and computational genomics, remote control of instruments, and distributed databases. Many of the applications are deployed to the PRAGMA grid testbed in routine basis experiments. Strategies for deploying applications without modifications, and those taking advantage of new programming models on the grid are explored and valuable lessons learned are reported. Comprehensive end to end solutions from PRAGMA member institutes that provide important grid middleware components and generalized models of integrating applications and instruments on the grid are also described