A Primer on High-Throughput Computing for Genomic Selection

High-throughput computing (HTC) uses computer clusters to solve advanced computational problems, with the goal of accomplishing high-throughput over relatively long periods of time. In genomic selection, for example, a set of markers covering the entire genome is used to train a model based on known data, and the resulting model is used to predict the genetic merit of selection candidates. Sophisticated models are very computationally demanding and, with several traits to be evaluated sequentially, computing time is long, and output is low. In this paper, we present scenarios and basic principles of how HTC can be used in genomic selection, implemented using various techniques from simple batch processing to pipelining in distributed computer clusters. Various scripting languages, such as shell scripting, Perl, and R, are also very useful to devise pipelines. By pipelining, we can reduce total computing time and consequently increase throughput. In comparison to the traditional data processing pipeline residing on the central processors, performing general-purpose computation on a graphics processing unit provide a new-generation approach to massive parallel computing in genomic selection. While the concept of HTC may still be new to many researchers in animal breeding, plant breeding, and genetics, HTC infrastructures have already been built in many institutions, such as the University of Wisconsin–Madison, which can be leveraged for genomic selection, in terms of central processing unit capacity, network connectivity, storage availability, and middleware connectivity. Exploring existing HTC infrastructures as well as general-purpose computing environments will further expand our capability to meet increasing computing demands posed by unprecedented genomic data that we have today. We anticipate that HTC will impact genomic selection via better statistical models, faster solutions, and more competitive products (e.g., from design of marker panels to realized genetic gain). Eventually, HTC may change our view of data analysis as well as decision-making in the post-genomic era of selection programs in animals and plants, or in the study of complex diseases in humans

An energy-aware scheduling approach for resource-intensive jobs using smart mobile devices as resource providers

The ever-growing adoption of smart mobile devices is a worldwide phenomenon that positions smart-phones and tablets as primary devices for communication and Internet access. In addition to this, the computing capabilities of such devices, often underutilized by their owners, are in continuous improvement. Today, smart mobile devices have multi-core CPUs, several gigabytes of RAM, and ability to communicate through several wireless networking technologies. These facts caught the attention of researchers who have proposed to leverage smart mobile devices aggregated computing capabilities for running resource intensive software. However, such idea is conditioned by key features, named singularities in the context of this thesis, that characterize resource provision with smart mobile devices.These are the ability of devices to change location (user mobility), the shared or non-dedicated nature of resources provided (lack of ownership) and the limited operation time given by the finite energy source (exhaustible resources).Existing proposals materializing this idea differ in the singularities combinations they target and the way they address each singularity, which make them suitable for distinct goals and resource exploitation opportunities. The latter are represented by real life situations where resources provided by groups of smart mobile devices can be exploited, which in turn are characterized by a social context and a networking support used to link and coordinate devices. The behavior of people in a given social context configure a special availability level of resources, while the underlying networking support imposes restrictionson how information flows, computational tasks are distributed and results are collected. The latter constitutes one fundamental difference of proposals mainly because each networking support ?i.e., ad-hoc and infrastructure based? has its own application scenarios. Aside from the singularities addressed and the networking support utilized, the weakest point of most of the proposals is their practical applicability. The performance achieved heavily relies on the accuracy with which task information, including execution time and/or energy required for execution, is provided to feed the resource allocator.The expanded usage of wireless communication infrastructure in public and private buildings, e.g., shoppings, work offices, university campuses and so on, constitutes a networking support that can be naturally re-utilized for leveraging smart mobile devices computational capabilities. In this context, this thesisproposal aims to contribute with an easy-to-implement  scheduling approach for running CPU-bound applications on a cluster of smart mobile devices. The approach is aware of the finite nature of smart mobile devices energy, and it does not depend on tasks information to operate. By contrast, it allocatescomputational resources to incoming tasks using a node ranking-based strategy. The ranking weights nodes combining static and dynamic parameters, including benchmark results, battery level, number of queued tasks, among others. This node ranking-based task assignment, or first allocation phase, is complemented with a re-balancing phase using job stealing techniques. The second allocation phase is an aid to the unbalanced load provoked as consequence of the non-dedicated nature of smart mobile devices CPU usage, i.e., the effect of the owner interaction, tasks heterogeneity, and lack of up-to-dateand accurate information of remaining energy estimations. The evaluation of the scheduling approach is through an in-vitro simulation. A novel simulator which exploits energy consumption profiles of real smart mobile devices, as well as, fluctuating CPU usage built upon empirical models, derived from real users interaction data, is another major contribution. Tests that validate the simulation tool are provided and the approach is evaluated in scenarios varying the composition of nodes, tasks and nodes characteristics including different tasks arrival rates, tasks requirements and different levels of nodes resource utilization.Fil: Hirsch Jofré, Matías Eberardo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Tandil. Instituto Superior de Ingeniería del Software. Universidad Nacional del Centro de la Provincia de Buenos Aires. Instituto Superior de Ingeniería del Software; Argentin

Scheduling in virtual infrastructure

For the execution of the scientific applications, different methods have been proposed to dynamically provide execution environments for such applications that hide the complexity of underlying distributed and heterogeneous infrastructures. Recently virtualization has emerged as a promising technology to provide such environments. Virtualization is a technology that abstracts away the details of physical hardware and provides virtualized resources for high-level scientific applications. Virtualization offers a cost-effective and flexible way to use and manage computing resources. Such an abstraction is appealing in Grid computing and Cloud computing for better matching jobs (applications) to computational resources. This work applies the virtualization concept to the Condor dynamic resource management system by using Condor Virtual Universe to harvest the existing virtual computing resources to their maximum utility. It allows existing computing resources to be dynamically provisioned at run-time by users based on application requirements instead of statically at design-time thereby lay the basis for efficient use of the available resources, thus providing way for the efficient use of the available resources.En la ejecución de aplicaciones científicas, existen diversas propuestas cuyo objetivo es proporcionar entornos adecuados de ejecución que oculten la complejidad de las infraestructuras distribuidas y heterogéneas subyacentes a las aplicaciones. Recientemente, la virtualización ha emergido como una prometedora tecnología que permite abstraer los detalles del hardware, mediante la asignación de recursos virtualizados a las aplicaciones científicas de altas necesidades de cómputo. La virtualización ofrece una solución rentable y además permite una gestión flexible de recursos. Este nivel de abstracción es deseable en entornos de Grid Computing y Cloud Computing para obtener una planificación adecuada de tarea (aplicaciones) sobre los recursos computacionales. Este trabajo aplica el concepto de virtualización al sistema gestor dinámico de recursos Condor, mediante la utilización de Condor Virtual Universe para conseguir una máxima utilización de los recursos computacionales virtuales. Además, permite que los recursos de cómputo existentes sean proporcionados dinámicamente en tiempo de ejecución por los usuarios, en función de los requisitos de la aplicación, en lugar de mantener la definición estática definida en tiempo de diseño, y así sentar las bases del uso eficiente de los recursos disponibles.En l'execució d'aplicacions científiques, existeixen diverses propostes amb l'objectiu de proporcionar entorns adequats d'execució que amaguin la complexitat de les infraestructures distribuïdes i heterogènies subjacents a les aplicacions. Recentment, la virtualització ha sorgit com una prometedora tecnologia que ha de permetre abstraure els detalls del hardware, mitjançant l'assignació de recursos virtualitzats a les aplicacions científiques amb altes necessitats de còmput. La virtualizatzació ofereix una solució rentable i a més permet una gestió flexible de recursos. Aquest nivell d'abstracció es desitjable en entorns de Grid Computing i Cloud Computing per a obtenir una planificació adequada del treball (aplicacions) sobre els recursos computacionals. Aquest treball aplica el concepte de virtualització al sistema gestor dinàmic de recursos Condor, mitjançant la utilització de Condor Virtual Universe per aconseguir una màxima utilització dels recursos computacionals virtuals. A més, permet que els recursos de còmput existents siguin proporcionats dinàmicament en temps d'execució pels usuaris, en funció dels requisits de l'aplicació, en lloc de mantenir la definició estàtica definida en temps de disseny, i així assentar unes bases per l'ús eficient dels recursos disponibles

Catalog of selected heavy duty transport energy management models

A catalog of energy management models for heavy duty transport systems powered by diesel engines is presented. The catalog results from a literature survey, supplemented by telephone interviews and mailed questionnaires to discover the major computer models currently used in the transportation industry in the following categories: heavy duty transport systems, which consist of highway (vehicle simulation), marine (ship simulation), rail (locomotive simulation), and pipeline (pumping station simulation); and heavy duty diesel engines, which involve models that match the intake/exhaust system to the engine, fuel efficiency, emissions, combustion chamber shape, fuel injection system, heat transfer, intake/exhaust system, operating performance, and waste heat utilization devices, i.e., turbocharger, bottoming cycle

Research and Education in Computational Science and Engineering

Over the past two decades the field of computational science and engineering (CSE) has penetrated both basic and applied research in academia, industry, and laboratories to advance discovery, optimize systems, support decision-makers, and educate the scientific and engineering workforce. Informed by centuries of theory and experiment, CSE performs computational experiments to answer questions that neither theory nor experiment alone is equipped to answer. CSE provides scientists and engineers of all persuasions with algorithmic inventions and software systems that transcend disciplines and scales. Carried on a wave of digital technology, CSE brings the power of parallelism to bear on troves of data. Mathematics-based advanced computing has become a prevalent means of discovery and innovation in essentially all areas of science, engineering, technology, and society; and the CSE community is at the core of this transformation. However, a combination of disruptive developments---including the architectural complexity of extreme-scale computing, the data revolution that engulfs the planet, and the specialization required to follow the applications to new frontiers---is redefining the scope and reach of the CSE endeavor. This report describes the rapid expansion of CSE and the challenges to sustaining its bold advances. The report also presents strategies and directions for CSE research and education for the next decade.Comment: Major revision, to appear in SIAM Revie

Urubu: energy scavenging in wireless sensor networks

For the past years wireless sensor networks (WSNs) have been coined as one of the most promising technologies for supporting a wide range of applications. However, outside the research community, few are the people who know what they are and what they can offer. Even fewer are the ones that have seen these networks used in real world applications. The main obstacle for the proliferation of these networks is energy, or the lack of it. Even though renewable energy sources are always present in the networks environment, designing devices that can efficiently scavenge that energy in order to sustain the operation of these networks is still an open challenge. Energy scavenging, along with energy efficiency and energy conservation, are the current available means to sustain the operation of these networks, and can all be framed within the broader concept of “Energetic Sustainability”. A comprehensive study of the several issues related to the energetic sustainability of WSNs is presented in this thesis, with a special focus in today’s applicable energy harvesting techniques and devices, and in the energy consumption of commercially available WSN hardware platforms. This work allows the understanding of the different energy concepts involving WSNs and the evaluation of the presented energy harvesting techniques for sustaining wireless sensor nodes. This survey is supported by a novel experimental analysis of the energy consumption of the most widespread commercially available WSN hardware platforms.Há já alguns anos que as redes de sensores sem fios (do Inglês Wireless Sensor Networks - WSNs) têm sido apontadas como uma das mais promissoras tecnologias de suporte a uma vasta gama de aplicações. No entanto, fora da comunidade científica, poucas são as pessoas que sabem o que elas são e o que têm para oferecer. Ainda menos são aquelas que já viram a sua utilização em aplicações do dia-a-dia. O principal obstáculo para a proliferação destas redes é a energia, ou a falta dela. Apesar da existência de fontes de energia renováveis no local de operação destas redes, continua a ser um desafio construir dispositivos capazes de aproveitar eficientemente essa energia para suportar a operação permanente das mesmas. A colheita de energia juntamente com a eficiência energética e a conservação de energia, são os meios disponíveis actualmente que permitem a operação permanente destas redes e podem ser todos englobados no conceito mais amplo de “Sustentabilidade Energética”. Esta tese apresenta um estudo extensivo das várias questões relacionadas com a sustentabilidade energética das redes de sensores sem fios, com especial foco nas tecnologias e dispositivos explorados actualmente na colheita de energia e no consumo energético de algumas plataformas comercias de redes de sensores sem fios. Este trabalho permite compreender os diferentes conceitos energéticos relacionados com as redes de sensores sem fios e avaliar a capacidade das tecnologias apresentadas em suportar a operação permanente das redes sem fios. Este estudo é suportado por uma inovadora análise experimental do consumo energético de algumas das mais difundidas plataformas comerciais de redes de sensores sem fios