AI Technical Considerations:Data Storage, Cloud usage and AI Pipeline

Artificial intelligence (AI), especially deep learning, requires vast amounts of data for training, testing, and validation. Collecting these data and the corresponding annotations requires the implementation of imaging biobanks that provide access to these data in a standardized way. This requires careful design and implementation based on the current standards and guidelines and complying with the current legal restrictions. However, the realization of proper imaging data collections is not sufficient to train, validate and deploy AI as resource demands are high and require a careful hybrid implementation of AI pipelines both on-premise and in the cloud. This chapter aims to help the reader when technical considerations have to be made about the AI environment by providing a technical background of different concepts and implementation aspects involved in data storage, cloud usage, and AI pipelines

A First Look at the Auriga-California Giant Molecular Cloud With Herschel and the CSO: Census of the Young Stellar Objects and the Dense Gas

We have mapped the Auriga/California molecular cloud with the Herschel PACS and SPIRE cameras and the Bolocam 1.1 mm camera on the Caltech Submillimeter Observatory (CSO) with the eventual goal of quantifying the star formation and cloud structure in this Giant Molecular Cloud (GMC) that is comparable in size and mass to the Orion GMC, but which appears to be forming far fewer stars. We have tabulated 60 compact 70/160um sources that are likely pre-main-sequence objects and correlated those with Spitzer and WISE mid-IR sources. At 1.1 mm we find 18 cold, compact sources and discuss their properties. The most important result from this part of our study is that we find a modest number of additional compact young objects beyond those identified at shorter wavelengths with Spitzer. We also describe the dust column density and temperature structure derived from our photometric maps. The column density peaks at a few x 10^22 cm^-2 (N_H2) and is distributed in a clear filamentary structure along which nearly all the pre-main-sequence objects are found. We compare the YSO surface density to the gas column density and find a strong non-linear correlation between them. The dust temperature in the densest parts of the filaments drops to ~10K from values ~ 14--15K in the low density parts of the cloud. We also derive the cumulative mass fraction and probability density function of material in the cloud which we compare with similar data on other star-forming clouds.Comment: in press Astrophysical Journal, 201

Trusted resource allocation in volunteer edge-cloud computing for scientific applications

Data-intensive science applications in fields such as e.g., bioinformatics, health sciences, and material discovery are becoming increasingly dynamic and demanding with resource requirements. Researchers using these applications which are based on advanced scientific workflows frequently require a diverse set of resources that are often not available within private servers or a single Cloud Service Provider (CSP). For example, a user working with Precision Medicine applications would prefer only those CSPs who follow guidelines from HIPAA (Health Insurance Portability and Accountability Act) for implementing their data services and might want services from other CSPs for economic viability. With the generation of more and more data these workflows often require deployment and dynamic scaling of multi-cloud resources in an efficient and high-performance manner (e.g., quick setup, reduced computation time, and increased application throughput). At the same time, users seek to minimize the costs of configuring the related multi-cloud resources. While performance and cost are among the key factors to decide upon CSP resource selection, the scientific workflows often process proprietary/confidential data that introduces additional constraints of security postures. Thus, users have to make an informed decision on the selection of resources that are most suited for their applications while trading off between the key factors of resource selection which are performance, agility, cost, and security (PACS). Furthermore, even with the most efficient resource allocation across multi-cloud, the cost to solution might not be economical for all users which have led to the development of new paradigms of computing such as volunteer computing where users utilize volunteered cyber resources to meet their computing requirements. For economical and readily available resources, it is essential that such volunteered resources can integrate well with cloud resources for providing the most efficient computing infrastructure for users. In this dissertation, individual stages such as user requirement collection, user's resource preferences, resource brokering and task scheduling, in lifecycle of resource brokering for users are tackled. For collection of user requirements, a novel approach through an iterative design interface is proposed. In addition, fuzzy interference-based approach is proposed to capture users' biases and expertise for guiding their resource selection for their applications. The results showed improvement in performance i.e. time to execute in 98 percent of the studied applications. The data collected on user's requirements and preferences is later used by optimizer engine and machine learning algorithms for resource brokering. For resource brokering, a new integer linear programming based solution (OnTimeURB) is proposed which creates multi-cloud template solutions for resource allocation while also optimizing performance, agility, cost, and security. The solution was further improved by the addition of a machine learning model based on naive bayes classifier which captures the true QoS of cloud resources for guiding template solution creation. The proposed solution was able to improve the time to execute for as much as 96 percent of the largest applications. As discussed above, to fulfill necessity of economical computing resources, a new paradigm of computing viz-a-viz Volunteer Edge Computing (VEC) is proposed which reduces cost and improves performance and security by creating edge clusters comprising of volunteered computing resources close to users. The initial results have shown improved time of execution for application workflows against state-of-the-art solutions while utilizing only the most secure VEC resources. Consequently, we have utilized reinforcement learning based solutions to characterize volunteered resources for their availability and flexibility towards implementation of security policies. The characterization of volunteered resources facilitates efficient allocation of resources and scheduling of workflows tasks which improves performance and throughput of workflow executions. VEC architecture is further validated with state-of-the-art bioinformatics workflows and manufacturing workflows.Includes bibliographical references

Velocity-resolved [CII] emission and [CII]/FIR Mapping along Orion with Herschel

We present the first 7.5'x11.5' velocity-resolved map of the [CII]158um line toward the Orion molecular cloud-1 (OMC-1) taken with the Herschel/HIFI instrument. In combination with far-infrared (FIR) photometric images and velocity-resolved maps of the H41alpha hydrogen recombination and CO J=2-1 lines, this data set provides an unprecedented view of the intricate small-scale kinematics of the ionized/PDR/molecular gas interfaces and of the radiative feedback from massive stars. The main contribution to the [CII] luminosity (~85%) is from the extended, FUV-illuminated face of the cloud G_0>500, n_H>5x10^3 cm^-3) and from dense PDRs (G_0~10^4, n_H~10^5 cm^-3) at the interface between OMC-1 and the HII region surrounding the Trapezium cluster. Around 15% of the [CII] emission arises from a different gas component without CO counterpart. The [CII] excitation, PDR gas turbulence, line opacity (from [13CII]) and role of the geometry of the illuminating stars with respect to the cloud are investigated. We construct maps of the [CII]/FIR and FIR/M_Gas ratios and show that [CII]/FIR decreases from the extended cloud component (10^-2-10^-3) to the more opaque star-forming cores (10^-3-10^-4). The lowest values are reminiscent of the "[CII] deficit" seen in local ultra-luminous IR galaxies hosting vigorous star formation. Spatial correlation analysis shows that the decreasing [CII]/FIR ratio correlates better with the column density of dust through the molecular cloud than with FIR/M_Gas. We conclude that the [CII] emitting column relative to the total dust column along each line of sight is responsible for the observed [CII]/FIR variations through the cloud.Comment: 21 pages, 17 figures. Accepted for publication in the Astrophysical Journal (2015 August 12). Figures 2, 6 and 7 are bitmapped to lower resolution. This is version 2 after minor editorial changes. Notes added after proofs include