Heuristics for periodical batch job scheduling in a MapReduce computing framework

Task scheduling has a significant impact on the performance of the MapReduce computing framework. In this paper, a scheduling problem of periodical batch jobs with makespan minimization is considered. The problem is modeled as a general two-stage hybrid flow shop scheduling problem with schedule-dependent setup times. The new model incorporates the data locality of tasks and is formulated as an integer program. Three heuristics are developed to solve the problem and an improvement policy based on data locality is presented to enhance the methods. A lower bound of the makespan is derived. 150 instances are randomly generated from data distributions drawn from a real cluster. The parameters involved in the methods are set according to different cluster setups. The proposed heuristics are compared over different numbers of jobs and cluster setups. Computational results show that the performance of the methods is highly dependent on both the number of jobs and the cluster setups. The proposed improvement policy is effective and the impact of the input data distribution on the policy is analyzed and tested.This work is supported by the National Natural Science Foundation of China (No. 61272377) and the Specialized Research Fund for the Doctoral Program of Higher Education (No. 20120092110027). Ruben Ruiz is partially supported by the Spanish Ministry of Economy and Competitiveness, under the project "RESULT - Realistic Extended Scheduling Using Light Techniques" (No. DPI2012-36243-C02-01) partially financed with FEDER funds.Xiaoping Li; Tianze Jiang; Ruiz García, R. (2016). Heuristics for periodical batch job scheduling in a MapReduce computing framework. Information Sciences. 326:119-133. https://doi.org/10.1016/j.ins.2015.07.040S11913332

Kernel-Based Tests for Likelihood-Free Hypothesis Testing

Given $n$ observations from two balanced classes, consider the task of labeling an additional $m$ inputs that are known to all belong to \emph{one} of the two classes. Special cases of this problem are well-known: with complete knowledge of class distributions ($n=\infty$) the problem is solved optimally by the likelihood-ratio test; when $m=1$ it corresponds to binary classification; and when $m\approx n$ it is equivalent to two-sample testing. The intermediate settings occur in the field of likelihood-free inference, where labeled samples are obtained by running forward simulations and the unlabeled sample is collected experimentally. In recent work it was discovered that there is a fundamental trade-off between $m$ and $n$: increasing the data sample $m$ reduces the amount $n$ of training/simulation data needed. In this work we (a) introduce a generalization where unlabeled samples come from a mixture of the two classes -- a case often encountered in practice; (b) study the minimax sample complexity for non-parametric classes of densities under \textit{maximum mean discrepancy} (MMD) separation; and (c) investigate the empirical performance of kernels parameterized by neural networks on two tasks: detection of the Higgs boson and detection of planted DDPM generated images amidst CIFAR-10 images. For both problems we confirm the existence of the theoretically predicted asymmetric $m$ vs $n$ trade-off.Comment: 36 pages, 6 figure

Roles of thermal energy storage technology for carbon neutrality

Abstract In order to achieve global carbon neutrality in the middle of the 21st century, efficient utilization of fossil fuels is highly desired in diverse energy utilization sectors such as industry, transportation, building as well as life science. In the energy utilization infrastructure, about 75% of the fossil fuel consumption is used to provide and maintain heat, leading to more than 60% waste heat of the input energy discharging to the environment. Types of low-grade waste heat recovery technologies are developed to increase the energy efficiency. However, due to the spatial and temporal mismatch between the need and supply of the thermal energy, much of the waste thermal energy is difficult to be recovered. Thermal energy storage (TES) technologies in the forms of sensible, latent and thermochemical heat storage are developed for relieving the mismatched energy supply and demand. Diverse TES systems are developed in recent years with the superior features of large density, long-term, durable and low-cost. These technologies are vital in efficient utilization of low-grade waste heat and expected for building a low or zero carbon emission society. This paper reviews the thermal storage technologies for low carbon power generation, low carbon transportation, low carbon building as well as low carbon life science, in addition, carbon capture, utilization, and storage are also considered for carbon emission reduction. The conclusion and perspective are raised after discussing the specific technologies. This study is expected to provide a reference for the TES technologies in achieving zero-carbon future

Characterization of a Mass-Produced SiPM at Liquid Nitrogen Temperature for CsI Neutrino Coherent Detectors

Silicon Photomultiplier (SiPM) is a sensor that can detect low-light signals lower than the single-photon level. In order to study the properties of neutrinos at a low detection threshold and low radioactivity experimental background, a low-temperature CsI neutrino coherent scattering detector is designed to be read by the SiPM sensor. Less thermal noise of SiPM and more light yield of CsI crystals can be obtained at the working temperature of liquid nitrogen. The breakdown voltage (Vbd) and dark count rate (DCR) of SiPM at liquid nitrogen temperature are two key parameters for coherent scattering detection. In this paper, a low-temperature test is conducted on the mass-produced ON Semiconductor J-Series SiPM. We design a cryogenic system for cooling SiPM at liquid nitrogen temperature and the changes of operating voltage and dark noise from room to liquid nitrogen temperature are measured in detail. The results show that SiPM works at the liquid nitrogen temperature, and the dark count rate drops by six orders of magnitude from room temperature (120 kHz/mm2) to liquid nitrogen temperature (0.1 Hz/mm2)

Petroleum Geology and Exploration of Deep-Seated Volcanic Condensate Gas Reservoir around the Penyijingxi Sag in the Junggar Basin

Many types of volcanic rock oil and gas reservoirs have been found in China, showing great petroleum exploration potential. Volcanic reservoir also is one of the key fields of exploration in the Junggar Basin and mainly concentrated in the middle and shallow layers, while the deep volcanic rock and natural gas fields have not been broken through. Based on comprehensive analysis of core observation, single well analysis, reservoir description, source rocks evaluation, combined with seismic data and time-frequency electromagnetic technology, multiple volcanic rock exploration targets were identified, and industrial oil and gas flow was obtained in the well SX 16 of the Penyijingxi Sag, western Junggar Basin. It is believed that the deep Permian source rocks have relatively higher natural gas generation potential and volcanic breccia usually have large reservoir space. And the mudstone of the Upper Wuerhe Formation played as the role of caprock. The success of exploration well SX16 has achieved a major breakthrough in natural gas exploration in the Penyijingxi Sag, which has essential guiding significance for the exploration of deep volcanic rocks and large-scale gas exploration in the Junggar Basin

Effect of H2O Molecule Adsorption on the Electronic Structure and Optical Properties of the CsI(Na) Crystal

We investigated H2O molecule adsorption that had an effect on the luminescence properties of the CsI(Na) crystal using experiments and first-principle calculations. We measured the emission spectra of the CsI(Na) crystal at different exposure times under gamma ray excitation. The experimental results showed that the energy resolution of the CsI(Na) crystal was worse when the crystal surface adsorbed more H2O molecules, and the crystal surface deliquescence decreased the luminescence efficiency of the CsI(Na) crystal. We studied the band structure, density of states, and optical properties changes caused by H2O molecule adsorption on the CsI(Na) (010) surface. The generalized gradient approximation (GGA) was used to describe the exchange and correlation potential between the electrons. Our calculation results showed that the band gap width of the CsI(Na) (010) surface decreased after adsorbing H2O molecules, while three new peaks appeared in the valence band, and the absorption coefficient decreased from 90,000 cm−1 to 65,000 cm−1, and the reflection coefficient decreased from 0.195 to 0.105. Further, the absorption coefficient was reduced by at least 25% because of H2O molecule adsorption, which led to the luminescence degradation of the CsI(Na) crystal

Additional file 1 of Analysis of long non-coding RNA expression profile of bovine monocyte-macrophage infected by Mycobacterium avium subsp. paratuberculosis

Additional file 1