Optimizing MapReduce for Highly Distributed Environments

MapReduce, the popular programming paradigm for large-scale data processing, has traditionally been deployed over tightly-coupled clusters where the data is already locally available. The assumption that the data and compute resources are available in a single central location, however, no longer holds for many emerging applications in commercial, scientific and social networking domains, where the data is generated in a geographically distributed manner. Further, the computational resources needed for carrying out the data analysis may be distributed across multiple data centers or community resources such as Grids. In this paper, we develop a modeling framework to capture MapReduce execution in a highly distributed environment comprising distributed data sources and distributed computational resources. This framework is flexible enough to capture several design choices and performance optimizations for MapReduce execution. We propose a model-driven optimization that has two key features: (i) it is end-to-end as opposed to myopic optimizations that may only make locally optimal but globally suboptimal decisions, and (ii) it can control multiple MapReduce phases to achieve low runtime, as opposed to single-phase optimizations that may control only individual phases. Our model results show that our optimization can provide nearly 82% and 64% reduction in execution time over myopic and single-phase optimizations, respectively. We have modified Hadoop to implement our model outputs, and using three different MapReduce applications over an 8-node emulated PlanetLab testbed, we show that our optimized Hadoop execution plan achieves 31-41% reduction in runtime over a vanilla Hadoop execution. Our model-driven optimization also provides several insights into the choice of techniques and execution parameters based on application and platform characteristics

Serendipitous discovery of a projected pair of QSOs separated by 4.5 arcsec on the sky

We present the serendipitous discovery of a projected pair of quasi-stellar objects (QSOs) with an angular separation of $\Delta\theta =4.50$ arcsec. The redshifts of the two QSOs are widely different: one, our programme target, is a QSO with a spectrum consistent with being a narrow line Seyfert 1 AGN at $z=2.05$. For this target we detect Lyman-$\alpha$, \ion{C}{4}, and \ion{C}{3]}. The other QSO, which by chance was included on the spectroscopic slit, is a Type 1 QSO at a redshift of $z=1.68$, for which we detect \ion{C}{4}, \ion{C}{3]} and \ion{Mg}{2}. We compare this system to previously detected projected QSO pairs and find that only about a dozen previously known pairs have smaller angular separation.Comment: 4 pages, 3 figures. Accepted for publication in A

Determining the fraction of reddened quasars in COSMOS with multiple selection techniques from X-ray to radio wavelengths

The sub-population of quasars reddened by intrinsic or intervening clouds of dust are known to be underrepresented in optical quasar surveys. By defining a complete parent sample of the brightest and spatially unresolved quasars in the COSMOS field, we quantify to which extent this sub-population is fundamental to our understanding of the true population of quasars. By using the available multiwavelength data of various surveys in the COSMOS field, we built a parent sample of 33 quasars brighter than $J=20$ mag, identified by reliable X-ray to radio wavelength selection techniques. Spectroscopic follow-up with the NOT/ALFOSC was carried out for four candidate quasars that had not been targeted previously to obtain a 100\% redshift completeness of the sample. The population of high $A_V$ quasars (HAQs), a specific sub-population of quasars selected from optical/near-infrared photometry, is found to contribute $21\%^{+9}_{-5}$ of the parent sample. The full population of bright spatially unresolved quasars represented by our parent sample consists of $39\%^{+9}_{-8}$ reddened quasars defined by having $A_V>0.1$, and $21\%^{+9}_{-5}$ of the sample having $E(B-V)>0.1$ assuming the extinction curve of the Small Magellanic Cloud. We show that the HAQ selection works well for selecting reddened quasars, but some are missed because their optical spectra are too blue to pass the $g-r$ color cut in the HAQ selection. This is either due to a low degree of dust reddening or anomalous spectra. We find that the fraction of quasars with contributing light from the host galaxy is most dominant at $z \lesssim 1$. At higher redshifts the population of spatially unresolved quasars selected by our parent sample is found to be representative of the full population at $J<20$ mag. This work quantifies the bias against reddened quasars in studies that are based solely on optical surveys.Comment: 22 pages, 10 figures, accepted for publication in A&A. The ArXiv abstract has been shortened for it to be printabl

Energetic Cost of Ichthyophonus Infection in Juvenile Pacific Herring (Clupea pallasii)

The energetic costs of fasting and Ichthyophonus infection were measured in juvenile Pacific herring (Clupea pallasii) in a lab setting at three temperatures. Infected herring incurred significant energetic costs, the magnitude of which depended on fish condition at the time of infection (fat versus lean). Herring that were fed continually and were in relatively good condition at the time of infection (fat) never stored lipid despite ad libitum feeding. In feeding herring, the energetic cost of infection was a 30% reduction in total energy content relative to controls 52 days post infection. Following food deprivation (lean condition), infection caused an initial delay in the compensatory response of herring. Thirty-one days after re-feeding, the energetic cost of infection in previously-fasted fish was a 32% reduction in total energy content relative to controls. Body composition of infected herring subsequently recovered to some degree, though infected herring never attained the same energy content as their continuously fed counterparts. Fifty-two days after re-feeding, the energetic cost of infection in previously-fasted fish was a 6% reduction in total energy content relative to controls. The greatest impacts of infection occurred in colder temperatures, suggesting Ichthyophonus-induced reductions in body condition may have greater consequences in the northern extent of herring's range, where juveniles use most of their energy reserves to survive their first winter

Spectroscopy at B-factories Using Hard Photon Emission

The process of hard photon emission by initial electrons (positrons) at B-factories is discussed. It is shown that studies of the bottomonium spectroscopy will be feasible for the planned integrated luminosity of the B-factory experiments.Comment: 9 pages, Latex, 1 fugure, Submitted to Int.Jour.Mod.Phys.

Prospects for detection of Υ(1D)→Υ(1S)ππ\Upsilon(1D) \to \Upsilon(1S) \pi \pi via Υ(3S)→Υ(1D)+X\Upsilon(3S) \to \Upsilon(1D) + X

At least one state in the first family of D-wave $b \bar b$ quarkonium levels has been discovered near the predicted mass of 10.16 GeV/$c^2$. This state is probably the one with J=2. This state and the ones with J=1 and J=3 may contribute a detectable amount to the decay $\Upsilon(1D) \to \Upsilon(1S) \pi \pi$, depending on the partial widths for these decays for which predictions vary considerably. The prospects for detection of the chain $\Upsilon(3S) \to \Upsilon(1D) + X \to \Upsilon \pi \pi + X$ are discussed.Comment: 4 pages, LaTeX, 1 figure, to be published in Phys. Rev. D, comment added after Eq. (2

Effects of Environmental Temperature on the Dynamics of Ichthyophoniasis in Juvenile Pacific Herring (Clupea pallasii)

The effects of temperature and infection by Ichthyophonus were examined in juvenile Pacific herring (Clupea pallasii) maintained under simulated overwinter fasting conditions. In addition to defining parameters for a herring bioenergetics model (discussed in Vollenweider et al. this issue), these experiments provided new insights into factors influencing the infectivity and virulence of the parasite Ichthyophonus. In groups of fish with established disease, temperature variation had little effect on disease outcome. Ichthyophonus mortality outpaced that resulting from starvation alone. In newly infected fish, temperature variation significantly changed the mortality patterns related to disease. Both elevated and lowered temperatures suppressed disease-related mortality relative to ambient treatments. When parasite exposure dose decreased, an inverse relationship between infection prevalence and temperature was detected. These findings suggest interplay between temperature optima for parasite growth and host immune function and have implications for our understanding of how Ichthyophonus infections are established in wild fish populations

Debris disks in main sequence binary systems

We observed 69 A3-F8 main sequence binary star systems using the Multiband Imaging Photometer for Spitzer onboard the Spitzer Space Telescope. We find emission significantly in excess of predicted photospheric flux levels for 9(+4/-3)% and 40(+7/-6)% of these systems at 24 and 70 microns, respectively. Twenty two systems total have excess emission, including four systems that show excess emission at both wavelengths. A very large fraction (nearly 60%) of observed binary systems with small (<3 AU) separations have excess thermal mission. We interpret the observed infrared excesses as thermal emission from dust produced by collisions in planetesimal belts. The incidence of debris disks around main sequence A3-F8 binaries is marginally higher than that for single old AFGK stars. Whatever combination of nature (birth conditions of binary systems) and nurture (interactions between the two stars) drives the evolution of debris disks in binary systems, it is clear that planetesimal formation is not inhibited to any great degree. We model these dust disks through fitting the spectral energy distributions and derive typical dust temperatures in the range 100--200 K and typical fractional luminosities around 10^-5, with both parameters similar to other Spitzer-discovered debris disks. Our calculated dust temperatures suggest that about half the excesses we observe are derived from circumbinary planetesimal belts and around one third of the excesses clearly suggest circumstellar material. Three systems with excesses have dust in dynamically unstable regions, and we discuss possible scenarios for the origin of this short-lived dust.Comment: ApJ, in press. 57 pages, including 7 figures (one of which is in color