Stocator: A High Performance Object Store Connector for Spark

We present Stocator, a high performance object store connector for Apache Spark, that takes advantage of object store semantics. Previous connectors have assumed file system semantics, in particular, achieving fault tolerance and allowing speculative execution by creating temporary files to avoid interference between worker threads executing the same task and then renaming these files. Rename is not a native object store operation; not only is it not atomic, but it is implemented using a costly copy operation and a delete. Instead our connector leverages the inherent atomicity of object creation, and by avoiding the rename paradigm it greatly decreases the number of operations on the object store as well as enabling a much simpler approach to dealing with the eventually consistent semantics typical of object stores. We have implemented Stocator and shared it in open source. Performance testing shows that it is as much as 18 times faster for write intensive workloads and performs as much as 30 times fewer operations on the object store than the legacy Hadoop connectors, reducing costs both for the client and the object storage service provider

AIOps for a Cloud Object Storage Service

With the growing reliance on the ubiquitous availability of IT systems and services, these systems become more global, scaled, and complex to operate. To maintain business viability, IT service providers must put in place reliable and cost efficient operations support. Artificial Intelligence for IT Operations (AIOps) is a promising technology for alleviating operational complexity of IT systems and services. AIOps platforms utilize big data, machine learning and other advanced analytics technologies to enhance IT operations with proactive actionable dynamic insight. In this paper we share our experience applying the AIOps approach to a production cloud object storage service to get actionable insights into system's behavior and health. We describe a real-life production cloud scale service and its operational data, present the AIOps platform we have created, and show how it has helped us resolving operational pain points.Comment: 5 page

Establishing local temporal heap safety properties with applications to compile-time memory management

AbstractWe present a framework for statically reasoning about temporal heap safety properties. We focus on local temporal heap safety properties, in which the verification process may be performed for a program object independently of other program objects. We apply our framework to produce new conservative static algorithms for compile-time memory management, which prove for certain program points that a memory object or a heap reference will not be needed further. These algorithms can be used for reducing space consumption of Java programs. We have implemented a prototype of our framework, and used it to verify compile-time memory management properties for several small, but interesting example programs, including JavaCard programs

Post-Replication Repair Suppresses Duplication-Mediated Genome Instability

RAD6 is known to suppress duplication-mediated gross chromosomal rearrangements (GCRs) but not single-copy sequence mediated GCRs. Here, we found that the RAD6- and RAD18-dependent post-replication repair (PRR) and the RAD5-, MMS2-, UBC13-dependent error-free PRR branch acted in concert with the replication stress checkpoint to suppress duplication-mediated GCRs formed by homologous recombination (HR). The Rad5 helicase activity, but not its RING finger, was required to prevent duplication-mediated GCRs, although the function of Rad5 remained dependent upon modification of PCNA at Lys164. The SRS2, SGS1, and HCS1 encoded helicases appeared to interact with Rad5, and epistasis analysis suggested that Srs2 and Hcs1 act upstream of Rad5. In contrast, Sgs1 likely functions downstream of Rad5, potentially by resolving DNA structures formed by Rad5. Our analysis is consistent with models in which PRR prevents replication damage from becoming double strand breaks (DSBs) and/or regulates the activity of HR on DSBs

Provenance of north Gondwana Cambrian-Ordovician sandstone: U-Pb SHRIMP dating of detrital zircons from Israel and Jordan

A vast sequence of quartz-rich sandstone was deposited over North Africa and Arabia during Early Palaeozoic times, in the aftermath of Neoproterozoic Pan-African orogeny and the amalgamation of Gondwana. This rock sequence forms a relatively thin sheet (1–3 km thick) that was transported over a very gentle slope and deposited over a huge area. The sense of transport indicates unroofing of Gondwana terranes but the exact provenance of the siliciclastic deposit remains unclear. Detrital zircons from Cambrian arkoses that immediately overlie the Neoproterozoic Arabian–Nubian Shield in Israel and Jordan yielded Neoproterozoic U–Pb ages (900–530 Ma), suggesting derivation from a proximal source such as the Arabian–Nubian Shield. A minor fraction of earliest Neoproterozoic and older age zircons was also detected. Upward in the section, the proportion of old zircons increases and reaches a maximum (40%) in the Ordovician strata of Jordan. The major earliest Neoproterozoic and older age groups detected are 0.95–1.1, 1.8–1.9 and 2.65–2.7 Ga, among which the 0.95–1.1 Ga group is ubiquitous and makes up as much as 27% in the Ordovician of Jordan, indicating it is a prominent component of the detrital zircon age spectra of northeast Gondwana. The pattern of zircon ages obtained in the present work reflects progressive blanketing of the northern Arabian–Nubian Shield by Cambrian–Ordovician sediments and an increasing contribution from a more distal source, possibly south of the Arabian–Nubian Shield. The significant changes in the zircon age signal reflect many hundreds of kilometres of southward migration of the provenance

Domain-size control by global feedback in bistable systems

We study domain structures in bistable systems such as the Ginzburg-Landau equation. The size of domains can be controlled by a global negative feedback. The domain-size control is applied for a localized spiral pattern

Convection in Binary Fluid Mixtures. II. Localized Traveling Waves. (Physical Review E, in press)

Nonlinear, spatially localized structures of traveling convection rolls are investigated in quantitative detail as a function of Rayleigh number for two different Soret coupling strengths (separation ratios) with Lewis and Prandtl numbers characterizing ethanol-water mixtures. A finite-difference method was used to solve the full hydrodynamic field equations numerically. Structure and dynamics of these localized traveling waves (LTW) are dominated by the concentration field. Like in the spatially extended convective states ( cf. accompanying paper), the Soret-induced concentration variations strongly influence, via density changes, the buoyancy forces that drive convection. The spatio-temporal properties of this feed-back mechanism, involving boundary layers and concentration plumes, show that LTW's are strongly nonlinear states. Light intensity distributions are determined that can be observed in side-view shadowgraphs. Detailed analyses of all fields are made using colour-coded isoplots, among others. In the frame comoving with their drift velocity, LTW's display a nontrivial spatio-temporal symmetry consisting of time-translation by half an oscillation period combined with vertical reflection through the horizontal midplane of the layer. A time-averaged concentration current is driven by a phase difference between the waves of concentration and vertical velocity in the bulk of the LTW state. The associated large-scale concentration redistribution stabilizes the LTW and controls its drift velocity into the quiescent fluid by generating a buoyancy-reducing concentration "barrier" ahead of the leading LTW front. The selection of the width of the LTW's is investigated and comparisons with experiments are presented.Comment: 18 pages and 6 figures as uuencoded Postscript file (using uufiles) 1 color figure as uuencoded Postscript file, a high resolution version of the color figure (about 10MB) can be requested from [email protected] or [email protected].: (Barten)present address: PSI, CH-5232 Villigen PSI, Switzerlan

Convective Fingering of an Autocatalytic Reaction Front

We report experimental observations of the convection-driven fingering instability of an iodate-arsenous acid chemical reaction front. The front propagated upward in a vertical slab; the thickness of the slab was varied to control the degree of instability. We observed the onset and subsequent nonlinear evolution of the fingers, which were made visible by a {\it p}H indicator. We measured the spacing of the fingers during their initial stages and compared this to the wavelength of the fastest growing linear mode predicted by the stability analysis of Huang {\it et. al.} [{\it Phys. Rev. E}, {\bf 48}, 4378 (1993), and unpublished]. We find agreement with the thickness dependence predicted by the theory.Comment: 11 pages, RevTex with 3 eps figures. To be published in Phys Rev E, [email protected], [email protected], [email protected]