Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management

As users of big data applications expect fresh results, we witness a new breed of stream processing systems (SPS) that are designed to scale to large numbers of cloud-hosted machines. Such systems face new challenges: (i) to benefit from the pay-as-you-go model of cloud computing, they must scale out on demand, acquiring additional virtual machines (VMs) and parallelising operators when the workload increases; (ii) failures are common with deployments on hundreds of VMs - systems must be fault-tolerant with fast recovery times, yet low per-machine overheads. An open question is how to achieve these two goals when stream queries include stateful operators, which must be scaled out and recovered without affecting query results. Our key idea is to expose internal operator state explicitly to the SPS through a set of state management primitives. Based on them, we describe an integrated approach for dynamic scale out and recovery of stateful operators. Externalised operator state is checkpointed periodically by the SPS and backed up to upstream VMs. The SPS identifies individual operator bottlenecks and automatically scales them out by allocating new VMs and partitioning the check-pointed state. At any point, failed operators are recovered by restoring checkpointed state on a new VM and replaying unprocessed tuples. We evaluate this approach with the Linear Road Benchmark on the Amazon EC2 cloud platform and show that it can scale automatically to a load factor of L=350 with 50 VMs, while recovering quickly from failures. Copyright © 2013 ACM

Egocentric Spatial Representation in Action and Perception

Neuropsychological findings used to motivate the “two visual systems” hypothesis have been taken to endanger a pair of widely accepted claims about spatial representation in visual experience. The first is the claim that visual experience represents 3-D space around the perceiver using an egocentric frame of reference. The second is the claim that there is a constitutive link between the spatial contents of visual experience and the perceiver’s bodily actions. In this paper, I carefully assess three main sources of evidence for the two visual systems hypothesis and argue that the best interpretation of the evidence is in fact consistent with both claims. I conclude with some brief remarks on the relation between visual consciousness and rational agency

The Role of Consciousness in Memory

Conscious events interact with memory systems in learning, rehearsal and retrieval (Ebbinghaus 1885/1964; Tulving 1985). Here we present hypotheses that arise from the IDA computional model (Franklin, Kelemen and McCauley 1998; Franklin 2001b) of global workspace theory (Baars 1988, 2002). Our primary tool for this exploration is a flexible cognitive cycle employed by the IDA computational model and hypothesized to be a basic element of human cognitive processing. Since cognitive cycles are hypothesized to occur five to ten times a second and include interaction between conscious contents and several of the memory systems, they provide the means for an exceptionally fine-grained analysis of various cognitive tasks. We apply this tool to the small effect size of subliminal learning compared to supraliminal learning, to process dissociation, to implicit learning, to recognition vs. recall, and to the availability heuristic in recall. The IDA model elucidates the role of consciousness in the updating of perceptual memory, transient episodic memory, and procedural memory. In most cases, memory is hypothesized to interact with conscious events for its normal functioning. The methodology of the paper is unusual in that the hypotheses and explanations presented are derived from an empirically based, but broad and qualitative computational model of human cognition

Engineering of an Extreme Rainfall Detection System using Grid Computing

This paper describes a new approach for intensive rainfall data analysis. ITHACA's Extreme Rainfall Detection System (ERDS) is conceived to provide near real-time alerts related to potential exceptional rainfalls worldwide, which can be used by WFP or other humanitarian assistance organizations to evaluate the event and understand the potentially floodable areas where their assistance is needed. This system is based on precipitation analysis and it uses rainfall data from satellite at worldwide extent. This project uses the Tropical Rainfall Measuring Mission Multisatellite Precipitation Analysis dataset, a NASA-delivered near real-time product for current rainfall condition monitoring over the world. Considering the great deal of data to process, this paper presents an architectural solution based on Grid Computing techniques. Our focus is on the advantages of using a distributed architecture in terms of performances for this specific purpos

Temporal Mental Qualities and Selective Attention

This article presents an argument for the view that we can perceive temporal features without awareness. Evidence for this claim comes from recent empirical work on selective visual attention. An interpretation of selective attention as a mechanism that processes high-level perceptual features is offered and defended against one particular objection. In conclusion, time perception likely has an unconscious dimension and temporal mental qualities can be instantiated without ever being conscious

Summer CO2 evasion from streams and rivers in the Kolyma River basin, north-east Siberia

Inland water systems are generally supersaturated in carbon dioxide (CO2) and are increasingly recognized as playing an important role in the global carbon cycle. The Arctic may be particularly important in this respect, given the abundance of inland waters and carbon contained in Arctic soils; however, a lack of trace gas measurements from small streams in the Arctic currently limits this understanding.We investigated the spatial variability of CO2 evasion during the summer low-flow period from streams and rivers in the northern portion of the Kolyma River basin in north-eastern Siberia. To this end, partial pressure of carbon dioxide (pCO2) and gas exchange velocities (k) were measured at a diverse set of streams and rivers to calculate CO2 evasion fluxes. We combined these CO2 evasion estimates with satellite remote sensing and geographic information system techniques to calculate total areal CO2 emissions. Our results show that small streams are substantial sources of atmospheric CO2 owing to high pCO2 and k, despite being a small portion of total inland water surface area. In contrast, large rivers were generally near equilibrium with atmospheric CO2. Extrapolating our findings across the Panteleikha-Ambolikha sub-watersheds demonstrated that small streams play a major role in CO2 evasion, accounting for 86% of the total summer CO2 emissions from inland waters within these two sub-watersheds. Further expansion of these regional CO2 emission estimates across time and space will be critical to accurately quantify and understand the role of Arctic streams and rivers in the global carbon budget