Approximation with Error Bounds in Spark

We introduce a sampling framework to support approximate computing with estimated error bounds in Spark. Our framework allows sampling to be performed at the beginning of a sequence of multiple transformations ending in an aggregation operation. The framework constructs a data provenance tree as the computation proceeds, then combines the tree with multi-stage sampling and population estimation theories to compute error bounds for the aggregation. When information about output keys are available early, the framework can also use adaptive stratified reservoir sampling to avoid (or reduce) key losses in the final output and to achieve more consistent error bounds across popular and rare keys. Finally, the framework includes an algorithm to dynamically choose sampling rates to meet user specified constraints on the CDF of error bounds in the outputs. We have implemented a prototype of our framework called ApproxSpark, and used it to implement five approximate applications from different domains. Evaluation results show that ApproxSpark can (a) significantly reduce execution time if users can tolerate small amounts of uncertainties and, in many cases, loss of rare keys, and (b) automatically find sampling rates to meet user specified constraints on error bounds. We also explore and discuss extensively trade-offs between sampling rates, execution time, accuracy and key loss

The Impact of Social Culture Environment for Modern Science Development: Based on the Understanding of Merton's Dissertation

In the ages of Big Science and under the situation of advocating an innovative society, it’s very significant to re-read the Merton’s Dissertation named ″Science, Technology and Society in the 17th Century England″ and explore fascination of thought and modern value in the classics which was recognized as a work of sociology of science. Along the thinking logic of Merton’s text, we make the history of reduction and in-depth analysis of the England social culture environment in 17th century from three aspects of politics, economy and culture. And with this a base point, some important influences of Social Culture Environment to the rise and development of Modern Science are fully discussed from three perspectives which are the driving force of modern science, the turn of scientific research fields and its interaction mechanism