The Family of MapReduce and Large Scale Data Processing Systems

In the last two decades, the continuous increase of computational power has produced an overwhelming flow of data which has called for a paradigm shift in the computing architecture and large scale data processing mechanisms. MapReduce is a simple and powerful programming model that enables easy development of scalable parallel applications to process vast amounts of data on large clusters of commodity machines. It isolates the application from the details of running a distributed program such as issues on data distribution, scheduling and fault tolerance. However, the original implementation of the MapReduce framework had some limitations that have been tackled by many research efforts in several followup works after its introduction. This article provides a comprehensive survey for a family of approaches and mechanisms of large scale data processing mechanisms that have been implemented based on the original idea of the MapReduce framework and are currently gaining a lot of momentum in both research and industrial communities. We also cover a set of introduced systems that have been implemented to provide declarative programming interfaces on top of the MapReduce framework. In addition, we review several large scale data processing systems that resemble some of the ideas of the MapReduce framework for different purposes and application scenarios. Finally, we discuss some of the future research directions for implementing the next generation of MapReduce-like solutions.Comment: arXiv admin note: text overlap with arXiv:1105.4252 by other author

emgr - The Empirical Gramian Framework

System Gramian matrices are a well-known encoding for properties of input-output systems such as controllability, observability or minimality. These so-called system Gramians were developed in linear system theory for applications such as model order reduction of control systems. Empirical Gramian are an extension to the system Gramians for parametric and nonlinear systems as well as a data-driven method of computation. The empirical Gramian framework - emgr - implements the empirical Gramians in a uniform and configurable manner, with applications such as Gramian-based (nonlinear) model reduction, decentralized control, sensitivity analysis, parameter identification and combined state and parameter reduction

Improving reconfigurable systems reliability by combining periodical test and redundancy techniques: a case study

This paper revises and introduces to the field of reconfigurable computer systems, some traditional techniques used in the fields of fault-tolerance and testing of digital circuits. The target area is that of on-board spacecraft electronics, as this class of application is a good candidate for the use of reconfigurable computing technology. Fault tolerant strategies are used in order for the system to adapt itself to the severe conditions found in space. In addition, the paper describes some problems and possible solutions for the use of reconfigurable components, based on programmable logic, in space applications

A field portable ultrasound system for bovine tissue characterization

Efforts by beef producers and beef packers to reduce carcass losses and to move the industry to a value-based marketing system have renewed their interest in the development of new electronic grading techniques for assessing beef carcass composition. A custom ultrasonic data acquisition system has been developed for the purpose of investigating the feasibility of beef tissue characterization. The unit developed is a compact, hand-held, six channel battery-operated data logger capable of sampling A-mode ultrasound signals at a rate of 15 megasamples per second. A custom epoxy encapsulated ultrasound transducer array was developed to fit the inner curvature of the thoracic cavity on top of the intercostal muscle between the 12th and 13th rib. Data collected by the ultrasound unit is temporarily stored in a handheld data terminal/computer for retrieval and analysis on a personal computer at a later time. Ultrasound samples from 39 carcasses were analyzed for backscatter energy content. The ultrasound records were partitioned into two groups based on the contact characteristics between the probe and tissue. Preliminary fat estimation in the longissimus dorsi muscle have resulted in correlation coefficients on two partitioned data sets of 0.81 and 0.82. Preliminary marbling estimation in the longissimus dorsi muscle have resulted in correlation coefficients on two partitioned groups of 0.59 and 0.70

Inspection System And Method For Bond Detection And Validation Of Surface Mount Devices Using Sensor Fusion And Active Perception

A hybrid surface mount component inspection system which includes both vision and infrared inspection techniques to determine the presence of surface mount components on a printed wiring board, and the quality of solder joints of surface mount components on printed wiring boards by using data level sensor fusion to combine data from two infrared sensors to obtain emissivity independent thermal signatures of solder joints, and using feature level sensor fusion with active perception to assemble and process inspection information from any number of sensors to determine characteristic feature sets of different defect classes to classify solder defects.Georgia Tech Research Corporatio