Optimal Elephant Flow Detection

Monitoring the traffic volumes of elephant flows, including the total byte count per flow, is a fundamental capability for online network measurements. We present an asymptotically optimal algorithm for solving this problem in terms of both space and time complexity. This improves on previous approaches, which can only count the number of packets in constant time. We evaluate our work on real packet traces, demonstrating an up to X2.5 speedup compared to the best alternative.Comment: Accepted to IEEE INFOCOM 201

Optimizing Lossy Compression Rate-Distortion from Automatic Online Selection between SZ and ZFP

With ever-increasing volumes of scientific data produced by HPC applications, significantly reducing data size is critical because of limited capacity of storage space and potential bottlenecks on I/O or networks in writing/reading or transferring data. SZ and ZFP are the two leading lossy compressors available to compress scientific data sets. However, their performance is not consistent across different data sets and across different fields of some data sets: for some fields SZ provides better compression performance, while other fields are better compressed with ZFP. This situation raises the need for an automatic online (during compression) selection between SZ and ZFP, with a minimal overhead. In this paper, the automatic selection optimizes the rate-distortion, an important statistical quality metric based on the signal-to-noise ratio. To optimize for rate-distortion, we investigate the principles of SZ and ZFP. We then propose an efficient online, low-overhead selection algorithm that predicts the compression quality accurately for two compressors in early processing stages and selects the best-fit compressor for each data field. We implement the selection algorithm into an open-source library, and we evaluate the effectiveness of our proposed solution against plain SZ and ZFP in a parallel environment with 1,024 cores. Evaluation results on three data sets representing about 100 fields show that our selection algorithm improves the compression ratio up to 70% with the same level of data distortion because of very accurate selection (around 99%) of the best-fit compressor, with little overhead (less than 7% in the experiments).Comment: 14 pages, 9 figures, first revisio

Real-Time Dense Stereo Matching With ELAS on FPGA Accelerated Embedded Devices

For many applications in low-power real-time robotics, stereo cameras are the sensors of choice for depth perception as they are typically cheaper and more versatile than their active counterparts. Their biggest drawback, however, is that they do not directly sense depth maps; instead, these must be estimated through data-intensive processes. Therefore, appropriate algorithm selection plays an important role in achieving the desired performance characteristics. Motivated by applications in space and mobile robotics, we implement and evaluate a FPGA-accelerated adaptation of the ELAS algorithm. Despite offering one of the best trade-offs between efficiency and accuracy, ELAS has only been shown to run at 1.5-3 fps on a high-end CPU. Our system preserves all intriguing properties of the original algorithm, such as the slanted plane priors, but can achieve a frame rate of 47fps whilst consuming under 4W of power. Unlike previous FPGA based designs, we take advantage of both components on the CPU/FPGA System-on-Chip to showcase the strategy necessary to accelerate more complex and computationally diverse algorithms for such low power, real-time systems.Comment: 8 pages, 7 figures, 2 table

A Simple Baseline for Travel Time Estimation using Large-Scale Trip Data

The increased availability of large-scale trajectory data around the world provides rich information for the study of urban dynamics. For example, New York City Taxi Limousine Commission regularly releases source-destination information about trips in the taxis they regulate. Taxi data provide information about traffic patterns, and thus enable the study of urban flow -- what will traffic between two locations look like at a certain date and time in the future? Existing big data methods try to outdo each other in terms of complexity and algorithmic sophistication. In the spirit of "big data beats algorithms", we present a very simple baseline which outperforms state-of-the-art approaches, including Bing Maps and Baidu Maps (whose APIs permit large scale experimentation). Such a travel time estimation baseline has several important uses, such as navigation (fast travel time estimates can serve as approximate heuristics for A search variants for path finding) and trip planning (which uses operating hours for popular destinations along with travel time estimates to create an itinerary).Comment: 12 page

Minimising latency of pitch detection algorithms for live vocals on low-cost hardware

A pitch estimation device was proposed for live vocals to output appropriate pitch data through the musical instrument digital interface (MIDI). The intention was to ideally achieve unnoticeable latency while maintaining estimation accuracy. The projected target platform was low-cost, standalone hardware based around a microcontroller such as the Microchip PIC series. This study investigated, optimised and compared the performance of suitable algorithms for this application. Performance was determined by two key factors: accuracy and latency. Many papers have been published over the past six decades assessing and comparing the accuracy of pitch detection algorithms on various signals, including vocals. However, very little information is available concerning the latency of pitch detection algorithms and methods with which this can be minimised. Real-time audio introduces a further latency challenge that is sparsely studied, minimising the length of sampled audio required by the algorithms in order to reduce overall total latency. Thorough testing was undertaken in order to determine the best-performing algorithm and optimal parameter combination. Software modifications were implemented to facilitate accurate, repeatable, automated testing in order to build a comprehensive set of results encompassing a wide range of test conditions. The results revealed that the infinite-peak-clipping autocorrelation function (IACF) performed better than the other autocorrelation functions tested and also identified ideal parameter values or value ranges to provide the optimal latency/accuracy balance. Although the results were encouraging, testing highlighted some fundamental issues with vocal pitch detection. Potential solutions are proposed for further development

Accurate Jitter Decomposition in High-Speed Links

In a high-speed digital communication system, jitter performance plays a crucial role in Bit-Error Rate (BER). It is important to accurately derive each type of jitter as well as total jitter (TJ) and to identify the root causes of jitter by jitter decomposition. In this work, we propose new jitter decomposition techniques in high-speed links testing. The background of jitter decomposition is described in chapter 1. In chapter 2, duty cycle distortion jitter amplification is introduced. As channel loss results in both ISI and jitter amplification, DCD amplification is a big concern in high-speed links. The derivation of a formula of DCD amplification for data channels is included and the calculation result matches the time-domain simulation in the system. Chapter 3 provides an accurate jitter decomposition algorithm using Least Squares (LS) which simultaneously separates ISI, RJ, and PJ. A new time domain ISI model is proposed, which is faster and more accurate than the conventional ISI model. This algorithm obtains the estimated individual jitter component value with fine accuracy by using less samples of total jitter data compared with conventional methods. The simulation and measurement show the accuracy and efficiency of this algorithm with less data samples. In chapter 4, a low-cost comparator-based jitter decomposition algorithm is proposed. Instead of using TIE jitter sequence to decompose, it uses a low cost and simple comparator network to identify the deviation of current sampling positions from the ideal sampling positions to represent the TIE. It simultaneously separates ISI, DCD, and PJ and can achieve similar accuracy compared to the instrument test. Both the simulation and measurement show the decomposition algorithm with great accuracy and efficiency. In chapter 5, a low cost and simple dithering method to improve the test of linearity of analog-to-digital converter (ADC) is proposed. This method exhibits an improvement and enhancement for the ultra-fast segmented model identification of linearity error (uSMILE) algorithm which reduces 99% of the test time compared to the conventional method. In this study, we proposed three types of distribution dithering methods adding to the ramp input signal to reduce the estimation error when uSMILE was applied in low resolution ADCs. The fix pattern distribution was proved as the most efficient and cost-effective method by comparing with the Gaussian, uniform, and fix-pattern distributions. Both the simulation results and hardware measurement indicate that the estimation error can be significantly reduced in 12-bit SAR ADC with effective dithering

Concepts for on-board satellite image registration. Volume 2: IAS prototype performance evaluation standard definition

Problems encountered in testing onboard signal processing hardware designed to achieve radiometric and geometric correction of satellite imaging data are considered. These include obtaining representative image and ancillary data for simulation and the transfer and storage of a large quantity of image data at very high speed. The high resolution, high speed preprocessing of LANDSAT-D imagery is considered

Overview of methods to analyse dynamic data

This book gives an overview of existing data analysis methods to analyse the dynamic data obtained from full scale testing, with their advantages and drawbacks. The overview of full scale testing and dynamic data analysis is limited to energy performance characterization of either building components or whole buildings. The methods range from averaging and regression methods to dynamic approaches based on system identification techniques. These methods are discussed in relation to their application in following in situ measurements: -measurement of thermal transmittance of building components based on heat flux meters; -measurement of thermal and solar transmittance of building components tested in outdoor calorimetric test cells; -measurement of heat transfer coefficient and solar aperture of whole buildings based on co-heating or transient heating tests; -characterisation of the energy performance of whole buildings based on energy use monitoring

Longitudinal flying qualities criteria for single-pilot instrument flight operations

Modern estimation and control theory, flight testing, and statistical analysis were used to deduce flying qualities criteria for General Aviation Single Pilot Instrument Flight Rule (SPIFR) operations. The principal concern is that unsatisfactory aircraft dynamic response combined with high navigation/communication workload can produce problems of safety and efficiency. To alleviate these problems. The relative importance of these factors must be determined. This objective was achieved by flying SPIFR tasks with different aircraft dynamic configurations and assessing the effects of such variations under these conditions. The experimental results yielded quantitative indicators of pilot's performance and workload, and for each of them, multivariate regression was applied to evaluate several candidate flying qualities criteria