Hybrid tables for speeding-up data accesses in hybrid database management systems

Peer ReviewedPostprint (published version

Flow Fair Sampling Based on Multistage Bloom Filters

Network traffic distribution is heavy-tailed. Most of network flows are short and carry very few packets, and the number of large flows is small. Traditional random sampling tends to sample more large flows than short ones. However, many applications depend on per-flow traffic other than just large flows. A flow fair sampling based on multistage Bloom filters is proposed. The total measurement interval is divided into n child time intervals. In each child time interval, employ multistage Bloom filters to query the incoming packet’s flow whether exists in flow information table or not, if exists, sample the packet with static sampling rate which is inversely proportional to the estimation flow traffic up to the previous time interval. If it is a new flow’s first packet, create its flow information and insert it into the multistage Bloom filters. The results show that the proposed algorithm is accurate especially for short flows and easy to extend

Approximate filtering of redundant RFID data streams in mobile environment

U zadnje vrijeme RFID tehnologija (Radio Frequency Identification Technology) se naveliko rabi u mnogim aplikacijama kao što su nadgledanje i praćenje objekta, zahvaljujući jedinstvenim značajkama kao što su beskontaktna, brza i simultana identifikacija više ciljeva. Međutim, zbog interferencije faktora okoline i potrebe za detekcijom u realnom vremenu, podaci koje su RFID čitači prikupili često su puni redundancije, a to može smanjiti učinkovitost obrade RFID aplikacijskih servera, pa čak rezultirati i donošenjem krivih zaključaka. Stoga je neophodno potrebno filtrirati redundantne podatke u RFID sustavima prije nego se prenesu do naprednijih aplikacija. U svrhu podržavanja aproksimativnog filtriranja RFID nizova podataka u mobilnom okruženju, u radu se pokušava analizirati mehanizam za učinkovito redundantno filtriranje modelom kliznog prozora. Najprije se daje razvoj aplikacije RFID nizova podataka i arhitektura RFID sustava utemeljeni na međusoftveru. Zatim se predlaže vremensko-prostorni Bloom filtar utemeljen na kliznim prozorima koji proširuje niz podataka s jednom dimenzijom u standardnom Bloom filtru na filtar s dvije dimenzije, pohranjujući i čitača IDs-a i promatrane vremenske oznake originalnih promatranih stavki. U međuvremenu, kako bi se osigralo da se lažno pozitivna brzina ne poveća zbog toga što se popunio prostor filtra, predlažemo strategiju slučajnog nestajanja za brisanje zastarjelih elemenata. Relativno učestale pogreške predloženog filtra, uključujući lažno pozitivne i lažno negativne, teorijski se analiziraju. Eksperimentalni rezultati pokazuju da predloženi filtar može učinkovito filtrirati vremenski redundantne podatke te uspješno locirati RFID objekte.Recently, RFID technology has been widely used in many applications such as object monitoring and tracing due to the unique features such as non-contact, automatic, fast and multi-target identification simultaneously. However, because of the interference of environmental factors and the requirement of real-time detection, the data collected by the RFID readers are often full of redundancy, which may reduce the processing efficiency of RFID application servers, even lead to making false decisions. Therefore, it is of definite necessity to filter the redundant data in RFID systems before transmitting them to the upper applications. In order to support approximate filtering of RFID data streams in mobile environment, this paper intends to study effective redundant filtering mechanism in the sliding window model. Firstly, we introduce the application background of RFID data streams and the RFID system architecture based on middleware. Then, we propose a temporal-spatial Bloom filter based on sliding windows, which extends the one-dimension array in the standard bloom filter to a two-dimension array, storing both reader IDs and the observed timestamps of original observation items. Meanwhile, in order to guarantee the false positive rate does not increase due to the reason that the space of the filter becomes full, we suggest a random decay strategy for deleting the expired elements. The error rates of the suggested filter, including false positives and false negatives, are analysed in theory. Experimental results show that the suggested filter can filter time redundant data effectively and has a good performance to deal with location movement of RFID objects

Hardware acceleration for power efficient deep packet inspection

The rapid growth of the Internet leads to a massive spread of malicious attacks like viruses and malwares, making the safety of online activity a major concern. The use of Network Intrusion Detection Systems (NIDS) is an effective method to safeguard the Internet. One key procedure in NIDS is Deep Packet Inspection (DPI). DPI can examine the contents of a packet and take actions on the packets based on predefined rules. In this thesis, DPI is mainly discussed in the context of security applications. However, DPI can also be used for bandwidth management and network surveillance. DPI inspects the whole packet payload, and due to this and the complexity of the inspection rules, DPI algorithms consume significant amounts of resources including time, memory and energy. The aim of this thesis is to design hardware accelerated methods for memory and energy efficient high-speed DPI. The patterns in packet payloads, especially complex patterns, can be efficiently represented by regular expressions, which can be translated by the use of Deterministic Finite Automata (DFA). DFA algorithms are fast but consume very large amounts of memory with certain kinds of regular expressions. In this thesis, memory efficient algorithms are proposed based on the transition compressions of the DFAs. In this work, Bloom filters are used to implement DPI on an FPGA for hardware acceleration with the design of a parallel architecture. Furthermore, devoted at a balance of power and performance, an energy efficient adaptive Bloom filter is designed with the capability of adjusting the number of active hash functions according to current workload. In addition, a method is given for implementation on both two-stage and multi-stage platforms. Nevertheless, false positive rates still prevents the Bloom filter from extensive utilization; a cache-based counting Bloom filter is presented in this work to get rid of the false positives for fast and precise matching. Finally, in future work, in order to estimate the effect of power savings, models will be built for routers and DPI, which will also analyze the latency impact of dynamic frequency adaption to current traffic. Besides, a low power DPI system will be designed with a single or multiple DPI engines. Results and evaluation of the low power DPI model and system will be produced in future

Enhancing counting bloom filters through Huffman-coded multilayer structures

n/

Network Processors and Next Generation Networks: Design, Applications, and Perspectives

Network Processors (NPs) are hardware platforms born as appealing solutions for packet processing devices in networking applications. Nowadays, a plethora of solutions exists, with no agreement on a common architecture. Each vendor has proposed its specific solution and no official standard still exists. The common features of all proposals are a hierarchy of processors, with a general purpose processor and several units specialized for packet processing, a series of memory devices with different sizes and latencies, a low-level programmability. The target is a platform for networking applications with low time to market and high time in market, thanks to a high flexibility and a programmability simpler than that of ASICs, for example. After about ten years since the "birth" of network processors, this research activity wants to make an analytical balance of their development and usage. Many authoritative opinions suggest that NPs have been "outdated" by multicore or manycore systems, which provide general purpose environments and some specialized cores. The main reasons of these negative opinions are the hard programmability of NPs, which often requires the knowledge of private microcode, or the excessive architectural limits, such as reduced memories and minimal instruction store. Our research shows that Network Processors can be appealing for different applications in networking area, and many interesting solutions can be obtained, which present very high performance, outscoring current solutions. However, the issues of hard programming and remarkable limits exist, and they could be alleviated only by providing almost a comprehensive programming environment and a proper design in terms of processing and memory resources. More e cient solutions can be surely provided, but the experience of network processors has produced an important legacy in developing packet processing engines. In this work, we have realized many devices for networking purposes based on NP platform, in order to understand the complexity of programming, the flexibility of design, the complexity of tasks that can be implemented, the maximum depth of packet processing, the performance of such devices, the real usefulness of NPs in network devices. All these features have been accurately analyzed and will be illustrated in this thesis. Many remarkable results have been obtained, which confirm the Network Processors as appealing solutions for network devices. Moreover, the research on NPs have lead us to analyze and solve more general issues, related for instance to multiprocessor systems or to processors with no big available memory. In particular, the latter issue lead us to design many interesting data structures for set representation and membership query, which are based on randomized techniques and allow for big memory savings

Behaviour recognition and monitoring of the elderly using wearable wireless sensors. Dynamic behaviour modelling and nonlinear classification methods and implementation.

In partnership with iMonSys - an emerging company in the passive care field - a new system, 'Verity', is being developed to fulfil the role of a passive behaviour monitoring and alert detection device, providing an unobtrusive level of care and assessing an individual's changing behaviour and health status whilst still allowing for independence of its elderly user. In this research, a Hidden Markov Model incorporating Fuzzy Logic-based sensor fusion is created for the behaviour detection within Verity, with a method of Fuzzy-Rule induction designed for the system's adaptation to a user during operation. A dimension reduction and classification scheme utilising Curvilinear Distance Analysis is further developed to deal with the recognition task presented by increasingly nonlinear and high dimension sensor readings, and anomaly detection methods situated within the Hidden Markov Model provide possible solutions to identification of health concerns arising from independent living. Real-time implementation is proposed through development of an Instance Based Learning approach in combination with a Bloom Filter, speeding up the classification operation and reducing the storage requirements for the considerable amount of observation data obtained during operation. Finally, evaluation of all algorithms is completed using a simulation of the Verity system with which the behaviour monitoring task is to be achieved