Enhanced User-driven Ranking System with Splay Tree

E-learning is one of the information and communication technology products used for teaching and learning process [35]. An efficient and effective way to construct trust relationship among peer users in e-learning environment is ranking. User-driven ranking systems are based only on the feedback or rating provided by the users. In [46-48] the authors provide a variety of trust and reputation methods. Certified Belief in Strength (CBS) [45] is a novel trust measurement method based on reputation and strength. In [38] author presents a recommendation system based on the relevant feedback review to predict the user's interests, that are ranked based on the recommendations history they provide previously. Users with higher rating obtain high reputation compared to less scored users. In question answering websites like StackOverflow, new or low scored users are ignored by the community. This discourage them and their involvement with the community reduces further down, as power law states, alleged low users are pushed to the bottom of the ranking list. Avoid this condition by encouraging less reputed users and prevent them from moving further down in ranking level. Thus, low reputed users are provided with few more chances to participate actively in the e-learning environments. A splay tree is a Binary Search Tree with self-balancing skill. The splay tree brings the recently accessed item to the top of the tree, thus active users are always on the top of the tree. A splay tree is used to represent user's ranks, and to semi-splay low ranked users again in the tree thus preventing them from further drowning in the ranking list. The focus of this research work is to find and enhance low reputed users in reputation system by providing few more chances to take part actively in the e-learning environment using the splay tree. Normalized discounted cumulative gain (NDCG) acts as a decision part for identifying drowning users

New Algorithms and Lower Bounds for Sequential-Access Data Compression

This thesis concerns sequential-access data compression, i.e., by algorithms that read the input one or more times from beginning to end. In one chapter we consider adaptive prefix coding, for which we must read the input character by character, outputting each character's self-delimiting codeword before reading the next one. We show how to encode and decode each character in constant worst-case time while producing an encoding whose length is worst-case optimal. In another chapter we consider one-pass compression with memory bounded in terms of the alphabet size and context length, and prove a nearly tight tradeoff between the amount of memory we can use and the quality of the compression we can achieve. In a third chapter we consider compression in the read/write streams model, which allows us passes and memory both polylogarithmic in the size of the input. We first show how to achieve universal compression using only one pass over one stream. We then show that one stream is not sufficient for achieving good grammar-based compression. Finally, we show that two streams are necessary and sufficient for achieving entropy-only bounds.Comment: draft of PhD thesi

Early Packet Rejection Using Dynamic Binary Decision Diagram

A firewall is a hardware or software device that performs inspection on a given incoming/outgoing packets and decide whether to allow/deny the packet from entering/leaving the system. Firewall filters the packets by using a set of rules called firewall policies. The policies define what type of packets should be allowed or discarded. These policies describe the field values that the packet header must contain in order to match a policy in the firewall. The decision for any given packet is made by finding the first matching firewall policy, if any. In a traditional firewall, the packet filter goes through each policy in the list until a matching rule is found; the same process is again repeated for every packet that enters the firewall. The sequential lookup that the firewall uses to find the matching rule is time consuming and the total time it takes to perform the lookup increases as the policy in the list increases. Nowadays, a typical enterprise based firewall will have 1000+ firewall policy in it, which is normal. A major threat to network firewalls is specially crafted malicious packets that target the bottom rules of the firewall’s entire set of filtering rules. This attack’s main objective is to overload the firewall by processing a flood of network traffic that is matched against almost all the filtering rules before it gets rejected by a bottom rule. As a consequence of this malicious flooding network traffic, the firewall performance will decrease and the processing time of network traffic may increase significantly The current research work is based on the observation that an alternative method for the firewall policies can provide a faster lookup and hence a better filtering performance. The method proposed in this research relies on a basic fact that the policy c a n be represented as a simple Boolean expression. Thus, Binary Decision Diagrams (BDDs) are used as a basis for the representation of access list in this study. The contribution of this research work is a proposed method for representing firewall Policies using BDDs to improve the performance of packet filtering. The proposed mechanism is called Static Shuffling Binary Decision Diagram (SS-BDD), and is based on restructuring of the Binary Decision Diagram (BDD) by using byte-wise data structure instead of using Field-wise data structure. Real world traffic is used during the simulation phase to prove the performance of packet filtering. The numerical results obtained by the simulation shows that the proposed technique improves the performance for packet filtering significantly on medium to long access lists. Furthermore, using BDDs for representing the firewall policies provides other Useful characteristics that makes this a beneficial approach to in real world

SUTMS - Unified Threat Management Framework for Home Networks

Home networks were initially designed for web browsing and non-business critical applications. As infrastructure improved, internet broadband costs decreased, and home internet usage transferred to e-commerce and business-critical applications. Today’s home computers host personnel identifiable information and financial data and act as a bridge to corporate networks via remote access technologies like VPN. The expansion of remote work and the transition to cloud computing have broadened the attack surface for potential threats. Home networks have become the extension of critical networks and services, hackers can get access to corporate data by compromising devices attacked to broad- band routers. All these challenges depict the importance of home-based Unified Threat Management (UTM) systems. There is a need of unified threat management framework that is developed specifically for home and small networks to address emerging security challenges. In this research, the proposed Smart Unified Threat Management (SUTMS) framework serves as a comprehensive solution for implementing home network security, incorporating firewall, anti-bot, intrusion detection, and anomaly detection engines into a unified system. SUTMS is able to provide 99.99% accuracy with 56.83% memory improvements. IPS stands out as the most resource-intensive UTM service, SUTMS successfully reduces the performance overhead of IDS by integrating it with the flow detection mod- ule. The artifact employs flow analysis to identify network anomalies and categorizes encrypted traffic according to its abnormalities. SUTMS can be scaled by introducing optional functions, i.e., routing and smart logging (utilizing Apriori algorithms). The research also tackles one of the limitations identified by SUTMS through the introduction of a second artifact called Secure Centralized Management System (SCMS). SCMS is a lightweight asset management platform with built-in security intelligence that can seamlessly integrate with a cloud for real-time updates

Models, Algorithms, and Architectures for Scalable Packet Classification

The growth and diversification of the Internet imposes increasing demands on the performance and functionality of network infrastructure. Routers, the devices responsible for the switch-ing and directing of traffic in the Internet, are being called upon to not only handle increased volumes of traffic at higher speeds, but also impose tighter security policies and provide support for a richer set of network services. This dissertation addresses the searching tasks performed by Internet routers in order to forward packets and apply network services to packets belonging to defined traffic flows. As these searching tasks must be performed for each packet traversing the router, the speed and scalability of the solutions to the route lookup and packet classification problems largely determine the realizable performance of the router, and hence the Internet as a whole. Despite the energetic attention of the academic and corporate research communities, there remains a need for search engines that scale to support faster communication links, larger route tables and filter sets and increasingly complex filters. The major contributions of this work include the design and analysis of a scalable hardware implementation of a Longest Prefix Matching (LPM) search engine for route lookup, a survey and taxonomy of packet classification techniques, a thorough analysis of packet classification filter sets, the design and analysis of a suite of performance evaluation tools for packet classification algorithms and devices, and a new packet classification algorithm that scales to support high-speed links and large filter sets classifying on additional packet fields

On Line Trace Synchronization for Large Scale Distributed Systems

RÉSUMÉ Les systèmes distribués en réseau fournissent une plate-forme informatique polyvalente pour soutenir diverses applications, telles que des algorithmes de routage dans les réseaux de télécommunication, les systèmes bancaires dans les applications de réseau, les systèmes de contrôle d'aéronefs dans le contrôle de processus en temps réel, ou le calcul scientifique, y compris les grilles et grappes de calcul en calcul parallèle. Ces systèmes sont généralement supervisés afin de détecter, de déboguer et d'éviter les problèmes de sécurité ou de performance. Un outil de traçage est une des méthodes les plus efficaces et précises, avec laquelle toutes les informations détaillées pour chaque noeud individuel dans le système peuvent être extraites et étudiées. Typiquement, une tâche énorme est divisée en de nombreuses tâches, qui sont distribuées et exécutées sur plusieurs ordinateurs coopérant en réseau. Ainsi, afin de contrôler la fonctionnalité des systèmes distribués actuels, toutes les informations sont collectées à partir de plusieurs systèmes et appareils embarqués pour une analyse et une visualisation à la fois en ligne et hors ligne. Cette information de traçage, générée à un rythme effarant, est livrée avec estampilles de temps générées localement sur chaque noeud. Ces estampilles sont généralement fondées sur des compteurs de cycle, avec une granularité du niveau de la nanoseconde. Toutefois, les horloges de chaque noeud sont indépendantes et donc asynchrones les unes des autres. Néanmoins, les utilisateurs s'attendent à voir la sortie de l'analyse en temps réel, sur un axe de référence de temps commun, afin d'être en mesure de diagnostiquer les problèmes plus facilement. La portée de l'oeuvre proposée ici est la synchronisation efficace et en direct de traces générées dans un environnement de grande grappe d'ordinateurs avec des estampilles de temps de granularité du niveau de la nanoseconde, produites par des horloges non synchronisées. Par ailleurs, le modèle de trafic du réseau, le nombre de noeuds informatiques disponibles et même la topologie du réseau peuvent changer. En effet, les grands centres de données roulent un ensemble diversifié et en constante évolution d'applications. Les noeuds peuvent échouer ou revenir en ligne à tout moment, et même le réseau peut être reconfiguré dynamiquement. Ainsi, motivé par la grande échelle des systèmes ciblés, le volume élevé de flux de traces de données associés, la limitation des tampons mémoire et la nécessité d'une analyse en direct, et la haute précision de synchronisation requise, nous avons conçu une nouvelle approche incrémentale pour synchroniser les traces de plusieurs ordinateurs connectés à un réseau dynamique à grande échelle. Tout d'abord, nous présentons une nouvelle technique de synchronisation en direct des connexions individuelles basée sur la classification rapide des paquets échangés, soit comme des paquets précis ou des paquets inintéressants. Cette méthode permet d'obtenir à la fois le plus bas coût de calcul, une latence minimale et une meilleure précision. Deuxièmement, nous avons proposé un algorithme efficace pour calculer incrémentalement l'arbre couvrant minimum des liaisons réseau avec la meilleure précision (plus faible inexactitude) afin de permettre le calcul efficace de paramètres de synchronisation transitive entre deux noeuds qui ne sont pas connectés directement. Ce problème est un défi multiple puisque l'exactitude des liens change au fur et à mesure que des paquets sont échangés entre deux noeuds, de nouveaux liens peuvent apparaître lorsque les noeuds commencent à échanger des paquets, et de nouveaux noeuds peuvent aussi apparaître. Enfin, nous avons proposé un nouvel algorithme pour identifier efficacement et mettre à jour le noeud de référence optimal dans l'arbre couvrant minimum, afin d'utiliser ce noeud comme référence de temps pour l'analyse et la visualisation des traces de plusieurs noeuds. En résumé, nous avons conçu et mis en oeuvre une nouvelle procédure efficace et complète pour la synchronisation de trace optimale, dans un environnement de très grande grappe d'ordinateurs, en direct. Le Linux Trace Toolkit next generation (LTTng), développé à l'École Polytechnique de Montréal, offre une trace d'exécution détaillée des systèmes Linux avec faible surcharge. Notre nouvelle procédure a été programmée et validée par la synchronisation en ligne d'énormes traces LTTng dans de grands réseaux dynamiques.----------ABSTRACT Networked distributed systems provide a versatile computing platform for supporting various applications, such as routing algorithms in telecommunication networks, banking systems in network applications, aircraft control systems in real-time process control, or scientific computing including cluster and grid computing in parallel computation. These systems are typically monitored to detect, debug and avoid security or performance problems. A tracing tool is one of the most efficient and precise methods, in which all the detailed information for every individual node in the system can be extracted and studied. Typically, a particular huge task is divided into many tasks, which are distributed and run on several cooperating networked computers. Hence, in order to monitor the functionality of current distributed systems, all information is collected, from multiple systems and embedded devices, for both online and a posteriori offline analysis and viewing. This tracing information, generated at a staggering rate, comes with timestamps locally generated on each node. These timestamps are typically based on cycle counters, with a nanosecond level granularity. However, the clocks in each node are independent and thus asynchronous from one another. Nonetheless, users expect to see the analysis output in real-time, on a common time reference axis, in order to be able to diagnose problems more easily. The scope of the work proposed here is the efficient and live synchronization of traces generated in distributed systems with nanosecond granularity timestamps produced by unsynchronized clocks. Moreover, the pattern of network traffic, the number of available computer nodes and even the network topology can change. Indeed, distributed systems run a diverse and changing set of applications, nodes may fail or come back online at any time, and even the network can be reconfigured dynamically. Thus, motivated by the large scale of targeted systems, the high volume of associated trace data streams, the data buffering limitations, and the need for live analysis and high synchronization precision, we designed a new incremental approach to synchronize traces from multiple connected computers in a large scale dynamic network. First, we present a novel schema for live synchronization of individual connections based on the fast classification of exchanged packets as either accurate packets or uninteresting packets. This method achieves at the same time the lowest computing cost, lowest latency and best accuracy. Secondly, we proposed an efficient algorithm to incrementally compute the minimum spanning tree of network links with the best precision (lowest inaccuracy) in order to allow the efficient computation of synchronization parameters transitively between two nodes which are not connected directly. This problem is a multiple challenge since the accuracy of links changes as more packets are exchanged between two nodes, new links may appear when nodes start exchanging packets, and new nodes may appear as well. Finally, we proposed a new algorithm to efficiently identify and update the optimal reference node in the minimum spanning tree, in order to use this node as time reference when analyzing and visualizing traces from multiple nodes. In summary, we designed and implemented a new efficient procedure for optimum trace synchronization in a live distributed systems. The Linux Trace Toolkit next generation (LTTng), developed at Polytechnique Montreal, provides a detailed execution trace of Linux systems with low overhead. Our new procedure was programmed and validated through the online synchronization of huge LTTng traces in large dynamic networks

Enhancing snort IDs performance using data mining

Intrusion detection systems (IDSs) such as Snort apply deep packet inspection to detect intrusions. Usually, these are rule-based systems, where each incoming packet is matched with a set of rules. Each rule consists of two parts: the rule header and the rule options. The rule header is compared with the packet header. The rule options usually contain a signature string that is matched with packet content using an efficient string matching algorithm. The traditional approach to IDS packet inspection checks a packet against the detection rules by scanning from the first rule in the set and continuing to scan all the rules until a match is found. This approach becomes inefficient if the number of rules is too large and if the majority of the packets match with rules located at the end of the rule set. In this thesis, we propose an intelligent predictive technique for packet inspection based on data mining. We consider each rule in a rule set as a ‘class’. A classifier is first trained with labeled training data. Each such labeled data point contains packet header information, packet content summary information, and the corresponding class label (i.e. the rule number with which the packet matches). Then the classifier is used to classify new incoming packets. The predicted class, i.e. rule, is checked against the packet to see if this packet really matches the predicted rule. If it does, the corresponding action (i.e. alert) of the rule is taken. Otherwise, if the prediction of the classifier is wrong, we go back to the traditional way of matching rules. The advantage of this intelligent predictive packet matching is that it offers much faster rule matching. We have proved, both analytically and empirically, that even with millions of real network traffic packets and hundreds of rules, the classifier can achieve very high accuracy, thereby making the IDS several times faster in making matching decisions

Computer Aided Verification

This open access two-volume set LNCS 13371 and 13372 constitutes the refereed proceedings of the 34rd International Conference on Computer Aided Verification, CAV 2022, which was held in Haifa, Israel, in August 2022. The 40 full papers presented together with 9 tool papers and 2 case studies were carefully reviewed and selected from 209 submissions. The papers were organized in the following topical sections: Part I: Invited papers; formal methods for probabilistic programs; formal methods for neural networks; software Verification and model checking; hyperproperties and security; formal methods for hardware, cyber-physical, and hybrid systems. Part II: Probabilistic techniques; automata and logic; deductive verification and decision procedures; machine learning; synthesis and concurrency. This is an open access book

Effective techniques for understanding and improving data structure usage

Turing Award winner Niklaus Wirth famously noted, `Algorithms + Data Structures = Programs', and it follows that data structures should be carefully considered for effective application development. In fact, data structures are the main focus of program understanding, performance engineering, bug detection, and security enhancement, etc. Our research is aimed at providing effective techniques for analyzing and improving data structure usage in fundamentally new approaches: First, detecting data structures; identifying what data structures are used within an application is a critical step toward application understanding and performance engineering. Second, selecting efficient data structures; analyzing data structures' behavior can recognize improper use of data structures and suggest alternative data structures better suited for the current situation where the application runs. Third, detecting memory leaks for data structures; tracking data accesses with little overhead and their careful analysis can enable practical and accurate memory leak detection. Finally, offloading time-consuming data structure operations; By leveraging a dedicated helper thread that executes the operations on the behalf of the application thread, we can improve the overall performance of the application.Ph.D