Robust and secure monitoring and attribution of malicious behaviors

Worldwide computer systems continue to execute malicious software that degrades the systemsâ performance and consumes network capacity by generating high volumes of unwanted traffic. Network-based detectors can effectively identify machines participating in the ongoing attacks by monitoring the traffic to and from the systems. But, network detection alone is not enough; it does not improve the operation of the Internet or the health of other machines connected to the network. We must identify malicious code running on infected systems, participating in global attack networks. This dissertation describes a robust and secure approach that identifies malware present on infected systems based on its undesirable use of network. Our approach, using virtualization, attributes malicious traffic to host-level processes responsible for the traffic. The attribution identifies on-host processes, but malware instances often exhibit parasitic behaviors to subvert the execution of benign processes. We then augment the attribution software with a host-level monitor that detects parasitic behaviors occurring at the user- and kernel-level. User-level parasitic attack detection happens via the system-call interface because it is a non-bypassable interface for user-level processes. Due to the unavailability of one such interface inside the kernel for drivers, we create a new driver monitoring interface inside the kernel to detect parasitic attacks occurring through this interface. Our attribution software relies on a guest kernelâ s data to identify on-host processes. To allow secure attribution, we prevent illegal modifications of critical kernel data from kernel-level malware. Together, our contributions produce a unified research outcome --an improved malicious code identification system for user- and kernel-level malware.Ph.D.Committee Chair: Giffin, Jonathon; Committee Member: Ahamad, Mustaque; Committee Member: Blough, Douglas; Committee Member: Lee, Wenke; Committee Member: Traynor, Patric

Infrastructural Security for Virtualized Grid Computing

The goal of the grid computing paradigm is to make computer power as easy to access as an electrical power grid. Unlike the power grid, the computer grid uses remote resources located at a service provider. Malicious users can abuse the provided resources, which not only affects their own systems but also those of the provider and others. Resources are utilized in an environment where sensitive programs and data from competitors are processed on shared resources, creating again the potential for misuse. This is one of the main security issues, since in a business environment competitors distrust each other, and the fear of industrial espionage is always present. Currently, human trust is the strategy used to deal with these threats. The relationship between grid users and resource providers ranges from highly trusted to highly untrusted. This wide trust relationship occurs because grid computing itself changed from a research topic with few users to a widely deployed product that included early commercial adoption. The traditional open research communities have very low security requirements, while in contrast, business customers often operate on sensitive data that represents intellectual property; thus, their security demands are very high. In traditional grid computing, most users share the same resources concurrently. Consequently, information regarding other users and their jobs can usually be acquired quite easily. This includes, for example, that a user can see which processes are running on another user´s system. For business users, this is unacceptable since even the meta-data of their jobs is classified. As a consequence, most commercial customers are not convinced that their intellectual property in the form of software and data is protected in the grid. This thesis proposes a novel infrastructural security solution that advances the concept of virtualized grid computing. The work started back in 2007 and led to the development of the XGE, a virtual grid management software. The XGE itself uses operating system virtualization to provide a virtualized landscape. Users’ jobs are no longer executed in a shared manner; they are executed within special sandboxed environments. To satisfy the requirements of a traditional grid setup, the solution can be coupled with an installed scheduler and grid middleware on the grid head node. To protect the prominent grid head node, a novel dual-laned demilitarized zone is introduced to make attacks more difficult. In a traditional grid setup, the head node and the computing nodes are installed in the same network, so a successful attack could also endanger the user´s software and data. While the zone complicates attacks, it is, as all security solutions, not a perfect solution. Therefore, a network intrusion detection system is enhanced with grid specific signatures. A novel software called Fence is introduced that supports end-to-end encryption, which means that all data remains encrypted until it reaches its final destination. It transfers data securely between the user´s computer, the head node and the nodes within the shielded, internal network. A lightweight kernel rootkit detection system assures that only trusted kernel modules can be loaded. It is no longer possible to load untrusted modules such as kernel rootkits. Furthermore, a malware scanner for virtualized grids scans for signs of malware in all running virtual machines. Using virtual machine introspection, that scanner remains invisible for most types of malware and has full access to all system calls on the monitored system. To speed up detection, the load is distributed to multiple detection engines simultaneously. To enable multi-site service-oriented grid applications, the novel concept of public virtual nodes is presented. This is a virtualized grid node with a public IP address shielded by a set of dynamic firewalls. It is possible to create a set of connected, public nodes, either present on one or more remote grid sites. A special web service allows users to modify their own rule set in both directions and in a controlled manner. The main contribution of this thesis is the presentation of solutions that convey the security of grid computing infrastructures. This includes the XGE, a software that transforms a traditional grid into a virtualized grid. Design and implementation details including experimental evaluations are given for all approaches. Nearly all parts of the software are available as open source software. A summary of the contributions and an outlook to future work conclude this thesis

Intrusion Detection Systems in Cloud Computing: A Contemporary Review of Techniques and Solutions

Rapid growth of resources and escalating cost of infrastructure is leading organizations to adopt cloud computing. Cloud computing provides high performance, efficient utilization, and on-demand availability of resources. However, the cloud environment is vulnerable to different kinds of intrusion attacks which involve installing malicious software and creating backdoors. In a cloud environment, where businesses have hosted important and critical data, the security of underlying technologies becomes crucial. To mitigate the threat to cloud environments, Intrusion Detection Systems (IDS) are a layer of defense. The aim of this survey paper is to review IDS techniques proposed for the cloud. To achieve this objective, the first step is defining the limitations and unique characteristics of each technique. The second step is establishing the criteria to evaluate IDS architectures. In this paper, the criteria used is derived from basic characteristics of cloud. Next step is a comparative analysis of various existing intrusion detection techniques against the criteria. The last step is on the discussion of drawbacks and open issues, comprehended from the evaluation, due to which implementation of IDS in cloud environment face hurdles

Autoscopy Jr.: Intrusion Detection for Embedded Control Systems

Securing embedded control systems within the power grid presents a unique challenge: on top of the resource restrictions inherent to these devices, SCADA systems must also accommodate strict timing requirements that are non-negotiable, and their massive scale greatly amplifies costs such as power consumption. These constraints make the conventional approach to host intrusion detection--namely, employing virtualization in some manner--too costly or impractical for embedded control systems within critical infrastructure. Instead, we take an in-kernel approach to system protection, building upon the Autoscopy system developed by Ashwin Ramaswamy that places probes on indirectly-called functions and uses them to monitor its host system for behavior characteristic of control-flow-altering malware, such as rootkits. In this thesis, we attempt to show that such a method would indeed be a viable method of protecting embedded control systems. We first identify several issues with the original prototype, and present a new version of the program (dubbed Autoscopy Jr.) that uses trusted location lists to verify that control is coming from a known, trusted location inside our kernel. Although we encountered additional performance overhead when testing our new design, we developed a kernel profiler that allowed us to identify the probes responsible for this overhead and discard them, leaving us with a final probe list that generated less than 5% overhead on every one of our benchmark tests. Finally, we attempted to run Autoscopy Jr. on two specialized kernels (one with an optimized probing framework, and another with a hardening patch installed), finding that the former did not produce enough performance benefits to preclude using our profiler, and that the latter required a different method of scanning for indirect functions for Autoscopy Jr. to operate. We argue that Autoscopy Jr. is indeed a feasible intrusion detection system for embedded control systems, as it can adapt easily to a variety of system architectures and allows us to intelligently balance security and performance on these critical devices

Studying a Virtual Testbed for Unverified Data

It is difficult to fully know the effects a piece of software will have on your computer, particularly when the software is distributed by an unknown source. The research in this paper focuses on malware detection, virtualization, and sandbox/honeypot techniques with the goal of improving the security of installing useful, but unverifiable, software. With a combination of these techniques, it should be possible to install software in an environment where it cannot harm a machine, but can be tested to determine its safety. Testing for malware, performance, network connectivity, memory usage, and interoperability can be accomplished without allowing the program to access the base operating system of a machine. After the full effects of the software are understood and it is determined to be safe, it could then be run from, and given access to, the base operating system. This thesis investigates the feasibility of creating a system to verify the security of unknown software while ensuring it will have no negative impact on the host machine

System Design for Software Packet Processing

The role of software in computer networks has never been more crucial than today, with the advent of Internet-scale services and cloud computing. The trend toward software-based network dataplane—as in network function virtualization—requires software packet processing to meet challenging perfomance requirements, such as supporting exponentially increasing link bandwidth and microsecond-order latency. Many architectural aspects of existing software systems for packet processing, however, are decades old and ill-suited totoday’s network I/O workloads.In this dissertation, we explore the design space of high-performance software packet processing systems in the context of two application domains, . First, we start by discussingthe limitations of BSD Socket, which is a de-facto standard in network I/O for server applications. We quantify its performance limitations and propose a clean-slate API, called MegaPipe, as an alternative to BSD Socket. In the second part of this dissertation, we switch our focus to in-network software systems for network functions, such as network switches and middleboxes. We present Berkeley Extensible Software Switch (BESS), a modular framework for building extensible network functions. BESS introduces various novel techniques to achieve high-performance software packet processing, without compromising on either programmability or flexibility

Securing Virtualized System via Active Protection

Virtualization is the predominant enabling technology of current cloud infrastructure