A Survey on the Integration of NAND Flash Storage in the Design of File Systems and the Host Storage Software Stack

With the ever-increasing amount of data generate in the world, estimated to reach over 200 Zettabytes by 2025, pressure on efficient data storage systems is intensifying. The shift from HDD to flash-based SSD provides one of the most fundamental shifts in storage technology, increasing performance capabilities significantly. However, flash storage comes with different characteristics than prior HDD storage technology. Therefore, storage software was unsuitable for leveraging the capabilities of flash storage. As a result, a plethora of storage applications have been design to better integrate with flash storage and align with flash characteristics. In this literature study we evaluate the effect the introduction of flash storage has had on the design of file systems, which providing one of the most essential mechanisms for managing persistent storage. We analyze the mechanisms for effectively managing flash storage, managing overheads of introduced design requirements, and leverage the capabilities of flash storage. Numerous methods have been adopted in file systems, however prominently revolve around similar design decisions, adhering to the flash hardware constrains, and limiting software intervention. Future design of storage software remains prominent with the constant growth in flash-based storage devices and interfaces, providing an increasing possibility to enhance flash integration in the host storage software stack

A Survey on the Integration of NAND Flash Storage in the Design of File Systems and the Host Storage Software Stack

With the ever-increasing amount of data generate in the world, estimated to reach over 200 Zettabytes by 2025, pressure on efficient data storage systems is intensifying. The shift from HDD to flash-based SSD provides one of the most fundamental shifts in storage technology, increasing performance capabilities significantly. However, flash storage comes with different characteristics than prior HDD storage technology. Therefore, storage software was unsuitable for leveraging the capabilities of flash storage. As a result, a plethora of storage applications have been design to better integrate with flash storage and align with flash characteristics. In this literature study we evaluate the effect the introduction of flash storage has had on the design of file systems, which providing one of the most essential mechanisms for managing persistent storage. We analyze the mechanisms for effectively managing flash storage, managing overheads of introduced design requirements, and leverage the capabilities of flash storage. Numerous methods have been adopted in file systems, however prominently revolve around similar design decisions, adhering to the flash hardware constrains, and limiting software intervention. Future design of storage software remains prominent with the constant growth in flash-based storage devices and interfaces, providing an increasing possibility to enhance flash integration in the host storage software stack

Exploiting intrinsic flash properties to enhance modern storage systems

The longstanding goals of storage system design have been to provide simple abstractions for applications to efficiently access data while ensuring the data durability and security on a hardware device. The traditional storage system, which was designed for slow hard disk drive with little parallelism, does not fit for the new storage technologies such as the faster flash memory with high internal parallelism. The gap between the storage system software and flash device causes both resource inefficiency and sub-optimal performance. This dissertation focuses on the rethinking of the storage system design for flash memory with a holistic approach from the system level to the device level and revisits several critical aspects of the storage system design including the storage performance, performance isolation, energy-efficiency, and data security. The traditional storage system lacks full performance isolation between applications sharing the device because it does not make the software aware of the underlying flash properties and constraints. This dissertation proposes FlashBlox, a storage virtualization system that utilizes flash parallelism to provide hardware isolation between applications by assigning them on dedicated chips. FlashBlox reduces the tail latency of storage operations dramatically compared with the existing software-based isolation techniques while achieving uniform lifetime for the flash device. As the underlying flash device latency is reduced significantly compared to the conventional hard disk drive, the storage software overhead has become the major bottleneck. This dissertation presents FlashMap, a holistic flash-based storage stack that combines memory, storage and device-level indirections into a unified layer. By combining these layers, FlashMap reduces critical-path latency for accessing data in the flash device and improves DRAM caching efficiency significantly for flash management. The traditional storage software incurs energy-intensive storage operations due to the need for maintaining data durability and security for personal data, which has become a significant challenge for resource-constrained devices such as mobiles and wearables. This dissertation proposes WearDrive, a fast and energy-efficient storage system for wearables. WearDrive treats the battery-backed DRAM as non-volatile memory to store personal data and trades the connected phone’s battery for the wearable’s by performing large and energy-intensive tasks on the phone while performing small and energy-efficient tasks locally using battery-backed DRAM. WearDrive improves wearable’s battery life significantly with negligible impact to the phone’s battery life. The storage software which has been developed for decades is still vulnerable to malware attacks. For example, the encryption ransomware which is a malicious software that stealthily encrypts user files and demands a ransom to provide access to these files. Prior solutions such as ransomware detection and data backups have been proposed to defend against encryption ransomware. Unfortunately, by the time the ransomware is detected, some files already undergo encryption and the user is still required to pay a ransom to access those files. Furthermore, ransomware variants can obtain kernel privilege to terminate or destroy these software-based defense systems. This dissertation presents FlashGuard, a ransomware-tolerant SSD which has a firmware-level recovery system that allows effective data recovery from encryption ransomware. FlashGuard leverages the intrinsic flash properties to defend against the encryption ransomware and adds minimal overhead to regular storage operations.Ph.D

Big Data Security (Volume 3)

After a short description of the key concepts of big data the book explores on the secrecy and security threats posed especially by cloud based data storage. It delivers conceptual frameworks and models along with case studies of recent technology

Detection and Mitigation of Steganographic Malware

A new attack trend concerns the use of some form of steganography and information hiding to make malware stealthier and able to elude many standard security mechanisms. Therefore, this Thesis addresses the detection and the mitigation of this class of threats. In particular, it considers malware implementing covert communications within network traffic or cloaking malicious payloads within digital images. The first research contribution of this Thesis is in the detection of network covert channels. Unfortunately, the literature on the topic lacks of real traffic traces or attack samples to perform precise tests or security assessments. Thus, a propaedeutic research activity has been devoted to develop two ad-hoc tools. The first allows to create covert channels targeting the IPv6 protocol by eavesdropping flows, whereas the second allows to embed secret data within arbitrary traffic traces that can be replayed to perform investigations in realistic conditions. This Thesis then starts with a security assessment concerning the impact of hidden network communications in production-quality scenarios. Results have been obtained by considering channels cloaking data in the most popular protocols (e.g., TLS, IPv4/v6, and ICMPv4/v6) and showcased that de-facto standard intrusion detection systems and firewalls (i.e., Snort, Suricata, and Zeek) are unable to spot this class of hazards. Since malware can conceal information (e.g., commands and configuration files) in almost every protocol, traffic feature or network element, configuring or adapting pre-existent security solutions could be not straightforward. Moreover, inspecting multiple protocols, fields or conversations at the same time could lead to performance issues. Thus, a major effort has been devoted to develop a suite based on the extended Berkeley Packet Filter (eBPF) to gain visibility over different network protocols/components and to efficiently collect various performance indicators or statistics by using a unique technology. This part of research allowed to spot the presence of network covert channels targeting the header of the IPv6 protocol or the inter-packet time of generic network conversations. In addition, the approach based on eBPF turned out to be very flexible and also allowed to reveal hidden data transfers between two processes co-located within the same host. Another important contribution of this part of the Thesis concerns the deployment of the suite in realistic scenarios and its comparison with other similar tools. Specifically, a thorough performance evaluation demonstrated that eBPF can be used to inspect traffic and reveal the presence of covert communications also when in the presence of high loads, e.g., it can sustain rates up to 3 Gbit/s with commodity hardware. To further address the problem of revealing network covert channels in realistic environments, this Thesis also investigates malware targeting traffic generated by Internet of Things devices. In this case, an incremental ensemble of autoencoders has been considered to face the ''unknown'' location of the hidden data generated by a threat covertly exchanging commands towards a remote attacker. The second research contribution of this Thesis is in the detection of malicious payloads hidden within digital images. In fact, the majority of real-world malware exploits hiding methods based on Least Significant Bit steganography and some of its variants, such as the Invoke-PSImage mechanism. Therefore, a relevant amount of research has been done to detect the presence of hidden data and classify the payload (e.g., malicious PowerShell scripts or PHP fragments). To this aim, mechanisms leveraging Deep Neural Networks (DNNs) proved to be flexible and effective since they can learn by combining raw low-level data and can be updated or retrained to consider unseen payloads or images with different features. To take into account realistic threat models, this Thesis studies malware targeting different types of images (i.e., favicons and icons) and various payloads (e.g., URLs and Ethereum addresses, as well as webshells). Obtained results showcased that DNNs can be considered a valid tool for spotting the presence of hidden contents since their detection accuracy is always above 90% also when facing ''elusion'' mechanisms such as basic obfuscation techniques or alternative encoding schemes. Lastly, when detection or classification are not possible (e.g., due to resource constraints), approaches enforcing ''sanitization'' can be applied. Thus, this Thesis also considers autoencoders able to disrupt hidden malicious contents without degrading the quality of the image

Santa Fe New Mexican, 04-21-1910

https://digitalrepository.unm.edu/sfnm_news/1196/thumbnail.jp

New Secure IoT Architectures, Communication Protocols and User Interaction Technologies for Home Automation, Industrial and Smart Environments

Programa Oficial de Doutoramento en Tecnoloxías da Información e das Comunicacións en Redes Móbiles. 5029V01Tese por compendio de publicacións[Abstract] The Internet of Things (IoT) presents a communication network where heterogeneous physical devices such as vehicles, homes, urban infrastructures or industrial machinery are interconnected and share data. For these communications to be successful, it is necessary to integrate and embed electronic devices that allow for obtaining environmental information (sensors), for performing physical actuations (actuators) as well as for sending and receiving data (network interfaces). This integration of embedded systems poses several challenges. It is needed for these devices to present very low power consumption. In many cases IoT nodes are powered by batteries or constrained power supplies. Moreover, the great amount of devices needed in an IoT network makes power e ciency one of the major concerns of these deployments, due to the cost and environmental impact of the energy consumption. This need for low energy consumption is demanded by resource constrained devices, con icting with the second major concern of IoT: security and data privacy. There are critical urban and industrial systems, such as tra c management, water supply, maritime control, railway control or high risk industrial manufacturing systems such as oil re neries that will obtain great bene ts from IoT deployments, for which non-authorized access can posse severe risks for public safety. On the other hand, both these public systems and the ones deployed on private environments (homes, working places, malls) present a risk for the privacy and security of their users. These IoT deployments need advanced security mechanisms, both to prevent access to the devices and to protect the data exchanged by them. As a consequence, it is needed to improve two main aspects: energy e ciency of IoT devices and the use of lightweight security mechanisms that can be implemented by these resource constrained devices but at the same time guarantee a fair degree of security. The huge amount of data transmitted by this type of networks also presents another challenge. There are big data systems capable of processing large amounts of data, but with IoT the granularity and dispersion of the generated information presents a new scenario very di erent from the one existing nowadays. Forecasts anticipate that there will be a growth from the 15 billion installed devices in 2015 to more than 75 billion devices in 2025. Moreover, there will be much more services exploiting the data produced by these networks, meaning the resulting tra c will be even higher. The information must not only be processed in real time, but data mining processes will have to be performed to historical data. The main goal of this Ph.D. thesis is to analyze each one of the previously described challenges and to provide solutions that allow for an adequate adoption of IoT in Industrial, domestic and, in general, any scenario that can obtain any bene t from the interconnection and exibility that IoT brings.[Resumen] La internet de las cosas (IoT o Internet of Things) representa una red de intercomunicaciones en la que participan dispositivos físicos de toda índole, como vehículos, viviendas, electrodomésticos, infraestructuras urbanas o maquinaria y dispositivos industriales. Para que esta comunicación se pueda llevar a cabo es necesario integrar elementos electr onicos que permitan obtener informaci on del entorno (sensores), realizar acciones f sicas (actuadores) y enviar y recibir la informaci on necesaria (interfaces de comunicaciones de red). La integración y uso de estos sistemas electrónicos embebidos supone varios retos. Es necesario que dichos dispositivos presenten un consumo reducido. En muchos casos deberían ser alimentados por baterías o fuentes de alimentación limitadas. Además, la gran cantidad de dispositivos que involucra la IoT hace necesario que la e ciencia energética de los mismos sea una de las principales preocupaciones, por el coste e implicaciones medioambientales que supone el consumo de electricidad de los mismos. Esta necesidad de limitar el consumo provoca que dichos dispositivos tengan unas prestaciones muy limitadas, lo que entra en conflicto con la segunda mayor preocupación de la IoT: la seguridad y privacidad de los datos. Por un lado existen sistemas críticos urbanos e industriales, como puede ser la regulación del tráfi co, el control del suministro de agua, el control marítimo, el control ferroviario o los sistemas de producción industrial de alto riesgo, como refi nerías, que son claros candidatos a benefi ciarse de la IoT, pero cuyo acceso no autorizado supone graves problemas de seguridad ciudadana. Por otro lado, tanto estos sistemas de naturaleza publica, como los que se desplieguen en entornos privados (viviendas, entornos de trabajo o centros comerciales, entre otros) suponen un riesgo para la privacidad y también para la seguridad de los usuarios. Todo esto hace que sean necesarios mecanismos de seguridad avanzados, tanto de acceso a los dispositivos como de protección de los datos que estos intercambian. En consecuencia, es necesario avanzar en dos aspectos principales: la e ciencia energética de los dispositivos y el uso de mecanismos de seguridad e ficientes, tanto computacional como energéticamente, que permitan la implantación de la IoT sin comprometer la seguridad y la privacidad de los usuarios. Por otro lado, la ingente cantidad de información que estos sistemas puede llegar a producir presenta otros dos retos que deben ser afrontados. En primer lugar, el tratamiento y análisis de datos toma una nueva dimensión. Existen sistemas de big data capaces de procesar cantidades enormes de información, pero con la internet de las cosas la granularidad y dispersión de los datos plantean un escenario muy distinto al actual. La previsión es pasar de 15.000.000.000 de dispositivos instalados en 2015 a más de 75.000.000.000 en 2025. Además existirán multitud de servicios que harán un uso intensivo de estos dispositivos y de los datos que estos intercambian, por lo que el volumen de tráfico será todavía mayor. Asimismo, la información debe ser procesada tanto en tiempo real como a posteriori sobre históricos, lo que permite obtener información estadística muy relevante en diferentes entornos. El principal objetivo de la presente tesis doctoral es analizar cada uno de estos retos (e ciencia energética, seguridad, procesamiento de datos e interacción con el usuario) y plantear soluciones que permitan una correcta adopción de la internet de las cosas en ámbitos industriales, domésticos y en general en cualquier escenario que se pueda bene ciar de la interconexión y flexibilidad de acceso que proporciona el IoT.[Resumo] O internet das cousas (IoT ou Internet of Things) representa unha rede de intercomunicaci óns na que participan dispositivos físicos moi diversos, coma vehículos, vivendas, electrodomésticos, infraestruturas urbanas ou maquinaria e dispositivos industriais. Para que estas comunicacións se poidan levar a cabo é necesario integrar elementos electrónicos que permitan obter información da contorna (sensores), realizar accións físicas (actuadores) e enviar e recibir a información necesaria (interfaces de comunicacións de rede). A integración e uso destes sistemas electrónicos integrados supón varios retos. En primeiro lugar, é necesario que estes dispositivos teñan un consumo reducido. En moitos casos deberían ser alimentados por baterías ou fontes de alimentación limitadas. Ademais, a gran cantidade de dispositivos que se empregan na IoT fai necesario que a e ciencia enerxética dos mesmos sexa unha das principais preocupacións, polo custo e implicacións medioambientais que supón o consumo de electricidade dos mesmos. Esta necesidade de limitar o consumo provoca que estes dispositivos teñan unhas prestacións moi limitadas, o que entra en con ito coa segunda maior preocupación da IoT: a seguridade e privacidade dos datos. Por un lado existen sistemas críticos urbanos e industriais, como pode ser a regulación do tráfi co, o control de augas, o control marítimo, o control ferroviario ou os sistemas de produción industrial de alto risco, como refinerías, que son claros candidatos a obter benefi cios da IoT, pero cuxo acceso non autorizado supón graves problemas de seguridade cidadá. Por outra parte tanto estes sistemas de natureza pública como os que se despreguen en contornas privadas (vivendas, contornas de traballo ou centros comerciais entre outros) supoñen un risco para a privacidade e tamén para a seguridade dos usuarios. Todo isto fai que sexan necesarios mecanismos de seguridade avanzados, tanto de acceso aos dispositivos como de protección dos datos que estes intercambian. En consecuencia, é necesario avanzar en dous aspectos principais: a e ciencia enerxética dos dispositivos e o uso de mecanismos de seguridade re cientes, tanto computacional como enerxéticamente, que permitan o despregue da IoT sen comprometer a seguridade e a privacidade dos usuarios. Por outro lado, a inxente cantidade de información que estes sistemas poden chegar a xerar presenta outros retos que deben ser tratados. O tratamento e a análise de datos toma unha nova dimensión. Existen sistemas de big data capaces de procesar cantidades enormes de información, pero coa internet das cousas a granularidade e dispersión dos datos supón un escenario moi distinto ao actual. A previsión e pasar de 15.000.000.000 de dispositivos instalados no ano 2015 a m ais de 75.000.000.000 de dispositivos no ano 2025. Ademais existirían multitude de servizos que farían un uso intensivo destes dispositivos e dos datos que intercambian, polo que o volume de tráfico sería aínda maior. Do mesmo xeito a información debe ser procesada tanto en tempo real como posteriormente sobre históricos, o que permite obter información estatística moi relevante en diferentes contornas. O principal obxectivo da presente tese doutoral é analizar cada un destes retos (e ciencia enerxética, seguridade, procesamento de datos e interacción co usuario) e propor solucións que permitan unha correcta adopción da internet das cousas en ámbitos industriais, domésticos e en xeral en todo aquel escenario que se poda bene ciar da interconexión e flexibilidade de acceso que proporciona a IoT

Categorising Network Telescope data using big data enrichment techniques

Network Telescopes, Internet backbone sampling, IDS and other forms of network-sourced Threat Intelligence provide researchers with insight into the methods and intent of remote entities by capturing network traffic and analysing the resulting data. This analysis and determination of intent is made difficult by the large amounts of potentially malicious traffic, coupled with limited amount of knowledge that can be attributed to the source of the incoming data, as the source is known only by its IP address. Due to the lack of commonly available tooling, many researchers start this analysis from the beginning and so repeat and re-iterate previous research as the bulk of their work. As a result new insight into methods and approaches of analysis is gained at a high cost. Our research approaches this problem by using additional knowledge about the source IP address such as open ports, reverse and forward DNS, BGP routing tables and more, to enhance the researcher's ability to understand the traffic source. The research is a BigData experiment, where large (hundreds of GB) datasets are merged with a two month section of Network Telescope data using a set of Python scripts. The result are written to a Google BigQuery database table. Analysis of the network data is greatly simplified, with questions about the nature of the source, such as its device class (home routing device or server), potential vulnerabilities (open telnet ports or databases) and location becoming relatively easy to answer. Using this approach, researchers can focus on the questions that need answering and efficiently address them. This research could be taken further by using additional data sources such as Geo-location, WHOIS lookups, Threat Intelligence feeds and many others. Other potential areas of research include real-time categorisation of incoming packets, in order to better inform alerting and reporting systems' configuration. In conclusion, categorising Network Telescope data in this way provides insight into the intent of the (apparent) originator and as such is a valuable tool for those seeking to understand the purpose and intent of arriving packets. In particular, the ability to remove packets categorised as non-malicious (e.g. those in the Research category) from the data eliminates a known source of `noise' from the data. This allows the researcher to focus their efforts in a more productive manner