Technologies and Applications for Big Data Value

This open access book explores cutting-edge solutions and best practices for big data and data-driven AI applications for the data-driven economy. It provides the reader with a basis for understanding how technical issues can be overcome to offer real-world solutions to major industrial areas. The book starts with an introductory chapter that provides an overview of the book by positioning the following chapters in terms of their contributions to technology frameworks which are key elements of the Big Data Value Public-Private Partnership and the upcoming Partnership on AI, Data and Robotics. The remainder of the book is then arranged in two parts. The first part “Technologies and Methods” contains horizontal contributions of technologies and methods that enable data value chains to be applied in any sector. The second part “Processes and Applications” details experience reports and lessons from using big data and data-driven approaches in processes and applications. Its chapters are co-authored with industry experts and cover domains including health, law, finance, retail, manufacturing, mobility, and smart cities. Contributions emanate from the Big Data Value Public-Private Partnership and the Big Data Value Association, which have acted as the European data community's nucleus to bring together businesses with leading researchers to harness the value of data to benefit society, business, science, and industry. The book is of interest to two primary audiences, first, undergraduate and postgraduate students and researchers in various fields, including big data, data science, data engineering, and machine learning and AI. Second, practitioners and industry experts engaged in data-driven systems, software design and deployment projects who are interested in employing these advanced methods to address real-world problems

Technologies and Applications for Big Data Value

This open access book explores cutting-edge solutions and best practices for big data and data-driven AI applications for the data-driven economy. It provides the reader with a basis for understanding how technical issues can be overcome to offer real-world solutions to major industrial areas. The book starts with an introductory chapter that provides an overview of the book by positioning the following chapters in terms of their contributions to technology frameworks which are key elements of the Big Data Value Public-Private Partnership and the upcoming Partnership on AI, Data and Robotics. The remainder of the book is then arranged in two parts. The first part “Technologies and Methods” contains horizontal contributions of technologies and methods that enable data value chains to be applied in any sector. The second part “Processes and Applications” details experience reports and lessons from using big data and data-driven approaches in processes and applications. Its chapters are co-authored with industry experts and cover domains including health, law, finance, retail, manufacturing, mobility, and smart cities. Contributions emanate from the Big Data Value Public-Private Partnership and the Big Data Value Association, which have acted as the European data community's nucleus to bring together businesses with leading researchers to harness the value of data to benefit society, business, science, and industry. The book is of interest to two primary audiences, first, undergraduate and postgraduate students and researchers in various fields, including big data, data science, data engineering, and machine learning and AI. Second, practitioners and industry experts engaged in data-driven systems, software design and deployment projects who are interested in employing these advanced methods to address real-world problems

NLP-Based Techniques for Cyber Threat Intelligence

In the digital era, threat actors employ sophisticated techniques for which, often, digital traces in the form of textual data are available. Cyber Threat Intelligence~(CTI) is related to all the solutions inherent to data collection, processing, and analysis useful to understand a threat actor's targets and attack behavior. Currently, CTI is assuming an always more crucial role in identifying and mitigating threats and enabling proactive defense strategies. In this context, NLP, an artificial intelligence branch, has emerged as a powerful tool for enhancing threat intelligence capabilities. This survey paper provides a comprehensive overview of NLP-based techniques applied in the context of threat intelligence. It begins by describing the foundational definitions and principles of CTI as a major tool for safeguarding digital assets. It then undertakes a thorough examination of NLP-based techniques for CTI data crawling from Web sources, CTI data analysis, Relation Extraction from cybersecurity data, CTI sharing and collaboration, and security threats of CTI. Finally, the challenges and limitations of NLP in threat intelligence are exhaustively examined, including data quality issues and ethical considerations. This survey draws a complete framework and serves as a valuable resource for security professionals and researchers seeking to understand the state-of-the-art NLP-based threat intelligence techniques and their potential impact on cybersecurity

Towards Secure and Intelligent Diagnosis: Deep Learning and Blockchain Technology for Computer-Aided Diagnosis Systems

Cancer is the second leading cause of death across the world after cardiovascular disease. The survival rate of patients with cancerous tissue can significantly decrease due to late-stage diagnosis. Nowadays, advancements of whole slide imaging scanners have resulted in a dramatic increase of patient data in the domain of digital pathology. Large-scale histopathology images need to be analyzed promptly for early cancer detection which is critical for improving patient's survival rate and treatment planning. Advances of medical image processing and deep learning methods have facilitated the extraction and analysis of high-level features from histopathological data that could assist in life-critical diagnosis and reduce the considerable healthcare cost associated with cancer. In clinical trials, due to the complexity and large variance of collected image data, developing computer-aided diagnosis systems to support quantitative medical image analysis is an area of active research. The first goal of this research is to automate the classification and segmentation process of cancerous regions in histopathology images of different cancer tissues by developing models using deep learning-based architectures. In this research, a framework with different modules is proposed, including (1) data pre-processing, (2) data augmentation, (3) feature extraction, and (4) deep learning architectures. Four validation studies were designed to conduct this research. (1) differentiating benign and malignant lesions in breast cancer (2) differentiating between immature leukemic blasts and normal cells in leukemia cancer (3) differentiating benign and malignant regions in lung cancer, and (4) differentiating benign and malignant regions in colorectal cancer. Training machine learning models, disease diagnosis, and treatment often requires collecting patients' medical data. Privacy and trusted authenticity concerns make data owners reluctant to share their personal and medical data. Motivated by the advantages of Blockchain technology in healthcare data sharing frameworks, the focus of the second part of this research is to integrate Blockchain technology in computer-aided diagnosis systems to address the problems of managing access control, authentication, provenance, and confidentiality of sensitive medical data. To do so, a hierarchical identity and attribute-based access control mechanism using smart contract and Ethereum Blockchain is proposed to securely process healthcare data without revealing sensitive information to an unauthorized party leveraging the trustworthiness of transactions in a collaborative healthcare environment. The proposed access control mechanism provides a solution to the challenges associated with centralized access control systems and ensures data transparency and traceability for secure data sharing, and data ownership

Ontologies for the Interoperability of Heterogeneous Multi-Agent Systems in the scope of Energy and Power Systems

Tesis por compendio de publicaciones[ES]El sector eléctrico, tradicionalmente dirigido por monopolios y poderosas empresas de servicios públicos, ha experimentado cambios significativos en las últimas décadas. Los avances más notables son una mayor penetración de las fuentes de energía renovable (RES por sus siglas en inglés) y la generación distribuida, que han llevado a la adopción del paradigma de las redes inteligentes (SG por sus siglas en inglés) y a la introducción de enfoques competitivos en los mercados de electricidad (EMs por sus siglas en inglés) mayoristas y algunos minoristas. Las SG emergieron rápidamente de un concepto ampliamente aceptado en la realidad. La intermitencia de las fuentes de energía renovable y su integración a gran escala plantea nuevas limitaciones y desafíos que afectan en gran medida las operaciones de los EMs. El desafiante entorno de los sistemas de potencia y energía (PES por sus siglas en inglés) refuerza la necesidad de estudiar, experimentar y validar operaciones e interacciones competitivas, dinámicas y complejas. En este contexto, la simulación, el apoyo a la toma de decisiones, y las herramientas de gestión inteligente, se vuelven imprescindibles para estudiar los diferentes mecanismos del mercado y las relaciones entre los actores involucrados. Para ello, la nueva generación de herramientas debe ser capaz de hacer frente a la rápida evolución de los PES, proporcionando a los participantes los medios adecuados para adaptarse, abordando nuevos modelos y limitaciones, y su compleja relación con los desarrollos tecnológicos y de negocios. Las plataformas basadas en múltiples agentes son particularmente adecuadas para analizar interacciones complejas en sistemas dinámicos, como PES, debido a su naturaleza distribuida e independiente. La descomposición de tareas complejas en asignaciones simples y la fácil inclusión de nuevos datos y modelos de negocio, restricciones, tipos de actores y operadores, y sus interacciones, son algunas de las principales ventajas de los enfoques basados en agentes. En este dominio, han surgido varias herramientas de modelado para simular, estudiar y resolver problemas de subdominios específicos de PES. Sin embargo, existe una limitación generalizada referida a la importante falta de interoperabilidad entre sistemas heterogéneos, que impide abordar el problema de manera global, considerando todas las interrelaciones relevantes existentes. Esto es esencial para que los jugadores puedan aprovechar al máximo las oportunidades en evolución. Por lo tanto, para lograr un marco tan completo aprovechando las herramientas existentes que permiten el estudio de partes específicas del problema global, se requiere la interoperabilidad entre estos sistemas. Las ontologías facilitan la interoperabilidad entre sistemas heterogéneos al dar un significado semántico a la información intercambiada entre las distintas partes. La ventaja radica en el hecho de que todos los involucrados en un dominio particular los conocen, comprenden y están de acuerdo con la conceptualización allí definida. Existen, en la literatura, varias propuestas para el uso de ontologías dentro de PES, fomentando su reutilización y extensión. Sin embargo, la mayoría de las ontologías se centran en un escenario de aplicación específico o en una abstracción de alto nivel de un subdominio de los PES. Además, existe una considerable heterogeneidad entre estos modelos, lo que complica su integración y adopción. Es fundamental desarrollar ontologías que representen distintas fuentes de conocimiento para facilitar las interacciones entre entidades de diferente naturaleza, promoviendo la interoperabilidad entre sistemas heterogéneos basados en agentes que permitan resolver problemas específicos de PES. Estas brechas motivan el desarrollo del trabajo de investigación de este doctorado, que surge para brindar una solución a la interoperabilidad de sistemas heterogéneos dentro de los PES. Las diversas aportaciones de este trabajo dan como resultado una sociedad de sistemas multi-agente (MAS por sus siglas en inglés) para la simulación, estudio, soporte de decisiones, operación y gestión inteligente de PES. Esta sociedad de MAS aborda los PES desde el EM mayorista hasta el SG y la eficiencia energética del consumidor, aprovechando las herramientas de simulación y apoyo a la toma de decisiones existentes, complementadas con las desarrolladas recientemente, asegurando la interoperabilidad entre ellas. Utiliza ontologías para la representación del conocimiento en un vocabulario común, lo que facilita la interoperabilidad entre los distintos sistemas. Además, el uso de ontologías y tecnologías de web semántica permite el desarrollo de herramientas agnósticas de modelos para una adaptación flexible a nuevas reglas y restricciones, promoviendo el razonamiento semántico para sistemas sensibles al contexto

Architecture-centric support for security orchestration and automation

Security Orchestration, Automation and Response (SOAR) platforms leverage integration and orchestration technologies to (i) automate manual and repetitive labor-intensive tasks, (ii) provide a single panel of control to manage various types of security tools (e.g., intrusion detection system, antivirus and firewall) and (iii) streamline complex Incident Response Process (IRP) responses. SOAR platforms increase the operational efficiency of overwhelmed security teams in a Security Operation Centre (SOC) and accelerate the SOC’s defense and response capacity against ever-growing security incidents. Security tools, IRPs and security requirements form the underlying execution environment of SOAR platforms, which are changing rapidly due to the dynamic nature of security threats. A SOAR platform is expected to adapt continuously to these dynamic changes. Flexible integration, interpretation and interoperability of security tools are essential to ease the adaptation of a SOAR platform. However, most of the effort for designing and developing existing SOAR platforms are ad-hoc in nature, which introduces several engineering challenges and research challenges. For instance, the advancement of a SOAR platform increases its architectural complexity and makes the operation of such platforms difficult for end-users. These challenges come from a lack of a comprehensive view, design space and architectural support for SOAR platforms. This thesis aims to contribute to the growing realization that it is necessary to advance SOAR platforms by designing, implementing and evaluating architecture-centric support to address several of the existing challenges. The envisioned research and development activities require the identification of current practices and challenges of SOAR platforms; hence, a Multivocal Literature Review (MLR) has been designed, conducted and reported. The MLR identifies the functional and non-functional requirements, components and practices of a security orchestration domain, along with the open issues. This thesis advances the domain of a SOAR platform by providing a layered architecture, which considers the key functional and non-functional requirements of a SOAR platform. The proposed architecture is evaluated experimentally with a Proof of Concept (PoC) system, Security Tool Unifier (STUn), using seven security tools, a set of IRPs and playbooks. The research further identifies the need for and design of (i) an Artificial Intelligence (AI) based integration framework to interpret the activities of security tools and enable interoperability automatically, (ii) a semantic-based automated integration process to integrate security tools and (iii) AI-enabled design and generation of a declarative API from user query, namely DecOr, to hide the internal complexity of a SOAR platform from end-users. The experimental evaluation of the proposed approaches demonstrates that (i) consideration of architectural design decisions supports the development of an easy to interact with, modify and update SOAR platform, (ii) an AI-based integration framework and automated integration process provides effective and efficient integration and interpretation of security tools and IRPs and (iii) DecOr increases the usability and flexibility of a SOAR platform. This thesis is a useful resource and guideline for both practitioners and researchers who are working in the security orchestration domain. It provides an insight into how an architecture-centric approach, with incorporation of AI technologies, reduces the operational complexity of SOAR platforms.Thesis (Ph.D.) -- University of Adelaide, School of Computer Science, 202

Towards Responsible Plant Data Linkage: Data Challenges for Agricultural Research and Development

This open access book provides the first systematic overview of existing challenges and opportunities for responsible data linkage, and a cutting-edge assessment of which steps need to be taken to ensure that plant data are ethically shared and used for the benefit of ensuring global food security – one of the UN’s Sustainable Development Goals. The volume focuses on the contemporary contours of such challenges through sustained engagement with current and historical initiatives and discussion of best practices and prospective future directions for ensuring responsible plant data linkage. The volume is divided into four sections that include case studies of plant data use and linkage in the context of particular research projects, breeding programs, and historical research. It address technical challenges of data linkage in developing key tools, standards and infrastructures, and examines governance challenges of data linkage in relation to socioeconomic and environmental research and data collection. Finally, the last section addresses issues raised by new data production and linkage methods for the inclusion of agriculture’s diverse stakeholders. This book brings together leading experts in data curation, data governance and data studies from a variety of fields, including data science, plant science, agricultural research, science policy, data ethics and the philosophy, history and social studies of plant science