Distributed resource discovery using a context sensitive infrastructure

Distributed Resource Discovery in a World Wide Web environment using full-text indices will never scale. The distinct properties of WWW information (volume, rate of change, topical diversity) limits the scaleability of traditional approaches to distributed Resource Discovery. An approach combining metadata clustering and query routing can, on the other hand, be proven to scale much better. This paper presents the Content-Sensitive Infrastructure, which is a design building on these results. We also present an analytical framework for comparing scaleability of different distribution strategies

Big-Data Solutions for Manufacturing Health Monitoring and Log Analytics

Modern semiconductor manufacturing is a complex process with a multitude of software applications. This application landscape has to be constantly monitored, since the communication and access patterns provide important insights. Because of the high event rates of the equipment log data stream in modern factories, big-data tools are required for scalable state and history analytics. The choice of suitable big-data solutions and their technical realization remains a challenging task. This thesis compares big-data architectures and discovers solutions for log-data ingest, enrichment, analytics and visualization. Based on the use cases and requirements of developers working in this field, a comparison of a custom assembled stack and a complete solution is made. Since the complete stack is a preferable solution, Datadog, Grafana Loki and the Elastic 8 Stack are selected for a more detailed study. These three systems are implemented and compared based on the requirements. All three systems are well suited for big-data logging and fulfill most of the requirements, but show different capabilities when implemented and used.:1 Introduction 1.1 Motivation 1.2 Structure 2 Fundamentals and Prerequisites 2.1 Logging 2.1.1 Log level 2.1.2 CSFW log 2.1.3 SECS log 2.2 Existing system and data 2.2.1 Production process 2.2.2 Log data in numbers 2.3 Requirements 2.3.1 Functional requirements 2.3.2 System requirements 2.3.3 Quality requirements 2.4 Use Cases 2.4.1 Finding specific communication sequence 2.4.2 Watching system changes 2.4.3 Comparison with expected production path 2.4.4 Enrichment with metadata 2.4.5 Decoupled log analysis 3 State of the Art and Potential Software Stacks 3.1 State of the art software stacks 3.1.1 IoT flow monitoring system 3.1.2 Big-Data IoT monitoring system 3.1.3 IoT Cloud Computing Stack 3.1.4 Big-Data Logging Architecture 3.1.5 IoT Energy Conservation System 3.1.6 Similarities of the architectures 3.2 Selection of software stack 3.2.1 Components for one layer 3.2.2 Software solutions for the stack 4 Analysis and Implementation 4.1 Full stack vs. a custom assembled stack 4.1.1 Drawbacks of a custom assembled stack 4.1.2 Advantages of a complete solution 4.1.3 Exclusion of a custom assembled stack 4.2 Selection of full stack solutions 4.2.1 Elastic vs. Amazon 4.2.2 Comparison of Cloud-Only-Solutions 4.2.3 Comparison of On-Premise-Solutions 4.3 Implementation of selected solutions 4.3.1 Datadog 4.3.2 Grafana Loki Stack 4.3.3 Elastic 8 Stack 5 Comparison 5.1 Comparison of components 5.1.1 Collection 5.1.2 Analysis 5.1.3 Visualization 5.2 Comparison of requirements 5.2.1 Functional requirements 5.2.2 System requirements 5.2.3 Quality requirements 5.3 Results 6 Conclusion and Future Work 6.1 Conclusion 6.2 Future WorkDie moderne Halbleiterfertigung ist ein komplexer Prozess mit einer Vielzahl von Softwareanwendungen. Diese Anwendungslandschaft muss ständig überwacht werden, da die Kommunikations- und Zugriffsmuster wichtige Erkenntnisse liefern. Aufgrund der hohen Ereignisraten des Logdatenstroms der Maschinen in modernen Fabriken werden Big-Data-Tools für skalierbare Zustands- und Verlaufsanalysen benötigt. Die Auswahl geeigneter Big-Data-Lösungen und deren technische Umsetzung ist eine anspruchsvolle Aufgabe. Diese Arbeit vergleicht Big-Data-Architekturen und untersucht Lösungen für das Sammeln, Anreicherung, Analyse und Visualisierung von Log-Daten. Basierend auf den Use Cases und den Anforderungen von Entwicklern, die in diesem Bereich arbeiten, wird ein Vergleich zwischen einem individuell zusammengestellten Stack und einer Komplettlösung vorgenommen. Da die Komplettlösung vorteilhafter ist, werden Datadog, Grafana Loki und der Elastic 8 Stack für eine genauere Untersuchung ausgewählt. Diese drei Systeme werden auf der Grundlage der Anforderungen implementiert und verglichen. Alle drei Systeme eignen sich gut für Big-Data-Logging und erfüllen die meisten Anforderungen, zeigen aber unterschiedliche Fähigkeiten bei der Implementierung und Nutzung.:1 Introduction 1.1 Motivation 1.2 Structure 2 Fundamentals and Prerequisites 2.1 Logging 2.1.1 Log level 2.1.2 CSFW log 2.1.3 SECS log 2.2 Existing system and data 2.2.1 Production process 2.2.2 Log data in numbers 2.3 Requirements 2.3.1 Functional requirements 2.3.2 System requirements 2.3.3 Quality requirements 2.4 Use Cases 2.4.1 Finding specific communication sequence 2.4.2 Watching system changes 2.4.3 Comparison with expected production path 2.4.4 Enrichment with metadata 2.4.5 Decoupled log analysis 3 State of the Art and Potential Software Stacks 3.1 State of the art software stacks 3.1.1 IoT flow monitoring system 3.1.2 Big-Data IoT monitoring system 3.1.3 IoT Cloud Computing Stack 3.1.4 Big-Data Logging Architecture 3.1.5 IoT Energy Conservation System 3.1.6 Similarities of the architectures 3.2 Selection of software stack 3.2.1 Components for one layer 3.2.2 Software solutions for the stack 4 Analysis and Implementation 4.1 Full stack vs. a custom assembled stack 4.1.1 Drawbacks of a custom assembled stack 4.1.2 Advantages of a complete solution 4.1.3 Exclusion of a custom assembled stack 4.2 Selection of full stack solutions 4.2.1 Elastic vs. Amazon 4.2.2 Comparison of Cloud-Only-Solutions 4.2.3 Comparison of On-Premise-Solutions 4.3 Implementation of selected solutions 4.3.1 Datadog 4.3.2 Grafana Loki Stack 4.3.3 Elastic 8 Stack 5 Comparison 5.1 Comparison of components 5.1.1 Collection 5.1.2 Analysis 5.1.3 Visualization 5.2 Comparison of requirements 5.2.1 Functional requirements 5.2.2 System requirements 5.2.3 Quality requirements 5.3 Results 6 Conclusion and Future Work 6.1 Conclusion 6.2 Future Wor

The 'what' and 'how' of learning in design, invited paper

Previous experiences hold a wealth of knowledge which we often take for granted and use unknowingly through our every day working lives. In design, those experiences can play a crucial role in the success or failure of a design project, having a great deal of influence on the quality, cost and development time of a product. But how can we empower computer based design systems to acquire this knowledge? How would we use such systems to support design? This paper outlines some of the work which has been carried out in applying and developing Machine Learning techniques to support the design activity; particularly in utilising previous designs and learning the design process

Techniques for organizational memory information systems

The KnowMore project aims at providing active support to humans working on knowledge-intensive tasks. To this end the knowledge available in the modeled business processes or their incarnations in specific workflows shall be used to improve information handling. We present a representation formalism for knowledge-intensive tasks and the specification of its object-oriented realization. An operational semantics is sketched by specifying the basic functionality of the Knowledge Agent which works on the knowledge intensive task representation. The Knowledge Agent uses a meta-level description of all information sources available in the Organizational Memory. We discuss the main dimensions that such a description scheme must be designed along, namely information content, structure, and context. On top of relational database management systems, we basically realize deductive object- oriented modeling with a comfortable annotation facility. The concrete knowledge descriptions are obtained by configuring the generic formalism with ontologies which describe the required modeling dimensions. To support the access to documents, data, and formal knowledge in an Organizational Memory an integrated domain ontology and thesaurus is proposed which can be constructed semi-automatically by combining document-analysis and knowledge engineering methods. Thereby the costs for up-front knowledge engineering and the need to consult domain experts can be considerably reduced. We present an automatic thesaurus generation tool and show how it can be applied to build and enhance an integrated ontology /thesaurus. A first evaluation shows that the proposed method does indeed facilitate knowledge acquisition and maintenance of an organizational memory

The Semantic Grid: A future e-Science infrastructure

e-Science offers a promising vision of how computer and communication technology can support and enhance the scientific process. It does this by enabling scientists to generate, analyse, share and discuss their insights, experiments and results in an effective manner. The underlying computer infrastructure that provides these facilities is commonly referred to as the Grid. At this time, there are a number of grid applications being developed and there is a whole raft of computer technologies that provide fragments of the necessary functionality. However there is currently a major gap between these endeavours and the vision of e-Science in which there is a high degree of easy-to-use and seamless automation and in which there are flexible collaborations and computations on a global scale. To bridge this practice–aspiration divide, this paper presents a research agenda whose aim is to move from the current state of the art in e-Science infrastructure, to the future infrastructure that is needed to support the full richness of the e-Science vision. Here the future e-Science research infrastructure is termed the Semantic Grid (Semantic Grid to Grid is meant to connote a similar relationship to the one that exists between the Semantic Web and the Web). In particular, we present a conceptual architecture for the Semantic Grid. This architecture adopts a service-oriented perspective in which distinct stakeholders in the scientific process, represented as software agents, provide services to one another, under various service level agreements, in various forms of marketplace. We then focus predominantly on the issues concerned with the way that knowledge is acquired and used in such environments since we believe this is the key differentiator between current grid endeavours and those envisioned for the Semantic Grid

DSpace Manual: Software version 1.5

DSpace is an open source software platform that enables organizations to: - Capture and describe digital material using a submission workflow module, or a variety of programmatic ingest options - Distribute an organization's digital assets over the web through a search and retrieval system - Preserve digital assets over the long term This system documentation includes a functional overview of the system, which is a good introduction to the capabilities of the system, and should be readable by nontechnical personnel. Everyone should read this section first because it introduces some terminology used throughout the rest of the documentation. For people actually running a DSpace service, there is an installation guide, and sections on configuration and the directory structure. Note that as of DSpace 1.2, the administration user interface guide is now on-line help available from within the DSpace system. Finally, for those interested in the details of how DSpace works, and those potentially interested in modifying the code for their own purposes, there is a detailed architecture and design section

The Advanced Framework for Evaluating Remote Agents (AFERA): A Framework for Digital Forensic Practitioners

Digital forensics experts need a dependable method for evaluating evidence-gathering tools. Limited research and resources challenge this process and the lack of multi-endpoint data validation hinders reliability in distributed digital forensics. A framework was designed to evaluate distributed agent-based forensic tools while enabling practitioners to self-evaluate and demonstrate evidence reliability as required by the courts. Grounded in Design Science, the framework features guidelines, data, criteria, and checklists. Expert review enhances its quality and practicality

Space Plug-and-Play Architecture Networking: A Self-Configuring Heterogeneous Network Architecture

The Space Plug-and-Play Architecture (SPA) networking approach outlined in this dissertation is an improvement over the previous approach used by the Satellite Data Model(SDM). The first improvement is the introduction of a SPA network model based on the Open Systems Interconnection (OSI) model. Second, a new addressing and routing scheme is presented, which places the burden of routing on the network infrastructure instead of the network endpoints. These improvements have been implemented in a software infrastructure called the SPA Services Manager (SSM). The SSM was developed under an International Organization for Standardization (ISO) 9001 certified development process, the details of which are presented. A collection of network timing graphs that measure latency and jitter of the SPA network is contained in this dissertation, as well as a runtime memory footprint. The maturity of the development process and these initial performance measurements demonstrate that the SSM is qualified for spaceflight

A Dynamic Service Lookup and Discovery Scheme using a Self-Organizing Overlay Network for Indoor Location-Based Service

REACTION 2012. 1st International workshop on Real-time and distributed computing in emerging applications. December 4th, 2012, San Juan, Puerto Rico.Indoor location-based service (LBS) has various challenges, including that of numerous service lookups being requested concurrently and frequently from several locations, and that the network infrastructure needs to support high scalability, such as inserting or deleting network nodes anytime and anywhere. In general, indoor LBS resources are generally located in close proximity to the requested point. However, a traditional centralized LBS system needs to maintain a geographical map of the entire building or complex in its central server, which can cause low scalability and traffic congestion. This paper presents a self-organizing and fully distributed indoor LBS platform through regional cooperation among devices, and a service lookup algorithm that searches for the shortest physical path to the service resource. An evaluation of the performance of the proposed platform has been compared to the traditional centralized method regarding the service turnaround time according to the number of concurrent lookup increases.This work was supported by the IT R&D program of MKE/KEIT. [10041145, Self-Organized Softwareplatform(SOS) for welfare devices