Self-organized Data Ecologies for Pervasive Situation-Aware Services: the Knowledge Networks Approach

Pervasive computing services exploit information about the physical world both to adapt their own behavior in a context-aware way and to deliver to users enhanced means of interaction with their surrounding environment. The technology to acquire digital information about the physical world is increasingly available, making services at risk of being overwhelmed by such growing amounts of data. This calls for novel approaches to represent and automatically organize, aggregate, and prune such growing amounts of data before delivering it to services. In particular, individual data items should form a sort of self-organized ecology in which, by linking and combining with each other into sorts of “knowledge networks”, they can be able to provide to services compact and easy to be managed higher-level knowledge about situations occurring in the environment. In this context, the contribution of this paper is twofold. First, with the help of a simple case study, we motivate the need to evolve from models of “context-awareness” towards models of “situation-awareness” via proper self-organized “knowledge networks” tools, and introduce a general reference architecture for knowledge networks. Second, we describe the design and implementation of a knowledge network toolkit we have developed, and exemplify algorithms for knowledge self-organization integrated within it. Open issues and future research directions are also discussed

Logging Statements Analysis and Automation in Software Systems with Data Mining and Machine Learning Techniques

Log files are widely used to record runtime information of software systems, such as the timestamp of an event, the name or ID of the component that generated the log, and parts of the state of a task execution. The rich information of logs enables system developers (and operators) to monitor the runtime behavior of their systems and further track down system problems in development and production settings. With the ever-increasing scale and complexity of modern computing systems, the volume of logs is rapidly growing. For example, eBay reported that the rate of log generation on their servers is in the order of several petabytes per day in 2018 [17]. Therefore, the traditional way of log analysis that largely relies on manual inspection (e.g., searching for error/warning keywords or grep) has become an inefficient, a labor intensive, error-prone, and outdated task. The growth of the logs has initiated the emergence of automated tools and approaches for log mining and analysis. In parallel, the embedding of logging statements in the source code is a manual and error-prone task, and developers often might forget to add a logging statement in the software's source code. To address the logging challenge, many e orts have aimed to automate logging statements in the source code, and in addition, many tools have been proposed to perform large-scale log le analysis by use of machine learning and data mining techniques. However, the current logging process is yet mostly manual, and thus, proper placement and content of logging statements remain as challenges. To overcome these challenges, methods that aim to automate log placement and content prediction, i.e., `where and what to log', are of high interest. In addition, approaches that can automatically mine and extract insight from large-scale logs are also well sought after. Thus, in this research, we focus on predicting the log statements, and for this purpose, we perform an experimental study on open-source Java projects. We introduce a log-aware code-clone detection method to predict the location and description of logging statements. Additionally, we incorporate natural language processing (NLP) and deep learning methods to further enhance the performance of the log statements' description prediction. We also introduce deep learning based approaches for automated analysis of software logs. In particular, we analyze execution logs and extract natural language characteristics of logs to enable the application of natural language models for automated log le analysis. Then, we propose automated tools for analyzing log files and measuring the information gain from logs for different log analysis tasks such as anomaly detection. We then continue our NLP-enabled approach by leveraging the state-of-the-art language models, i.e., Transformers, to perform automated log parsing